SGQN论文：自适应动作网格未必是性能提升的关键
Selective Growing Q-Networks (SGQN): An Analysis of Adaptive Action Grids in Decoupled Q-Learning
作者亲手用消融证明自己的核心机制没用，真正有效的是粗网格起步和 epsilon-greedy，这类论文不多见。
SGQN 在解耦 Q-learning（DecQN）基础上，把 GQN 的均匀网格增长改为自适应扩展：在估计 Q 值高的 bin 旁添加新 bin、剪掉低 Q 值 bin，动作按 epsilon-greedy 选择。在 DeepMind Control Suite 的 8 个任务（动作维度 2 到 12）上，SGQN 在 4 个任务取得对比方法中最高的平均最终回报，3 个任务与最佳方法差距在 1.1% 以内，仅 Walker Run 上比 Bang-Bang MPO 低 3.4%，平均每维使用 9 个 bin 中的 7.5 个。但消融实验显示自适应网格并非增益来源：增长机制实际会激活所有候选 bin，行为等同均匀增长，去掉剪枝后回报变化不超过 1.1%。真正起作用的两个因素是：从粗网格起步（一开始就用全部 9 个 bin 会导致种子训练崩溃），以及 epsilon-greedy 选择（换成退火 softmax 会让回报下降 23% 到 100%）。论文同时指出 SGQN 相比 GQN 和 DecQN 基线存在未控制的差异——每环境步做 1 次梯度更新而基线是每 8 步 1 次，因此不把优势归因于单一组件。
Selective Growing Q-Networks (SGQN): An Analysis of Adaptive Action Grids in Decoupled Q-Learning
Value-based Q-learning for continuous control can discretise the action space, but the resolution of the discretisation trades early exploration against final precision. Growing Q-Networks (GQN) grow the resolution uniformly from coarse to fine. We introduce Selective Growing Q-Networks (SGQN), which keeps decoupled Q-learning and replaces uniform growth with an adaptive action grid: bins are added next to bins with high estimated Q-value, bins with low Q-value are pruned, and actions are chosen epsilon-greedily. On eight DeepMind Control Suite (DMControl) tasks with 2 to 12 action dimensions, SGQN has the highest mean final return among the compared methods on four, is within 1.1% of the best method on three more, and is 3.4% below Bang-Bang MPO on Walker Run, using on average 7.5 of 9 available bins per dimension. We then test whether the adaptive grid explains these results, and find that it does not. Growth activates every candidate bin and behaves like uniform growth, and removing pruning changes the return by at most 1.1%. Two other choices matter: starting from a coarse grid, since starting with all 9 bins produces collapsed seeds, and epsilon-greedy selection, since an annealed softmax lowers the return by 23% to 100%. SGQN is insensitive to its growth and pruning hyperparameters. It also differs from the GQN and DecQN baselines in other respects, notably one gradient update per environment step instead of one per eight, which we did not control for, so we do not attribute its advantage to a single component. We present SGQN as a competitive value-based method and as an analysis of which ingredients of a growing action grid matter.
