论文

CoC-CFG:用步级引导量化 VLA 推理对动作的影响

How Much Should a VLA Trust Its Reasoning? Step-Level Guidance as a Measured Action-Value Function

精选理由

教 VLA 模型别太信自己的推理链:每步测出该给推理多大权重,驾驶闭环从 252 提到 262,还顺手在 Waymo 榜上刷了 SOTA。

CoC-CFG 在 VLA 策略解码时,对含推理与不含推理的上下文之间施加 classifier-free guidance,用每个 Euler 步一个权重来衰减或放大推理的影响,w=1 时恢复原始策略。论文把权重的选择建模为 Markov 决策过程,动作值通过冻结专家、固定噪声直接测量得到而非自举,再用小 critic 拟合这些值做 Q-adaptive 引导。在 Alpasim 驾驶模拟器的闭环测试中,该方法通过 262/300 个场景,原始策略为 252 个,并超过固定权重与 SAC、GRPO 在线 RL。在 Waymo 端到端驾驶基准的 held-out 帧上,其前三个阶段也超过固定权重、四种自适应引导方法和 SAC、GRPO。在隐藏测试集上,冻结的 Alpamayo 1.5 配合规则化相机与车辆转换、Q-adaptive 引导,以 8.135159 的 rater feedback 分数超过已发表最佳方法的 8.079,创下已发表方法中的新纪录。

原文 · arXiv cs.LG

How Much Should a VLA Trust Its Reasoning? Step-Level Guidance as a Measured Action-Value Function

Reasoning vision-language-action (VLA) policies write a reasoning trace and then decode the action from it. A wrong trace yields a wrong action, and even a correct-looking trace can mislead the plan. Existing remedies ask whether the reasoning is right. We ask how strongly it should steer. CoC-CFG applies classifier-free guidance between the policy's context and the same context without the reasoning span. One weight per Euler step attenuates or amplifies the reasoning, and w=1 recovers the original policy. We treat the choice of this weight as a Markov decision process whose action values are measured, not bootstrapped. With the expert frozen and the noise fixed, applying a weight for one step and finishing the plan under the current controller returns that weight's value exactly. Q-adaptive guidance fits these values with small critics and improves the controller conservatively. In the Alpasim driving simulator it outperforms every fixed weight and online RL (SAC, GRPO) in open loop, and in closed loop it passes 262 of 300 scenes, against 252 for the original policy. On held-out frames of the Waymo end-to-end driving benchmark, its first three stages outperform every fixed weight, four adaptive-guidance methods, SAC and GRPO. On the hidden test set, a frozen Alpamayo 1.5 with rule-based camera and vehicle conversions and Q-adaptive guidance sets a new state of the art among published methods: 8.135159 rater feedback score, against 8.079 for the best published method.