论文提出视觉可归因的在线自蒸馏强化学习方法 VAG
Reinforcement Learning with Visually Attributable On-Policy Self-Distillation
视觉推理 RL 训练里错误定位一直很难，这篇论文的 VAG 用证据对比来过滤教师修正，在 6 个基准上超过了 GRPO，做训练的可以看看。
论文针对多模态 RLVR 训练中难以定位视觉推理错误的问题，提出 VAG（Visual-Attribution Gating）方法。该方法将特权教师给出的 token 级修正与“有证据/无证据”两种视觉视图做对比验证，只保留方向上可归因于视觉证据的修正。在 6 个视觉推理基准上，VAG 的平均成绩超过 GRPO 和 OPSD 方法，创下新的 SOTA 平均表现。论文编号为 arXiv:2610.26856v1。
Reinforcement Learning with Visually Attributable On-Policy Self-Distillation
Multimodal reinforcement learning with verifiable rewards (RLVR) provides reliable task-level supervision, yet offers limited guidance for localizing errors in visual reasoning. On-policy self-distillation (OPSD) complements this supervision with dense token-level corrections from a privileged teacher equipped with an evidence-centered visual view. However, the resulting corrections may conflate visual evidence with non-visual shifts induced by linguistic priors, response style, or the privileged view itself. We formalize this phenomenon as privileged-correction contamination: a token-level correction is contaminated when its update direction cannot be consistently attributed to the privileged visual evidence. Directly distilling contaminated corrections may therefore reinforce erroneous token-level updates. To address this issue, we propose reinforcement learning with visually attributable on-policy self-distillation, an evidence-consistent paradigm that augments task-level RLVR supervision with privileged token-level corrections grounded in visual evidence. We instantiate this paradigm with Visual-Attribution Gating (VAG), which uses the verifier-derived advantage as the optimization anchor and validates each privileged correction against the contrast between evidence-present and evidence-degraded teacher views. By admitting only corrections directionally supported by this visual contrast, VAG preserves visually grounded guidance while suppressing inconsistent residuals. Experiments across six visual reasoning benchmarks show that VAG consistently outperforms GRPO and OPSD methods, establishing new state-of-the-art average performance.
