浙大论文：决策模型自身做不好任务，也能当步级评分器训练更强的智能体
浙大这篇论文挺有意思：一个自己做任务只有 42.9% 成功率的 JEV 决策模型，转行当裁判反而把 Qwen2.5-1.5B 从 72.8% 带到 82.9%。做智能体 RL 训练的人可以直接抄它的评分器选法，不用再为逐步打分标注数据。
浙江大学的新论文提出了一个反直觉结论：一个自己执行任务很差的 AI 模型，仍能通过事后复盘把另一个模型训练得更好。决策模型 JEV 在 ALFWorld 上自己只能完成 42.9% 的模拟家务任务，但作为评分器使用时，能把 Qwen2.5-1.5B 策略的成功率从 GRPO 结果奖励下的 72.8% 提升到 82.9%。
技术机制在于解决 Agent RL 的信用分配问题：传统方法只奖励最终结果，每个动作拿到相同的信用；而逐步打分通常需要逐环境的标签或每一步调用 LLM 裁判。论文的方案是让决策模型（返回选项概率分布而非文本）看完整个运行记录后，对每个动作是否有助于任务完成逐一打分，分数再注入 GRPO 训练。
研究在 ALFWorld 和 WebShop 两个基准上验证了决策模型作为执行者和评分者的双重角色。对开发者的实操建议是：挑选步级评分器时看它评判已录制的运行轨迹的能力，而不是它自己的执行能力，并且评分要针对单个动作而非最终结果。
AI model that is itself bad at a task can still train another AI to do it well.
New paper shows that a decision model called JEV, an AI model built to pick between options instead of writing text, completes only 42.9% of simulated household tasks on its own, yet by reviewing another AI’s moves afterward it raised that AI’s success from 72.8% to 82.9%.
New Zhejiang University paper shows that an AI model that is bad at a task can still train another AI to do it well, by reviewing each of its moves after the attempt ends.
Agent RL usually rewards only the final outcome, so every step gets the same credit. Scoring each step usually needs per-environment labels or an LLM judge at every action.
They tested decision models, which return a choice with probabilities instead of text, as actors and as step graders on ALFWorld and WebShop. As a grader, the model sees the whole run and scores whether each action helped, and those scores are added to GRPO training.
The decision model JEV completes only 42.9% of ALFWorld tasks when acting. As a grader, it lifts a Qwen2.5-1.5B policy to 82.9% success, up from 72.8% with GRPO outcome rewards alone.
Choose step graders by how well they judge recorded runs, not how well they act, and ask about each action, not the outcome.
本条情报归属 AITOP「论文」核心栏目，重点聚焦中国 AI 产业与国产大模型生态。在跨区域产业观察坐标中，该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线，反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括：JEV、Qwen2.5、GRPO、ALFWorld，读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明：本文原始线索采编自「rohanpaul_ai」（@rohanpaul_ai），经 AITOP 多维评分管线结构化提炼。截至目前，全网已有 2 家独立信源对该事件进行交叉印证与跟进报道（含 elvis 等），属于近期高关注度行业信号。
- elvis11-13 14:47原文
