MCGPO：基于匹配上下文的 LLM 智能体训练新方法
Matched-Context Group Policy Optimization for LLM Agent Training
训练智能体只在终点给反馈、中间决策没法打分？这篇 MCGPO 直接复用已有轨迹做加权偏好，Qwen3-8B 在 WebShop 上提了 5 个点，还不额外加 critic。
论文提出 Matched-Context Group Policy Optimization（MCGPO），从已采集轨迹中构造加权动作偏好，用于长程 LLM 智能体的稀疏反馈训练。方法用动作前状态摘要和规则进度定义对比上下文，在固定折叠中将候选决策与终端结果分离，保持无 critic 训练。在 ALFWorld 和 WebShop 两个环境上配合 Qwen2.5-1.5B/7B、Qwen3-4B/8B 共六种设置测试，相比 GraphGPO 基线，ALFWorld/WebShop 在 Qwen2.5 上提升 0.47-1.46 个百分点，WebShop 在 Qwen3-4B 和 8B 上分别提升 3.96 和 5.16 个百分点。三种子因子对照实验验证了对折证据和证据加权的有效性。
Matched-Context Group Policy Optimization for LLM Agent Training
Training long-horizon language model agents requires assigning credit to intermediate decisions from sparse terminal feedback. Within a finite group of sibling rollouts, useful action comparisons depend on their context, evidence for both alternatives, and how that evidence enters learning. We propose Matched-Context Group Policy Optimization (MCGPO), a procedure for constructing weighted action preferences from already collected trajectories. MCGPO defines comparison contexts with pre-action state summaries and observed rubric progress, annotated by task rules. Two-action support selects among ordered exact and coarse matching rules. Fixed folds separate candidate decisions from the terminal outcomes used to orient their preferences, and support-dependent evidence weights scale an auxiliary pairwise objective alongside shared GraphGPO task learning. MCGPO reuses rollout data and each response's original history, preserving critic-free training. Across six environment-backbone settings with three training seeds each, MCGPO improves mean success over matched GraphGPO: by 0.47-1.46 percentage points on ALFWorld and WebShop with Qwen2.5-1.5B and 7B, and by 3.96 and 5.16 points on WebShop with Qwen3-4B and 8B. The ALFWorld gains build on baseline success above 92%, where remaining headroom is limited. Integrated success improves in three of four Qwen2.5 settings. Three-seed factorial controls support opposite-fold evidence and evidence weighting; joint state-rubric matching achieves the highest mean in all four representation-control settings.
