OpenBMB 发布 PostTrainBench 分析：训练智能体策略固化问题
清华团队测了1300多条训练轨迹，发现智能体只会埋头干活不会换思路，人类一句话能白捡17个点，做自动训练的都该看看。
OpenBMB 基于 PostTrainBench 的 1,338 条完整训练轨迹，拆解了后训练智能体的执行能力与策略能力，覆盖 7 个基准、4 个基础模型和 Claude Code / Codex CLI / OpenCode 三种框架。智能体平均分从 33.34 提升到 48.55，但 3,557 组相邻实验中只有 2.1% 真正切换过策略，策略层面的建议零采纳。额外经验和推理算力无法解锁策略切换，AIME 2025 上约 1400 万额外 token 只多解出 1 题。而在中途加一句人类指引切换到备选策略，GSM8K 提升 17.44 个百分点、HumanEval 提升 10.80 个百分点、AIME 2025 提升 6.67 个百分点。
An agent that can run a full post-training pipeline almost never reconsiders its initial strategy. We analyze 1,338 complete training trajectories from PostTrainBench — 7 benchmarks, 4 base models, Claude Code / Codex CLI / OpenCode, 10h on one H100 — and split capability into EXECUTION vs STRATEGY.
1️⃣ Qualified executor, locked strategy. Agents improve over base models on every benchmark; with experience, avg score goes 33.34 → 48.55. ALL execution-level suggestions were adopted, but NONE of the strategy-level suggestions (e.g., switching to RL, adding SFT warmup) were adopted. Only 2.1% of 3,557 adjacent experiment pairs actually switch strategy. Defaults follow the agent, not the task: Claude Code starts full-parameter SFT 71.9% of the time; Codex CLI starts PEFT/LoRA 89.5%.
2️⃣ Experience and compute don't unlock it. Extra experience and memory raise execution but never leave the locked strategy; more reasoning tokens buy finer execution rather than better direction — on AIME 2025, ~14M extra tokens solve just 1 more problem.
3️⃣ A mid-run human guidance wins everywhere. At the same checkpoint, same remaining budget, one sentence switching to the alternative strategy beats autonomous continuation: GSM8K +17.44pp, HumanEval +10.80pp, AIME 2025 +6.67pp.
The missing ability isn't executing a new strategy — it's the trigger to spontaneously restart strategy decisions mid-trajectory.
📄 Paper: https://t.co/yuI2gosUXC #AIAgent #PostTraining #LLM #RLHF #GRPO #PPO #RSI #AgentBenchmark #PostTrainBench #THUNLP
