Alibaba 与 Cornell 研究编程智能体验证机制：独立 critic 性价比最高
Alibaba 和 Cornell 下了 17 种验证方案的对比，做智能体框架的可以看看：结尾加个独立 critic 几乎不加钱就能多拿 3.9 到 6.6 分。
Alibaba 和 Cornell 的论文对比了 17 种验证设计，覆盖七个前沿编程模型，测试基准为 Terminal-Bench 和 SWE-bench Verified。验证设计分为四个位置：初始指令、运行中提醒、完成前门控和结尾独立 critic。结果显示独立 reviewer 在 Terminal-Bench 上提升 5.9 到 9.5 分，成本仅为基线的 1.13 到 1.25 倍。独立 critic 在 SWE-bench Verified 上加 3.9 到 6.6 分且几乎无额外成本，是单位成本收益最高的方案。增加 critic 轮数最多还能再加 3.5 分。
Great paper from Alibaba and Cornell on verification in coding-agent harnesses.
(bookmark it)
If you build agent harnesses, this one is worth your time.
They find that a separate reviewer agent adds 5.9 to 9.5 points on Terminal-Bench at 1.13 to 1.25x the baseline cost.
They compared 17 verification designs across seven frontier coding models on Terminal-Bench and SWE-bench Verified. The designs check the agent's work at four points: the initial instructions, reminders during the run, a gate before the agent can finish, and an independent critic at the end.
Instructions and reminders gave small, model-dependent gains. Completion gates gave larger gains, sometimes at much higher cost. The critic gave the best accuracy per dollar, adding 3.9 to 6.6 points on SWE-bench Verified at almost no extra cost. More review rounds added up to 3.5 more points on Terminal-Bench.
https://t.co/uYBKCDSd50
Chat with Paper: https://t.co/Y3G0M6WpPo
