论文精选

Google 论文:优化器可在冻结 Gemini 3.8 Flash 上重写 agent harness 实现 self-improvement

精选理由

Google 这篇论文把 agent harness 拆成四个模块,让优化器每次只改一个,token 花费从 451.6k 降到 46.7k,性能只掉 2.5 个点,还能直接迁移到 Gemini 4 Argon。做 agent 开发的值得看看 gated 编辑这套流程,论文里的失败模式清单也很实用。

Google 发布一篇关于 self-improving agent harness 的论文,在 SWE-bench Lite 上让优化器围绕冻结的 Gemini 3.8 Flash 重写 harness,并对比三种方案。直接改 prompt 规则文件反而使 held-out 准确率从 5.0% 降到 2.5%;整体重写 harness 达到 42.5%,但每个任务消耗 451.6k tokens;每次只改动一个模块的 gated 方案达到 40.0%,token 降到 46.7k。

正文

gated 方案把 harness 拆成 context、control、tools、verification 四个模块,每轮只 patch 一个模块,在 held-out 任务上设置 token 上限做验证,通过后才部署。这个 gated harness 还零搜索迁移到 Gemini 4 Argon,解决 37.5% 的问题,高于整体重写版的 30.0%。论文同时列出 harness 自编辑时的失败模式,包括 prompt bloat、benchmark gaming 和 evaluator tampering。

作者提醒测试集只有 40 道 held-out 问题,数字只能作方向性参考。对做 agent 工程的人来说,这篇的价值在于用具体数据说明:与其整体重写,不如把 harness 模块化并加验证门控,token 成本相差近一个数量级而性能只差 2.5 个百分点。

原文 · elvis

Another very interesting paper from Google on self-improving agent harnesses. (bookmark it) They let an optimizer rewrite the harness around a frozen Gemini 3.8 Flash on SWE-bench Lite and compared three approaches. - A prompt rule file lowered held-out accuracy from 5.0% to 2.5%. - A full rewrite of the harness reached 42.5% at 451.6k tokens per task. - Gated edits to one module at a time reached 40.0% at 46.7k tokens per task. The gated version splits the harness into four parts (context, control, tools, and verification). Each round it patches one part, checks the patch on held-out tasks under a token cap, and deploys it only if it passes. The gated harness also transferred to Gemini 4 Argon with no new search, solving 37.5% against 30.0% for the full rewrite. The test set is small (40 held-out issues), so treat the numbers as directional. The paper also lists failure modes to guard against when a harness edits itself, including prompt bloat, benchmark gaming, and evaluator tampering. arxiv.org/abs/2611.22299 Chat with Paper: academy.dair.ai/papers/harness…📊 13 ⚡

产业坐标与信源核验

本条情报归属 AITOP「论文」核心栏目,重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:Gemini、SWE-bench、agent harness、论文,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始线索采编自「elvis」(@omarsar0),经 AITOP 多维评分管线结构化提炼。