论文

ICL 加 GRPO 让 LLM Agent 测试时成功率升至 51.6%

Where Should LLM Agents Adapt at Test Time? In Context, in Weights, or Both

精选理由

如果你在做多轮 Agent 或经验复用类项目,这篇论文给出的 51.6% 数字很扎实。它把 ICL 和 GRPO 放在同一套协议里对比,结论清楚:上下文管大头,强化学习修细节。做 SWE-bench 或 SQL Agent 的开发者尤其该看看。

论文系统比较了 LLM Agent 在测试时通过上下文(ICL)与权重更新(GRPO、SDPO)两条通道利用经验的差异。实验基于 Qwen3.5-9B,在来自 SWE-bench Verified、AppWorld、ScienceWorld、BIRD-Critic-SQLite 四个环境的 109 个任务上进行,共评估 14 种适应机制。

正文

结果显示 ICL 是最大的独立增益来源:最佳策略把平均成功率从 28.8% 提升到 49.4%;在 ICL 基础上叠加 GRPO 后达到最高的 51.6% 平均成功率。轨迹分析表明 ICL 让 Agent 减少了环境查询、缩短了尝试步骤,说明它在复用任务间的共享结构,但也常跳过对当前任务的检查与验证。

叠加 GRPO 能修复这些被跳过的检查行为,同时保留 ICL 减少查询的优势,而 SDPO 的自蒸馏效果则好坏参半。论文由此主张将上下文管理与权重更新作为联合问题研究,对构建长期在代码仓库、数据库等环境中反复工作的 Agent 有直接参考价值。

原文 · arXiv cs.AI

Where Should LLM Agents Adapt at Test Time? In Context, in Weights, or Both

LLM agents operating repeatedly in the same environment, such as a code repository, a database, or an application suite, encounter tasks that share latent structure. Experience can reveal this structure, allowing agents to reuse environment knowledge and learn from environmental feedback. Agents can adapt at test-time using this experience through two channels: retaining it in context and updating their weights. These channels are usually studied separately, leaving unclear what each contributes and when they complement each other. We compare in-context learning (ICL), reinforcement learning with verifiable rewards using GRPO, and self-distillation using SDPO, alone and in combination, using a shared online protocol and modular framework. Agents encounter matched task streams, are evaluated on each task before adapting to it, and learn only from their own attempts and environment feedback. We evaluate 14 adaptation mechanisms and a stateless baseline with Qwen3.5-9B on 109 tasks from four environments drawn from SWE-bench Verified, AppWorld, ScienceWorld, and BIRD-Critic-SQLite, using five task orders per environment. ICL provides the largest standalone gain: its best policy raises mean success across environments from 28.8% to 49.4%. Adding GRPO to this policy achieves the highest mean success, 51.6%. While adding GRPO improves the overall mean under every ICL policy, SDPO has mixed effects. Trace analyses show fewer lookups and shorter attempts under ICL, consistent with reusing knowledge of the shared structure. However, ICL also frequently skips inspection and verification for the current task. Adding GRPO to ICL helps restore these behaviors while preserving its reduced environment lookups. These observations suggest that reinforcement learning with verifiable rewards can refine how agents use retained experience, motivating the joint study of context management and weight updates.

产业坐标与信源核验

本条情报归属 AITOP「论文」核心栏目,重点聚焦中国 AI 产业与国产大模型生态。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:GRPO、ICL、Qwen3.5-9B、SWE-bench,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始线索采编自「arXiv cs.AI」(@Parsa Mirtaheri, Mohammadreza Armandpour, Keivan Alizadeh, Iman Mirzadeh, Parshin Shojaee, Aryo Lotfi, David Harrison, Mehrdad Farajtabar),经 AITOP 多维评分管线结构化提炼。截至目前,全网已有 2 家独立信源对该事件进行交叉印证与跟进报道(含 rohanpaul_ai 等),属于近期高关注度行业信号。