论文

IBM 团队发布智能体自蒸馏记忆管线:记忆多少该由模型决定

How Much Should an Agent Remember? Deployment Lessons from Self-Distilled Memory for LLM Agents

精选理由

做智能体记忆的团队应该看看这篇:IBM 开源了整套蒸馏管线,还给 DeepSeek-V3.2 和 gpt-oss-120b 各配了最佳记忆量。最值钱的是踩坑部分,比如日志正常但任务全挂的 API 错误。

IBM 研究团队在 arXiv 发表部署经验报告,介绍一套让智能体从自身轨迹中学习的方法:先从每条轨迹提炼 1-3 条 lesson,再合并成 guidelines,每条记录合并了多少 lesson,推理时注入全部或子集。管线已开源为 altk-evolve 库,并部署在 IBM 的 CUGA 智能体和 watsonx Orchestrate 平台上。

正文

评估基于 AppWorld 基准,覆盖基线任务完成率 40%-92% 的多个模型。核心发现是:合适的记忆量取决于模型本身。DeepSeek-V3.2 使用全部 191 条 guidelines 后任务完成率提升 9.5 个百分点、场景完成率提升 16.1 个百分点;而 gpt-oss-120b 基线仅 39.9%,用全部 199 条 guidelines 到 51.2%,但精选的约 29 条反而达到 55.4%(两次运行均值)。

文章还详细复盘了踩坑过程:一条高频 guideline 反而有害、合并时来源信息被静默丢失、被吞掉的 API 错误让所有任务失败但日志看起来正常,以及一个几乎导致误导性对比的基线混淆。这些教训被整理成面向部署团队的建议。

原文 · arXiv: DeepSeek

How Much Should an Agent Remember? Deployment Lessons from Self-Distilled Memory for LLM Agents

LLM agents on long-horizon tasks fail in recurring, procedural ways. They mis-paginate an API, resolve the wrong entity, or answer when no answer was asked. We built a pipeline that lets an agent learn from its own history. It first mines short \emph{lessons} from the agent's own trajectories, one to three per episode. It then consolidates those lessons into a smaller set of \emph{guidelines}, where each guideline records how many lessons it merged. At inference the agent receives either all guidelines or a selected subset. The pipeline ships in the open-source altk-evolve library and is deployed in IBM's CUGA agent and watsonx Orchestrate platform. We report what we learned building, measuring, and productizing it. Our primary instrument is the AppWorld benchmark, across models spanning 40--92\% baseline task completion. The central finding is that the right amount of memory is a property of the model, not the memory. A strong model keeps improving as guidelines are added. DeepSeek-V3.2 gains +9.5 points of task completion and +16.1 points of scenario completion with its full 191-guideline set. A weak model gains less from the same full set than from a small selected subset. gpt-oss-120b starts at 39.9\% task completion. It reaches 51.2\% with all 199 of its guidelines but 55.4\% (mean of two runs) with a selected subset of about 29. Around that result we report what went wrong: a high-support guideline that backfired, provenance silently dropped during consolidation, a swallowed API error that failed every task while producing a plausible log, and a baseline confound that nearly produced a misleading comparison with a prior method. We distill these into recommendations for teams deploying agent memory.

产业坐标与信源核验

本条情报归属 AITOP「论文」核心栏目,重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」「中美大模型技术代差与商业化对照」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:DeepSeek-V3.2、gpt-oss-120b、AppWorld、智能体,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始信息发布自一手权威信源「arXiv: DeepSeek」(Tier-1 一手发布),具备高确定性参考价值。