MaxIM：用强化学习优化增量摘要的信息保留机制
MaxIM: Maximally Informative Incremental Summarization via Reinforcement Learning
这篇论文解决了智能体长对话记忆压缩的老难题。它不靠人类偏好标注，用互信息做密集奖励，比 SFT 和普通 RL 基线效果都好。做长上下文应用或智能体框架的人可以仔细看看方法部分。
新加坡国立大学等机构的研究团队发布 MaxIM（Maximally Informative Incremental Memory）框架，把增量摘要建模为容量受限的信息论奖励塑形问题，用于在有限记忆内维护无限长的交互历史。该方法通过最大化未来任务相关目标的预测对数似然，给出预测充分性的变分下界；同时引入递归一致性条件，以摘要与观测历史之间的互信息（MI）最大化来保证摘要有据可依。针对稀疏延迟奖励导致的信用分配瓶颈，论文设计了参数化非对称 critic，逐步计算变分 MI 下界生成密集奖励。在长程科学阅读和多轮对话基准上，MaxIM 在事实 grounding、综合 autorater 指标和下游任务效用上超过 RL、SFT 与 prompt 基线，且无需昂贵的人类偏好标注。
技术核心在于奖励设计：通过约束序列信息损失，把下游任务效用转化为可优化的互信息下界，使 RL 训练获得逐步反馈而非仅靠最终结果。这与常见依赖人类偏好数据的 RLHF 路线不同，MaxIM 的信号完全来自信息论度量。
对开发者而言，该框架指向长上下文场景中一条无需人工标注的记忆压缩路径，适用于智能体长会话、科学文献处理等 token 预算受限的应用。
MaxIM: Maximally Informative Incremental Summarization via Reinforcement Learning
AI systems must often maintain bounded memory of unbounded interaction histories. However, optimizing the memory representation, and its updates, for downstream task utility is difficult for standard reinforcement learning (RL) due to severe credit assignment bottlenecks induced by sparse, delayed rewards. We introduce MaxIM (Maximally Informative Incremental Memory), a framework that formulates incremental summarization as a capacity-constrained, information-theoretic reward-shaping problem. Grounded in value suboptimality bounds, we show that bounding sequential information loss motivates maximizing the predictive log-likelihood of future, task-relevant targets, a variational lower bound on predictive sufficiency. We introduce a recursive consistency condition to ensure summaries are grounded by maximizing mutual information (MI) with the observed history. We mitigate credit assignment issues with a parameterized asymmetric critic that computes step-wise variational MI bounds for dense reward shaping. Empirical evaluation on long-horizon scientific reading and multi-turn dialogue benchmarks shows that MaxIM outperforms RL, supervised fine-tuning, and prompt-focused baselines on factual grounding, comprehensive autorater metrics, and downstream task utility across strict token budgets and extended horizons, without the need for expensive human preference labels.
本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中，该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线，反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括：MaxIM、强化学习、RAG、长上下文，读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明：本文原始线索采编自「arXiv cs.AI」（@Jihwan Jeong, Guy Tennenholtz, Yinlam Chow, Chih-Wei Hsu, Craig Boutilier），经 AITOP 多维评分管线结构化提炼。