arXiv 论文：上下文压缩对后续问题几乎无帮助
Evaluating Context Compaction on the Questions That Come Later
研究上下文压缩的人该看看：为当前问题删上下文，对后续问题基本没用，LLMLingua-2 按词删反而好 12.5 个点。
一篇 arXiv 论文评估了语言模型智能体的上下文压缩方法在后续问题上的表现，覆盖 MultiRC、FEVER、Evidence Inference、τ-bench 四个语料。结果显示，在 MultiRC 上，用 gpt-6-luna 为当前问题选句可将该问题准确率提高 11.4 个点（25% token 预算），但对后续问题与随机删句没有可检测的差异。LLMLingua-2 按词删除反而比随机删句在后续问题上高 12.5 个点，因为它保留了约两倍的实词比例。论文还指出 FEVER 的标签准确率会高估保留文本的真实支撑，且各方法排名随设置变化，为当前问题优化的压缩分数不能反映后续表现。
Evaluating Context Compaction on the Questions That Come Later
Language-model agents compact their context for the question at hand, but the compacted context must also serve questions that come later. We evaluate compaction on these later questions, using four corpora that annotate the evidence of several questions per text and recorded agent trajectories, and compare every method with random deletion at the same budget. On MultiRC, keeping the sentences that gpt-6-luna (through the OpenAI Decisions API) judges necessary for the current question raises its accuracy on that question by 11.4 points at 25% of the tokens. On the later questions, however, no method that selects sentences for the current question does detectably better than deleting sentences at random. Deleting words instead of sentences does better: LLMLingua-2, which ignores the question, beats random sentence deletion at the same token count by 12.5 points on the later MultiRC questions. It keeps about twice random deletion's share of the evidence's content words, and at equal content-word retention its gain shrinks to 7.3 points (exploratory). On FEVER, label accuracy can overstate what the kept text supports: the gain of sentence selection is no longer detected when answers are scored against the kept evidence, and the 21.0-point gain of LLMLingua-2 is not shown to rest on the kept text. The ranking also depends on the setting: on Evidence Inference sentence selection beats LLMLingua-2 at a nominal 25% budget, and on $τ$-bench trajectories, whose later tool calls need exact identifiers, it beats random deletion while LLMLingua-2 shows no detected gain. Scores on the question a compaction was made for do not reliably show how it serves later ones.
