论文提出 LYRA 方法破解长上下文检索中的 Proximity Trap
The Sirens' Song: When Proximal Background Context Overshadows Distant Evidence
一篇很实在的论文：作者发现长文档检索差的真正元凶是近处无关文字抢注意力，给出 LYRA 方法外加 ProxBench 基准，做了 RAG 的话值得看他们的实验数据。
arXiv 论文指出，长上下文模型检索远距离证据失效的主因不是距离本身，而是近处大量与任务无关背景文本的竞争，作者称之为 Proximity Trap。论文提出 LYRA，一种基于 t 分布的方向匹配机制，把注意力质量重新导向任务相关证据，同时保留相对位置信息。在 LongBench-v2、RULER 和 LongBench 三个基准上，LYRA 在不同上下文长度和任务类别中均取得一致提升。作者还发布 ProxBench，用于评估近端背景干扰增强时远距离证据的利用能力。
The Sirens' Song: When Proximal Background Context Overshadows Distant Evidence
Long-context LLMs focus on retrieving distant evidence from extensive context, yet existing work has largely focused on overcoming distance alone. In this work, we identify the Proximity Trap, insufficient attention to distant evidence often arises less from distance itself than from cumulative competition with abundant, task-irrelevant proximal background. To address the Proximity Trap, we introduce LYRA (Long-context heavY-tailed Relevance Alignment), a t-distributed directional matching mechanism that reshapes the context retrieval distribution, directing more attention mass toward task-relevant evidence, while preserving the relative positional information encoded. Extensive experiments on LongBench-v2, RULER, and LongBench demonstrate consistent improvements across context lengths and task categories. We further introduce ProxBench, a multi-level fine-grained benchmark for evaluating distant evidence utilization under increasing proximal background interference. Project page: https://xiaoyuyoung.github.io/LYRA/
