论文提出共享全局 KV 加层级本地历史方案，降低语言模型困惑度
Shared Global KV with Layer-Specific Local History
arXiv 上一篇讲 KV 缓存的新论文，用本地历史替换当前 token 分支，126M 模型上困惑度降了约 1.4%，做推理优化的人可以看看。
一项针对 Decoder-only Transformer 的研究探讨 KV 缓存中本地记忆应保留什么内容，将历史内容与形成它的输入来源分开。在 126M 参数、2K 上下文设置下，八种子实验显示本地历史方案比当前 token 本地分支的测试困惑度低约 1.4%。与 GQA 和相邻层 KV 共享对比，该方法在等学习率搜索下取得更好的同源似然。研究还给出一个充分后缀调度，在保持精确算术完整缓存的同时减少上层的构建工作量。
Shared Global KV with Layer-Specific Local History
Decoder-only Transformer language models cache keys and values (KV) to reuse past computation during generation. Sharing KV across layers saves storage but reduces the diversity of representations available across depth. We study what local memory should retain alongside shared global KV, separating historical content from the input source used to form it. At 126M parameters and 2K context, an eight-seed study finds about 1.4% lower held-out test perplexity with local history than with a current-token local branch. Capacity, entry-count and training-compute controls support the value of historical content. In a two-seed comparison, this value persists when adjacent layers share local inputs while retaining independent projections; source sharing also shortens exact cache-construction dependencies. Against GQA and adjacent-layer KV sharing, equal bounded learning-rate searches and new-seed confirmation yield better same-source likelihood with larger caches and higher long-request latency. The ordering against adjacent-layer sharing persists after equal-token adaptation to 8K, with a short-context cost. The eight-seed external-book history effect remains uncertain, and downstream outcomes vary by task. We derive a sufficient suffix schedule that reduces upper-layer construction work while preserving the complete cache in exact arithmetic.
