MLIPS：跨层共享索引池，vLLM 实测长上下文最高提速 1.40 倍
MLIPS: Multi-Layer Indexer Pool Sharing
这篇论文教你让稀疏注意力模型多层共用一套候选池，不用重训，vLLM 里给 DeepSeek-V4-Flash 实测提速 1.4 倍还省电，做推理优化的值得看。
论文提出 MLIPS，一种免训练的稀疏注意力优化方法：由生产者层发布一个大于注意力预算 K 的 top-M 候选池，后续各层从同一池中自行打分选出自己的 top-K。作者在 vLLM 中为 DeepSeek-V4-Flash 实现了逐层共享、解码阶段持久复用和粗粒度生产者。在四个长上下文测试套件上，每个生产者服务三个消费层可带来 1.14 到 1.40 倍加速，GPU 能耗降低 14.8% 到 27.4%，单次运行准确率变化在 -1.49 到 +0.29 个百分点之间。论文还分析了 prefill 采样轨迹中跨查询组与层复现的 KV 位置，并据此提出探索性的 Top-G 有界池构造方法。
MLIPS: Multi-Layer Indexer Pool Sharing
Sparse attention limits the keys used by each query, but finding those keys can still require a full-prefix scan at every layer. We introduce \textbf{MLIPS (Multi-Layer Indexer Pool Sharing)}, a training-free method that shares candidate search while retaining each consumer's own scoring and selection. A producer publishes a top-$M$ pool larger than the attention budget $K$; subsequent layers select their own top-$K$ from that pool. Measurements across three sparse-attention model families show substantially higher coverage from broad pools than overlap between final selections. We implement layerwise sharing, persistent decode reuse, and a coarse producer in vLLM for DeepSeek-V4-Flash. Across four long-context suites, three consumer layers per producer give $1.14$--$1.40\times$ speedups and $14.8$--$27.4\%$ lower GPU energy, with observed accuracy changes of $-1.49$ to $+0.29$ percentage points in single runs. More extensive sharing exposes task-dependent quality losses. Sampled prefill traces also reveal recurring selected KV positions across query groups and layers. We characterize this shared working-set structure and use it to motivate Top-$G$, an exploratory bounded pool construction for prefill. Its deployment, task accuracy, and serving benefits remain open.
