Multi-layer Indexer 让 DeepSeek 稀疏注意力长文解码提速 14.5%
Multi-layer Indexer: Overlapping Selection and Attention in DeepSeek Sparse Attention
DeepSeek 和 GLM 在用的稀疏注意力有解码瓶颈，这篇提出相邻层复用索引结果，不改权重就能在 SGLang 里提速最多 14.5%，做长上下文推理的可以看看。
DeepSeek Sparse Attention（DSA）是 DeepSeek-V3.2 和 GLM-5 系列采用的稀疏注意力机制，每层前置的 lightning indexer 扫描整个缓存并保留得分最高的 k=2048 个 token，长上下文下 indexer 扫描成为解码耗时瓶颈。由于相邻层选中的 token 几乎相同，Multi-layer Indexer（MLI）让每层直接复用上一层 indexer 的选择，使 indexer 与本层注意力并行执行。MLI 不需要重新训练、不引入新权重，默认实现无需改动 kernel，可直接跑在 SGLang 的生产解码图里。与 GLM-5.2 的 IndexCache 组合后，在相同保留率下 DeepSeek-V3.2 解码最高快 14.5%，GLM-5 快 9.8%；在 128K 上下文、1/4 保留率下达到原模型解码吞吐的 1.32 倍。LongBench、RULER 和 LongBench v2 的精度与 IndexCache 相差在几个百分点以内。
Multi-layer Indexer: Overlapping Selection and Attention in DeepSeek Sparse Attention
Sparse attention cuts the cost of long-context inference by making the attention matrix sparse: each query attends only a subset of the sequence, chosen by a fixed pattern or, in recent production LLMs, learned per query. In DeepSeek Sparse Attention (DSA), the attention of DeepSeek-V3.2 and the GLM-5 family, every layer is preceded by a lightning indexer that scores the whole cache and keeps the k=2048 highest-scoring tokens for the layer to attend. At long context this indexer scan, not the sparse attention it feeds, is where the decode time goes. But adjacent layers select nearly the same tokens, so a layer can attend the selection of the previous indexer layer instead of its own. This removes the dependency: each indexer then runs concurrently with the attention of its layer. The resulting Multi-layer Indexer (MLI) needs no retraining and no new weights. Its default realization changes no kernel, running inside SGLang's production decode graph; an optional fused decode kernel accelerates it further on DeepSeek-V3.2. MLI composes with IndexCache (the cross-layer reuse GLM-5.2 ships as IndexShare): IndexCache drops the indexer from some layers, and MLI hides the scan on the rest. We serve DeepSeek-V3.2 and GLM-5 under IndexCache's protocol at retentions 1/2 to 1/32 (the fraction of layers that keep an indexer) and contexts 10K to 195K. At equal retention the composition decodes faster than IndexCache: up to 14.5% faster on DeepSeek-V3.2 and 9.8% on GLM-5 at 1/2, and 7.9% and 5.1% at 1/4, largest where IndexCache keeps the most indexers and growing with context. At retention 1/r it is faster still than IndexCache at 1/(2r) while keeping twice the fresh selections: at 128K it reaches 1.32x the unmodified model's decode throughput at 1/4, above IndexCache's 1.28x at 1/8. Its LongBench, RULER and LongBench v2 accuracy stays within a few percentage points of IndexCache's.
