研究提出 SQD 方案优化异构系统下子二次注意力推理效率
Rethinking Heterogeneous System Disaggregation for Subquadratic Attention
这个论文研究了一种叫 SQD 的方法，专门针对子二次注意力模型，能显著提升在异构系统上的推理效率，比单纯用 GPU 的方法好很多。
本文提出 SQD (SubQuadratic Disaggregation) 细粒度异构解耦方案，针对子二次注意力 LLMs 的独特算术强度和内存占用进行解耦，而非按算子类型。在模拟的 8xB200 异构系统上，该方案使 GLM 5.2、Nemotron 3 Ultra 和 Gemma 4 31B 的能耗效率分别提升 53%、31% 和 56%，优于纯 GPU 基准。在固定功耗的 Rubin+LPX 系统分析中，可实现 1.2 到 1.5 倍更低的延迟和 3.6 倍更高的吞吐量。
Rethinking Heterogeneous System Disaggregation for Subquadratic Attention
Frontier language models are more aggressively using subquadratic attention to reduce the memory footprint and compute requirements during inference while still delivering frontier accuracy. While existing systems make dense attention-centric disaggregated serving decisions, we show that disaggregating inference around the unique arithmetic intensity and memory footprint of subquadratic attention LLMs can achieve significant throughput and energy efficiency gains on emerging DRAM-based and SRAM-only heterogeneous systems. We introduce SQD (SubQuadratic Disaggregation), a fine-grained heterogeneous disaggregation scheme that splits decode by quadratic and subquadratic attention rather than by operator type, and that applies across subquadratic attention variants. For sparse attention LLMs, we disaggregate decode into top-k selection, which must index through the full KV, and top-k attention plus FFN, which have static memory footprints. For linear and sliding-window attention LLMs, we disaggregate decode into dense attention layers and subquadratic attention layers plus FFN. In an adjusted 8xB200 heterogeneous system proxy, we observe average tokens/J improvements of 53% on GLM 5.2, 31% on Nemotron 3 Ultra, and 56% on Gemma 4 31B over the strongest GPU-only baselines. In an analytical model of a Rubin plus LPX system with fixed power budgets, we observe 1.2x to 1.5x tighter achievable latencies and up to 3.6x higher throughput over the best baseline of attention-FFN disaggregation. Our experiments also reveal architectural insights on chip and interconnect provisioning for next-generation heterogeneous systems serving subquadratic attention.