LinQuant：基于函数不变性的线性递归低精度量化框架
LinQuant: Functional Invariances for Low-Precision Linear Recurrences
arXiv 上的一篇量化研究，专门解决 Mamba 这类线性递归模型的低精度推理问题，在 Mamba-2 和 Nemotron-H 上验证了效果。
线性递归架构正被用作 softmax attention 的替代方案，但其低精度推理方法远不如 attention 成熟。论文提出 LinQuant，归纳了七种保持全精度计算的函数不变性，包括旋转、对角缩放以及递归衰减和写入因子的不同放置方式。作者推导了这些变换在线性、门控和 delta-rule 等常见递归族上何时有效。将该框架应用于 Mamba-2 的低精度 prefill 管线 SSDi8 后，在 Mamba-2 1.3B、2.7B、8B 和 Nemotron-H 8B 上均以相同延迟获得更低困惑度。
LinQuant: Functional Invariances for Low-Precision Linear Recurrences
Linear recurrences are emerging as practical alternatives to softmax attention in large language models, appearing in both standalone and hybrid architectures. However, despite their rapid emergence, low-precision inference remains much less developed for these models than for softmax attention. During prefill, chunkwise-parallel algorithms cast much of the recurrence as dense matrix multiplications, suggesting a natural path to low-precision execution. Simply quantizing these operations in their native form, however, can be fragile. In this work, we identify functional invariances that preserve the full-precision computation while making the operands more amenable to quantization. Building on this observation, we introduce LinQuant, a framework for low-precision recurrent prefill based on seven such invariances, including rotations, diagonal rescalings, and alternative placements of recurrent decay and write factors. We derive when these transformations remain valid across common recurrence families spanning linear, gated, and delta-rule updates, and show that they provide complementary degrees of freedom for improving low-precision accuracy. Finally, we translate this framework to production by augmenting SSDi8, an existing low-precision prefill pipeline for Mamba-2, with the applicable invariances. LinQuant improves perplexity at matched latency on Mamba-2 1.3B, 2.7B, and 8B and Nemotron-H 8B, demonstrating the practical utility of invariance-based quantization for recurrent models.
