面向 GDN 循环状态低比特量化的 GGA 与纯写滑动窗口方法
Low-Bit GDN Recurrent-State Quantization: Global Gradient-aware Allocation and Pure-Write Sliding Windows
Qwen3.5 上的低比特量化新论文，GGA 加滑动窗口比 STEPQuant 高 40 个百分点，还能让输出短三分之一，做长上下文部署的可以看看。
线性注意力以固定大小的循环状态存储历史信息，降低状态精度可减少长上下文推理的存储开销，但量化写入误差会影响长推理任务表现。论文提出全局梯度感知位宽分配 GGA 与纯写滑动窗口 PW：GGA 结合量化残差与梯度敏感度在全模型混合精度预算下分配头位宽；PW 用 FP8 向量对独立维护近期纯写入，离开窗口后再合并进低比特历史状态。在 Qwen3.5-4B/9B 的七项评测上，GGA 单独使用比 STEPQuant@4 高 19.48/29.84 个百分点，叠加 PW-S8 后宏平均达 69.83%/78.63%，领先 38.39/41.21 个百分点。在 @6 设置下 GGA 将平均生成长度缩短 10.45%/14.65%，再加 PW-S8 进一步缩短 32.32%/36.77%，缓解了重复生成与输出过长问题。
Low-Bit GDN Recurrent-State Quantization: Global Gradient-aware Allocation and Pure-Write Sliding Windows
Linear attention stores historical information in a recurrent state of fixed size, providing a compact memory representation for long-context inference. Reducing state precision further lowers per-request storage requirements, but quantized writeback errors enter subsequent recurrences, degrade prediction quality on long-reasoning tasks, and can be accompanied by repetitive generation and excessively long outputs. We propose Global Gradient-aware Allocation (GGA) and the Pure-Write Sliding Window (PW) to address precision allocation and recent-information maintenance. GGA combines candidate quantization residuals with gradient sensitivity to future prediction loss to allocate head bit widths under a model-wide mixed-precision budget. PW independently maintains recent pure writes as FP8 vector pairs, propagates them through the full GDN state transition, and merges them into the low-bit historical state only after they leave the window. Across seven evaluations on Qwen3.5-4B/9B, GGA allocation alone improves over STEPQuant@4 by 19.48/29.84 percentage points. Adding PW-S8 yields seven-task macro-averages of 69.83%/78.63%, improving over STEPQuant@4 by 38.39/41.21 percentage points, respectively. Both precision reallocation and window maintenance improve task scores while mitigating excessively long outputs: at @6, GGA reduces mean generation length, averaged equally across tasks, by 10.45%/14.65% relative to STEPQuant; adding PW-S8 to GGA@4 further reduces mean generation length by 32.32%/36.77%. Comparisons at similar storage budgets further show that using additional capacity to maintain recent writes yields higher task scores and shorter mean outputs than solely increasing the bit width of the accumulated state.
