LEMS 与 KFAC-SVD 改进 SVD 压缩方法降低 LLM 计算成本
Advancing SVD-based LLM Compression via Layer-Wise Error Model Search
想在单卡跑大模型又怕掉点的话，这篇给出更聪明的 SVD 压缩方案，比现有方法准确率高不少，代码也放出来了。
一篇 arXiv 论文提出 Layer-wise Error Modeling Search（LEMS）和 KFAC-SVD 两种方法，改进低秩 SVD 模型压缩。LEMS 通过逐层误差代理结合 ILP 求解实现更精准的全局秩分配，KFAC-SVD 利用 token 级统计量缓解 Fisher 方法中的秩坍缩问题。在 Mistral、Qwen3 和 Llama 3 三个模型家族上，LEMS 带来最高 4.8 个百分点的零样本准确率提升，并验证到 70B 规模，KFAC-SVD 平均降低困惑度 15%。代码与项目页已开源。
Advancing SVD-based LLM Compression via Layer-Wise Error Model Search
Low-rank SVD-based compression offers a powerful strategy to reduce the computational costs of LLMs. However, existing methods face two key limitations: (i) global rank allocation, where uncalibrated error proxies fail to capture complex error propagation, and (ii) decomposition quality, where Fisher-based estimators suffer from severe rank collapse. In this work, we address these limitations by introducing Layer-wise Error Modeling Search (LEMS) and KFAC-SVD. LEMS advances rank allocation by introducing a layer-wise error surrogate that integrates local and global layer importance alongside a propagation bias, enabling effective global rank allocation via an ILP formulation. KFAC-SVD improves decomposition quality by utilizing token-wise statistics, mitigating the rank deficiency observed in prior Fisher-based SVD approaches. Across Mistral, Qwen3, and Llama 3 model families, we show that LEMS consistently outperforms existing search strategies, delivering significant zero-shot accuracy gains of up to 4.8 p.p. that generalize to model sizes of 70B parameters, while KFAC-SVD achieves an average perplexity improvement of 15%. Project page and code: https://lems-svd.github.io