TraceKV：免训练低秩 KV 压缩，5.64 倍压缩保推理精度

方法上，TraceKV 在同一个策略保持原则下统一三项技术：在线策略校准（on-policy calibration）、敏感度感知 SVD 分解和全局秩分配，显著降低推理过程中的 policy-KL 散度。结合量化后可在 5.64 倍 KV 缓存压缩下保持近乎无损的推理准确率；优化后的低秩注意力核还带来最高 1.83 倍的端到端解码加速。

该工作针对的是推理链越来越长的多模态模型在显存和解码速度上的瓶颈，代码将开源在 GitHub（AK-DREAM/TraceKV），对部署推理型 VLM 的开发者可直接复用。

TraceKV: Policy-Preserving Low-Rank KV Cache Compression for Reasoning Vision-Language Models

Reasoning vision-language models (VLMs) impose substantial KV cache demands from both large multimodal contexts and growing reasoning traces. Low-rank KV compression has emerged as a promising paradigm that stores low-dimensional latents of KV states and can be combined with quantization. However, existing low-rank methods often induce growing predictive policy drift from the dense model when compressing reasoning KV states, resulting in reasoning failures. We identify two key causes of this degradation: calibration-state mismatch between static contexts and on-policy reasoning states, and unequal error sensitivity across KV states. To address these challenges, we propose TraceKV, a training-free low-rank KV compression framework that unifies on-policy calibration, sensitivity-aware SVD, and global rank allocation under a single policy-preserving principle, substantially reducing policy-KL divergence during reasoning. Experiments across multiple VLMs and visual reasoning benchmarks demonstrate superior accuracy--memory trade-offs, achieving near-lossless reasoning accuracy with 5.64$\times$ KV cache compression when combined with quantization. Optimized low-rank attention kernels further deliver up to 1.83$\times$ end-to-end decoding speedup. Code will be available at https://github.com/AK-DREAM/TraceKV.

本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中，该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线，反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括：TraceKV、VLM、KV缓存压缩、推理模型，读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明：本文原始线索采编自「arXiv cs.LG」（@Ke Li, Zishan Shao, Yiran Chen），经 AITOP 多维评分管线结构化提炼。