KV-Cache 淘汰策略可按风险目标认证：Llama 上 SnapKV 认证通过 75% 保留率
Risk-Controlled KV-Cache Eviction: From Memory Budgets to Risk Targets
这论文把 KV-cache 压缩变成了能签合同的事：给定风险目标，自动算出能砍多少缓存，砍不了就回退全 KV。做长文本推理部署的可以看看。
一篇 arXiv 论文将 KV-cache 淘汰从平均质量-内存权衡重构为部署级风险控制问题：当淘汰导致单个请求任务效用下降超过容忍阈值时记为一次风险事件，部署风险即这类事件的频率。论文提出与压缩器无关的事后认证程序，根据校准数据在有限样本保证下选择保留策略，无法认证时回退到完整 KV。在 Llama 和 Mistral 模型上测试，针对 LongBench 与 RULER-32K 两套基准，同一风险契约给出不同结论：Llama 上 SnapKV 在 LongBench 通过 75% 保留率的认证，但在 RULER-32K 上没有受测压缩策略通过认证，触发完整 KV 回退。实验还显示经验退化率低于 5% 目标的策略仍可能未通过有限样本认证，经验阈值法在 Llama LongBench 上选出的策略比固定预算方法少保留 5-10 个百分点的缓存。
Risk-Controlled KV-Cache Eviction: From Memory Budgets to Risk Targets
KV-cache eviction is typically evaluated through average quality-memory trade-offs, yet a small average loss can hide requests whose utility degrades materially. We reformulate eviction as a deployment risk-control problem: a material degradation occurs when eviction lowers task utility by more than a deployment-specified tolerance relative to full-KV inference on the same request, and deployment risk is the population frequency of such events. Given a reliability contract specifying a target risk level and confidence requirement, we use a compressor-agnostic post-hoc certification procedure to select a retention policy from calibration data with a finite-sample guarantee, falling back to full KV when no compressed policy is certified. Across multiple eviction methods, Llama and Mistral models, and LongBench and RULER-32K, the same contract supports substantially different levels of eviction: on Llama, it certifies SnapKV at 75% retention on LongBench but no tested compressed policy on RULER-32K, triggering full-KV fallback. Policies with empirical degradation rates below the 5% target can still fail finite-sample certification; on Llama LongBench, empirical thresholding selects uncertified policies that retain 5-10 percentage points less cache across fixed-budget methods. The proposed framework converts a deployment-level reliability requirement into a KV-memory operating point.
