RidgeFill：免训练的后期层缓存填充方法加速长文本 prefill
RidgeFill: Training-Free Late-Layer Cache Fill for Faster Prefill
不用重新训练模型，靠一次岭回归就能把长文本 prefill 提速到 1.5 倍以上，LLM 推理部署的可以看看这套测量方法。
RidgeFill 是一种免训练的 prefill 加速方法：模型下半部分层正常计算，从分割层的状态用一次岭回归拟合出的映射预测后续各层输入，直接填充 KV 缓存。在 Qwen3.8-27B 上的 llama.cpp 分支中，16K 到 120K token 场景下首 token 时间加快 1.25 到 1.31 倍，RULER 下降 1.2 到 1.4 分；在 Qwen3.8-Flash-Next 的 vLLM 上达到 1.50 到 1.60 倍加速，RULER 检索任务得分 99.8 对比完整计算的 100.0。与 YOCO 和 DeepSeek-V4.1-Flash 需要把投影训练进模型不同，RidgeFill 无需训练即可改装到已有模型上。
RidgeFill: Training-Free Late-Layer Cache Fill for Faster Prefill
RidgeFill shortens prefill by not computing most of a model's upper layers for the prompt. The lower layers run exactly up to a split; from the split layer's state a per-layer map predicts every later layer's input, and each late layer writes its cache from that prediction. The design follows YOCO and DeepSeek-V4.1-Flash, where the projection is trained into the model. RidgeFill retrofits it onto models that were never trained for it: the map is a ridge regression fitted once in closed form from one pass over calibration text, the layer's own projections run unchanged, and a chosen set of rows, the last few thousand tokens, and every decoding step run the full model. Nothing is trained and no token is dropped. Most of the work was in measurement, finding a repeatable methodology. Loss averaged over all tokens hides RidgeFill's error. Needle retrieval in the filled region, loss on tokens that depend on distant context, and splice tests that swap filled keys or values for exact ones show which component carries it: the keys on Llama-3.1-8B, half of it in their scale, and the values and the recurrent state on Qwen3.5-9B. On Qwen3.8-27B in a llama.cpp fork on an A100, time to first token is 1.25 to 1.31x faster at 16K to 120K tokens with the maps in VRAM; RULER changes by -1.2 and -1.4 points and LongBench v2 by -1.4; the 95 % intervals include zero but do not rule out losses of a few points. On Qwen3.8-Flash-Next in vLLM it is 1.50 to 1.60x faster than exact at 9K to 32K tokens, with RULER's needle tasks at 99.8 against 100.0 and perplexity on novel tokens 2.65 % higher [-0.02 %, +4.93 %]. On three 8 to 9B reference models, at the same split and tail as the identity fill, the fitted map retrieves 49.1 and 49.6 more filled-region RULER points on the two dense models (at that split Llama's loss is worse), and its long-document loss is +/-0.03 nats of the released SwiftKV checkpoint's.