EAGLE-4 推测解码达 863.8 tokens/s，比 DFlash 2 快 1.28 倍
EAGLE-4 这个推测解码组件能在不改模型输出的前提下把解码速度提到约 3 倍，比 DFlash 2 还快 1.28 倍，做推理部署的可以看看。
EAGLE-4 是一个推理加速组件，在 27B 模型单请求上解码速度达 863.8 tokens/s，比 DFlash 2 快 1.28 倍，比 DSpark 快 1.40 倍。它的原理是用一个小的草稿模型一次猜出多个 token，目标模型在一次前向传播中批量验证，输出结果与逐 token 生成完全一致。相比之前版本，EAGLE-4 保留了草稿-验证循环，但在草稿生成侧做了优化，采用自适应草稿结构和更便宜的草稿 LM head。在测试环境下整体速度约为原来的 3 倍。
Some of the most useful inference work right now is happening in the speculative decoding component.
It has become one of the most dependable ways to get more tokens per second out of a model you’ve already chosen.
EAGLE-4 decodes 863.8 tokens/s on a 27B model on a single request, 1.28X DFlash 2 and 1.40X DSpark.
EAGLE-4 is a speed-up add-on for running AI chatbots: a tiny helper model guesses the next few words, and the big model checks all those guesses at once instead of writing one word at a time. The answers come out identical, just about 3× faster on the tested setup.
EAGLE-4 keeps the draft-and-verify loop and puts its work into the drafting side, with an adaptive draft structure and a cheaper draft LM head.
The loop:
- A small draft model proposes several tokens ahead - The target model checks all of them in one forward pass - Matching tokens are kept, and the first mismatch is replaced with the target’s own token - Repeat from there
