Google 为 Gemma 4 内置多 token 预测起草器,推测解码最高提速 3.1 倍
做 Agent 服务的人该看看:Gemma 4 把 drafter 塞进了模型本体,不用外挂、不用额外 prefill,手机 GPU 也能跑出 2.2 倍速度。和传统独立 drafter 方案比,部署简单得多,微调场景还省前向计算,端侧开发者尤其受益。
Google 团队为 Gemma 4 推出内置式推测解码方案,把多 token 预测起草器直接嵌进模型,而非外挂独立 drafter。服务器端大模型解码最高提速 3.1 倍,小模型在 H100 上达 2.3 倍,在智能手机 GPU 上达 2.2 倍。
技术关键是一个共享起草头:它直接读取 backbone 自身的激活值和已有的 KV cache,因此省去了独立 drafter 所需的单独 prefill 步骤。起草阶段再用聚类词表投影(clustered vocabulary projection)降低每步草稿成本,对手机等小设备尤其有效。
团队还测试了领域微调场景:backbone 适配新领域时,同步蒸馏 drafter 所需的目标模型前向传播次数比单独调 backbone 少最多 1.83 倍,领域适配代码已公开。对跑长输出的智能体服务来说,解码加速同时降低延迟和 serving 成本。
Brilliant work from Google on speculative decoding for Gemma 4.
This matters a lot if your agents generate long outputs, because faster decoding lowers both latency and serving cost.
The team built a multi-token prediction drafter directly into the model. It uses one shared drafting head that reads the backbone's own activations and its existing KV cache. That removes the separate prefill step a standalone drafter needs.
A clustered vocabulary projection also cuts the cost of each draft step, which helps most on small devices.
The speedups reach 3.1x for large models on servers. Small models get up to 2.3x on an H100 and up to 2.2x on a smartphone GPU.
They also tested fine-tuning.
When the backbone is adapted to a new domain, distilling the drafter with it needs up to 1.83x fewer target forward passes than tuning the backbone alone.
The code for the domain adaptation recipes is public.
https://arxiv.org/abs/2611.25813
Chat with Paper: https://academy.dair.ai/papers/efficient-and-robust-drafting-with-multi-token-prediction-2611.25813
本条情报归属 AITOP「模型」核心栏目,重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」「中美大模型技术代差与商业化对照」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:Gemma 4、Google、推测解码、多模态,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明:本文原始线索采编自「DAIR.AI」(@dair_ai),经 AITOP 多维评分管线结构化提炼。