Raptor:基于拒绝反馈的树状投机解码加速框架
Raptor: Rejection-aware Adaptive Tree based on Speculative Decoding
推理 VLM 太慢的话可以看看这篇:Raptor 不用训练,靠拒绝反馈动态调树深度和分支宽度,实测平均加速到 2.39 倍。做推理优化的工程师值得把方法搬进自己的 pipeline 里试。
论文提出 Raptor,一个免训练的树状投机解码加速框架,面向视觉语言模型(VLMs)。在两个 VLM、七个多模态基准上的实验显示,相对自回归解码最高取得 2.39 倍平均加速。
核心洞察是:仅靠接受长度无法区分“候选覆盖不足”和“深层扩展过度”这两种情况,而二者需要不同的资源分配策略。Raptor 从拒绝反馈中提取互补信号,刻画投机进度、前沿覆盖与冗余扩展,用带动量平滑的有界更新来动态调整树的深度、分支宽度和草稿预算,并通过执行控制限制不必要的草稿生成、在合适时机逐步恢复扩展。
树状投机解码本身通过并行验证多个候选序列来加速 VLM,但长视觉上下文会抬高投机计算成本,使候选覆盖与冗余扩展成为树拓扑自适应之外必须权衡的问题。Raptor 不需要任何训练,直接在推理阶段调节资源分配,对部署多模态推理服务的团队有直接参考价值。
Raptor: Rejection-aware Adaptive Tree based on Speculative Decoding
Tree-structured speculative decoding accelerates vision-language models (VLMs) by verifying multiple candidate continuations in parallel. However, long visual contexts increase the cost of speculative computation, making candidate coverage and redundant expansion important considerations beyond tree topology adaptation. We propose Raptor, a training-free framework that uses rejection feedback to jointly regulate candidate coverage and drafting effort. Our key insight is that acceptance length alone cannot distinguish insufficient candidate coverage from excessive deep expansion, and these conditions require different allocation decisions. Raptor therefore extracts complementary rejection-aware signals to characterize speculative progress, frontier coverage, and redundant expansion. These signals jointly guide bounded, momentum-smoothed updates of tree depth, branching width, and drafting budget, together with execution control that limits unnecessary drafting and progressively restores expansion when appropriate. Extensive experiments with two VLMs across seven multimodal benchmarks demonstrate the effectiveness of Raptor, achieving up to 2.39x average speedup over autoregressive decoding.
本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:speculative decoding、VLM、推理加速、Raptor,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明:本文原始线索采编自「arXiv cs.AI」(@Qishu Hu, Haozhao Wang, Zicong Hong, Yufeng Zhan, Yuanqing Xia),经 AITOP 多维评分管线结构化提炼。