ForeSpark：并行投机解码中草稿与验证预算的联合调度方案
ForeSpark: Joint Draft and Verification Budget Scheduling for Parallel Speculative Decoding
做 LLM 推理服务的人可以看看这篇。vLLM 上跑的 ForeSpark 把草稿和验证长度一起调度，吞吐比 DSpark 高 1.77 倍，高负载下草稿浪费的算力能省下来。
论文提出工作保持投机（WCS）设计，并给出原型系统 ForeSpark，用于在 LLM serving 中为每个请求联合分配 draft 与 verification 预算。研究发现高负载下并行 drafter 的草稿生成最多消耗 40% 的每步时间，其中大部分从未被验证而被丢弃。
ForeSpark 在 vLLM 上实现，包含三个组件：基于目标模型上下文特征的 pre-draft 估计器，可预测各候选长度的接受率；通过全局排序同时选定 draft 与 verification 长度的轻量联合调度器；以及将验证预算分配给最有希望 token 的 post-draft 分配机制。
在匹配单请求生成速率的条件下，ForeSpark 吞吐量相比当时最优的 DSpark 提升最高 1.77x，相比自回归解码提升 2.49x，且输出分布不变；在延迟 SLO 约束下可承受 DSpark 1.51x 的请求速率。
ForeSpark: Joint Draft and Verification Budget Scheduling for Parallel Speculative Decoding
Parallel drafters accelerate LLM decoding by proposing a whole block of draft tokens in a single forward pass, making them attractive for LLM serving. Recent systems optimize goodput (i.e., accepted tokens per second) because rejected drafts steal compute from other requests. However, these systems control only how many tokens are verified, so every request still drafts a full block. At high serving load, we find that drafting consumes up to 40% of step time, of which a majority is discarded because it is never verified. Yet, draft-length control from autoregressive drafters does not carry over, because a parallel drafter commits to its length before its single pass. Moreover, draft and verification lengths are coupled across the batch, so they must be selected jointly for every request while keeping the decision computationally cheap at every step. In this paper, we propose work-conserving speculation (WCS), a design that jointly allocates draft and verification budgets per request in LLM serving, and present ForeSpark as its prototype. ForeSpark realizes WCS with (i) a context-aware pre-draft estimator that predicts acceptance for every candidate length from the target model's context features, (ii) a lightweight joint draft-verify scheduler that selects both lengths through a global sort, and (iii) a post-draft budget distribution that spends verification on the most promising tokens. Implemented on vLLM, ForeSpark improves throughput by up to 1.77x over the state-of-the-art DSpark and by 2.49x over autoregressive decoding at a matched per-request generation rate, without changing the output distribution. It also tolerates higher load, sustaining up to 1.51x the request rate of DSpark under latency service level objectives (SLOs).
本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中，该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线，反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括：ForeSpark、vLLM、Speculative Decoding、推理优化，读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明：本文原始线索采编自「arXiv cs.LG」（@Jaeyong Song, Daniel Wang, Pratyush Patel, Jonny Kong, Preyas Shah, Shobhit Kanaujia），经 AITOP 多维评分管线结构化提炼。