DWISE 调度引擎让扩散语言模型推理延迟降低至 1/5.4
DWISE: Denoising Workload-Informed Scheduling Engine
扩散语言模型跑得慢可能不是模型的问题，是调度策略拖了后腿。这篇把延迟砍了 5 倍以上，做推理服务的值得看看。
扩散语言模型（dLLM）采用现有 FCFS 和 prefill 优先策略会浪费其并行去噪的效率优势。DWISE 按剩余工作量调度请求，让 prefill 和去噪共享同一轮次，并用提示词预测器加第一轮去噪隐状态精修来预估未知负载。该引擎基于 SGLang 实现，在 3 个 dLLM 家族的 5 个模型上测试，平均将端到端延迟相对 SGLang 降低 5.4 倍，最高达 14.9 倍，任务精度不受影响。
DWISE: Denoising Workload-Informed Scheduling Engine
Unlike autoregressive large language models (AR-LLMs), diffusion LLMs (dLLMs) generate text by iteratively denoising many tokens in parallel, offering a promising path to efficient inference. Existing serving frameworks, however, schedule dLLM requests with policies designed for AR models, namely first-come-first-served (FCFS) admission and prefill priority. We find that both policies are ill-suited to dLLMs, wasting much of their speed. FCFS ignores that a dLLM request's decoding cost is highly heterogeneous and unknown at arrival, causing severe head-of-line blocking. Prefill priority, widely used in AR-LLM serving to keep time-to-first-token low, sharply cuts dLLM efficiency, as rounds that run only prompts leave most of the batch idle. We present DWISE, a Denoising Workload-Informed Scheduling Engine for efficient dLLM serving. DWISE schedules requests by their remaining work, measured in a block-level unit shared by prefill and denoising, so both phases share a round. To predict the unknown denoising workload, DWISE combines a lightweight prompt-based predictor at admission with a single refinement from the first denoising round's hidden states, which capture model-specific information absent from the prompt and require no additional model execution. Implemented in SGLang and evaluated on five models from three dLLM families under bursty workloads, DWISE reduces mean end-to-end latency over SGLang by 5.4$\times$ on average and up to 14.9$\times$ while preserving task accuracy. The code will be released publicly.
