Turing 提出 C-MOPD 蒸馏方法，Turing-20B-A2B 整合得分达 0.9988
Turing LLM Post-Training: Calibrated Distillation and Efficient Dynamic-MoE Execution
Turing 这篇论文讲怎么把多个专家模型蒸馏进一个 MoE，80 多行细节，还把训练步骤快了 2.86 倍，做训练的人可以看看。
论文研究动态路由 MoE 模型的后训练，提出校准多教师在线蒸馏方法 C-MOPD。在 Turing-20B-A2B 上，四域归一化整合得分达到 0.9988，每个域保留至少 90.16% 的专家增益。在匹配 8K 响应上限的设置下，模型在四个数学与代码推理基准上的无权重平均分为 85.92，高于 Qwen3-8B 的 80.33 和共享 SFT 的 74.21。执行层面结合有界宽度分发与异步重叠调度后，整合步骤耗时从 398.7 秒降至 139.3 秒，观测到 2.86 倍加速。
Turing LLM Post-Training: Calibrated Distillation and Efficient Dynamic-MoE Execution
We study post-training for dynamically routed mixture-of-experts (MoE) models, focusing on integrating independently optimized specialists while keeping consolidation efficient. Our approach combines calibrated multi-teacher on-policy distillation (C-MOPD), which matches each selected teacher on a local Top-$K$ support with teacher-wise loss normalization and clipping, with an execution path that uses post-capacity bounded-width dispatch, final-route replay, and asynchronous rollout--training overlap. On Turing-20B-A2B, the selected model achieves a four-domain normalized integration score of 0.9988 and retains at least 90.16\% of the measured specialist gain in every domain. At matched $K=32$, the full calibration objective scores 0.9962, compared with 0.9724 without calibration and 0.9834 without clipping. At a matched 8K response cap, the model exceeds both shared SFT and Qwen3-8B on all four reasoning-enabled mathematics and code benchmarks (unweighted mean: 85.92, versus 80.33 for Qwen3-8B and 74.21 for SFT), and improves over SFT on ten of twelve broader evaluation metrics. In a PPU-based profile, the combined execution configuration reduces consolidation step time from 398.7 to 139.3 seconds, an observed 2.86$\times$ speedup over serial execution. These results demonstrate strong specialist integration alongside more efficient consolidation, while reflecting the complete training and execution framework rather than any single component.
