Nash-MOPD：用局部极小极大博弈优化多教师在线蒸馏的任务分配
Nash-MOPD: Optimizing Task Trade-offs in Multi-Teacher On-Policy Distillation
一篇蒸馏调度的论文，把多个教师的训练预算分配问题建模成博弈，在 DeepSeek 和 Qwen 的蒸馏模型上比固定配比效果更好，做蒸馏的可以看看。
多教师在线蒸馏（MOPD）要把多个领域专家教师的能力聚合到一个学生模型中，但各领域共享学生模型有限的优化步数和 rollout 预算。论文提出 Nash-MOPD，基于局部极小极大博弈构建自适应领域调度器，利用已有的 reverse-KL 统计量求解均衡，让提升较慢的领域获得更多训练样本。在 DeepSeek-R1-Distill-Qwen-1.5B 上，第 100 步时 Nash-MOPD 就超过 Fixed 和 D$^3$-MOPD 两个基线的最终数学与代码成绩；在 Qwen3-8B-Base 上达到 Fixed 的最终水平，并主导 9 项指标中的 8 项。该方法不增加教师查询或 rollout 开销。
Nash-MOPD: Optimizing Task Trade-offs in Multi-Teacher On-Policy Distillation
Multi-teacher on-policy distillation (MOPD) aggregates the capabilities of multiple domain-expert teachers into a single student model. However, all domains in MOPD share the student's finite budget of optimization steps and rollouts. Fixed uses constant domain sampling ratios based on corpus sizes; by contrast, D$^3$-MOPD adjusts the ratios using the teacher-student reverse-KL divergence and its recent decrease. However, favoring large recent decreases can direct more training toward domains that already improve quickly, while domains that improve more slowly may receive too few examples. To address inflexible allocation and the risk of giving insufficient training to domains that improve more slowly, we propose Nash-MOPD, an adaptive domain scheduler based on a local minimax game using existing reverse-KL statistics. The equilibrium maximizes the smallest predicted proportional reduction under the local model and gives more prompts to domains that need more examples for comparable improvement. Consequently, Nash-MOPD responds to changing learning progress and supports domains that improve more slowly without extra teacher queries or rollouts. At step 100, Nash-MOPD exceeds both baselines' final mathematics and code checkpoint scores on DeepSeek-R1-Distill-Qwen-1.5B and reaches Fixed's on Qwen3-8B-Base. Furthermore, it leads eight of nine reported metrics and is best or tied best on all six mathematics and code accuracy metrics, respectively.