Δ-MOPD：以教师相对偏移改进多教师在线蒸馏
Composing What Each Teacher Learned: Multi-Teacher On-Policy Distillation through Teacher-Relative Shifts
蒸馏时直接抄老师的最终输出会混进老师基座的偏好，这篇论文改用老师训练前后的偏移量，三个老师组合下 Math 高了 4 分多，做模型蒸馏的可以看看。
论文提出 Δ-MOPD 方法，在多教师在线蒸馏中不直接迁移各教师的最终策略，而是迁移教师减去其基座模型的 logit 偏移，并重新锚定到学生模型的冻结初始状态。该方法能消除继承自教师基座的偏好拉扯，降低教师项范数比值与目标-学生 KL。在三教师组合蒸馏中，Δ-MOPD 相比终点组合方法在 Math 上高出 4.11 分，在五基准合计上高出 1.95 分。在分阶段路由下，它将不同阶段顺序间的性能差距从 10.50 分缩小到 6.42 分。
Composing What Each Teacher Learned: Multi-Teacher On-Policy Distillation through Teacher-Relative Shifts
Multi-teacher on-policy distillation (MOPD) is used in two settings. In common-domain composition, several teachers score each student rollout from one prompt domain and their signals form a single target; in routed-domain distillation, prompts from different domains are assigned to the corresponding specialist. Both settings usually transfer each teacher's endpoint policy, which mixes what post-training changed with preferences inherited from the teacher's base. We introduce $Δ$-MOPD, which transfers each teacher's teacher-minus-base logit shift re-anchored at the student's frozen initialization, and compare it with endpoint supervision in both settings while holding teacher selection fixed. We first expose the mechanism that impedes endpoint transfer: inherited base pull can exceed the post-training shift. Removing it reduces the teacher-term norm ratio and target--student KL. Across our experiments, the results suggest that shift targets are particularly useful when teacher signals are combined at a state. With three composed teachers, $Δ$-MOPD exceeds endpoint composition by $4.11$ Math and $1.95$ five-benchmark points; with two, it matches endpoint accuracy. Under phased routing, it achieves higher mean performance in both phase orders and reduces the observed order gap from $10.50$ to $6.42$ points. Under interleaved routing, where each update involves one teacher, the two targets perform comparably. The phased results provide supporting evidence that the benefit may extend to signals accumulated across training phases. Target construction is thus an independent design axis in MOPD, complementary to teacher selection.
