CompassOPD：跨家族模型蒸馏新方法

50个百分点。对于MoE教师模型，通过减少专家激活直接构建参考点，无需单独参考检查点即可实现3.

43百分点的性能提升。

CompassOPD: Cross-Family On-Policy Distillation via Within-Family Likelihood Shifts

On-policy distillation (OPD) provides dense token-level supervision on student-generated trajectories. Although OPD performs strongly when teacher and student belong to the same model family, we find that its effectiveness degrades in cross-family settings even after tokenizer alignment, with substantially stronger external teachers offering little additional improvement. To understand this disconnect, we decompose the cross-family OPD signal into two components: an offset between a low-capability teacher-family reference and the student, and the within-family log-likelihood shift from that reference to the strong teacher. Standard OPD transfers both components together, allowing the offset to dominate the update direction and obscure the changes associated with teacher capability improvements. We propose CompassOPD, which removes this offset and transfers the within-family shift, while a frozen student reference anchors updates to the student's initial policy. Thus, both teacher-side and student-side changes are measured within their respective model families. Experiments across three student families and multiple teacher families show that CompassOPD consistently outperforms standard cross-family OPD, improving average reasoning accuracy by up to 5.50 points. For an MoE teacher, we further construct the reference directly from the teacher checkpoint by reducing expert activation, eliminating the need for a separate reference checkpoint while retaining a 3.43-point gain over OPD.

本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中，该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」「各国 AI 监管法案、合规治理与主权算力政策」研究主线，反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括：CompassOPD、OPD、模型蒸馏、推理准确率，读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明：本文原始线索采编自「arXiv cs.LG」（@Naibin Gu, Qingyi Si, Chenxu Yang, Chuanyu Qin, Junhao Zhou, Peng Fu, Zheng Lin, Weiping Wang），经 AITOP 多维评分管线结构化提炼。