研究显示 MoE 模型合并后的路由漂移不能直接判定为路由失败
Routing Drift Alone Does Not Diagnose Failure in Merged MoE LLMs
合并 MoE 模型时看到路由乱了先别急着修，这篇用 DeepSeekMoE 等三个模型证明漂移不等于失败，还给了工具包和修复代码。
arXiv 论文针对 DeepSeekMoE、OLMoE、Qwen3-MoE 三种 MoE 模型，提出一套路由分析工具包，通过反事实干预和 token 级分析研究合并后的路由漂移。结果显示，大部分专家重新分配来自输入变化而非同层参数变化，且源相对路由差异难以预测恢复源路由后 next-token 似然的提升。论文将路由失败操作化定义为：在固定非路由参数下，特定路由干预可挽回的任务损失，并据此提出 Selective Router Repair（SRR）案例。分析工具包与 SRR 代码已开源。
Routing Drift Alone Does Not Diagnose Failure in Merged MoE LLMs
Model merging efficiently combines specialized large language models (LLMs) without joint retraining, but can substantially alter expert routing in Mixture-of-Experts (MoE) models. Such \emph{routing drift} is often interpreted as routing failure, raising a fundamental question that remains unclear: \emph{does routing drift after MoE merging actually indicate routing failure, and what evidence should justify repair?} We investigate these questions across DeepSeekMoE, OLMoE, and Qwen3-MoE proposing a routing analysis toolkit for controlled counterfactual interventions and token-level analysis. By crossing source and merged router inputs and parameters, we attribute most expert reassignments to input shifts rather than parameter changes at the same layer. However, source-relative routing differences poorly predict next-token likelihood gains from source-route restoration, and different expert selections can produce directionally similar mixture outputs. We therefore operationalize routing failure as \textit{task loss recoverable under a specified routing intervention, with non-routing parameters fixed.} These tests detect recoverable loss under deliberate router corruption, whereas source-route restoration does not establish reliable task benefits in the evaluated merged models. Motivated by these, we propose \emph{Selective Router Repair (SRR)} as a case study, and find that source-specialist token-likelihood advantages do not reliably identify beneficial local corrections. Together, these findings show that \textbf{routing drift alone is insufficient evidence of routing failure}: source-informed corrections must be judged by their task-level intervention effects. The analysis toolkit and SRR code are released.
