Muon 在什么条件下才真正胜过 AdamW？一项配对预算基准测试
When Does Muon Actually Win? A Paired Data-, Time-, and Tuning-Budget Benchmark
训练小模型的可以看看：Muon 省数据但在语言任务上按时间算反而输给 AdamW，瓶颈是优化器本身太耗时，文中数字很细。
arXiv 论文在单张 A100 GPU 上对比 Muon 与 AdamW，模型覆盖 17.7M/33.6M 参数因果语言模型和 2.70M 参数视觉 Transformer，用统一的 23 候选配置预算和五个随机种子评估。相同数据量下 Muon 将验证损失曲线下面积降低 3.25%–22.44%，20 组主测试全部降低测试损失。按时间归一化后，Muon 在视觉任务上提升 19.68%–22.31%，但语言任务上 AdamW 反而好 3.38%–6.96%。原因在于 Muon 优化器调用（含辅助 AdamW 更新）占训练时间的 56.1%–69.6%，而 AdamW 基线只占 6.1%–9.1%。论文结论限于参考实现的小模型，未对更大模型或优化实现下排序结论。
When Does Muon Actually Win? A Paired Data-, Time-, and Tuning-Budget Benchmark
Muon's gains in data efficiency motivate evaluating its performance under explicit data, time, and tuning budgets. On a single A100 GPU, we compare a reference single-device Muon implementation with AdamW across 17.7M- and 33.6M-parameter causal language models and 2.70M-parameter vision transformers. A common 23-candidate development budget selects configurations evaluated with five matched seeds, full validation trajectories, optimizer-step timing, and a frozen one-shot test. Muon reduces validation-loss area under the curve by 3.25--22.44\% at equal data and lowers test loss in all 20 primary pairs. In this configuration, Muon improves time-normalized loss area} by 19.68--22.31\% on vision, while AdamW is better by 3.38--6.96\% on language. The full Muon optimizer call, including auxiliary AdamW updates, consumes 56.1--69.6\% of training time, compared with 6.1--9.1\% for the AdamW baseline. Muon also has higher expected regret under restricted access to these candidate pools. These results characterize small models with reference kernels; their wall-clock ordering does not establish a ranking for larger models or optimized implementations.
