论文

Muon 优化器何时最优:径向脆弱性与极小极大几何分析

When Is Muon Optimal? Radial Fragility and Minimax Geometry

精选理由

搞优化器的人都该看看这篇,它把 Muon 为什么有效从曲率角度做了严格的几何刻画,还指出了原有解释的脆弱点。

该论文研究 Muon 优化器中梯度正交化更新的最优性条件。作者提出 RR-ICM(随机半径各向同性曲率模型)和 Polar-Factor Admissibility Set,刻画任意径向律下缩放极因子最优的几何条件。结果证明所有非标量可容许响应都来自与曲率折点对齐的径向原子,而所有无原子径向律呈现零容量刚性。论文进一步提出 Minimax RR-ICM,证明全局极小值点必落在梯度的极射线上,其几何为正半定 trace 平板结构。

原文 · arXiv cs.LG

When Is Muon Optimal? Radial Fragility and Minimax Geometry

Muon replaces a matrix gradient by an approximate polar factor and has shown strong empirical performance in neural-network training. The Isotropic Curvature Model (ICM) provides a curvature-based explanation for this update by asking when gradient orthogonalization minimizes a one-step local-loss surrogate. We investigate the stability of this explanation when the fixed input radius of the ICM is replaced by a general radial law. We introduce the Random-Radius Isotropic Curvature Model (RR-ICM) and the Polar-Factor Admissibility Set, which encodes exact scaled-polar optimality as a geometric condition on the positive polar factor of the gradient. We derive an exact characterization of this set for arbitrary radial laws and show that all nonscalar admissible responses arise from radial atoms aligned with curvature kinks. Consequently, every atomless radial law exhibits zero-capacity rigidity: exact scaled-polar optimality is possible only when the gradient is already a scaled column isometry. To connect the curvature-based framework with the established spectral geometry of Muon, we introduce the Minimax RR-ICM by replacing directional averaging with worst-case optimization over probability laws on the sphere. We prove that every nonzero global minimizer lies on the polar ray of the gradient, while its scale is determined by a one-dimensional convex problem. The corresponding admissibility geometry is a positive-semidefinite trace slab, in sharp contrast to the spherical-zonoid geometry of the RR-ICM. Together, these results expose the radial fragility of average-curvature explanations and provide a new minimax variational characterization of Muon's polar direction.

Muon 优化器何时最优:径向脆弱性与极小极大几何分析|AI热报