Muon polar update optimal only under specific radial conditions
When Is Muon Optimal? Radial Fragility and Minimax Geometry
Artificial IntelligenceMachine Learning
Summary
Training neural networks involves adjusting models to improve accuracy. The authors study Muon, a method that changes these adjustments to use an approximation called the polar factor. They find that Muon's effectiveness depends heavily on assumptions about how input data distances are distributed. When these assumptions change, the method only works optimally under very strict conditions. Their work introduces new ways to understand when Muon’s approach is best and reveals limitations of previous explanations.
What this means in practice
- •For neural network engineers: Know when Muon’s polar update works best to improve gradient steps in training neural networks under specific data assumptions.
- •For optimization algorithm designers: Use new geometric characterizations of polar updates to design or analyze gradient-based optimization schemes with structured curvature models.
A theory result. No direct application yet.
Authors
Ao Xu
Abstract
Muon replaces a matrix gradient by an approximate polar factor and has shown strong empirical performance in neural-network training. The Isotropic Curvature Model (ICM) provides a curvature-based explanation for this update by asking when gradient orthogonalization minimizes a one-step local-loss surrogate. We investigate the stability of this explanation when the fixed input radius of the ICM is replaced by a general radial law. We introduce the Random-Radius Isotropic Curvature Model (RR-ICM) and the Polar-Factor Admissibility Set, which encodes exact scaled-polar optimality as a geometric condition on the positive polar factor of the gradient. We derive an exact characterization of this set for arbitrary radial laws and show that all nonscalar admissible responses arise from radial atoms aligned with curvature kinks. Consequently, every atomless radial law exhibits zero-capacity rigidity: exact scaled-polar optimality is possible only when the gradient is already a scaled column isometry. To connect the curvature-based framework with the established spectral geometry of Muon, we introduce the Minimax RR-ICM by replacing directional averaging with worst-case optimization over probability laws on the sphere. We prove that every nonzero global minimizer lies on the polar ray of the gradient, while its scale is determined by a one-dimensional convex problem. The corresponding admissibility geometry is a positive-semidefinite trace slab, in sharp contrast to the spherical-zonoid geometry of the RR-ICM. Together, these results expose the radial fragility of average-curvature explanations and provide a new minimax variational characterization of Muon's polar direction.