Derivative free updates enable matrix parameter tuning without gradients
Derivative-Free Structured Updates for Muon
Machine Learning
Summary
Training some neural networks needs adjusting many numbers in matrices, which usually relies on calculating derivatives or gradients. This paper shows how to update those matrices without needing gradients by using special ways to guess changes from just the output values. The authors try different approaches to do this guessing efficiently and accurately, even when gradients are unreliable or missing. Their experiments show some methods save effort but might be less precise, and overall, their approach can be useful in certain black-box scenarios where gradients are unavailable.
What this means in practice
- •For machine learning engineers: Optimize neural networks in situations where gradient computations are unreliable or unavailable by using structured finite difference updates.
- •For control system developers: Tune matrix parameters of policies in reinforcement learning tasks when only noisy or approximate feedback signals are accessible.
Authors
Pengcheng Xie
Abstract
Muon updates matrix-valued neural-network parameters by orthogonalizing a gradient-based momentum matrix. Its reliance on derivatives limits its use when gradients are unavailable or unreliable. We develop a derivative-free framework that constructs Muon-style updates from structured finite differences. Four variants are considered: full entrywise recovery, random low-rank surrogates, basis-aligned rank-one probing, and direct structured search. Exhaustive basis-aligned probing is equivalent, up to positive scaling before ideal polar orthogonalization, to coordinate finite differences. Matrix-regression experiments show that random rank-one probing can reduce the number of function evaluations substantially, at the cost of less accurate updates. Controlled noisy-gradient experiments on regression and a neural network illustrate when accurate function values can compensate for an unreliable gradient oracle. A small CartPole study further examines orthogonal rank-one probes under a fixed episode budget. These results support structured probing as a practical option for selected black-box problems; they do not establish a general convergence guarantee or an advantage over accurate, inexpensive gradients.