Dual-anchor method improves stochastic monotone root finding speed
Dual-Anchor Acceleration Is Near-Optimal for Stochastic Monotone Root-Finding
Machine Learning
Summary
Finding solutions to certain mathematical problems can be slow when noise is involved. The authors studied a method called dual-anchoring, which helps speed up this process more reliably than older techniques. They improved it further by combining it with new tricks to reduce error and control randomness. As a result, their approach gets close to the best possible speed for these noisy problems. This helps find solutions faster when the problem involves uncertainty or randomness.
What this means in practice
- •For machine learning engineers: Speed up training algorithms that rely on solving noisy optimization or equilibrium problems with monotone operators.
- •For operations research teams: Improve efficiency in large-scale stochastic equilibrium computations within supply chain and economic modeling.
A theory result. No direct application yet.
Authors
TaeHo Yoon, Nicolas Loizou
Abstract
Among distinct optimal acceleration mechanisms for deterministic monotone root-finding problems and fixed-point problems, dual-anchoring has recently been shown to admit a more robust direct stochastic extension than standard anchor acceleration. However, without additional strong monotonicity, the existing stochastic dual-anchoring guarantee has two limitations: first, it requires cocoercivity in expectation, and second, it attains only $O(ε^{-3})$ oracle complexity, leaving a gap to the near-optimal $\tilde{O}(ε^{-2})$ complexity achieved by other methods. In this work, we address both of these limitations by combining dual-anchoring with stochastic resolvent approximation and optimized variance control. For unbiased stochastic oracles with variance bounded by $σ^2$, where sample operators are monotone and uniformly $L$-Lipschitz, our algorithm finds a point with $ε$-residual with a near-optimal oracle complexity of $O ( (LD / ε) \ell + (σ^2 / ε^2) \ell^2)$, where $\ell = \log (1 + LD / ε)$ and $D$ is the initial distance to a solution. This result improves the best known oracle complexity in the noise-dominated regime under these samplewise assumptions, reducing the poly-logarithmic factor from cubic to quadratic.