Optimal risk descent speed follows a cycloid shape for fastest arrival

Least-time Gradient Flow

Machine Learning

Summary

When trying to improve something step-by-step by following its steepest downhill path, the authors study how to choose the speed at which to move. They find a special way to set this speed so that the overall error decreases at a precise rate, no matter the problem's complexity. To do this best, the speed follows a certain curved path called a cycloid, which they describe exactly. This result helps understand how to reach the goal in the least possible time with smooth speed changes.

What this means in practice

  • For machine learning engineers: Design faster convergence schedules in training by adjusting step speeds along risk landscapes based on theory-backed optimal timing.
  • For optimization algorithm designers: Incorporate the derived cycloid-based speed profiles into algorithms to ensure minimum time to reduce error under smoothness constraints.

A theory result. No direct application yet.

Authors

Alessandro Betti, Marco Gori, Stefano Melacci, Jinwei Zhao

Abstract

Prescribing the speed of gradient flow on the risk itself, by the dynamics $\dot w=-u(E(w))\nabla E(w)/\abs{\nabla E(w)}^{2}$, makes the risk $e(t)=E(w(t))$ obey $\dot e=-u(e)$ exactly, whatever the landscape~$E$; the time needed to reach zero risk from $e_0$ is $\int_0^{e_0}\dd e/u(e)$. Minimizing this time alone is ill posed, and we study the regularized problem $\inf\{\int_0^{e_0}(\tfrac\lambda2\abs{u'}^{2}+1/u)\,\dd e:\ u\in H^{1}(0,e_0),\ u\ge0,\ u(0)=0\}$, $λ>0$. We prove that the minimizer exists, is unique, and is a linearly scaled cycloid, and we show that the optimal rate behaves like $u^{*}(e)\sim(9/(2λ))^{1/3}e^{2/3}$ near zero risk: the exponent $2/3$ is the one found in \cite{betti2026holder} by a power-law ansatz, and it lies in the Hölder window $(\tfrac12,1)$ where the arrival is in finite time with vanishing weight speed. The proof follows the classical route: existence by the direct method, uniqueness by strict convexity, positivity of the minimizer away from the origin, and the explicit integration of the Euler-Lagrange equation.