Adaptive loop computation on the hypersphere improves reasoning accuracy
Spherical Fixed-Point Reasoners: Adaptive Loop Halting on the Hypersphere
Artificial IntelligenceMachine Learning
Summary
Solving puzzles and problems using AI often involves repeating steps many times, but deciding when to stop is tricky. The authors introduce a new method that works on a mathematical shape called a hypersphere, where stopping criteria and adjustments are simpler and more reliable. This approach leads to better performance on challenging tasks like Sudoku and mazes, and the way the AI processes inputs can be understood by looking at angles between tokens. This method also uses computation time more effectively by adapting to difficulty.
What this means in practice
- •For machine learning engineers: Improve efficiency and accuracy of AI models by adapting computation dynamically during inference using spherical fixed-point reasoning.
- •For game ai developers: Use adaptive loop halting methods to optimize game agents solving puzzles and mazes, enabling faster and more reliable gameplay reasoning.
Authors
Arber Zela, Artavazd Maranjyan, Pol Puigdemont, David Berghaus, Valentyn Boreiko, Volkan Cevher
Abstract
Looped (weight-tied) transformers enable adaptive computation by allocating more iterations to harder inputs. Fixed-Point Reasoners (FPRM) achieve this with a damped fixed-point solver, but operate in an unbounded Euclidean residual space with coupled rescaling and relative-norm halting. We introduce Spherical Fixed-Point Reasoners (SFPRM), which perform the same computation on the unit hypersphere of normalized transformers (nGPT). Input injection, damping, and halting become geodesic operations, yielding boundedness by construction and eliminating coefficient coupling. We prove fixed-point existence, local convergence under an explicit step-size condition, and global convergence under strong injection. On Sudoku-Extreme, Maze-Hard, and Mini-ARC, SFPRM achieves 96.6%, 87.9%, and 59.1% test accuracy, respectively, outperforming FPRM and other baselines while allocating test-time compute more effectively. Its converged geometry is also interpretable: token-input angles capture puzzle structure and predict correctness.