Adaptive replay refinement boosts AI math problem solving efficiency
Adaptive Replay Refinement: Allocating Test-Time Compute Between Revision and Resampling
Computation and LanguageMachine Learning
Summary
When computers try to solve problems, they can either keep improving one guess or start fresh with a new guess. The authors found a way to smartly decide when to stop improving a current answer and try something new without extra training. This method, called Adaptive Replay Refinement, helps save computing effort while still solving tough math problems well. It works by checking if further improvements don’t make a difference, then moves on to new guesses more efficiently.
What this means in practice
- •For machine learning engineers: Improve efficiency of AI model inference by allocating compute dynamically between refining current outputs and generating new ones without extra training.
- •For ai system architects: Design AI systems that reduce latency and computational cost during problem solving by detecting when revising outputs no longer helps.
Authors
Ahsan Bilal, Muhammad Ahmed Mohsin, Muhammad Umer, Ayesha Mohsin, Sarmad Bilal, Dean F. Hougen
Abstract
Test-time scaling allocates additional inference compute to refining existing solutions or generating new ones. This raises a fundamental question: Should I keep improving this solution, or stop working on it and try a completely new solution? Existing methods make this decision using fixed refinement schedules, self-verification judgments, or learned controllers, which can waste compute or rely on signals that may be incorrect or require additional training. We introduce Adaptive Replay Refinement (ARR), a training-free method that detects when a stationary revision reaches a fixed point and reallocates the released compute to a new independent sample; replay decoding treats the current solution as a draft and verifies unchanged tokens in parallel, enabling efficient fixed-point detection. Across seven models from five families and seven benchmarks, ARR matches majority voting on three of five mathematical reasoning benchmarks at equal compute, while reducing critical-path serial passes by up to 2.4 times. On MATH500, ARR achieves 84.6% accuracy with only 41% of the serial passes required by majority voting. These results demonstrate that detecting refinement exhaustion enables adaptive reallocation from revision to independent sampling, improving inference efficiency while highlighting the limits of self-revision. Code is available at https://github.com/AhsanBilal7/adaptive_refine.