Ridge improves web question answering accuracy with smarter sampling
Follow the Mixed Groups: Ridge-Tracking Gaussian Sampling for Zero-Variance Web Rollouts
Machine Learning
Summary
Answering questions online often involves sampling different ways to find the correct response, but sometimes groups of answers are all right or all wrong, making it hard to learn from them. The authors propose Ridge, a method that tracks mixed-answer groups more effectively by adjusting how it samples questions based on statistical models of past results. This approach leads to better accuracy in answering math and retrieval-based questions while needing fewer attempts and tool uses. Ridge keeps the main learning rules but improves where to focus effort next, saving time and resources.
What this means in practice
- •For online question answering teams: Improve accuracy and reduce computational cost of web-based question answering by targeting mixed-result sample groups using Ridge sampling.
- •For ai tool developers: Optimize tool-call usage during AI model rollouts by adapting sampling weights based on posterior predictions with Ridge.
Authors
Tomas Duncun, Jonah Ellison, Owen Pell, Mateo Ruiz, Hanae Fujita, Ingrid Solberg, Caleb Nguyen
Abstract
Group relative policy optimization learns from a question only when the sampled responses disagree. A group that is entirely correct or entirely incorrect has zero reward variance and a zero advantage. On the Web, that group has already spent its searches and page fetches. Gaussian curricula aim the next batch at pass rates where a finite group can still mix. A Gaussian on the latest empirical rate treats every all-fail history as the same point, and a center stepped by the average signed change of those rates is pushed by prompts already stuck at 0 or 1. RIDGE keeps the Gaussian curriculum and changes the coordinate, the center, and the width. Each prompt carries a Beta posterior. The sampling weight is the marginal of a Gaussian target under a Gaussian approximation of that posterior, divided by a prediction of tool-call cost. The center and the width are an exponential smooth of the posterior means of groups that just came back mixed. All-correct and all-incorrect groups refresh the posterior and do not move the center. The GRPO update itself is unchanged. On two math backbones and a retrieval-augmented question answerer, RIDGE raises average accuracy by 3.4 and 2.9 points over GRPO and average exact match by 3.7 points, at 29.6\% fewer math rollouts and 32.7\% fewer tool calls than dynamic sampling.