Adaptive sampling improves translation quality and reduces compute costs
LAND: Learning an Adaptive Number of Draws for Bayesian Best-of-N Selection
Artificial Intelligence
Summary
Generating text often involves picking the best out of many tries, but this uses the same amount of computing power for every piece of text, even when some need fewer tries and others need more. The authors propose a method called LAND that smartly decides how many tries to use based on how helpful additional attempts are likely to be. LAND learns from past examples and stops trying once more attempts aren't worth the computing effort. In tests on translating languages, LAND matched or beat fixed-number attempts while using less computing power.
What this means in practice
- •For machine translation engineers: Optimize translation systems by dynamically allocating computation to text segments that benefit most from extra sampling, improving quality and reducing costs.
- •For natural language generation teams: Improve output quality and reduce generation time by adaptively deciding how many candidate texts to generate before selecting the best.
Authors
Koen Oostermeijer, Teryn Jones
Abstract
Best-of-N decoding improves generation quality by sampling multiple candidates and selecting the best, but it allocates equal compute to sources regardless of how much they benefit from additional sampling: the expected benefit is small for sources with low-variance score distributions and potentially much larger for those with high-variance distributions. To address this shortcoming, we propose LAND (Learning an Adaptive Number of Draws), which dynamically allocates samples according to their posterior-predictive marginal value. LAND learns a discrete empirical mixture of source-conditioned score distributions from calibration data, updates its belief about a source as completion scores are observed, and stops sampling when the expected improvement of another draw falls below a shared compute price. Across machine-translation experiments with multiple models and language pairs, LAND achieves better MetricX scores than fixed best-of-N at matched average compute. Conversely, at matched translation quality, LAND requires approximately 20-40% fewer candidate generations. Adaptive allocation therefore provides a simple way to retain the benefits of best-of-N while substantially improving its compute efficiency.