Ladders in Chaos: When, How, (and Perhaps Why) Does Test-Time Scaling Improve LLM Machine Translation
Computation and Language
Summary
The authors studied two ways to improve translations from large language models at test time: sequential sampling, where each new guess builds on previous ones, and parallel sampling, where guesses are made independently and then ranked. They found that sequential sampling can produce better and more natural translations, especially when only a few tries are possible. However, if many guesses are made, this method might reduce accuracy. Their analysis suggests that sequential sampling works well because the model uses more context from earlier outputs, but the method's success depends on how the context is constructed.
Authors
Di Wu, Sergey Troshin, Christof Monz, Antske Fokkens, Vlad Niculae
Abstract
Two forms of test-time scaling for Large Language Models (LLMs) have emerged as effective and widely adopted paradigms: sequential, in which later answer attempts depend on earlier ones, and parallel, such as i.i.d. sampling with reranking. In this study, we investigate their properties in translation. First, our study shows that sequential sampling has a higher performance ceiling, providing a more diverse and effective pool of samples, particularly under smaller sampling budgets. Second, we interrogate the nature of test-time scaling through a multidimensional manual analysis. Human analysis of the Best-of-$N$ translations demonstrates that sequential sampling substantially improves translation fluency and naturalness, but can degrade accuracy when inference budgets are large. Finally, we suggest an explanation of the mechanism through which sequential scaling improves machine translation. Our controlled analysis partially attributes the success of sequential self-improvement to the model's access to a larger target-side context. Ablation experiments on sequential sampling demonstrate its robustness across different sampling temperatures, while also revealing sensitivity to context construction, suggesting directions for future improvement.