Test-Time Augmentation for LLMs: When Input Diversity Beats Output Diversity at Matched Compute
2026-08-10 • Machine Learning
Machine LearningArtificial Intelligence
AI summaryⓘ
The authors look at ways to make large language models (LLMs) more accurate without increasing cost too much. They compare changing the model's input in different ways (Test-Time Augmentation) to the usual method of sampling multiple reasoning paths (self-consistency). They find that changing the input, especially by rephrasing questions, generally gives better accuracy for the same cost on most tasks tested. This works best for medium-sized models where very large, expensive ones are not an option. Overall, the authors show that mixing up the input helps models give better answers more efficiently than just trying multiple answers from the same input.
Large Language ModelsTest-Time AugmentationSelf-ConsistencyInference CostChain-of-Thought PromptingSemantic RephrasingLexical PerturbationsMulti-modal Question AnsweringPareto EfficiencyModel Scaling
Authors
Nikita Kozodoi, Zainab Afolabi, Jack Butler
Abstract
Test-time scaling improves LLM accuracy but multiplies inference cost, making the accuracy gained per unit of compute the metric that matters in deployment. Self-consistency is one of the established approaches, which spends this budget entirely on the output side by sampling repeated reasoning paths. We study Test-Time Augmentation (TTA), which extends self-consistency by also perturbing the input, aggregating predictions across transformed versions of the input, and ask whether input-side diversity converts compute into accuracy more efficiently than output-side diversity. We perform a systematic, matched-compute comparison: we evaluate three simple input-side strategies (semantic rephrasing, lexical perturbations, and visual transformations) across six datasets covering general and multilingual knowledge, mathematical reasoning, multi-modal question answering, and sentiment classification, against chain-of-thought prompting and self-consistency. Semantic rephrasing delivers consistent and statistically significant accuracy gains while Pareto-dominating self-consistency on cost-effectiveness, delivering roughly 1.8X more accuracy per dollar and outperforming it on five of six tasks. We further analyze the number of augmentations, multi-modal strategies, and base model scaling, finding that TTA is most cost-effective for mid-tier models where a stronger model is unavailable or too expensive. Our findings indicate that for current mid-tier LLMs, varying the input converts inference compute into accuracy more efficiently than varying the reasoning path alone. The TTA implementation is available at https://github.com/aws-samples/sample-genai-reflection-for-bedrock.