Multi-candidate reasoning improved by fixing answer selection issues
Correct but Unselectable: The Hidden Interface Tax in Multi-Candidate Reasoning
Machine Learning
Summary
Small language models that solve problems by generating multiple candidate answers often have trouble choosing the best one because some correct answers are hard for the selector to understand. The authors found that even when the right answer is there, it might be incomplete or confusing, causing it to be ignored. They propose a method called Compatibility-Aware Candidate Construction (CACC) that fixes these issues without retraining the system. This approach helps the model pick correct answers more often, improving performance on math and reasoning tasks.
What this means in practice
- •For machine learning engineers: Improve accuracy of small language models by applying CACC to fix answer formatting before selection.
- •For automated customer support teams: Increase correct answer rates in multi-turn AI chatbots by repairing and aligning generated candidate responses.
Authors
Qianli Ma, Zhiqing Tang, Fanshuai Meng, Zhi Yao, Hanshuai Cui, Weijia Jia
Abstract
Small language models deployed under cost, latency, or privacy constraints cannot simply switch to larger models when reasoning fails. Multi-candidate generate-then-rerank offers a natural test-time scaling route. However, more candidates do not always yield higher accuracy. We identify a previously overlooked bottleneck between generation and selection: a correct answer may appear in the candidate pool yet fail to be selected because it is incomplete, unparseable, or misaligned with the answer mode the fixed selector expects. We term this loss the hidden interface tax and formalize a coverage-conversion decomposition that distinguishes raw oracle coverage from selectable coverage, the subset of problems where a correct candidate can actually be consumed by the selector. We propose Compatibility-Aware Candidate Construction (CACC), an interface layer between the candidate source and the fixed selector that repairs fragments, aligns answer modes, and removes selector-confusing noise without retraining either component. On numeric reasoning (GSM8K, competition_math), CACC raises both oracle coverage and final accuracy. On GPQA Diamond, CACC alone improves final accuracy from 5.6% to 19.2% (+13.6 pp); combined with proposer-side strengthening it reaches 25.8% (+20.2 pp). On MMLU-Pro, CACC exposes a coverage-conversion gap: answer-mode-compatible construction raises oracle coverage, but final accuracy decreases by 0.85 pp under the original selector; adding proposer-side strengthening improves final accuracy.