Large language models improve accuracy by deciding when to abstain
When Should LLMs Abstain? Chain-of-Self-Questioning for Selective Risk Control
Computation and LanguageArtificial Intelligence
Summary
Large language models sometimes give confident but wrong answers when they don’t have enough reliable information. The authors introduce a method called Chain-of-Self-Questioning (CoSQ), which helps these models check if they have enough support before answering. This approach lets the model decide to answer or abstain, reducing wrong answers and improving accuracy. The method was tested across many models and question types, showing consistent improvements.
What this means in practice
- •For chatbot developers: Improve chatbot reliability by implementing CoSQ to let models abstain when uncertain, reducing incorrect responses.
- •For customer service teams: Use CoSQ-enabled language models to lower misinformation risk by deferring unclear queries for human review.
Authors
Ali Şenol
Abstract
Large language models can produce fluent answers when their factual support is weak. This paper introduces Chain-of-Self-Questioning (CoSQ), a prompt-only framework that makes answer commitment conditional on an explicit assessment of the information required to answer a question. We evaluate three CoSQ variants under seventeen conditions on the 817-item TruthfulQA multiple-choice validation set using eleven open-weight and hosted model families. In the final balanced-option protocol, Grounded-CoSQ at τ=0.90 reduces the mean unconditional wrong-commitment rate from 13.1% under chain-of-thought prompting to 8.9%, a 32.1% relative reduction, while increasing answered accuracy from 86.9% to 89.7% and answering 87.6% of questions. Both improvements hold for all eleven models and at every evaluated threshold. Critical-CoSQ and Adaptive-CoSQ provide neighboring operating points with 88.6% and 86.5% coverage, respectively, while remaining more reliable than the baseline. A secondary Natural Questions Short-Answer evaluation provides convergent open-form evidence. These findings show that self-assessment can support explicit, tunable answer-or-abstain decisions when an unsupported commitment is more costly than referral or review.