Model confidence guides chatbots to avoid making stuff up
Alleviating Hallucination in Reasoning Tasks with Training-Free Uncertainty-Guided Steering
Artificial Intelligence
Summary
Sometimes, large language models make up answers instead of giving correct ones. The authors show that by checking how sure the model is about its answers while it’s thinking, they can gently steer it toward better ones during the process. Their method doesn’t require retraining the model or extra labeled examples. This technique helps reduce the model’s tendency to hallucinate or invent incorrect facts across different tasks.
What this means in practice
- •For ai service providers: Improve the reliability of AI-generated answers by adjusting model outputs during inference to reduce hallucination without retraining models.$Commercial implications: Enables AI companies to offer more accurate conversational AI products by improving answer correctness dynamically at inference time.
- •For software developers: Integrate uncertainty-guided steering methods into apps to produce less fabricated information from language models in real-time applications.
Authors
Litian Liu, Qiqi Hou, Yubing Jian, Reza Pourreza, Mohammad Ghavamzadeh, Roland Memisevic, Yao Qin, Hong Cai
Abstract
Recent work on hallucination detection in large language models has shown that, for a fixed pre-trained model and reasoning task, it is possible to estimate the model's confidence in the correctness of its outputs. Such uncertainty estimates have primarily been used to improve truthfulness by detecting or filtering confabulations. In this work, we ask whether these signals can instead be used more proactively to directly improve the accuracy of model-generated answers. We propose USteer, a simple, training-free steering mechanism that adjusts a model's layer-wise activations during inference using the gradient of a confidence measure with respect to the activations. This procedure nudges generation toward outputs with lower uncertainty at inference time, without modifying model parameters or requiring additional supervision. We show that this approach consistently reduces hallucination across a range of tasks, demonstrating that confidence signals can be leveraged not only for detection, but also for effective inference-time control of model behavior.