Conversational question answering systems show error fixes need user answers
Not One Belief: Answer Payloads, Persistence, and Dissociated Confidence Signals in Conversational Question Answering
Computation and Language
Summary
Answering machines in chat can make mistakes, and sometimes users push back or ask again. The authors studied how these systems respond to user reactions by separating what the user says into types (like repeating a question or disagreeing) and what answers they provide. They found that just re-asking or agreeing rarely fixes wrong answers unless the user supplies the correct answer. Several ways to measure confidence in answers often disagree, and only giving true information reliably repairs errors. This helps understand how to make chat-based Q&A systems more reliable and trustworthy.
What this means in practice
- •For chatbot developers: Design chatbots that improve answer accuracy by prompting users for correct answers rather than just re-asking or agreeing.
- •For customer support teams: Use confidence signals to better detect when automated responses are trustworthy and when human intervention is needed.
Authors
Duc-Vinh Tran
Abstract
Conversational QA systems should fix errors when users push back and signal which answers to trust. We split each follow-up into its speech act (re-ask, agree, or challenge) and its payload (an answer string, correct or not), and measure four confidence signals on the same transcripts: verbalized confidence, length-normalized likelihood, a frozen hidden-state probe, and P(True). Answers change when the user supplies an answer string and almost never under the speech act alone: in every retained full-size cell, the correct answer converts at least four times as many wrong answers as re-asking or agreement. The four signals disagree. On Qwen2.5-7B the frozen probe rises with answer persistence rather than truth; this holds on held-out questions, with Llama-3-8B (cleaned answers) as support, but not on four other models. Among 81 Qwen2.5-7B errors that survive one re-ask, re-asking, a fresh chat, or abstain repair almost none, while a gold answer repairs 75-95%. In two preregistered replications, every repair rate without an answer string stays below 0.15; a larger post-hoc rerun excludes 0.15 except for the empty elicit. With BM25 TriviaQA evidence, sentences that contain the answer repair 67-76% of persistent errors versus at most 12% when they do not. An end-to-end contain-gate cuts harm on TriviaQA, but a deployable gate that does not know the gold answer repairs few errors.