Clinical question answering models struggle to fully verify cited claims
Verifiable by Construction: Claim-Level Evaluation of Verbatim Citation in Clinical Question Answering
Computation and Language
Summary
Answering medical questions quickly and accurately is hard, especially when doctors need proof for the answers. The authors tested how well language models provide exact quotes to back up every fact in their answers. While many models can often give quotes for most claims, those quotes don't always prove every part of the claim. This means there is still a gap in tools that can give clinicians precise, verifiable information they can trust without extra checking.
What this means in practice
- •For clinical informatics teams: Improve clinical QA systems by integrating models that provide exact quotes supporting each factual claim in answers, aiding trust and verification.
- •For medical chatbot developers: Develop chatbots that produce answers with exact, verifiable quotes from guidelines to support clinicians during time-sensitive decision-making.
Authors
Jiashuo Zhang, Yuling Chen, Yvonne Commodore-Mensah, Michael Oberst
Abstract
Large language models (LLMs) have been widely adopted for clinical question answering (QA). Current systems can attach citations to their answers, but these often point to broad texts, leaving time-pressed clinicians unable to verify them efficiently. An alternative is to ensure that responses are verifiable by construction: providing fine-grained verbatim quotes from reference material that substantiate claims, so users can verify an answer without opening other documents. In this paper, we evaluate the ability of current models to perform this task end-to-end: from providing citations for every factual claim, to producing verbatim quotes, to ensuring that those quotes fully substantiate the claims. To do so, we build a standardized harness over four clinical practice guidelines and evaluate twelve LLMs on 222 synthetic clinical questions, measuring each of these stages separately. We find that most models can attach verbatim quotes to over 90% of their claims from prompting alone, apart from some lightweight models such as claude-haiku-4.5. Yet these quotes often fail to substantiate every detail of the claims they accompany. For instance, claude-opus-5 produces verbatim quotes for 98.0% of its claims, but fully substantiates only 37.1%. Our work provides insights into the current capability gap of LLMs in building verifiable clinical QA systems, along with artifacts for future research.