Medical language models verify and revise answers from multiple sources
CLEAR: Cross-Source Evidence Adjudication for Large Language Models in Medicine
Artificial IntelligenceComputation and Language
Summary
Medical knowledge keeps changing, but large language models (LLMs) only know what they learned during training. To keep answers accurate, the authors suggest CLEAR, a system that gets information from three places: the LLM’s own memory, trusted local documents, and live searches online. It compares answers from all these sources, checks if they agree or conflict, and decides if it needs to keep or change the answer. If there’s disagreement, CLEAR performs extra searches and rechecks, aiming for more reliable medical answers.
What this means in practice
- •For clinical ai developers: Improve medical chatbot accuracy by combining stored knowledge with up-to-date and curated evidence adjudicated for conflicts and agreement.
- •For healthcare data teams: Integrate multiple medical data sources with automated quality checks to support better decision-making tools in clinical settings.
Authors
Shuai Wang, Yize Zhao, Qingyu Chen
Abstract
Medical knowledge evolves continuously, whereas the parametric knowledge encoded in large language models (LLMs) is fixed at training time. External retrieval, including retrieval-augmented generation (RAG), can provide access to newly available evidence, but retrieved information may be irrelevant, incomplete, or conflicting. As a result, external retrieval can in turn degrade the factual accuracy and evidence grounding of LLM outputs. To address this challenge, we propose \textbf{CLEAR}, an agentic framework for cross-source evidence adjudication in LLMs in medicine. CLEAR independently generates candidate answers from three complementary pathways---parametric knowledge, locally curated corpora, and dynamically retrieved evidence---reflecting three common sources of information available to LLMs. An aggregation verifier jointly evaluates the candidates, supporting evidence, provenance, and source-quality information to identify agreement and conflict across sources. An adjudication module then determines whether the current conclusion should be preserved or revised through complementary override-guard and challenge-audit mechanisms, while unresolved conflicts trigger targeted follow-up search and re-adjudication.