Medical ai learns when to trust external knowledge for safer answers
Learning When to Trust Medical Knowledge: Regression-Safe Routing for Clinical Reasoning
Artificial Intelligence
Summary
Clinical AI systems often use medical knowledge from external sources to answer questions, but sometimes that information can be wrong or misleading, causing correct answers to become incorrect. The authors created tests to catch tricky cases where the AI might be fooled by bad information. They designed a method called Med-TrustRouter that decides, for each question, whether to trust outside knowledge or rely on the AI’s original answer. This approach helps the system keep correct answers safe while still improving overall accuracy.
What this means in practice
- •For clinical ai developers: Integrate per-question decision-making to choose when to use external medical knowledge safely, reducing harmful errors in AI-assisted diagnosis.
- •For healthcare it teams: Deploy trusted routing layers to maintain accuracy when updating clinical AI systems with new or possibly unreliable medical data sources.
Authors
Hui Zhang, Yujia Zeng, Yuhao Xin
Abstract
Clinical reasoning LLMs need external medical knowledge, but retrieved evidence and reasoning chains can be outdated, poisoned, or hallucinated. Harmful knowledge can cause regression: changing a correct answer into an incorrect one. We construct a medical knowledge-poisoning benchmark covering retrieved evidence and reasoning chains, with seven attack families filtered by a three-judge "subtle-but-wrong" vote and paired clean exams that separate poisoning effects from question difficulty. No fixed knowledge-use strategy is robust across these settings: strong RAG methods trade robustness to poisoned evidence against accuracy when evidence is useful. We formulate knowledge use as per-instance routing and introduce Med-TrustRouter, a lightweight controller that keeps the solver frozen. From leakage-free runtime signals, it estimates each action's expected correctness gain and uncertainty, preserving the base answer unless another action is confidently better. Across seven medical reasoning settings, Med-TrustRouter achieves 78.9 macro and 69.2 worst-case accuracy, matching the strongest competitor on macro accuracy while substantially improving the worst-case floor. Its gains transfer across solvers and evidence judges, unlike a hand-coded router using the same signals. Similar performance across linear, tree, and neural action-value heads suggests that the benefit comes from routing rather than estimator complexity. These results support learning when to trust external knowledge for robust clinical reasoning.