MedTRACE improves clinical diagnosis with smart evidence checking
MedTRACE: Tool-Augmented Multimodal Clinical Reasoning Agents for Evidence-Grounded Decision-Making
Computation and Language
Summary
Doctors need to look at different types of medical information, like images and health records, to make diagnoses. MedTRACE is a system that carefully checks this medical evidence, uses tools to find important information, and keeps track of how confident it is. This method helps the system make better and clearer decisions about a patient’s condition. The authors show that using MedTRACE makes diagnoses more accurate and easier to understand.
What this means in practice
- •For clinical ai developers: Build diagnostic AI tools that actively verify evidence and explain their reasoning with reliable confidence.
- •For medical imaging software teams: Enhance imaging analysis software to highlight and confirm diagnosis-relevant image regions for clearer clinical decisions.
Authors
Ji Lu, Lifei Liu, Haoran Yu, Xianglong Wang, Yiru Fang, Kuo Yang, Huiran Duan, Jianping Gou
Abstract
Multimodal clinical decision-making requires reliable reasoning over heterogeneous evidence from electronic health records, medical images, and physiological signals. Existing models typically map these inputs directly to diagnoses without explicitly assessing evidence sufficiency, tool-use requirements, or diagnostic uncertainty. This paper presents MedTRACE, a tool-augmented multimodal clinical reasoning agent for evidence-grounded decision-making. MedTRACE uses modality-specific encoders to construct a unified patient-state representation and performs an iterative loop of hypothesis formation, toolaware deliberation, and evidence verification. It dynamically invokes visual grounding, evidence retrieval, and structured parsing tools to locate diagnosis-relevant regions, retrieve clinical knowledge and similar cases, and extract structured findings. The acquired evidence enters an evidence memory, where a consistency verifier confirms or revises the current hypothesis. MedTRACE outputs a diagnosis together with supporting evidence, an auditable reasoning trace, and calibrated confidence. Experiments on multiple multimodal clinical diagnosis benchmarks show that MedTRACE improves diagnostic accuracy by 5.4% and AUROC by 4.7 percentage points over the strongest baseline. It also improves evidenceselection F1 by 8.2 percentage points and visual-grounding IoU by 6.5 percentage points, reduces expected calibration error by 31.6%, and decreases unsupported diagnostic errors by 27.8%. These results demonstrate that active evidence acquisition and verification improve the accuracy, interpretability, and reliability of multimodal clinical decisionmaking.