VeriDx improves diagnosis by checking disease reasoning steps
VeriDx: Earning the Right to Diagnose with Disease-Centric Verification
Artificial IntelligenceComputation and Language
Summary
Diagnosing diseases correctly is not just about getting the right answer, but also about checking all the necessary steps and evidence along the way. The authors created VeriDx, a tool that tracks whether doctors or AI systems fulfill these disease-specific checks while diagnosing. It spots problems like missing tests or ignoring conflicting information. Their tests on respiratory disease cases show many errors come from earlier mistakes in reasoning.
What this means in practice
- •For medical ai developers: Evaluate diagnostic AI systems by verifying if disease-specific reasoning steps and obligations are met, improving reliability beyond final answers.
- •For clinical decision support teams: Use disease-centric verification to identify and address diagnostic reasoning gaps in clinical software assisting doctors.
Authors
Zhong Cao, Shuying Chen
Abstract
A correct diagnosis can still be reached for the wrong reasons. In clinical reasoning, every disease hypothesis creates obligations: key evidence must be checked, alternatives must be ruled out, contradictions must be resolved, useful tests must be considered, and closure must be justified. Current evaluations of medical LLMs mostly focus on final answers, local steps, or isolated facts, and therefore miss these hypothesis-induced commitments. We introduce \textbf{VeriDx}, a disease-centric verification framework that links free-form diagnostic reasoning to structured disease profiles. VeriDx tracks whether each hypothesis is satisfied, unresolved, or violated its clinical obligations, exposing failures such as missing critical tests, unresolved differentials, ignored contradictions, unsupported claims, and premature closure. We instantiate VeriDx for complex respiratory diagnosis using guideline-derived disease profiles and expert-annotated longitudinal cases. Our results show that many diagnostic errors are not isolated mistakes, but broken commitments made earlier in the reasoning process.