Clinical AI decides when to diagnose or ask for more evidence
Decide, Ask, or Defer: Clinical LLMs under Incomplete Evidence
Artificial Intelligence
Summary
Making medical decisions with AI is hard when information is incomplete. This paper shows that clinical AI models differ a lot in deciding whether to give a diagnosis, ask for more information, or defer to a human doctor. The authors propose a new way to separate asking for more evidence from simply deferring, which helps reveal subtle behaviors that were hidden before. They tested several AI models and found tradeoffs between safety and autonomous decisions, meaning the new approach helps catch some AI mistakes but also sometimes removes correct decisions.
What this means in practice
- •For clinical ai developers: Design clinical AI systems that better decide when to diagnose or ask for more evidence, improving decision safety and transparency.
- •For medical software engineers: Improve diagnostic tools by integrating separate ask and defer options to handle uncertain clinical cases more effectively.
Authors
Mingzhan Yang, Weili Wu
Abstract
Clinical LLMs must decide not only what diagnosis to produce, but also whether the available evidence is sufficient for autonomous decision making. Binary DECIDE/ABSTAIN formulations merge distinct non decision states and do not explicitly evaluate information acquisition. We introduce a DECIDE/ASK/DEFER formulation together with a blinded protocol that prevents models from using evidence completeness metadata. We evaluate Qwen, Gemini, and GPT on 200 matched clinical evidence states constructed from DDXPlus. The models show substantial differences in action selection under identical evidence, with disagreement in 137 of 200 states. For Qwen, a matched targeted versus random analysis shows that selected information changes the likelihood of a subsequent autonomous decision more clearly than diagnostic correctness. Its matched DECIDE/ABSTAIN baseline further reveals a safety autonomy tradeoff: the three action policy rescues some erroneous autonomous decisions but also removes some correct autonomous deci sions. These results show that separating information acquisition from clinician deferral exposes behavior that binary abstention hides, without yielding a uniformly improved decision policy.