LLMs more likely than doctors to suggest unnecessary care in clinical triage
Sense and Sensitivity: Benchmarking LLM Clinical Triage Recommendations with Physician Experts
Artificial Intelligence
Summary
This paper looks at how well large language models (LLMs) perform when deciding how urgently patients need care, compared to real doctors. The researchers created thousands of clinical examples and tested how both doctors and LLMs responded, especially when the wording of patient descriptions was slightly changed. They found that LLMs often recommend more care than necessary and that their decisions change more than doctors' decisions when the text is altered in ways that don't affect the actual medical situation. This means LLMs may be less reliable than doctors in real-world clinical settings.
What this means in practice
- •For healthcare it teams: Evaluate and monitor AI triage tools under varied clinical text inputs to reduce unnecessary care recommendations.
- •For clinical quality assurance teams: Use benchmark data to compare AI triage outputs with physician standards and identify sensitivity to irrelevant text changes.
Authors
Abinitha Gourabathina, Haoran Zhang, Yuexing Hao, Walter Gerych, Marzyeh Ghassemi
Abstract
As large language models (LLMs) are increasingly used in clinical settings, it is critical to evaluate their reliability under realistic variation in clinical text. We study this question in clinical triage, comparing LLMs to practicing physicians under text perturbations that preserve the underlying clinical setting. We introduce a benchmark of over 6,000 clinical scenarios, 7,000 physician annotations, and 225,000 model responses. Using this benchmark, we make two key observations. First, LLMs are more likely than physicians to recommend unnecessary care at baseline, and this tendency increases under perturbed inputs. Further, we find that LLM recommendations are more sensitive to gender and tone perturbations than human recommendations. Together, these results demonstrate that LLMs can vary under clinically irrelevant textual changes, highlighting the need for deployment-oriented evaluations grounded in expert physician behavior.