Psychiatric labels increase large language model hospital stay recommendations
Being judged through the record: A Rosenhan-inspired test of psychiatric label sensitivity in large language models
Computers and Society
Summary
Large language models can give different medical advice depending on whether they are told about a patient’s past mental health diagnoses. The researchers found that when these models knew a patient had schizophrenia before, they were much more likely to recommend keeping the patient in the hospital even if current symptoms didn’t support it. This shows that the models can be biased by psychiatric labels, which might affect their usefulness in clinical decisions. The study tested five popular language models to see this effect consistently.
What this means in practice
- •For clinical ai developers: Adjust model training or outputs to reduce bias from prior psychiatric diagnoses in clinical decision support tools.
- •For hospital risk management teams: Monitor AI recommendations for hospitalization to detect and mitigate bias caused by documented psychiatric history.
Authors
Shaked Rosenblum, Teddy Lazebnik, Ziv Ben-Zion
Abstract
The integration of large language models (LLMs) into clinical decision-making raises unresolved questions regarding diagnostic bias. We investigate whether a documented psychiatric history alters LLM recommendations when the current clinical evidence is identical. Using a counterfactual experiment inspired by Rosenhan's 1973 field study, we factorially varied diagnostic-history condition (no prior diagnosis stated vs. prior schizophrenia diagnosis) and current-evidence condition (voices ceased vs. voices persisted) within a clinical vignette. Across five LLMs (GPT-5.6 Sol, Claude Sonnet 5, Gemini 3.6 Flash, Grok 4.6, and Mistral Large 2512), we generated 20,000 model responses across independent queries. When evidence supported discharge, adding a prior schizophrenia diagnosis increased recommendations for continued hospitalization nearly fivefold, from 4.7% to 22.3% (risk difference, 17.5 percentage points; 95% CI, 16.2-18.8). This shift occurred across all five models (risk-difference range, 10.4-34.4 percentage points). In the same condition, reference concordance decreased from 95.3% to 77.7%, while stated decision confidence increased. Across all model-by-evidence contrasts, confidence increases corresponded to Cohen's d values of 0.35-1.09. Signal-detection analyses showed that the prior diagnosis shifted the decision criterion toward hospitalization across all five models and reduced discrimination between the two current-evidence conditions in three of them. When current evidence supported continued hospitalization, the label increased the recommendation for hospitalization from 92.6% to 97.0%.