EQUITRIAGE: A Fairness Audit of Gender Bias in LLM-Based Emergency Department Triage
2026-05-05 • Computation and Language
Computation and LanguageComputers and Society
AI summaryⓘ
The authors studied if large language models used to help decide patient urgency in emergency rooms show gender bias. They tested five models on thousands of cases where they swapped patients' gender to see if the assigned urgency scores changed unfairly. All models showed some level of gender-related inconsistency, with some under-triaging women more than men. They found that bias varied by model and that hiding gender information sometimes helped, but factors like age also influenced results. The authors conclude that fairness tests need to be specific to each model before using them in hospitals.
Emergency department triageAcuity scoreLarge language models (LLMs)Gender biasCounterfactual invarianceCalibrationFairness auditMIMIC-IV-EDPrompt engineeringChain-of-thought prompting
Authors
Richard J. Young, Alice M. Matthews
Abstract
Emergency department triage assigns patients an acuity score that determines treatment priority, and clinical evidence documents persistent gender disparities in human acuity assessment. As hospitals pilot large language models (LLMs) as triage decision support, a critical question is whether these models reproduce or mitigate known biases. We present EQUITRIAGE, a fairness audit of LLM-based ESI assignment evaluating five models (Gemini-3-Flash, Nemotron-3-Super, DeepSeek-V3.1, Mistral-Small-3.2, GPT-4.1-Nano) across 374,275 evaluations on 18,714 MIMIC-IV-ED vignettes under four prompt strategies. Of 9,368 originals, 9,346 are paired with a gender-swapped counterfactual. All five models produced flip rates above a pre-registered 5% threshold (9.9% to 43.8%). Two showed directional female undertriage (DeepSeek F/M 2.15:1, Gemini 1.34:1); two were near-parity; one had high sensitivity with weak male-direction asymmetry. DeepSeek's directional bias coexisted with a low outcome-linked calibration gap (0.013 against MIMIC-IV admission), a Chouldechova-style dissociation between within-group calibration and between-pair counterfactual invariance. Demographic blinding reduced Gemini's flip rate to 0.5%; an age-preserving blind variant left DeepSeek with residual F/M 1.25, implicating age as a residual channel. Chain-of-thought prompting degraded accuracy for all five models. A two-model ablation reveals opposite underlying mechanisms for the same directional phenotype: in Gemini the signal is emergent in the combined name+gender swap, while in DeepSeek the gender token alone carries it. EQUITRIAGE shows that group parity, counterfactual invariance, and gender calibration are distinct fairness properties, that intervention effectiveness is model-dependent, and that per-model counterfactual auditing should precede clinical deployment.