Language models vary recommendations for mental health by user type
Curbside Chatbots: Auditing the Mental Health Interventions Large Language Models Recommend to Clinicians, Caregivers, and Patients
Computers and SocietyComputation and LanguageHuman-Computer Interaction
Summary
People ask language models for mental health advice, whether they are doctors, caregivers, or patients themselves. The authors studied how 41 language models respond differently depending on who is asking, finding that doctors get more detailed advice while others get simpler, cautious suggestions. They also found that sometimes models suggest treatments they don't rate highly, often with qualifications. The authors created a large dataset and tools to keep checking these chatbots over time.
What this means in practice
- •For healthcare technology developers: Improve chatbot responses by tailoring mental health advice based on whether users are clinicians, caregivers, or patients themselves.
- •For mental health service providers: Audit and evaluate AI mental health tools to ensure appropriate and safe advice is given to diverse user types.
Authors
Matthew Flathers, Jill Noorily, Phuong Anh Nguyen, Mark Kalinich, Mina Fallah, Victoria Lisowski, Christine Shi, Rebekah M. Bodner, Anna Simms, Mihika Hete, Meiting Chen, Samuel K. Powell, Tatum Williamson, Laura Ospina-Pinillos, Haruka Notsu, Julian Herpertz, John Torous
Abstract
Users are asking language models mental health questions and are receiving answers. We audited what 41 language models recommend to users when the same clinical information reaches them from three footings: a clinician describing a patient, a caregiver describing a loved one, or a person describing themselves. Across 19 case vignettes and five questions, we collected 35,055 free-text responses, extracted every intervention named with a validated judge, and had 29 models rate the full option lists on a seven-point Likert scale for the same cases. Models largely recommend what they rate well, with recommendation behavior differing by footing, and many models offering clinicians a consultation-style response and lay users hedged, simplified advice. When asked about recommendations for a specific intervention type, models more often named options they themselves rated below zero as initial choices, usually with conditions attached. We will release the instrument, code, and full corpus on publication as a living audit.