Llm agents improve psychiatric interviews and diagnosis accuracy
PsyCIDRA: A Dual-Agent Framework for Psychiatric Interviewing and Diagnostic Reasoning
Artificial Intelligence
Summary
Psychiatric interviews are complex conversations where doctors ask questions to understand mental health issues. The authors created PsyCIDRA, a system using two AI agents—one that conducts the interview and another that reasons about the diagnosis based on the interview. Their system performed better than simply prompting large language models, agreeing more often with expert psychologists. This shows that AI could assist in mental health assessments by holding natural conversations and providing careful diagnostic suggestions.
What this means in practice
- •For psychiatrists and clinical psychologists: Support psychiatric assessments by automating parts of interviewing and reasoning to improve diagnostic accuracy while aiding expert review.
- •For mental health chatbot developers: Build psychiatric interview chatbots that conduct dynamic conversations and provide diagnostic insights consistent with clinical guidelines.
Authors
Milad Mohammadi, Fatemeh Akrami Shamsabadi, Zahra Mohseni, Amirhossein Safdarian, Malekfarhad Malek, Hadi Moradi, Hesham Faili
Abstract
Large language models show promise in clinical reasoning, but psychiatric interviewing requires guiding an evolving conversation. Their ability to carry out this interactive assessment remains less studied. We present PsyCIDRA, a dual-agent framework linking free-form psychiatric interviewing with diagnostic reasoning for expert review. Its interviewer agent uses tools to maintain working notes, load expert-written skills, and retrieve ICD-11 references to guide inquiry. Its diagnostic reasoning agent then receives the completed interview transcript and reports hypotheses alongside supporting, conflicting, and missing evidence, withholding a final hypothesis when none is sufficiently supported. Using patient profiles generated with PsyCPG, we first evaluate PsyCIDRA in simulation. Across four models on 53 evaluation cases, it achieves higher diagnostic agreement than direct prompting. On 81 held-out simulated cases, rank-1 accuracy is 60.5% versus 51.9%. In a blinded study of 101 human participants in separate arms, PsyCIDRA agrees with psychologists on whether to propose a diagnostic hypothesis in 79.6% of cases, compared with 65.4% for direct prompting. Together, these findings support the potential of LLM agents to assist psychiatric assessment through free-form dialogue. By examining diagnostic reasoning, interview quality, and safety together, this study contributes to understanding the capabilities and limitations of psychiatric interview agents.