Large language models classify and generate cognitive clinic dialogue
Understanding Clinical Cognitive Dialogues Using Large Language Models
Computation and Language
Summary
Conversations during cognitive tests between clinicians and patients involve many subtle back-and-forth interactions. The authors created a special dataset that labels these conversations in detail, capturing who says what and the type of dialogue act involved. They tested large language models to recognize these dialogue acts and predict what a patient might say next. While these models can generally grasp broad conversational goals, they still find it hard to distinguish very similar kinds of dialogue acts. This work provides useful tools and data to better understand and study clinical conversations but does not offer any diagnostic tools.
What this means in practice
- •For clinical ai developers: Build AI systems that recognize subtle patient responses in cognitive assessments by training on annotated clinical dialogue data.
- •For medical educator teams: Develop simulated patient dialogues to train clinicians using dialogue models fine-tuned on richly annotated clinical conversations.
Authors
Vishalakshi Arumugam, Dan Schumacher, Veronica Rammouz, Enrique Gonzalez Guerrero, Jeremy Davis, Anthony Rios
Abstract
In-person cognitive assessment is both a test and an interaction. Clinicians explain tasks, repair misunderstandings, and adapt to patient responses, while patients may hesitate, seek clarification, or disengage. Yet clinical dialogue resources rarely label the interaction structure needed to study these behaviors at scale. We present an de-identified corpus of 33 cognitive assessment conversations with 8,250 utterances annotated for three speaker roles and 56 dialogue acts. We use this corpus to benchmark large language models on fine-grained dialogue-act classification and next-patient-utterance generation. We also test whether out-of-domain instruction data and explanation-augmented training transfer to this clinical setting. Instruction tuning produces the strongest patient-utterance reference matching and improves classification accuracy. Reasoning-aware fine-tuning produces the strongest classification results among the LLaMA-3.1-8B variants. However, even the best models struggle to separate closely related dialogue acts, showing that broad conversational intent is easier to recognize than fine-grained communicative function. The corpus and benchmark make interaction structure measurable in cognitive assessments and support follow-up work on conversational markers, clinician education, and carefully validated simulated patients. This work does not make diagnostic claims. Instead, it provides the data and evaluation framework needed to study these applications.