DiaWhisper improves speaker and role detection in clinical interviews
DiaWhisper-DPO: Role-Attributed Transcription of Clinical Interviews via Failure-Mined Preference Optimization
Computation and Language
Summary
Understanding who is speaking during a medical interview is important to spot signs of depression accurately. The authors tested existing methods on two types of recorded interviews and found that current systems can make mistakes by splitting the task into separate steps. So, they created a new single system called DiaWhisper that listens and labels speech roles at the same time, learning from its own errors without extra human help. This system did better than previous methods on English and Chinese interview recordings, showing more reliable role detection.
What this means in practice
- •For clinical software developers: Integrate DiaWhisper to improve accuracy in assigning speech roles during automated depression assessments from audio interviews.
- •For mental health data analysts: Use DiaWhisper to reliably separate clinician and patient speech in multilingual interview datasets for better data analysis.
Authors
Weiming Li, Ana Catarina Fidalgo Barata, Miguel Constante, João Miguel Sanches
Abstract
Automated depression screening from clinical interviews requires attribution of utterances to the clinician or patient. We evaluate two datasets: DAIC-WOZ, where participant-only recordings require re-synthesizing both sides for controlled two-party evaluation, and PDCH-HAMD, comprising voice-converted real Chinese interviews for cross-lingual validation. Cascaded systems combine speaker diarization with role-assignment heuristics, so errors can propagate across stages. We propose an end-to-end model, which we named DiaWhisper, that fine-tunes Whisper-large-v3 with LoRA and an auxiliary frame-level role head for transcription and attribution, together with DiaWhisper-DPO, a failure-mined refinement that uses genuine decoding failures as DPO rejected completions without human preference annotation. On 29 DAIC-WOZ test sessions, DiaWhisper-DPO achieves 0.973 role accuracy and 0.119 DER, 72% below the strongest cascaded baseline, and reduces seed variation from σ = .205 to .002. Retrained on PDCH-HAMD, it achieves 0.757 role accuracy and improves all 78 session-seed pairs.