Audio language models verify removed speakers in psychiatric interviews
Role-guided Speaker Deletion Verification in Clinical Psychiatry Speech Recordings with Audio Language Models
Machine LearningSound
Summary
When researchers record psychiatric interviews, they sometimes need to remove all speech from either the patient or the clinician for privacy or consent reasons. This paper looks at how to automatically check if the removal of a particular speaker's voice was done correctly, without needing people to listen to every recording. The authors use advanced audio and language AI models to scan the edited recordings and find out if anyone's speech was accidentally left in. Their method improves the detection of missed deletions by combining multiple AI models and perspectives.
What this means in practice
- •For clinical research teams: Automatically verify speaker removal in large-scale psychiatric interview datasets to maintain consent protocol compliance without manual review.
- •For data privacy engineers: Detect missed speaker deletions in audio recordings to improve privacy safeguards in sensitive multi-speaker clinical data.
Tested on one dataset.
Authors
Joseph T Colonel, Daniel Katzman, Kelsey Kirker, Adam N Davidson, Shalaila S Haas, Cheryl Corcoran, René S Kahn, Guillermo Checci, Baihan Lin
Abstract
Clinical research in psychiatry increasingly relies on large scale collection of spoken language data to identify acoustic and linguistic biomarkers. Yet evolving consent and protocol requirements can oblige investigators to remove a designated speaker from multi-speaker recordings and to verify said removal at a scale infeasible for manual review of entire corpora. We study this verification problem for role-driven dyadic clinical dialogue in psychiatry and investigate it with two parallel, symmetric pipelines: confirming that clinician speech has been removed from psychiatric interview recordings, and confirming that patient speech has been removed from the same recordings. Each pipeline redacts the raw audio for its target role and then scans the surviving output with audio-language and large-language models to identify missed deletions. We evaluate this approach on a corpus of 48 dyadic recordings drawn from psychiatry settings, testing four open-weight models in an inference-only setting: Gemma-4-12B, Gemma-4-31B, Nemotron-3-Nano, and Nemotron-3-Nano-Omni. A disjunctive OR ensemble over fourteen model-view configurations had a combined F1 of 0.478 (precision 0.330, recall 0.870), an improvement over individual model estimates driven by recall gains that point to substantial complementarity across models and context views.