AegisVox creates consistent surrogate voices for safer clinical speech sharing
AegisVox: Factorized Privacy and Clinical Preservation for Long-Form Speech
Artificial Intelligence
Summary
Clinical speech can reveal who someone is, not just through the words they say but also through their voice and speaking style. The authors developed AegisVox, a tool that changes identifying parts of speech while keeping important clinical details clear. It replaces speaker voices consistently across conversations and edits turns that contain identifiers. Their tests show that better replacement of identifying speech parts doesn’t always mean better protection or clinical accuracy, but AegisVox helps balance these tradeoffs.
What this means in practice
- •For hospital data teams: Transform sensitive clinical conversations so they can be shared for research while reducing identity exposure but preserving clinical details.
- •For call center managers: Replace customer and agent voices with consistent surrogate speakers to protect identities during call monitoring and training.
Authors
Praphul Singh, Akshat Agarwal, Ratul Ghosh, Aashna Kanuga, Devashish Khatwani, Anurag Dwarakanath, Sri Gadde, Krishnaram Kenthapadi
Abstract
Clinical speech reveals identity through both spoken identifiers and voice, yet its pauses, delivery, and speaker roles can also carry useful clinical evidence. Protecting that speech therefore requires more than preserving its words. We present AegisVox, a factorized framework that assigns consistent surrogate voices and edits identifier-bearing turns using document-consistent replacements. Across 390 public simulated or non-patient conversations totaling 63.6 hours, a matched comparison of two speech renderers separates replacement realization from retained identifiers, speaker linkage, and clinical preservation. Text-to-speech rendering realizes 535/601 planned replacements versus 453/601 for the codec-based speech editor. However, similar aggregate clinical preservation conceals more model-labeled contradictions, and conversation-wide speaker scores conceal greater edited-speech linkage under a score-direction sensitivity analysis. Fresh transcription of original audio provides a reference for interpreting the clinical measurement chain; an independent judge tests sensitivity to model-based assessment. Better replacement realization can therefore coexist with worse clinical or speaker-linkage measurements. AegisVox supports use-specific evaluation of those tradeoffs, not a guarantee of identifier removal or clinical safety.