Machine unlearning shows limits for automatic speech recognition privacy
Evaluating Machine Unlearning in ASR
Computation and Language
Summary
Some laws say people can ask to have their personal data forgotten, which means computer programs should 'unlearn' that data. This paper looks at how well current unlearning methods work for speech recognition systems, which turn spoken words into text. The authors found that simpler methods do a better job balancing how much the system forgets and how well it still works. They also discovered common tests don’t always reveal if data was fully forgotten, especially when unlearning multiple pieces at once. This means future tools need better ways to check if the unlearning really works in speech systems.
What this means in practice
- •For speech system developers: Improve privacy features of speech recognition by selecting unlearning techniques that balance data removal and recognition accuracy.
- •For privacy compliance engineers: Design better evaluation methods for verifying whether speech systems have effectively forgotten requested user data.
Authors
Diogo Dinis, Francisco Teixeira, Bhiksha Raj, Alberto Abad, Isabel Trancoso
Abstract
Machine unlearning (MU) offers a path to compliance with "right to be forgotten" regulations. While MU has received increasing attention for speech tasks, it remains largely unexplored for Automatic Speech Recognition (ASR). In this work, we investigate whether existing MU algorithms and evaluation tools are suitable for ASR. We apply several MU techniques to an ASR model, evaluating privacy-utility trade-offs for single-subject unlearning, then assess the best algorithm under sequential and simultaneous unlearning. Results show that gradient ascent-based algorithms achieve strong utility-privacy trade-offs, whereas more complex approaches over-unlearn samples, making them easier to identify as unlearned. This suggests standard privacy evaluations based on simple Membership Inference attacks are insufficient to reliably assess unlearning success, motivating improved evaluation methods for MU in ASR. Finally, we show that both sequential and simultaneous unlearning yield worse privacy and utility than single-subject unlearning, underscoring the need for unlearning constructions better suited to these settings.