Speech denoising improves by correcting teacher noise estimates
SNOAR: Self-training with Noise filtering, Observation adding, and Remixing for Speech Denoising Without Clean References
Machine LearningSound
Summary
Cleaning up noisy speech recordings is hard without perfect examples to learn from. The authors improve a learning approach where a model teaches itself to remove noise by fixing mistakes made along the way. Their method cleans noise that slips into speech and restores parts of the speech that get accidentally removed. This way, the model gets better at separating speech from noise without needing clean recordings to learn from. It performs nearly as well as methods trained with clean data and works well even on real noisy recordings.
What this means in practice
- •For speech recognition system developers: Improve noise reduction models without clean training data, enhancing real-world speech recognition accuracy in noisy environments.
- •For telecommunications engineers: Integrate improved speech denoising that adapts to real noisy recordings, boosting call clarity without relying on synthetic training sets.
Authors
Samuele Cornell, Matthew Maciejewski
Abstract
Unsupervised speech denoising methods such as MixIT and noisy-target training can be trained directly on noisy recordings, avoiding the sim-to-real gap of synthetic training data, but remain well below supervised performance. Teacher--student self-training, as in RemixIT, narrows this gap by remixing a teacher's estimates into new mixtures that supervise a student; however, since the student is trained only toward the teacher's estimates, teacher errors, in particular over-suppressed speech, are never recovered and propagate as the teacher is updated from the student. We propose SNOAR, a self-training framework that corrects the teacher's estimates before use: the noise leaked into its speech estimate is removed by differential noise filtering (DNF), and part of the noisy observation is added back in to restore suppressed speech and attenuate artifacts. We show that the two corrections depend on each other, as without DNF the loop collapses to the identity mapping. Compared to RemixIT, SNOAR is more stable along training and outperforms it in signal-based, perceptual, and word error rate metrics, reaching 97\% of the SI-SDR improvement of a fully-supervised topline on WHAM!; on real CHiME-3 recordings, it improves every metric of three different teachers, while RemixIT degrades their perceptual quality.