Speech enhancement improved by noisy target training with drifting loss
Noisy Target Training Meets Drifting for Weakly-Supervised Speech Enhancement
Sound
Summary
Cleaning up noisy speech recordings usually requires matching noisy sounds with their clean versions, which can be hard to get from real life. The paper explains how the authors improve a method that uses only noisy sounds by teaching a computer to make noisy recordings less noisy using extra noise inputs. They add a special technique called a Sinkhorn drifting loss that helps the cleaned speech sound more like real clean speech without needing exact clean examples for training. Tests with real recordings show this approach works better at reducing noise.
What this means in practice
- •For audio software developers: Integrate improved weakly-supervised noise reduction models that reduce noise residuals without needing exact clean samples for training.
- •For call center operators: Enhance voice clarity on noisy phone lines by applying models trained on noisy targets and drifting loss to real conversation recordings.
Authors
Yoshiki Masuyama, Dominik Klement, Christoph Boeddeker, Julius Richter, Kohei Saijo, Gordon Wichern, Takahiro Edo, Jonathan Le Roux
Abstract
In speech enhancement, standard supervised training requires pairs of noisy and clean speech. Such pairs are typically created by artificially degrading clean speech, leading to a domain mismatch between synthetic training data and real recordings. Noisy target training (NyTT) mitigates this limitation by training an enhancement model to predict a noisy recording from a noisier version generated by adding extra noise. However, since the training target is noisy, NyTT might learn to leave residual noise in the enhanced output. In this paper, we address this issue in NyTT by incorporating a Sinkhorn drifting loss that matches the distribution of the enhanced speech to that of unpaired clean speech. The Sinkhorn drifting loss attracts enhanced speech toward high-density regions of the clean-speech distribution while repelling it from the current enhanced-speech distribution in latent space. This loss uses only unpaired clean speech rather than the clean counterpart of each degraded recording, preserving the weakly-supervised nature of NyTT. Experimental results on VoiceBank-DEMAND and CHiME-3 real recordings demonstrate the efficacy of the proposed combination.