Learnable null embeddings improve speech synthesis quality and control
Learnable Classifier-Free Guidance Null Embeddings for Enhanced Controllable Speech Synthesis
Machine Learning
Summary
Text-to-speech systems try to turn written words into spoken speech, but sometimes struggle with voice consistency and expressiveness. This paper shows that by allowing the system to learn a special 'null' embedding—an unconditional sound representation—speech quality and speaker likeness improve. The researchers found that using different learnable null embeddings for various parts of the system gives more precise control over voice and expressiveness. This approach makes synthetic speech sound more natural and stable, even when pushing the system to produce stronger effects.
What this means in practice
- •For voice assistant developers: Produce synthetic voices with better speaker similarity and expressive control for personalized and natural interactions.$Commercial implications: Enables improved customizable voice products for consumer devices and services with more natural and stable speech generation.
- •For audiobook production teams: Generate expressive and consistent narrator voices with finer control over speech style and speaker identity.
Authors
Biel Tura Vecino, Yoach Lacombe, Julian Weber, Zbigniew Łatka, Haitong Zhang, Logan Hart, Eren Gölge
Abstract
Classifier-free Guidance (CFG) is widely adopted in text-to-speech (TTS) systems to enhance generation quality and conditioning fidelity by interpolating between conditioned and unconditioned predictions. A common unconditional technique is to use an empty representation, in the form of a fixed null vector. In this work, we propose replacing this representation with a learnable unconditional embedding, optimized to represent a meaningful unconditional state. Objective and subjective evaluations demonstrate that learnable null embeddings consistently outperform fixed null embeddings across speaker similarity, speech stability, and expressiveness, while exhibiting greater robustness to larger guidance scales. We further show that learning a distinct unconditional embedding for each of the TTS conditioning modalities allows fine-grained control over speaker and text guidance, showcasing the trade-off between similarity and quality, and stability and expressiveness in the generated speech.