DriftTTS speeds up text-to-speech with single-step generation
DriftTTS: One-stage Single Step Text-to-Speech via Generative Drifting
Sound
Summary
Creating natural-sounding speech from text usually takes many small steps, which makes it slow. The authors developed DriftTTS, a new method that learns to generate speech in just one step, making it much faster. They solved technical challenges with speech length and showed their method produces speech as good as slower multi-step models. This work shows it’s possible to get high-quality speech without needing a complex multi-step training setup.
What this means in practice
- •For voice assistant developers: Generate natural speech responses quickly by using single-step text-to-speech synthesis to reduce processing time in devices.
- •For audiobook producers: Produce high-quality synthetic voices faster by applying single-step speech generation models for more efficient audiobook creation.
Authors
Jaehyun Ahn, Dohyun Kim, Heeseung Kim
Abstract
Text-to-speech models built on score matching and flow matching produce natural speech, but only by calling the decoder many times, which makes synthesis slow. Existing ways to cut the number of calls either distill from a multi-step teacher or add a stage on top of a pretrained model, leaving single-step TTS without a pretrained TTS model unexplored. We question whether pretraining a TTS model is necessary, and propose DriftTTS, which adopts generative drifting, an objective that moves iterative sampling from inference into training. Drifting assumes samples of fixed shape, whereas speech has variable length; we address this issue with segment drifting. On LJSpeech, DriftTTS matches the multi-step quality of Matcha-TTS and RapFlow-TTS in a single decoder call, while being faster. To our knowledge, this is the first application of generative drifting to text-to-speech.