Flow maps improve text to speech quality with fewer steps
Distill Locally, Schedule Globally: Flow Maps for Few-Step Text-to-Speech
SoundMachine Learning
Summary
Text-to-speech systems can sound very natural but usually take many steps to create audio, which slows them down. The authors found a way to teach these systems to make good speech in fewer steps by changing how they learn and organize their work. They also created a method to decide the best number of steps to use during speech generation without extra testing. This makes it possible to make fast, high-quality speech with much less computing needed.
What this means in practice
- •For voice assistant developers: Create faster voice outputs by reducing the steps needed without losing clarity, improving user interaction speed.$Commercial implications: Enables commercial voice assistants to deliver high-quality speech with lower latency, appealing to users on mobile devices.
- •For speech synthesis engineers: Improve text-to-speech models to achieve natural sound using fewer computational resources and training time.
Authors
Yentl Collin, Evan Dufraisse, Amr Mohamed, Amine Khelif Khelif, Dani Bouch, Guokan Shang
Abstract
Flow-matching text-to-speech (TTS) models achieve high synthesis quality but require many neural function evaluations (NFEs) to integrate their generative trajectories. Recent few-step flow-map distillation approaches for TTS construct targets from numerically integrated teacher trajectories, creating a trade-off between target accuracy and training cost. We propose Local Flow-Map Distillation (LFMD), which adapts Eulerian Map Distillation to conditional TTS and avoids teacher trajectory integration during target construction. For inference, we derive a sampling schedule (TD-DP) from teacher dynamics and consistency of the learned maps, with a single cost graph supporting multiple NFE budgets without external audio-metric evaluation. Because scheduling offers no flexibility at one NFE, we refine this regime with alignment-aware temporal self-distillation using soft-DTW. Across Seed-TTS and LibriSpeech-PC, LFMD improves low-NFE synthesis over a matched integral-distillation baseline. On Seed-TTS, the refined student reaches 1.80% WER with 1-NFE, compared with 1.76% for its 32-NFE teacher.