LeanTTS offers compact bilingual speech synthesis with voice cloning
LeanTTS: Efficient Full-Stack Bilingual TTS for Zero-Shot Cloning and Voice Customization
Sound
Summary
Creating synthetic voices usually needs big, complex systems, which can be slow and costly. The authors built LeanTTS, a compact system that can mimic new speakers from just a short voice sample or customize voices using special tokens. It works in both Mandarin and English while using far fewer resources than similar setups. This means it can run quickly on regular CPUs and still generate good quality speech in two languages.
What this means in practice
- •For software developers: Build lightweight bilingual text-to-speech apps that support new speaker voices with minimal computational resources.
- •For call center technology teams: Integrate efficient voice personalization in customer support systems to improve multilingual user experience with realistic synthetic voices.
Authors
Yuheng Lu, Qiulin Li, Xi Wang, Yang Wang, Shengpei Jiang, Liyin Liang, Yifeng Qian, Longbiao Wang
Abstract
While recent advances in text-to-speech (TTS) synthesis have substantially improved speech quality and modeling flexibility, the development of lightweight and computationally efficient systems remains comparatively underexplored. We present LeanTTS, a bilingual complete synthesis stack that supports two complementary regimes: reference-prompt zero-shot cloning for unseen speakers and token-conditioned customization for recurring voices. LeanTTS combines a compact Mandarin polyphone front end, a structurally compressed flow-matching acoustic model, and a lightweight Vocos decoder with four-step MeanFlow generation. The resulting system contains 18.9M learned parameters, 86.12% fewer than the 136.2M-parameter ZipVoice-Vocos pipeline. On Seed-TTS-Eval, its zero-shot mode obtains 2.02 CER/62.3 SIM on Mandarin and 2.32 WER/48.3 SIM on English, exposing the quality-size trade-off under aggressive compression. With token-conditioned customized voices, the same system reaches 1.57 CER/77.2 SIM on Mandarin and 1.93 WER/76.6 SIM on English, with UTMOS scores of 3.03 and 4.33, respectively. It further achieves mean/P90 CPU RTFs of 0.049/0.061, establishing a sub-20M operating point for bilingual personalization and real-time CPU deployment. Audio demos are available at https://github.com/notlu/leantts.