Tacit-TTS speeds up voice cloning without using transcripts

Tacit-TTS: From Autoregressive Decoding to Masked Prediction for Efficient Transcript-Free Voice Cloning

Computation and LanguageSound

Summary

Voice cloning systems can copy a person's voice by analyzing speech, but many need to process words one after another, which is slow, or require the text of what was said, which limits flexibility. The authors created Tacit-TTS, a system that copies voices quickly without needing the transcript of the original speech. This means it works well across different languages and even with sounds like baby talk or made-up words. Tacit-TTS is much faster than earlier methods while keeping similar quality in cloned voices.

What this means in practice

  • For mobile app developers: Create fast voice cloning features that work without needing text transcripts, enabling apps to mimic voices in many languages or sounds instantly.
  • For content creators: Generate personalized voiceovers quickly using just audio samples, even if the sample contains gibberish or non-English speech.

Authors

Jian Chen, You Zhang, Mark Vinton

Abstract

TTS systems with autoregressive semantic modeling have demonstrated strong zero-shot voice cloning performance and rich expressive variation, but their sequential decoding incurs substantial latency. Non-autoregressive alternatives offer much faster generation, yet often rely on more restrictive reference conditioning, such as requiring transcripts of the reference speech during inference. We present Tacit-TTS, an efficient transcript-free zero-shot voice cloning system distilled from IndexTTS2. Our model replaces autoregressive text-to-semantic decoding with masked non-autoregressive generation, introduces training-free acoustic length estimation, and accelerates the flow-matching renderer through ReFlow distillation. Across two English and two Mandarin datasets, Tacit-TTS achieves competitive zero-shot quality while generating speech over 10x faster than IndexTTS2 for utterances longer than 5 seconds. Its transcript-free conditioning further supports cross-lingual and non-lexical references. We validate this capability using references from eight other languages, infant babble, and synthetic gibberish, where transcript-dependent systems often degrade or fail due to unreliable ASR transcripts.