Taming errors in long text speech synthesis improves quality and consistency
Taming Long-form Text-to-Speech
Sound
Summary
Generating computer voices from very long text or long audio samples is hard because the quality and voice consistency often break down. The authors propose a method called Localized Attention-Constrained Inference (LACI) that quickly detects mistakes during speech generation and fixes them on the fly by rerunning problematic parts with temporary safeguards. This approach greatly reduces errors and helps long speech sound more natural and similar to the original speaker, even for very long texts or reference audio. They also introduce a new way to measure voice consistency that better identifies failures.
What this means in practice
- •For voice assistant developers: Improve the reliability and voice quality of long conversations or monologues generated by text-to-speech systems using real-time error correction.
- •For audiobook producers: Generate consistent and accurate synthesized narration from very long text inputs while maintaining the cloned voice quality from lengthy reference audio.
Authors
Rongxiang Wang, Berkin Durmus, Aysegul Orhon, Eduardo Pacheco, Atila Orhon
Abstract
Long-form text-to-speech (TTS) enables multi-turn conversations with consistent prosody and higher quality voice cloning from longer reference audio. Recent open-weights autoregressive TTS models such as Qwen3-TTS and VoxCPM2 attain state-of-the-art word error rate (WER) and speaker similarity (SIM) on short-form prompts but significantly deteriorate when used with long-form prompts. We propose Localized Attention-Constrained Inference (LACI), an inference-only method to detect TTS errors in near real-time, roll back to the error onset and regenerate with temporary guardrails, adding negligible computational overhead. Using LACI, we improve worst-of-N WER across 10 RNG seeds for Qwen3-TTS-0.6B from 35.2% to 3.4% on prompts longer than 1500 words, even surpassing its short-form reliability of 5.4\% on prompts with fewer than 500 words. To demonstrate the efficacy of LACI on voice cloning reliability, we propose a sliding-window version of the SIM metric that we call wSIM. wSIM exposes several novel failure patterns that are not captured by SIM. LACI improves worst-of-N wSIM from 0.01 to 0.47 on 120 seconds of reference audio while reducing the rate of catastrophic generations with WER above 30% from 26% to below 1%