Papers for

text-to-speech developers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Continuous-time modeling improves speech emotion tracking in text to speech

Continuous-Time Acoustic Modelling with Neural Controlled Differential Equations

Abstract: Text-to-speech (TTS) models commonly address text--speech alignment by expanding phone-level encoder states to frame-level decoder inputs using predicted durations. While this length-regulation step resolves alignment structurally, this use of duration typically changes only where and how often latent states appear, not the values of the states themselves. This paper proposes a continuous-time mechanism for duration-aware acoustic modelling in TTS using neural controlled differential equations (CDEs). We formulate the phone representation as a temporally parameterised control path and use a neural acoustic vector field to produce a continuous-time hidden state whose values evolve with phonetic content and duration-derived timing. The resulting trajectory can be sampled at discrete points and integrated into a standard acoustic decoder pipeline. Objective results contrast CDEs and typical recurrent models. Subjective results suggest that CDE-based models evaluating one phone per step can improve rank-order agreement between synthesised and reference emotion intensity while maintaining comparable emotion-expression quality to a strong baseline. Additional experiments with half-phone step-sizes suggest that temporal resolution changes the trade-off between style tracking and absolute calibration. These results position CDEs as a promising design space for continuous-time and duration-aware style-sensitive TTS.

Thu 10 SeptSoundArtificial Intelligence
The gist
When computers turn text into spoken words, they need to decide how long to say each part, like syllables or sounds. Usually, these lengths only change the timing but not the details of how the speech sounds. The authors used a new math tool called neural controlled differential equations to model the speech sounds continuously over time. This lets the speech better capture emotions and timing simultaneously, making the computer voices sound more expressive and natural. They showed that this method can track emotional intensity more accurately without losing quality.
Open 2609.11725v1