Simple method adds clear word emphasis to text to speech voices
EmpDPO: Enabling Emphasis Control in TTS via Direct Preference Optimization
Sound
Summary
Some words need extra emphasis when spoken to show meaning beyond just the text. This paper shows how to make a speech program emphasize words better, even if it wasn’t trained with emphasis before. The authors use two training steps with a tool that detects emphasis automatically, so no one has to label examples by hand. Their improved program puts the right emphasis on words while still sounding natural.
What this means in practice
- •For voice assistant developers: Add controllable word emphasis post training to improve assistant speech clarity without extra labeled data.
- •For audiobook producers: Enhance audiobook narration by enabling precise emphasis control on word level for more expressive readings.$Commercial implications: Allows selling more natural and expressive audiobook voices with fine-grained emphasis control.
Authors
Haonan Chen, Yunyun Wang, Megan Wei, Zeyu Jin
Abstract
Word-level emphasis conveys meaning that text alone cannot express and is therefore essential for a practical text-to-speech (TTS) system. In this work, we study how to add emphasis control post hoc to a pretrained TTS model that has never been trained with emphasis annotations. We present EmpDPO, a simple, efficient, and effective two-stage recipe that adds emphasis control to a multilingual zero-shot diffusion TTS model through supervised fine-tuning followed by direct preference optimization (DPO). The training signals for both stages come from an off-the-shelf multilingual emphasis detector, so that no manual emphasis annotation or task-specific reward design is needed. We find that DPO can substantially strengthen emphasis control with only a few thousand preference pairs and training steps. Objective and subjective evaluations against strong public systems show that our model produces perceptible emphasis while preserving naturalness and overall quality.