Method calibrates gender bias in instruction text to speech models
Training-Free Instruction TTS Gender Bias Calibration Using Model-Adaptive Steering
Sound
Summary
Text-to-speech systems sometimes change the voice gender based on style prompts like job titles, even when no gender is specified. This paper introduces a way to adjust these systems without retraining, to make the gender distribution in generated speech more balanced. The method uses adjustments to the model’s internal signals and avoids changing voices when explicit gender words are given. The approach was tested on multiple different text-to-speech models and showed it can reduce gender bias while keeping speech quality high.
What this means in practice
- •For tts system developers: Adjust gender characteristics in synthesized voices without retraining models or overriding user input.
- •For voice assistant engineers: Calibrate voice gender balance in assistant responses controlling bias from style prompts while preserving user intent.
Authors
Kuan-Yu Chen, Yi-Cheng Lin, Jeng-Lin Li, Jian-Jiun Ding
Abstract
Instruction text-to-speech (ITTS) systems systematically encode acoustic gender skews from descriptive style prompts, such as occupations or personas, even when demographic attributes are left unspecified. Calibrating the implicit gender distribution of the synthesized voices, across heterogeneous architectures and without model retraining or overriding explicit user prompts, is an open problem. In this work, we propose \textit{model-adaptive steering}, a training-free bias calibration method that steers post-encoder conditioning representations using a group deviation vector paired with a coarse-to-fine strength search on a development set. A deterministic lexical gate bypasses intervention whenever explicit gender keywords are detected, preserving intended prompt semantics. Evaluated on a 12,800-prompt held-out benchmark across four ITTS models (three architectures), the proposed method reduces aggregate calibration error from 11.5--28.1 to 0.8--5.9 percentage points (e.g., shifting Parler-TTS Mini from 78.1\% and PromptTTS++ from 27.3\% female to 50.8--52.4\%). Output rates stay within 0.9 pp across anchor set sizes at fixed operating points, while UTMOS decreases by at most 0.08 and WER degrades by at most 3.3 pp.