Bangla text to speech model achieves near-natural voice cloning with less data
BanglaBox: A Phonetically-Balanced Corpus and Data-Efficient Foundation-Model Adaptation for Bangla Text-to-Speech with Zero-Shot Voice Cloning
Sound
Summary
Making speech computer programs for languages with few recordings is hard. The authors created a special set of Bangla speech samples and smart training steps to teach a model trained on English to speak Bangla with many different voices. Their approach uses much less Bangla recording data than before but still sounds natural and clear. They tested it carefully with native speakers and compared it to other systems, finding it worked better. All parts of their work are openly shared so others can use or improve them.
What this means in practice
- •For voice assistant developers: Create Bangla-speaking assistants that can mimic multiple voices without needing large amounts of Bangla audio for training.
- •For localization teams: Add authentic Bangla voices quickly and efficiently to text-to-speech products by fine-tuning existing models with minimal data.
Authors
Emtiaz Uddin Ahmed, Araf Mahmud, Sajib Hossain, Tarikul Islam Tamiti, Sajid Fardin Dipto, Anomadarshi Barua
Abstract
We present a recipe for adapting English-pretrained autoregressive TTS foundation models to underrepresented languages, demonstrated on Bangladeshi Bangla. Existing Bangla TTS corpora are small and single-speaker, and to our knowledge no open zero-shot voice-cloning system is available for the Bangladeshi register. We contribute a phonetically- and gender-balanced two-tier Bangladeshi Bangla corpus balanced via a tiered Jensen-Shannon divergence objective over conjunct clusters (juktakkhor), together with three fine-tuning changes: a merge-consistent tokenizer extension, Bangla text normalization, and a prompt-masked dual-loss objective. These changes preserve zero-shot cloning across the language switch. Our BanglaEval protocol applies Wilcoxon signed-rank tests with Bonferroni correction over native-speaker ratings. BanglaBox attains near-natural Naturalness, outperforms commercial and open-source baselines on Naturalness, Speaker Similarity, and Clarity, and reaches speaker similarity comparable to prior few-shot results while using approximately 7x less Bangla fine-tuning audio. We further validate the recipe beyond our own test split using a seven-category stress set of difficult real-world text, the public BnTTS evaluation benchmarks, and naturally occurring Bangla that no language model wrote. All artifacts, including the corpus, weights, code, and complete evaluation materials, are released publicly and unconditionally.