Index-translate combines multilingual models for diverse translation tasks
Index-Translate: A Multilingual Translation Model Family -- Text, Speech, Controlled Dubbing, and Long-Document Translation
Computation and Language
Summary
Translation technology often struggles to handle many languages and complex tasks like speech translation or long documents. The authors introduce Index-Translate, a family of models trained to translate text and speech across 150 languages, follow instructions, and work on tasks like controlled dubbing and translating lengthy documents. Their models come in three sizes and perform as well as much larger models in tests. The work includes specialized versions for speech-to-text, speech-to-speech translation, and syllable-based dubbing.
What this means in practice
- •For software localization teams: Use Index-Translate to improve dubbing quality and timing by controlling syllable alignment in multilingual audio translations.
- •For content production companies: Translate and dub large multilingual multimedia content with a consistent system supporting text, speech, and long documents efficiently.$Commercial implications: Enables scalable multilingual content creation tools that integrate dubbing and translation workflows, appealing to media production businesses.
Authors
Tianjiao Li, Mengran Yu, Chenyu Shi, Lusheng Zhang, Qisi Chen, Yanshan Zhou, Ji Qi, Jingying Liu, Yuang Feng, Ziang Cui, Tianxing Yan
Abstract
We introduce Index-Translate, a multilingual translation model family that combines a shared multilingual foundation with specialized training for general translation, instruction following, speech translation, controlled dubbing, and long-document translation. It includes three model sizes, 2B, 9B, and 35B-A3B, and supports translation in 150 languages, with multilingual instruction following. Evaluations on general translation and complex translation instructions show that Index-Translate outperforms translation models of comparable size and achieves performance comparable to 100B-scale translation models and frontier models. Index-Echo provides end-to-end speech-to-text and speech-to-speech translation, outperforming existing end-to-end models and achieving performance comparable to frontier omni models. Index-Homura extends the family to syllable-controlled dubbing. Index-NativeLong introduces native long-document translation with a dedicated task formulation and benchmark. These capabilities support diverse translation tasks, including multilingual content production.