Self evolving multi agent system improves long form subtitle translation quality
Breaking Babel: A Self-Evolving Multi-Agent System for Long-Form Subtitle Translation
Computation and LanguageArtificial IntelligenceComputer Vision and Pattern RecognitionMultiagent Systems
Summary
Translating subtitles for entire series or long movies is tricky because it needs to keep terms consistent and respect culture and story context. Many current systems translate line by line and don't adapt based on the scene or series context. The authors created SMART, a system where multiple agents work together, learn from past translations during testing, and improve their translation steps dynamically. This approach was tested on a large variety of shows and languages, outperforming other systems in quality and human evaluations.
What this means in practice
- •For video localization teams: Improve subtitle translation quality and consistency across entire series with adaptive multi-agent workflows.
- •For media streaming platforms: Produce more culturally accurate and stylistically consistent subtitles for long video content to enhance user experience.$Commercial implications: Enables sale of advanced subtitle translation services improving streaming content localization quality.
Authors
Haibo Jin, Xinjie Li, Najmeh Sadoughi, Yang Liu, Yibo Wang, Zhu Liu, Yuzong Liu
Abstract
Long-form subtitle translation requires reasoning over discourse and cultural context spanning episodes or entire series, while maintaining consistent terminology and style. Existing single-LLM methods are largely sentence-level, and multi-agent systems often use static workflows that do not adapt to scene complexity or production context. We propose SMART, a Self-evolving Multi-Agent system for long-foRm subtitle Translation. During test-time training, SMART builds persistent series-level memory and translates a subset of sentences through a dynamic router and Mixture-of-Agents layer with tools for terminology verification, subtitle constraint validation, and contextual retrieval. A judge-refiner loop scores candidates and uses textual critiques to update agent prompts and routing policies without retraining the underlying LLMs. During test-time inference, the evolved configuration translates the remaining series. We also introduce Subtitle Arena, covering 14 genres, 2--198 episodes per series, production years 1959--2023, and 15 target locales, together with SubMQM, a subtitle-adapted MQM framework with seven dimensions and 19 error categories. SMART achieves the best overall MQM score in all 15 Subtitle Arena directions, reducing average penalty by 6.9% over the strongest competing agent system. On the public MuSC benchmark, SMART obtains the best model result across all four language pairs and also achieves the best human-evaluation result, with an overall score of 4.50/5.