Large language models improve translation of tagged internet texts
Tag-Aware Structured Text Translation: Towards a Systematic Understanding
Computation and LanguageArtificial Intelligence
Summary
Internet texts often contain special format tags that help structure and explain the content, but current translation tools struggle to keep these tags correct while also making the translation sound natural. The authors developed a new method that carefully creates training data balancing tag variety and natural language, breaks translation into smaller tasks for better learning, and uses three different ways to train the model to pay attention to language flow and tags. Their approach works better on many language pairs than previous methods and helps fix common errors when translating tagged texts.
What this means in practice
- •For localization engineers: Improve automatic translation of documents containing complex formatting tags, ensuring translated text preserves original structure and readability.
- •For software developers: Enhance multilingual content management systems handling tagged text by integrating translation models trained for tag-aware accuracy.
Authors
Zhanglin Wu, Hengchao Shang, Daimeng Wei, Jiaxin Guo, Zongyao Li, Tengfei Song, Ning Xie, Weidong Zhang
Abstract
Internet texts are replete with format tags that carry structural, semantic, and functional meaning. Current large language model (LLM)-based translation systems struggle to balance translation fluency with tag fidelity when processing tagged text. We argue that resolving this tension requires a systematic approach at three interconnected levels: data synthesis, capability building, and multi-objective alignment. At the data level, we identify and formalize a fundamental trade-off between structural tag diversity and translation naturalness in synthetic data generation; existing methods optimize for one at the expense of the other. We propose a hybrid synthesis strategy (Hy-LST) combining LLM-based synthesis tag method and Two-Stage LLM-based synthesis tag method to produce both diverse and natural tagged data. At the capability level, we decompose tag-aware translation into four sub-tasks of increasing difficulty in a multi-task supervised fine-tuning framework, enabling targeted capability acquisition and knowledge transfer. At the alignment level, we design three complementary reward functions under a group relative policy optimization framework, each targeting a distinct objective (fluency, tag fidelity, and tag-scoped translation quality), and show that joint optimization consistently outperforms single-reward alternatives. Experiments on six language directions (en2zh, en2ja, en2de, en2fr, en2ru, de2fr) demonstrate that each level contributes measurable improvements, and the complete system significantly outperforms existing methods. Qualitative analysis reveals specific error patterns and their mitigation after training with our method.