StalePO improves machine translation quality using old edits
StalePO: Anchored Token-Level Preference Optimization using Legacy Post-Edits in Machine Translation
Computation and LanguageMachine Learning
Summary
Machine translation systems often update to better models, but only have old human corrections from earlier systems to learn from, which may be outdated or worse than new outputs. The authors address this ‘Stale Preference’ problem by creating a method called StalePO that carefully adjusts the new model using these old edits without degrading its overall quality. Their approach controls changes at the word level and anchors the model’s outputs to avoid making things worse. Tests on English-to-Hindi and English-to-Turkish translations show StalePO improves style and fluency, confirmed by both AI and human quality checks.
What this means in practice
- •For machine translation teams: Enhance updated translation models by effectively using existing human edits from older model versions to improve style and fluency.
- •For localization service providers: Improve the consistency and quality of translated content in multiple languages by applying stale feedback to newer models without costly new editing cycles.
Authors
Rohit Dhaipule, Sukhdeep Singh Kharbanda, Prasanth Bathala, Pradyumna Lanka, Anubhav Shrimal
Abstract
Machine translation systems are periodically upgraded to stronger models, but the available preference signal is human post-edits of an older system's outputs, which the newer model may already surpass. Moreover, collecting fresh post-edits for every new model is prohibitively expensive. We call this the Stale Preference problem. Standard DPO can fail in this setting: it may increase the likelihood of inferior post-edits, erode the model's existing quality, and fail to provide the per-token control needed to correct localized errors. We introduce StalePO, an objective derived from three requirements this regime imposes. Likelihood movement must be downward on both responses, the policy must be anchored to its own base response, and the KL constraint must apply at the token level. These requirements are jointly necessary. In ablations, each mechanism in isolation leaves the model's performance indistinguishable from the base model, and only their combination converts stale feedback into gains. On English-to-Hindi and English-to-Turkish localization data, StalePO improves the fraction of segments passing all LLM-as-judge MQM quality checks by 14.9 and 4.6 percentage points, respectively, with gains concentrated on style and fluency. A human evaluation under the same framework confirms these gains on English-to-Hindi, raising the fraction of segments passing all seven human checks by 13.8 percentage points.