Papers for

content localization managers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Pterodactyl improves low-cost automated translation quality evaluation

Pterodactyl: Pushing the Limits of Efficient Automated Translation Evaluation

Abstract: In this paper, we present 1020 Group's submission to the WMT26 shared task on automated quality evaluation. Our system, Pterodactyl, was engineered specifically for subtasks 2 and 3, that is, segment-level quality score prediction and the detection of error-free segments. Pterodactyl is a highly customizable, ultra-low cost system optimized for binary classification in translation triage and data curation. It is based on an LLM-as-a-judge approach, with prompt engineering and model selection guided by ROC analysis. We also developed several variants of Pterodactyl that were specifically adapted for subtasks 3.

Tue 10 NovComputation and Language
The gist
Checking the quality of translated text can be expensive and slow. The authors created Pterodactyl, a tool that uses large language models to quickly decide if a translation is good or if it has errors. Their system is very customizable and cheap to run, making it easier to sort translations and find mistakes automatically. They also made special versions of the system to better spot perfect translations without errors.
Open → 2611.14480v1