Detecting translation errors in agricultural equipment manuals with fine-tuned AI
INVAR-Agri: Detecting Discrepancies in Multilingual Operator Manuals of Agricultural Implements with a Fine-Tuned LLM
Computation and Language
Summary
Agricultural machine manuals are translated into many languages, but checking these translations for mistakes is slow and difficult. The authors tested two ways to use a large language model to find errors: one that translates back to English for comparison and another that checks the original and translated texts together. They found that the checking method caught more mistakes, although some types of errors were still missed. This AI tool could help support human reviewers, but it isn’t ready to replace them yet.
What this means in practice
- •For translation quality teams: Help review teams find discrepancies in multilingual agricultural manuals using AI-assisted checking between source and translated texts.
- •For industrial documentation managers: Support verifying compliance and accuracy of multilingual product manuals to improve operational safety and regulatory adherence.
Authors
Timo Oksanen
Abstract
Operator manuals of agricultural implements are delivered in many languages, and the standards require that competent persons review every translation, which makes this review the bottleneck. This paper presents a proof of concept that compares two training workflows on the same LLM, EuroLLM-1.7B-Instruct, trained on operator manuals of one manufacturer. In approach A, a back-translator translates each language version back into English, and an English reader compares it with the English version. In approach B, a checker reads the English version and the translation together, where the checker reports differences. The test sets contain errors planted in numbers, units, opposite words, obligations, negations, and conditions in 24 languages, and they include unmodified text and meaning-preserving edits for alerts and false alarms. The trained back-translator concealed many planted errors, most of all weakened obligations, while translation-quality scores rose. In contrast, the separately trained checker detected most planted errors, although it was weak for units and for number changes unlike those in training. A check by the author confirmed the concealment. In addition, a simple rule check caught the number and unit changes, but it alerted on harmless rewrites. A check of alerts on real Finnish translations showed that most alerts were false, caused by language-specific wording, so the precision on real text remains a limitation. Within this setting, a trained checker might support, not replace, the review by competent persons, and firmer conclusions need more data.