Extracting translation data from old language documents to aid rare languages
Mobilizing the Archive: Bitext Extraction at Scale from Archival Documents for Extremely Low-resource Languages
Computation and Language
Summary
Many languages have little presence on the internet, making it hard to gather translations for machine translation technology. The authors show how old language materials like grammars and dictionaries can be scanned and processed to find example sentences and their translations. They use tools that read text from images and match text with the page layout to find pairs of sentences in different languages. These pairs help train translation systems for 20 low-resource languages. The approach also improves large language models when translating between rare and common languages.
What this means in practice
- •For machine translation developers: Create translation systems for languages with little online data using examples extracted from historical linguistic materials.
- •For natural language processing engineers: Improve large language model translation performance by augmenting in-context learning with extracted bitext pairs from archival documents.
Authors
Chutong Meng, Alec J. Bateman, Antonios Anastasopoulos
Abstract
While mining parallel sentences from web pages has proven effective for low-resource machine translation, the majority of the world's languages have little to no web presence. What they do have, though, are resources that are the result of centuries of efforts by linguists, such as grammars, dictionaries, and other documentation materials. We propose extracting translation examples from such linguistic resources, which offer parallel data for languages under-represented online and could potentially scale to thousands more languages. We develop a pipeline to achieve this, leveraging an Optical Character Recognition (OCR) model as well as a vision language model (VLM), to ensure identified content to be aligned with visual layout information. We demonstrate the utility of the extracted bitexts for bootstrapping translation systems across 20 languages, both by training machine translation models and by applying in-context learning for large language models. For in-context learning, we observe improvements of +15.78 and +6.94 chrF2++ over zero-shot translation for into low- and high-resource languages, respectively.