Fine tuning vision language models improves low resource Lakota OCR accuracy
Design Considerations in Low-Resource Archival OCR: A Case Study in Lakota
Computation and Language
Summary
Reading old scanned documents in rare languages can help communities keep their history alive. The authors looked at how small computer vision and language models can read Lakota documents, a language with limited digital resources. They found that training these models on just a small number of pages made them much better at understanding the text. But, when dealing with messy, automatically cut-up images, the error increased, showing that real-world challenges still exist. Their work helps guide future technology to support language preservation efforts.
What this means in practice
- •For digital archivists: Improve transcription accuracy of low-resource indigenous language documents using fine-tuned vision-language OCR models.
- •For natural language processing developers: Develop OCR tools that accommodate orthographic variation and noisy document scans in small data settings.
Authors
Téa Wright, Laura X. Ma, Claudia Iron Hawk, Hannah J. Haynie, Alexis Palmer, Alane Suhr, Abteen Ebrahimi, Katharina von der Wense
Abstract
Building and expanding access to digital archives supports language reclamation and creates lasting resources for community revitalization. Designing effective tools to support these efforts requires taking into account the community-, language-, and document-specific context. We investigate how small ($\leq$3B) vision-language models (VLMs) can be adapted as optical character recognition (OCR) tools for Lakota, a North American Indigenous language. In addition to benchmarking modeling approaches, we inspect four practical facets of low-resource archival OCR that arise in previous work and conversations with community members: constrained training budgets, segmentation noise, scan degradation, and orthographic variance. We demonstrate that fine-tuning VLMs for OCR post-correction on 35 pages of transcribed data decreases character error rate (CER) from 17.1 to 3.9\% on a bilingual corpus, and find that training only with nine pages increases CER by only 1.1\% over this. However, evaluating under realistic automatic segmentation nearly doubles the CER of the best-performing models, stressing the importance of taking realistic use cases into account. We hope this work encourages future research on developing language technologies grounded in the workflows and constraints of ongoing revitalization efforts.