Combining synthetic and real data improves Manchu OCR accuracy
Combining Synthetic and Real Data for Low-Resource Historical OCR: A Manchu Case Study
Machine Learning
Summary
Manchu is an endangered language with many historical documents that are hard to read by computers. This study found that using a mix of fake (synthetic) images and real historical images helps computer programs read Manchu text much better. The best models reached over 96% accuracy, and combining multiple models improved accuracy even further. This approach can help make old Manchu texts easier to search and study.
What this means in practice
- •For digital archives teams: Improve optical character recognition systems for low-resource historical languages like Manchu using combined synthetic and real training data.
- •For language technology developers: Build more accurate OCR models for endangered or low-resource languages by supplementing limited real data with synthetic images in training.
Authors
Yan Hon Michael Chung, Hanlin Wang
Abstract
Manchu, now critically endangered, was one of the principal languages of the Qing empire (1636-1912), and its extensive archival record is increasingly digitized but remains difficult to search and analyze at scale. Previous work showed that vision-language models (VLMs) trained only on synthetic Manchu word images can reach 87.4% word accuracy on real Qing manuscripts and prints, leaving a substantial synthetic-to-real gap. This study examines how synthetic and real historical training data should be combined for low-resource OCR. Using 60,000 synthetic and 20,306 real historical word images, we evaluate three pretrained VLMs and a compact convolutional recurrent neural network (CRNN) under four regimes: synthetic-only, real-only, joint synthetic-real, and sequential synthetic-to-real training, following a common checkpoint-selection and archival evaluation protocol. Introducing real training images raises the leading configurations to between 95.09% and 96.28% word accuracy, while no synthetic-only configuration exceeds 87.92%. Synthetic supplementation substantially improves all three VLMs, whereas its marginal effect for the CRNN is sensitive to the training objective. Joint and sequential training yield broadly similar archival accuracy under the tested practical pipelines. A compact CRNN also reaches the leading performance range once real images are available, showing that model scale alone does not determine recognition accuracy. Finally, complementary errors among strong recognizers allow voting to raise accuracy to 98.27% without additional training, while an eighteenth-century Manchu dictionary provides a principled rule for adjudicating disagreements.