Character error rates affect papyrus text tasks differently
Which papyrus HTR is good enough? Character-error-rate tolerance of four papyrological tasks on Greek texts
Computation and Language
Summary
Many ancient Greek papyrus texts are unread because they have not been digitized. The authors studied how much error in automatically reading these texts is acceptable for different research tasks. They created simulated errors in clean texts and tested how well models could still classify or search them. Some tasks like telling if texts are literary or documentary can tolerate more errors, while others like dating the texts need very low error. Training models on noisy text helps maintain accuracy even with more errors.
What this means in practice
- •For digital humanities teams: Choose appropriate error thresholds for automated transcription tools to support different scholarly tasks on Greek papyri.
- •For data annotation services: Improve training data by including typical transcription errors to build more robust models for ancient text recognition.
Authors
Anton Repushko, Elena Chepel
Abstract
Purpose: Most Greek papyri remain unpublished and undigitised; a handwritten text recognition (HTR) pipeline that transcribes them automatically would let scholars discover documents and literary works that have so far gone unread. Recognition systems for Ancient Greek papyri are in statu nascendi, and how accurate they must be for a given papyrological task has not been examined. To answer this and set a benchmark for Greek papyrus HTR, we test a range of character error rates (CER) against four papyrological tasks, using published editions as ground truth. Methods: From 63,846 current editions of Greek texts in papyri.info, we imitate a letters-only "perfect HTR" output by removing the editorial layer, then degrade it with a seeded algorithm to exact CERs of 1 - 50%, with lost lines and four error-shape variants. On these data we train small models (TF-IDF, fastText, a character CNN, ByT5-small) for document type, dating and documentary-versus-literary classification, and apply eight keyword search methods. We compare models trained on clean text with models retrained at a specific CER level, and evaluate across CERs. Results: Tolerance differs by task. With clean-trained models, documentary-versus-literary classification retains 90% of its metric up to 20% CER; document type up to 7.5%; subtypes and search up to 5%; dating only up to 3%. Retraining on text containing character errors largely eliminates the sharp degradation that otherwise sets in above 15% CER. Models generally tolerate concentrated damage in a long document better than small errors spread across a short text. Conclusion: The study provides a CER target for each of the four tasks and shows that models trained on noisy text make current, imperfect text recognition useful for them.