Handwritten text recognition halves errors on medieval latin manuscripts
Reading the Registers of Honorius III without Manual Line Transcription: Forced-Aligned HTR, End to End, Modular or Hybrid
Computer Vision and Pattern RecognitionInformation Retrieval
Summary
Medieval archives contain important handwritten documents that are hard to read and transcribe because the text is old and often abbreviated. The authors worked on letters from Pope Honorius III and used a new computer method to read and expand these writings automatically without needing manual line-by-line transcription. Their hybrid system greatly reduces errors compared to older methods by combining different recognition techniques and clever alignment tricks. This approach helps unlock valuable historical documents more efficiently.
What this means in practice
- •For digital archivists: Automatically transcribe and expand abbreviated medieval manuscripts from images without manual line segmentation for large-scale historical archives.
- •For historical text digitization teams: Integrate hybrid HTR models that reduce transcription errors to improve digital editions of Latin documents with complex scribal abbreviations.
Tested on one dataset.
Authors
Tamas Kovacs, Daniel Luger
Abstract
Unlocking medieval archives at scale is bottlenecked by the cost of line-level diplomatic transcriptions and manual abbreviation expansion. Focusing on the registers of Pope Honorius III (1216-1227) over 5,000 letters surviving in full only as manuscript images - we show that existing, line-break-free expert transcriptions (edition drafts) suffice to train end-to-end hand-written text recognition (HTR) that jointly transcribes and expands heavily abbreviated Latin. Using forced alignment on 860 unaligned edition entries across 394 pages of Reg. Vat. 9, we automatically bootstrap 13,122 training lines without manual alignment of segmented lines. While the generic CATMuS baseline struggles on this domain (18.9% CER) and a modular space-free HTR + word-segmentation pipeline (cf. Camps et al. 2021) plateaus at 15.6% WER, our from-scratch hybrid Conformer - combining CTC with bidirectional subword attention decoders, iterative self-realignment and n-best LM rescoring - achieves 4.7% CER and 8.7% WER (8.4% WER with ROVER system combination), nearly halving errors on clause-final words (22.7% to 12.2% WER). These figures are measured on the test lines that align with the transcription (86% of all test lines for the best hybrid). Furthermore, we address a tension between paleographic fidelity and editorial normalization: training on targets that preserve scribal name initials reduces the WER against the manuscript text on the 31 affected test lines from 13.3% to 9.2%, at a small cost against the expanded transcription (WER +0.3 points, CER +0.4). Finally, extensive ablations show that none of an auxiliary diplomatic CTC head, a copy mechanism, a permutation decoder or masked-image pretraining improves on direct end-to-end supervision, while strict padding-mask propagation and per-line beam termination make inference batch-invariant.