Symbol spotting helps read encrypted historical handwriting without labels
Unsupervised Domain Adaptation for Symbol Spotting in Historical Encrypted Manuscripts
Computer Vision and Pattern Recognition
Summary
Decoding old encrypted manuscripts is hard because you first need to recognize the secret alphabet they use. The authors created a method to find those alphabet symbols in handwritten texts by comparing them against clean digital fonts without needing any labeled examples. Their approach cleverly bridges the gap between neat digital fonts and messy handwriting using a special three-step process that learns shared features and adapts styles automatically. This method performs better than popular AI models and can even help identify unknown scripts, which could be very useful for historians and researchers working with mysterious documents.
unsupervised domain adaptationsymbol spottingencrypted manuscriptsSimCLRDANNembedding spaceCLIPpalaeographyzero-shot learning
Authors
Giuseppe De Gregorio, Alicia Fornés, Lei Kang, Beáta Megyesi
Abstract
The decipherment of historical encrypted manuscripts poses a fundamental challenge in Digital Humanities: before any transcription can begin, the symbol inventory of the underlying cipher alphabet must first be identified and characterized. We address this challenge through symbol spotting: given a candidate alphabet specified as a set of rendered font glyphs, the task is to determine whether and where its characters appear in an unseen handwritten document, without any labeled examples from the target script. The main difficulty lies in the domain gap between clean, digitally rendered font queries and degraded handwritten manuscript symbols. We propose a three-stage pipeline that bridges this gap without manual annotation, combining a joint SimCLR+DANN encoder for domain-invariant glyph representations with an embedding-space style-adaptation mechanism applied at retrieval time, requiring no re-training. Experiments on fourteen pages from seven encrypted manuscript collections show that our method outperforms zero-shot foundation models, including CLIP and DINOv2, by a large margin ($+0.194$ P@1 over CLIP ViT-L/14), and surpasses task-specific trained baselines by $+0.138$ P@1. We further demonstrate that the Raw-Cover metric, computed in a fully unsupervised setting, provides a meaningful script-family fingerprint that identifies the underlying alphabet of an unknown document. This capability is of direct practical relevance to palaeographers, historians, and other researchers working with undeciphered manuscripts.