From transcription to semantic corpus analysis: unsupervised learning of sentence representations for ancient languages
2026-07-27 • Computation and Language
Computation and LanguageDigital LibrariesInformation Retrieval
AI summaryⓘ
The authors study ways to teach computers to understand sentences in ancient languages using only raw text, without any labeled examples. They focus on detecting reused text in important ancient Latin and Greek works and break the problem into spotting if reuse happens and finding exactly which parts match. They compare two unsupervised training methods and find that each has strengths: one is better at detection with lots of data, the other better at retrieval with less data. Their improved models outperform existing approaches and are accessible through an easy-to-use online tool.
Automatic Text Recognitionancient languagessentence embeddingsunsupervised learningtext reusesemantic searchlanguage modelscontrastive learningdigital humanitiesUMAP
Authors
Th{é}otime de la Selle
Abstract
Automatic Text Recognition (ATR) now supplies digital humanities with large volumes of unstructured, heterogeneous, and often noisy text in ancient languages. Downstream semantic analysestext reuse identification, alignment, and semantic search-rely on sentence embeddings, yet existing methods transfer poorly to ancient languages: generic multilingual encoders underperform, specialized language models yield anisotropic representation spaces, and labeled similarity data is unavailable. We study two fully unsupervised strategies - TSDAE and contrastive sentence embedding (CSE) - that adapt a specialized token-level language model into a corpus-specific sentence encoder using only raw sentences. On the philologically central case of biblical reuse in patristic literature (2,935 expert-verified parallels in Latin and Ancient Greek, from Augustine, Jerome, and Athanasius), we decompose reuse identification into two separately evaluated tasks-binary detection and correspondence retrieval-and benchmark the adapted encoders against multilingual, specialized, distilled, and supervised fine-tuned baselines, as well as on artificially noised data simulating HTR artifacts and scribal abbreviations. The adapted encoders outperform all baselines on both tasks, with complementary profiles: TSDAE leads detection given a large in-domain corpus, while CSE leads retrieval, reaches its optimum with as few as 4-8k raw in-domain sentences-a few tens of seconds of training on a laptop GPU-and transfers across works and authors, including to noisy post-ATR text when retrained directly on it. UMAP atlases relate the geometric effect of each strategy to the measured gains, and the full pipeline-segmentation, fine-tuning, cross-corpus semantic search-is made available to non-specialists through the online tool Paraphrasis.