Hidden Markov model improves Ottoman poetry meter finding accuracy

A Lattice-Based HMM Approach to Aruz Scansion of Ottoman Poetry

Computation and Language

Summary

Ottoman poetry uses a special rhythm based on the length of syllables, but this is hard to detect automatically, especially in modern Turkish writing without special markers. The authors created a computer method that guesses the poem's rhythm pattern and matches it to the poem's syllables, even when the writing has errors or missing details. They trained their system on many lines of poetry and tested it against other programs, finding it much more accurate. Their tool and code are also available online for others to use.

What this means in practice

  • For digital humanities teams: Automatically identify and align rhythmic patterns in Ottoman Divan poetry texts without requiring detailed manual markers or annotations.
  • For natural language processing engineers: Improve meter detection tools for historical or modern poetry that lacks explicit metrical annotation by using a lattice-based HMM approach.

Authors

Mehmet Emre Tiryaki

Abstract

Ottoman Divan poetry is composed in aruz, a quantitative meter based on the weights of syllables. The meter is sensitive to specific characters recorded in the original writing in the Arabic script. This work is concerned with automatic meter finding and scansion alignment for Ottoman Divan poetry, especially on modern Turkish spelling without the special diacritics that scholarly transliterations of Ottoman preserve. We define two tasks on this input, first identifying the meter used in the poem, and second aligning the meter's weights with the syllabification of the poem, identifying the flaws and freedoms the poet has taken along the way. We build a corpus of 52,000 lines split by poet, with scansion annotations for 500 test lines, and a robustness benchmark generated at the deviation rates measured on real web and OCR copies. Our model is a hidden Markov model over the positions of each meter, decoded over a lattice of every candidate path, and trained by EM from the printed meter, without scansion labels. On a held-out set it scans 77.6% of the test lines fully correctly, against 26.8% for the available rule-based scanner, 55.0% for a neural tagger trained with the same supervision and 63.6% for Claude Opus 5.5 few-shot. The code is open-sourced and the tool is released on the web.