Optical Music Recognition for Real-World Manuscripts with Synthetic Data
2026-06-08 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionDigital Libraries
AI summaryⓘ
The authors explain that Optical Music Recognition (OMR) systems have improved a lot but mostly work well on clean, digital sheet music. However, real-world music libraries have many handwritten or old music manuscripts that look very different, making current OMR tools perform poorly. They show that using a mix of a little real manuscript data and many computer-generated fake images can help improve recognition without needing a lot of costly detailed labeling. This approach makes it easier to apply OMR to preserving historical music documents.
Optical Music RecognitionSheet musicHandwritten manuscriptsDomain adaptationFine-grained annotationMusic notationData synthesisCultural heritageMachine learningResource constraints
Authors
Jiří Mayer, Martina Dvořáková, Vojtěch Dvořák, Markéta Herzánová Vlková, Filip Bím, Pavel Pecina, Samuel Šomorjai, Petr Žabička, Jan Hajič
Abstract
Optical Music Recognition (OMR) has seen major progress in model design, with end-to-end methods now capable of recognising notation at all levels of complexity. However, the impact of this progress has been limited by the visual domains of available training datasets, which are largely born-digital. Existing large collections of sheet music in libraries and other heritage institutions contain predominantly manuscripts, whose visual domains are highly diverse and different, so existing OMR systems fail when applied in the real world. These institutions are often resource-constrained, so large in-domain datasets cannot be expected. We provide a first baseline on real-world manuscripts with complex piano notation in the resource-constrained scenario. Using fine-grained music notation graph (MuNG) annotations and the Smashcima synthesis tool, we then show that while some direct transcriptions of in-domain data remain essential, domain adaptation using synthetic musical manuscript images brings significant improvement. Furthermore, the symbols used do not need to be in-domain, so the expensive fine-grained annotation can be avoided. We thus bring OMR closer to one of its stated goals: preserving and promoting musical cultural heritage.