Statistical models improve piano note hand separation and fingering annotation
Statistical Models for Automatic Fingering-Annotated Piano Sheet Music Transcription
Sound
Summary
Transcribing piano music from audio not only involves recognizing which notes are played but also deciding which hand plays each note and suggesting finger placements. The authors developed eight different statistical methods to automatically label notes with hand and fingering information. Their best method correctly assigns hands 90.8% of the time and provides combined hand and fingering labels 56.6% of the time. They also created a full system that transcribes piano audio into sheet music with finger annotations, showing this task is achievable by machines.
What this means in practice
- •For music transcription software developers: Build transcription tools that output piano sheet music with hand and fingering information to assist learners.
- •For digital sheet music publishers: Automatically generate finger-annotated piano scores to enhance sheet music usability and accessibility.$Commercial implications: Finger-annotated scores can be sold to pianists and educators as enhanced digital sheet music products enabled by this method.
Authors
Daniel Penner, Abram Hindle
Abstract
Machine learning tools have significantly aided automatic piano music transcription; however, this domain has focused primarily on accurately predicting the pitches and timings of played notes. To produce sheet music for the piano, notes must be separated into two staves, one for each hand, and good sheet music often contains fingering annotations to guide the player when sight-reading or learning fast or complex pieces. We propose 8 statistical approaches for combined hand and fingering annotation of transcribed piano notes, including baseline hidden Markov models, rule-based methods, a synthesis of existing approaches, and N-gram language models. Furthermore, we develop a pipeline for complete transcription from piano audio to fingering-annotated sheet music. Evaluations with the PIG dataset demonstrate that our Synthesis model achieves a hand separation accuracy of 90.8\% and a joint hand and finger annotation accuracy of 56.6\%. These approaches serve as a new baseline for further research into this problem, while our pipeline demonstrates the feasibility of a combined system for automated note transcription, hand separation, and fingering annotation.