Machine learning improves automatic piano music transcription accuracy
A learning-based method of automatic music transcription
Sound
Summary
Transcribing music from recordings to sheet music is especially hard for piano because many notes can play at the same time. The authors treat the task like recognizing multiple notes at each moment and use special audio processing and neural networks that focus on time patterns. They improve the networks with attention mechanisms and handle the fact that most notes are silent most of the time. Their best model performs much better than a simpler method and creates accurate piano note transcriptions even in complex music passages.
What this means in practice
- •For music software developers: Integrate improved piano transcription models to convert audio recordings into digital sheet music with higher accuracy.
- •For music streaming platforms: Automatically generate symbolic music representations from piano recordings to enhance music search, analysis, or recommendation.
Authors
Phu-Anh-Tai Do, Huu-Hoang Nguyen, Duc-Hoang Nguyen, Thanh-Hai Le
Abstract
Automatic Music Transcription (AMT) seeks to convert acoustic music recordings into symbolic musical notation, a process that remains remarkably difficult for polyphonic piano due to concurrent polyphony, harmonic partial interference, expressive timing, and severe pitch activation sparsity. In this paper, we formulate polyphonic piano transcription as a frame-level multi-label classification problem across the standard 88 piano pitches (MIDI notes 21 to 108). Raw audio waveforms are processed using the Constant-Q Transform (CQT) to construct a geometrically spaced log-frequency time-frequency representation that aligns with the chromatic musical scale across seven octaves (252 frequency bins per frame). We systematically investigate recurrent sequence models, contrasting unidirectional Long Short-Term Memory (LSTM) with Bidirectional LSTM (BiLSTM) backbones, and propose two architectural adaptations tailored to recurrent temporal representations: a modified additive attention module and a modified multihead attention mechanism equipped with hyperbolic tangent activation and head-wise summation. Furthermore, to mitigate the severe class imbalance between inactive and active notes (approximately 41:1), we evaluate Weighted Binary Cross-Entropy (WBCE) alongside standard Binary Cross-Entropy (BCE). Rigorous empirical evaluations on the MAPS and MAESTRO datasets demonstrate steady, compounding gains from bidirectionality and attention mechanisms. Our proposed Multihead Attention-BiLSTM architecture attains an F1-score of 88.20%, an accuracy of 78.90%, a precision of 0.89, and a recall of 0.86, substantially outperforming the baseline unidirectional LSTM (F1-score of 71.20%) while maintaining a compact parameter budget of 4.43M. Qualitative piano-roll visualizations confirm that our model generates faithful transcriptions even in complex harmonic passages.