Phase information improves piano note detection in ensembles
Phase-Based Onset Enhancement for Piano Transcription in Ensemble Settings
Sound
Summary
It is hard to detect which piano notes are played when the piano is part of a larger group of instruments because sounds can overlap. The authors found that looking at how sound waves change over time (phase information), rather than just their volume levels, helps better identify piano notes in these situations. They created a new dataset with accurate piano recordings alongside orchestras to test their method. Their approach worked better than older methods that only used sound loudness to find piano notes in ensemble music.
What this means in practice
- •For music software developers: Improve automatic piano transcription tools to work accurately with recordings of full orchestras by incorporating phase information.
- •For digital audio workstation makers: Enhance plugin features to better separate piano notes from other instruments for mixing and editing in ensemble recordings.
Authors
Hyemi Kim, Jiyun Park, Juhan Nam
Abstract
Piano transcription models have achieved very high performance in recent state-of-the-art systems, but their performance tends to degrade in ensemble settings where the piano is played alongside other instruments. While some approaches attempt to address this issue by incorporating multi-instrumental datasets, the problem remains challenging. In particular, when other instruments produce high-energy sustained tones while relatively low-energy piano notes occur simultaneously, the magnitude components of their frequency representations overlap, making it difficult to accurately distinguish piano notes. To address this issue, we propose an improved piano transcription model that jointly leverages magnitude and phase information, enabling more robust piano transcription performance in ensemble settings. We evaluate the proposed model on two piano ensemble datasets with precise MIDI note annotations, which we obtained by revisiting and extending existing datasets. Notably, we constructed a new piano concerto dataset, PCD-K, by playing a piano part on a Disklavier and recording the audio and MIDI pairs with pre-recorded orchestral tracks. Experimental results demonstrate that phase information is highly effective in improving performance, outperforming magnitude-based approaches in ensemble settings.