Musical-time coordinates improve piano audio transcription of full pieces
A Learned Musical-Time Coordinate for Piano Audio-to-Score Transcription
Sound
Summary
Turning a full piano recording into written music notes is hard because computer systems usually need the music broken into small parts first. The authors created a new way to teach a system when each note happens in the musical bar without extra tools. This helps the system find bar boundaries by itself and write down notes for whole pieces, not just small clips. Their approach makes the resulting music scores keep the rhythm structure better and works for entire performances.
What this means in practice
- •For music software developers: Build piano transcription apps that process entire performances without manual segmentation using learned musical-time coordinates.
- •For digital music publishers: Automatically generate accurate digital sheet music from raw piano recordings including correct bar segmentation and rhythm notation.
Authors
Min-Hsing Wang, Tsung-Ren Huang
Abstract
End-to-end piano audio-to-score (A2S) transcription is the task of reconstructing a notated score from an audio recording in a single system. Existing A2S approaches usually rely on pre-segmented excerpts, and few handle whole pieces without external segmentation. This raises the question of whether a musical-time coordinate can guide score transcription while also helping the system segment whole pieces. Previous work on jazz saxophone A2S uses an external beat tracker to obtain such a coordinate for score prediction. We introduce downbeat phase, a musical-time coordinate learned jointly with score transcription that represents each audio frame's relative position within a notated bar. With this coordinate, our system can not only condition its score decoder on downbeat phase but also self-segment at bar boundaries and stitch the segments into a whole piece. Evaluation suggests that the coordinate improves metrical structure and supports whole-piece transcription with comparable notation quality.