Framework improves piano music creation from detailed text descriptions
TASTE: Taxonomy-Aware Symbolic Piano Generation from Textual Description
Sound
Summary
Creating music from text is hard because the text can describe many parts of music, and it's unclear how to train models well. The authors designed a way to break down music descriptions into clear categories like genre and emotion, making it easier to turn text into piano music. They built a large dataset of piano music files tagged in detail, and trained models that generate music matching the text more closely and with better quality. Their method shows improvements in controlling music elements and aligning music with text.
What this means in practice
- •For music software developers: Integrate nuanced text controls into software for creating piano music from descriptions.
- •For game audio designers: Generate adaptive piano music aligned with story or emotion cues described textually.
Authors
Mingyang Yao, Jingyue Huang, Ke Chen, Taylor Berg-Kirkpatrick, Shlomo Dubnov
Abstract
Text-conditioned symbolic music generation has attracted growing interest with the development of generative models. Yet two questions remain underexplored: what aspects of music the conditioning text should describe, and how the corresponding data should be collected and trained to improve controllability and text-music alignment. We address these challenges with TASTE, a taxonomy-aware framework for symbolic music generation with a main focus on piano pieces. First, we introduce a taxonomy of musical elements that specifies how a piece can be captured through textual description in a musically grounded way. The taxonomy comprises a set of metadata tags (genre, emotion, section plan, etc.) together with a free-form description of music imagery. This structure decomposes textual input to match the generation goal more coherently and precisely. Second, we build a data pipeline for collecting, annotating, and retrieving public symbolic piano resources, yielding a corpus of 432K MIDI files labeled at multiple levels of granularity. Finally, we design a staged training procedure that leverages this multi-granularity data across two model families: an autoregressive model and a non-autoregressive diffusion model. Both objective and subjective evaluations show that our models achieve superior musical-element control, text-description alignment, and overall music quality. We will release the full pipeline, all annotated datasets, and the model checkpoints.