Text to music models improved with rubric based optimization
Rubric-Based Optimization for Text-to-Music Generation
Sound
Summary
Generating music from text is hard because it's difficult to measure what makes music good. The authors study using detailed scorecards, called rubrics, from special audio-language models to judge music quality during training. They find that these rubric scores help improve musical qualities that are tricky to define, but for specific features like tempo or key, direct measurements work better. This approach helps train music generators that produce more pleasing music in several ways.
What this means in practice
- •For audio software developers: Create music generation tools that improve overall musical quality by optimizing models with rubric-based audio-language feedback.
- •For music recommendation engineers: Enhance recommendation systems by incorporating models trained to better capture broad, perceptual music qualities via rubric-based optimization.
Authors
Ping Wang, Guang Yang, Shao-Rong Su, Junkai Wu, Pang Wei Koh, Noah A. Smith
Abstract
Post-training text-to-music generation requires reward signals that capture multiple aspects of musical quality beyond what any single automatic metric can measure. We study structured, rubric-based rewards from pretrained audio-language models (ALMs) as training signals for both autoregressive and diffusion-based music generators. An ALM scores each generated clip against the rubric; we rank candidates generated for the same text prompt by their scores and convert these rankings into preference pairs for DPO on both MusicGen-small and ACE-Step~v1, and additionally use the rubric scores directly as scalar rewards for DiffusionNFT on ACE-Step~v1. On MusicCaps, rubric-based optimization improves CLAP, SongEval, and Audiobox-Aesthetics simultaneously, with the strongest gains obtained by DiffusionNFT on ACE-Step. By contrast, on MusicGen-small, building preferences from any one of these automatic evaluators produces clear cross-metric trade-offs: the targeted evaluator improves while other independent evaluators deteriorate. We further study tempo, key, and instrumentation, where precise objective rewards are available. Directly optimizing these specialized rewards reliably improves the target attributes, whereas ALM rubrics provide only partial transfer for tempo and instrumentation and no measurable improvement for key. Together, these results suggest a practical division of labor: ALM rubrics are effective for broad perceptual qualities that are difficult to formalize, while specialized objective rewards remain preferable when reliable measurements are available.