Flowgrpo improves vocal separation using new reward design
FlowGRPO for Vocal Separation: From Reward Analysis to Reward Design
Sound
Summary
Separating vocals from music tracks is tricky because the usual training methods don’t always match how we measure sound quality later. The authors study how to better guide the separation model by changing rewards it gets after it processes the whole sound. They focus on rewards that measure sound quality in different ways and improve these rewards to better match human hearing. This leads to a vocal separation method that picks out vocals cleaner and sounds better.
What this means in practice
- •For music streaming services: Enhance vocal isolation in music tracks to improve user-driven remixing or karaoke features with higher perceived audio quality.
- •For audio software developers: Integrate improved reward-based training for vocal separation models to deliver clearer vocal tracks in editing software.
Authors
Lishi Zuo, Chong-Xin Gan, Zezhong Jin, Youzhi Tu, Man-Wai Mak, KongAik Lee
Abstract
The Flow Matching objective supervises the velocity field at intermediate states, whereas waveform metrics are evaluated after decoding and may be non-differentiable. To align Flow Matching models with waveform-level and perceptual quality objectives during post-training, FlowGRPO optimizes their trajectories using endpoint rewards. In this work, we focus on music separation and use multi-resolution short-time Fourier transform (MRSTFT), scale-invariant signal-to-distortion ratio (SI-SDR), and Virtual Speech Quality Objective Listener (ViSQOL) as reward components. Before policy optimization, we analyze the relationship between data quality factors and the within-group gap between the maximum and mean rollout rewards for each component. This analysis motivates a revised ViSQOL reward that matches channel-wise root-mean-square (RMS) levels and evaluates the interval from the first to the last active frame, retaining inactive frames within that interval. We also apply a sample-quality filter based on predefined waveform checks. Together, the revised reward and quality filter enable FlowGRPO to improve the pretrained vocal separator in both separation performance and perceptual quality.