Transformer system improves music source separation accuracy
SubRoformer: Subband Time-Domain Representation for Music Source Separation
Sound
Summary
Separating vocals and instruments from a mixed music recording makes remixing and music analysis easier but is challenging due to how sound signals are processed. The authors propose SubRoFormer, a system that breaks music into smaller frequency parts for better separation and then reconstructs the sounds. By analyzing these parts with a Transformer model, it can better identify each instrument or voice. Their approach performs better than previous systems on a standard music separation test set.
What this means in practice
- •For audio engineers: Separate vocal and instrument tracks from mixed music recordings with higher accuracy using SubRoFormer’s frequency-localized approach.
- •For music app developers: Build apps that allow users to remix songs by isolating different music stems with improved separation quality from a single model trained on more data.$Commercial implications: Enables development of commercial remixing tools that provide clearer isolated tracks for consumers and professionals.
Authors
Chunyat Wu, Chuang Ding, Paris Smaragdis, Qiuqiang Kong
Abstract
Music source separation extracts vocals and instruments from a mixed recording, enabling remixing, content production, and music analysis. Existing systems commonly operate on waveforms or full-band spectrograms. Waveform inputs contain many samples even in short segments, whereas a single spectrogram resolution imposes a shared time-frequency resolution trade-off across frequency ranges. We propose SubRoFormer, a Transformer-based system comprising a subband module and a residual RoFormer module. The subband module decomposes the mixture into frequency-localized, lower-rate signals for separation. It then reconstructs the target waveforms from the estimated subbands. The residual RoFormer module models temporal and cross-band relationships on temporally compressed features. It combines this context with fine-rate features through gated residual fusion to estimate source-specific complex masks. On the MUSDB18-HQ test set, training only on MUSDB18-HQ yields an average signal-to-distortion ratio (SDR) of 10.63 dB across four stems using target-specific models, achieving state-of-the-art performance among the compared systems without extra training data. With 500 additional training songs, a single model jointly separating all four stems achieves an average SDR of 11.59 dB, establishing state-of-the-art performance among the compared single-model multi-stem systems trained with extra data. Code and checkpoints are released at https://github.com/ArchiMickey/SubRoformer.