Adaptive audio splitting improves sound separation quality
Learning Where to Split: Dynamic Band Splitting for Audio Source Separation
Sound
Summary
Separating different sounds from a mixed audio recording is tricky because different sounds have unique patterns across frequencies. The authors present a new method that adjusts how the audio spectrum is divided based on the actual sounds in the mix, instead of using fixed frequency boundaries. This adaptive approach lets the algorithm better capture important sound details and improves the separation quality. They tested it on several music and speech datasets and found it consistently outperforms fixed splitting methods.
What this means in practice
- •For audio software developers: Improve music source separation tools by implementing adaptive frequency splitting to enhance separation quality without increasing computational cost.
- •For speech recognition engineers: Enhance speech separation components in noisy environments by dynamically adjusting frequency boundaries for better extraction of speech signals.
Authors
Bingzhe Zhang, Kai Li, Xiaolin Hu
Abstract
Audio source separation models must preserve fine spectral structure while keeping the cost of the separator tractable. Band-split architectures reduce this cost by manually partitioning the spectrum into predefined frequency bands and encoding each band as a token, but the same boundaries are used for every mixture. Because informative spectral structure varies with source content, fixed boundaries can merge discriminative components or spend tokens where little resolution is needed. We introduce Dynamic Band Splitting (DBS), an end-to-end framework that adapts boundary locations and band widths to each mixture under a fixed band-token budget. DBS predicts content-conditioned boundary scores within frequency blocks, selects a prescribed number of boundaries using hard Top-$K$, and applies content-dependent pooling to obtain compact band tokens. After separation, it smooths and expands these tokens to the original STFT grid, refines them along frequency, and decodes complex masks with a frequency-shared head. DBS improved separation over the corresponding static models on DnR v2, MUSDB18-HQ, and EchoSet, with DBS-BSRNN achieving relative gain of 29.0% in average SI-SDR on DnR v2. These results and boundary analyses on DnR v2 show that content-adaptive frequency partitioning benefits multiple separator backbones under a fixed band-token budget. Code and audio demos are available at https://cslikai.cn/Dynamic-Band-Splitting