Real-time music source separation runs efficiently on low-power audio DSP
Real-Time Music Source Separation on a Low-Power Audio DSP
Sound
Summary
Separating different sounds from a music mix in real time usually needs powerful computers, but the authors show how to do it on a small, low-power audio processor. They found existing methods don’t fit the strict memory and speed limits of typical audio hardware. By changing how the model is trained and adding a new filter that controls delay, they made a system that works quickly and nearly as well as bigger setups. This opens the door for better music processing in small devices.
What this means in practice
- •For audio device manufacturers: Embed real-time music source separation into low-power devices with limited memory and compute resources.$Commercial implications: Enables new features in portable music and audio equipment that require live music separation with constrained hardware.
- •For mobile app developers: Create music apps that separate audio sources efficiently on standard mobile processors without heavy battery drain.
Authors
Jianan Li, Li Liu, Ken Malsky, Gabby Yi
Abstract
Real-time music source separation is validated on desktop CPUs and GPUs. Does any published system fit the embedded audio hardware it targets? On a commercial audio DSP (2 MB SRAM, 2.07 GMAC/s measured), none does, and the constraints eliminate different models: memory rules out the 16-51 M parameter TasNet/X-UMX family, per-frame compute rules out RT-STT, needing 5.5x the available MAC rate. Parameter count predicts neither: weight reuse spans 1x to 345x. We then build one that fits. Training on continuous rather than block-padded convolution context proves essential: a model scoring 3.93 dB block-wise otherwise collapses to silence within 2 s frame-by-frame. A gated complex FIR deep filter adds a latency knob, gaining 0.38 dB even when strictly causal. It reaches 4.70 dB cSDR on MUSDB18-HQ and runs in 10.43 ms of an 11.6 ms hop, 0.5-0.7 dB behind systems that do not fit.