Singer-Informed Vocal Source Separation for Multi-Singer Music Mixtures
2026-08-14 • Sound
Sound
AI summaryⓘ
The authors studied how to separate the singing voice of a specific person when there are multiple singers mixed together in a song. They used a short sample of the target singer’s voice to help their system focus on that singer, improving how well the system isolates them. Their method worked better than regular systems when dealing with two singers singing at once, making the target voice clearer and improving sound quality. They also created a special dataset to test their approach and shared their code publicly.
music source separationvocal source separationsinger embeddingfeature-wise linear modulation (FiLM)duet datasetSI-SDRFréchet Audio Distance (FAD)enrollment recordingtarget singer extraction
Authors
Jocelyn Xu, Minje Kim
Abstract
Music source separation systems typically extract a single vocal track and do not distinguish between multiple singers. We study singer-informed vocal source separation for multi-singer mixtures. Our framework introduces a short enrollment recording of a target singer to guide separation through a learned embedding. The singer embedding is incorporated using feature concatenation or feature-wise linear modulation (FiLM), enabling the model to focus on the target singer while suppressing interference. We construct a duet dataset based on DAMP-VSEP with quality filtering and non-overlapping enrollment segments. Experiments on solo and duet settings show that while baseline models perform well for single-singer mixtures, the proposed method improves target-singer extraction in multi-singer cases, increasing target-singer SI-SDR from 0.33 dB to 5.58 dB. Fréchet Audio Distance (FAD) further shows improved perceptual quality and better alignment with target audio distributions. Code and checkpoints are available at https://github.com/jocelynxu01/singer-separation-paper.