Papers for

audiovisual content editors

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Target speech extraction works better with music in the background

MuseFlow-TSE: Music-Aware Target Speech Extraction via Cross-Source Band-Splitting and Data-to-Data Flow Matching

Abstract: Target speech extraction (TSE) isolates a speaker's voice given an enrollment utterance, but existing work is validated almost exclusively without music. Discriminative methods are content-faithful yet masking-bounded; generative methods improve naturalness but may hallucinate, drift in identity, and require many steps. We present MuseFlow-TSE, a discriminative deletion, generative insertion framework for music-bearing scenes. Stage-1 is a music-aware cross-source band-split extractor that estimates target, interfering speakers, music, and noise under conservation and music-leakage constraints. Stage-2 is a data-to-data flow refiner initialized by the discriminative estimate, negatively conditioned on interference stems, and run for 1--4 difficulty-adaptive Euler steps with an end-to-end fidelity objective.

Wed 21 OctSound
The gist
Sometimes when people talk with music playing, it’s hard to pick out just one person's voice. This paper introduces a method that can separate a target speaker’s voice from other sounds including music, other people talking, and noise. The method works in two stages: first, it separates different sound sources including music, and then it improves the speech clarity using a refined process. The authors show their approach helps keep the speaker’s identity clear and makes the extraction more accurate even with music present.
Open → 2610.29119v1