Score helps separate orchestral instruments from recordings accurately
SCISSOR: Score-Conditioned Instrument Source Separation for Orchestral Recordings
SoundArtificial Intelligence
Summary
It can be hard to pick out individual instruments in an orchestral recording because many sounds overlap. The authors created SCISSOR, a method that uses a musical score to guide the process by telling the system when and which instruments play. This approach helps keep track of each instrument even if some notes are missing or mistaken in the score. Their method works better than others on real and synthetic recordings, and is still good even if the score information is not perfect.
What this means in practice
- •For music producers: Improve extracting individual instrument tracks from orchestral recordings using aligned scores for better mixing and editing.
- •For music information retrieval teams: Enhance automatic analysis of orchestral music by separating instrument sections even with overlapping sounds and imperfect scores.
Authors
Yiheng Lu, Hao-Wen Dong
Abstract
Orchestral separation recovers instrument sections from mixtures in which shared pitches, harmonics, and timbres obscure source identity. An aligned score provides instrument labels, note pitches, and activity times. A score-informed approach appends piano rolls to audio features before mask prediction. We introduce SCISSOR (Score-Conditioned Instrument Source Separation for Orchestral Recordings), which uses the score to form a frame-wise query for each source. Each query matches a shared audio representation, and a softmax over instrument and background slots jointly assigns overlapping time-frequency evidence. The queries retain instrument identity even when notes are missing from the score. After training on SynthSOD and a small set of URMP and PHENICX-Anechoic recordings, SCISSOR achieves the highest average SDR on held-out real recordings. With SynthSOD-only training, it leads on SynthSOD and zero-shot PHENICX-Anechoic, and improves on its audio-only control on zero-shot URMP. SCISSOR also degrades less under score corruption than the evaluated score-based baselines.