Music source restoration recovers original tracks from mixed and degraded recordings
Restore, Separate, Restore: A Modular Framework for Music Source Restoration
Sound
Summary
Mixed and processed music recordings often have all instruments combined and affected by effects like compression or noise, making it hard to hear each instrument clearly. The authors developed a three-part method to first fix these problems, then separate the instruments, and finally clean up each separated track further. This approach improves the quality of recovered original instrument recordings compared to earlier methods. Their method works on many types of instruments, such as vocals, guitars, drums, and more.
What this means in practice
- •For audio engineers: Restore and isolate clean individual instrument tracks from mixed and mastered recordings for remixing or remastering.
- •For music app developers: Build apps that provide users with high-quality separated and restored instrument tracks from degraded music files.$Commercial implications: Enables development of consumer software offering enhanced audio source recovery that can be marketed to music enthusiasts and professionals.
Authors
Tobias Morocutti, Emmanouil Karystinaios, Gerhard Widmer
Abstract
Music Source Restoration (MSR) seeks to recover original, unprocessed instrument stems from mixed, mastered, and possibly degraded recordings. Unlike conventional source separation, which treats the mixture as a linear sum of clean sources, MSR must additionally invert nonlinear production effects, such as equalization and compression, and transmission-related degradations, such as codec artifacts. We propose a three-stage framework built around this distinction: (1) a mixture restoration model that addresses degradation before separation, (2) a single model that separates the restored mixture into eight target stems (vocals, guitars, keyboards, synthesizers, bass, drums, percussion, and orchestra), and (3) stem-specific restoration experts fine-tuned on the separator's own residual artifacts. Each stage improves restoration quality over the previous one on the MSR Challenge test set. We release code and models to support future research in MSR at https://github.com/theMoro/music_source_restoration.