Orchestral historical music restoration improves with latent space models
End-to-End Historical Music Restoration in Latent Space
Machine LearningSound
Summary
Old recordings of orchestras often sound poor because the technology then was limited and damaged over time. Restoring these recordings is hard since there are no perfect original versions to compare with. The authors created a way to simulate how old recordings get damaged and then trained a computer model to fix these problems. Their method works better than previous ones in making old orchestral music sound clearer and more natural. They also shared a large test set and open code for others to use.
What this means in practice
- •For audio restoration engineers: Restore noisy or degraded orchestral recordings using a trained model based on simulated historical audio damage.
- •For music streaming platforms: Improve the quality of early-20th-century orchestral recordings for listeners through enhanced automated restoration.$Commercial implications: Enables commercial streaming services to offer better-sounding historical orchestral music, attracting more subscribers and differentiating catalogs.
Authors
Steven Cho, Junghyun Koo, Raphael Lafargue, Tushar Dhyani, Eloi Moliner, Yuki Mitsufuji
Abstract
Historical music restoration (HMR) has almost exclusively focused on constrained problems such as Super-Resolution or the restoration of solo pieces, under-exploring the general task of restoring orchestral historical music, which has multiple instruments. This under-exploration is largely because the HMR domain, early-20th-century recordings, has no pre-degradation ground-truth pairs, making the restoration task unsupervised and more challenging. This paper presents a supervised end-to-end orchestral HMR benchmark by exploring both the synthetic degradation functions and the end-to-end generative deep-learning restoration methods. We simulate the historical recording degradation chain more faithfully than prior work, which makes orchestral restoration into a tractable supervised problem. A latent flow-matching model trained on the resulting synthetic pairs outperforms existing HMR baselines on intrusive, non-intrusive, and subjective evaluations. We also curate and release a 9.3-hour license-free, unpaired, historical classical-music test set, along with code and audio demos.