Music Restoration via Latent Operator Optimization and Diffusion Model Priors
2026-08-03 • Artificial Intelligence
Artificial Intelligence
AI summaryⓘ
The authors present LOUDAR, a new method to fix audio recordings that have been messed up in unknown ways. Unlike previous methods that need examples of the specific problems, LOUDAR works by learning the problem directly from the damaged audio using a special hidden space inside a pretrained audio model. It then cleans the audio step-by-step by guessing what the original sounded like and adjusting its guess based on a model trained on clear audio examples. The authors tested LOUDAR on removing effects from singing and guitar sounds and found it improves the audio and works well compared to other methods.
audio restorationlatent spaceautoencoderlatent operatordiffusion modelunsupervised learningsinging voice effect removalguitar distortioninferencedenoising
Authors
Michal Švento, Eloi Moliner, Valtteri Kallinen, Lauri Juvela, Vesa Välimäki, Pavel Rajmic
Abstract
Music restoration seeks to recover a clean signal from an observed recording degraded by an unknown effect, distortion, or corruption. Existing systems often rely on paired training data and distortion-specific supervision, which limits their use when the forward process is not known in advance. We propose LOUDAR (Latent-space Optimization of Unknown Distortion for Audio Restoration) a general-purpose restoration method that operates in the latent space of a pretrained audio autoencoder and models the unknown distortion as a learnable latent operator. At inference time, LOUDAR alternates between estimating the clean latent variable and updating the latent operator parameters. An unconditional latent diffusion model provides a prior over clean audio and regularizes this inference by steering the latent estimate toward the manifold of clean recordings. Because the degradation model is adapted per input, the approach is broadly applicable across diverse restoration problems. We evaluate LOUDAR on singing voice effect removal and restoration, as well as guitar distortion removal, and show that it consistently improves over degraded inputs and is competitive with supervised and unsupervised baselines in waveform and latent domains.