Learning recovers optimal diagonal regularizers for noisy inverse problems
Why Learning Rediscovers the Closed-Form Diagonal Regularizer
Machine LearningRobotics
Summary
When solving problems where data is noisy and incomplete, a common technique adjusts solutions using simple diagonal weighting. The authors found that the best way to do this weighting follows a specific pattern set only by prior knowledge, not details of the problem. This pattern explains why different learned diagonal adjustments perform almost the same. However, methods that consider interactions between different parts start to improve beyond this limit. Testing was done on simulated sound problems and heat diffusion models.
What this means in practice
- •For acoustic engineers: Improve sound reconstruction in rooms by using known optimal diagonal regularizers for noisy measurements.
- •For computational physicists: Apply optimal diagonal regularization patterns to model heat diffusion with known Green's function corrections.
Tested on simulated data.
Authors
Jeahn Han, Pyojin Kim
Abstract
We identify a diagonal saturation principle in modal inverse problems: when truncation noise is isotropic, the Bayes-optimal Tikhonov shape is a closed-form power law Gamma_k proportional to lambda_k^|s| set by the prior alone, independent of the domain. Berry's random-wave conjecture decorrelates the truncation noise across modes, and Weyl's eigenvalue counting law supplies enough modes for the conclusion to survive empirical Berry violations. Together they predict an approximately flat loss landscape across the per-mode family, leaving narrow scope for a diagonal regularizer to robustly beat the closed form. On FEM-simulated acoustic rooms, the closed form is near-optimal relative to per-room oracle tuning across observation windows, and three diagonal architectures trained on the same data match its reconstruction error within 1 pp despite learning qualitatively different spectra. The framework extends to heat diffusion via a known exponential Green's function correction with no new free parameters. Saturation is restricted to the diagonal family: Learned Iterative Ridge crosses the boundary by exploiting cross-mode coupling, locating where learning starts to help.