Scalable method improves high dimensional Gaussian dependency estimation

SCORE: Spectral Correlation Estimation for Multivariate Gaussians

Machine Learning

Summary

Modeling complex relationships in data that follow a Gaussian pattern requires estimating how variables relate in a stable and efficient way. The authors propose SCORE, a method that breaks down the problem to learn simple parts of the data and its correlations separately, using a spectral approach that is fast and uses less memory. Their method can be trained reliably even when some data aspects degenerate and ensures consistent results across certain transformations. They tested it on tasks like predicting weather, estimating depth from images, and forecasting time series, finding better accuracy while saving computation.

What this means in practice

  • For time series forecasters: Improve forecasting models by capturing complex variable dependencies efficiently in Gaussian-based time series data.
  • For weather prediction teams: Enhance spatial weather prediction accuracy using fast and stable covariance estimation for Gaussian structured outputs.

Authors

Christopher Bülte, Emil Partow, Astha Gupta, Pascal Esser, Gitta Kutyniok

Abstract

Neural network-based predictive modeling with high-dimensional structured Gaussian targets requires an efficient and numerically stable, yet expressive approximation of the covariance matrix. We propose SCORE: a scalable framework, combining scoring rule training with an expressive covariance approximation learned in spectral space. For $d$-dimensional data, the learning task is decomposed into learning the marginal distributions and learning a structured correlation matrix, which enables dense dependencies with linear storage and $\mathcal{O}(d\log d)$ cost. We utilize the closed form Gaussian kernel score for training, which remains defined even for degenerate covariances and admits bounded gradients during optimization. We characterize kernel scores under invertible transforms and prove exact invariance under unitary transforms. At population level, our two-level objective recovers the true marginals and projects the target correlation onto the representable class; finite-sample PAC bounds show that the errors of the two stages enter additively. We evaluate our model on a variety of tasks with a commonly assumed Gaussian domain: Time-series forecasting, monocular depth estimation, and spatial weather prediction, showing improved performance at lower computational cost.