Scalable gaussian process models capture symmetry with uncertainty
Triply-Scalable Equivariant Gaussian Process Modeling
Machine Learning
Summary
Gaussian processes (GPs) are tools that help make predictions while showing how sure those predictions are, but they usually struggle with big, complex datasets. The authors found a way to make GPs handle large problems more efficiently by focusing on symmetrical patterns (equivariance) in the data. They created new GP models that work well with these symmetries and can scale up quickly without losing the ability to show uncertainty. Their methods were tested on both simulated fields and molecular data, where they accurately predicted properties like electric dipole moments with much less computing effort than traditional methods.
What this means in practice
- •For computational chemists: Predict molecular properties with uncertainty-aware, scalable Gaussian processes that respect molecular symmetries, improving efficiency over classical methods.
- •For machine learning engineers: Use scalable equivariant Gaussian process models to build uncertainty-aware predictors for large datasets with symmetrical structures like vector fields.
Authors
Tim Steinert, David Ginsbourger
Abstract
Gaussian processes (GPs) provide principled probabilistic predictions while encoding prior knowledge, including equivariances. Yet, their use in large-scale scientific problems is limited by computational cost. Equivariant neural networks are common but typically lack the uncertainty quantification offered by GPs, which is valuable in applications such as molecular research. High-dimensional inputs and large symmetry groups further demand scalability. We establish results pertaining to the interplay of GP equivariance and conditioning and leverage them to obtain equivariant sparse GPs through suitable mean functions and covariance kernels. We instantiate this framework with a flexible class of integration-free equivariant kernels, yielding scalable and data-efficient GP inference. In particular, we introduce triply scalable equivariant Gaussian processes. We employ equivariant sparse variational Gaussian processes for $\mathrm{SO}(2)$-equivariant vector fields and molecular property prediction. Alongside the SVGP, we develop a matrix-free equivariant full-GP implementation that combines an exact Kronecker reduction with preconditioned conjugate-gradient solves, enabling fast and scalable evaluation of the full joint predictive density. We further compare different approaches for selecting inducing points in the equivariant sparse GP models. Our test cases include synthetic $\mathrm{SO}(2)$-equivariant fields as well as the prediction of electric dipole moments of N-methylformamide based on quantum chemistry simulations, achieving accurate, uncertainty-aware predictions at a fraction of the computational cost of classical GP inference.