Efficient deep gaussian process method speeds up predictions for large datasets
Efficient Deep Gaussian Process Surrogates via Mini-Batch Latent MAP Estimation
Machine Learning
Summary
Predicting outcomes in complex experiments can be slow and tricky when using deep Gaussian processes, a type of computer model. The authors propose a new way to quickly estimate the hidden parts of these models using a technique called maximum-a-posteriori (MAP) estimation. Their approach called LaMP-DGP uses small data batches to speed up computation without losing accuracy. The method performs well compared to current best techniques and runs faster, making it useful for big data tasks. The authors also provide an easy-to-use software package in R.
What this means in practice
- •For machine learning engineers: Build faster and scalable surrogate models for nonstationary complex responses using LaMP-DGP on large datasets.
- •For environmental modelers: Improve prediction speed and accuracy for complex environmental simulations with nonstationary data by using the LaMP-DGP method.
Authors
Chun-Yi Chang, Chih-Li Sung
Abstract
Deep Gaussian processes (DGPs) are flexible surrogate models for computer experiments with nonstationary response surfaces, but inference over their latent representations remains computationally challenging. Existing methods typically rely on variational inference or Markov chain Monte Carlo (MCMC), which impose variational-family assumptions or require costly posterior sampling. We propose the latent maximum-a-posteriori deep Gaussian process (LaMP-DGP), which uses gradient-based optimization to obtain MAP estimates of the latent representations and enables efficient probabilistic prediction through linked-GP calculations. To scale LaMP-DGP to large datasets, we develop a mini-batch optimization algorithm and, under a polynomial mini-batch bias-decay condition, establish a stationarity bound for latent-state optimization that motivates a growing batch-size schedule. Across numerical experiments and real-data analyses, LaMP-DGP provides consistently competitive predictive performance relative to doubly stochastic variational inference, stochastic gradient Hamiltonian Monte Carlo, and fully Bayesian Vecchia DGP inference. Computationally, LaMP-DGP is substantially faster than the Vecchia MCMC approach while remaining broadly competitive with the stochastic-gradient alternatives. These results position LaMP-DGP as a competitive and computationally efficient alternative for DGP inference. An R package, "LaMPDGP", is available for implementing the proposed method.