Gaussian process designs reveal what model assumptions can be tested

What Can a Gaussian Process Design Test

Machine Learning

Summary

Gaussian process (GP) models fit data either because their assumptions are correct or because the test points can’t expose errors. The paper shows how, before collecting any data, the choice of inputs determines what kinds of model mistakes can be detected. Some patterns in the data must hold true if the model is right, and violations of these patterns provide strong evidence against the model assumptions. The authors also show how to choose test points to increase the chance of spotting discrepancies.

What this means in practice

  • For data scientists: Select experimental inputs to maximize the ability to detect if a Gaussian process model’s assumptions are violated before collecting data.
  • For statistical modelers: Use geometric insights from kernel matrices to design tests that can precisely detect structural inconsistencies in Gaussian process predictions.

Authors

Ivan De Boi, Marnix Van Soom

Abstract

A Gaussian process (GP) model can agree with the data for two reasons: its assumptions are right, or the chosen inputs could never have shown that they are wrong. The distinction can be checked from the design before any responses are observed. Every model implies relations that its noiseless responses must satisfy at the chosen inputs, such as the middle value lies on the line through its two neighbours. For GPs built from finitely many features, these relations are exactly the null space of the kernel matrix. Gale duality gives them a geometric interpretation, in which each observation has a vector and the smallest groups of observations that can expose an error are the circuits. For other kernels the relations become soft: response patterns may be improbable under the prior rather than algebraically impossible. A standard test then combines two kinds of evidence. Structural evidence comes from a violated relation and grows without limit as the noise falls. Prior-based evidence only says that a departure is improbable under the prior. With all inputs at the two ends of an interval, for example, a GP can reject a straight line against a large curvature, but only because the implied intercept is improbable, never because curvature was seen. In simulations the predicted power matched the observed rejection rates. Choosing the next input by predicted power raised the power against a localised discrepancy from 0.48 to 0.72, against 0.51 when choosing by predictive variance, and a grid in two dimensions contained exact tests of additivity that a Latin hypercube lacked. The test itself is classical. The contribution is the prospective reading of that test: before observing the responses, the design already determines what kind of contradiction it can produce.