Weighted kernel method improves multivariable function approximation accuracy
A Weighted Kernel Method for Approximation that Adapts to Learned Multivariable Structure
Machine Learning
Summary
Approximating complex functions that depend on multiple inputs is hard when you don’t know which inputs matter most or how they interact. The authors introduce a new method called total sensitivity kernels (TSKs) that learn how important each input and their combinations are while approximating the function. This method adjusts its focus based on these learned importances, leading to better accuracy. They also show mathematically that their method is sound and demonstrate it works well in experiments.
What this means in practice
- •For machine learning practitioners: Create more accurate predictive models for functions with many interacting inputs when input importance is unknown.
- •For engineering simulation teams: Improve surrogate models for complex simulations by adapting kernels to identify key input factors and interactions.
Authors
John E. Darges, Laura Weidensager
Abstract
Approximating the input-output behavior of a multivariable black-box function from limited data is challenging when blind to the importance of its inputs and their interactions. We introduce total sensitivity kernels (TSKs), a method based on families of weighted ANOVA kernels that learn and adapt to this multivariable structure. TSKs parameterize the weights on each multivariable component of the target function by factors for each input. We propose learning these factors directly from function evaluations by selecting the reproducing kernel Hilbert space (RKHS) in which the target function has minimum norm. Under suitable conditions, we show that this norm-minimization problem admits a unique solution, and we establish consistency of a finite-data formulation based on minimum-norm interpolation. The learned TSK factors characterize the participation of individual inputs across interactions and main effects, providing a kernel-dependent notion of input sensitivity related to total Sobol indices. Numerical experiments demonstrate that adapting the kernel to learned multivariable structure can substantially improve approximation accuracy over a standard product kernel.