Multi-objective optimization improves with adaptive nonstationary gaussian processes
Adapting Nonstationary Multi-output Gaussian Processes to Bayesian Optimization
Machine LearningArtificial Intelligence
Summary
Multi-objective Bayesian optimization helps find the best trade-offs between conflicting goals but usually uses models that assume stable patterns, limiting their accuracy. The authors studied a flexible model that can adapt to changing patterns but found it sometimes makes overconfident mistakes. They developed a new method that corrects these mistakes and combines shared learning across objectives with individual adjustments, leading to better and more reliable optimization results. Testing on various example problems showed their approach outperforms others in both accuracy and robustness.
What this means in practice
- •For machine learning engineers: Improve tuning multiple competing model parameters together using adaptive nonstationary Gaussian process surrogates to better handle complex and varying data relationships.
- •For aerospace design teams: Optimize multiple aircraft performance objectives simultaneously with more reliable models that adapt to changing aerodynamic factors during design.
Authors
Zikai Xie
Abstract
Multi-objective Bayesian optimization (MOBO) commonly relies on independent Gaussian processes (GPs) with stationary kernels, limiting its ability to represent nonstationary structure and share information between objectives. However, expressive nonstationary GPs do not necessarily make reliable BO decisions. We study this mismatch for the multi-output low-rank nonstationary (MO-LRN) GP: strong training fit can coexist with large off-design errors and optimistic acquisition predictions. We introduce MOLRN-BO, which combines a regularized shared-spectral surrogate with objective-specific residuals, prequential mean correction and tempered covariance scaling, and Pareto-local qLogEHVI optimization with periodic global search. Experiments on 12 deterministic bi-objective benchmarks show that MOLRN-BO substantially improves upon the original MO-LRN and achieves the best average problem ranks for final normalized hypervolume and normalized inverted generational distance among nine evaluated algorithms. It also achieves the strongest adverse-tail performance while remaining competitive with the leading baselines in anytime optimization. Ablation studies further show that the shared spectral construction improves off-design prediction, the local--global decision policy improves optimization performance, and hierarchical calibration reduces systematic candidate bias. These results demonstrate that nonstationary multi-output surrogates can deliver strong and robust MOBO performance when their structure and use are explicitly adapted to the demands of sequential optimization.