Active-learning kriging optimizers may underestimate failure probabilities
Inner optimizers in active-learning Kriging reliability analysis: a pre-specified comparison and the unconservative estimates left by unlearned failure modes
Computational Engineering, Finance, and ScienceMachine Learning
Summary
This paper looks at how different methods choose sample points when using active-learning Kriging, a way to predict system failures more efficiently. The authors found that some inner optimization methods can miss important failure scenarios, leading to underestimates of how likely failures are. Specifically, certain optimizers sometimes focus too much on one area and fail to explore others, causing inaccurate predictions. The study suggests using default pool selection methods and checking coverage to avoid these problems.
What this means in practice
- •For reliability engineers: Improve failure probability estimates by using default pool selection and coverage checks when conducting active-learning Kriging.
- •For simulation analysts: Avoid underestimating risks in systems modeling by choosing sample points to better detect rare failure modes.
Authors
Wei-Chang Yeh
Abstract
Active-learning Kriging often replaces candidate pools by continuous metaheuristic search. A pre-specified comparison sharing surrogate, population and stopping rule isolated this inner optimizer: five budget-matched continuous optimizers, including hybrid swarm optimization with L-BFGS-B polishing, against candidate pools on eleven problems (dimension 2 to 100). The primary hypothesis was not supported: against optimizers sharing its criterion the hybrid won 6, tied 41 and lost 19 of 66 Holm-corrected accuracy comparisons, and against pool methods 0, 23 and 21 of 44. With sequential selection on three series systems, multistart L-BFGS-B left a failure mode unlearned in 59 of 60 runs, more often than swarm, genetic-algorithm or AK-MCS. In all 198 runs (22 post-freeze) with an unlearned mode, the learned modes did not compensate and failure probability was underestimated (exploratory). The error criterion can accept substantial underestimation: it accepted 17 runs (13 pool-selected) at 0.38 to 0.89 of the reference; 181 exhausted the budget. Follow-up analyses on single-design-point problems indicate underestimation when selections concentrate near the design point. Kriging-believer pool selection satisfied the surrogate stopping criterion with fewer calls than the best continuous method in all four informative batch cells (two post-freeze). We recommend default pool selection and coverage checks for any selection.