Efficient multi-level optimization technique handles complex constrained problems
MF-SCBO : Multi-fidelity Scalable Constrained Bayesian Optimization
Machine Learning
Summary
Some problems need lots of computer tests that are costly to run, especially when the problem is complex and has many limits or rules. The authors introduce a new method called MF-SCBO that uses data from many levels of test detail to find the best solution faster. This method works well even when the problem involves many variables and unknown restrictions. Their tests showed it usually finds good solutions faster than earlier methods that only use one test level or can't handle many variables well.
What this means in practice
- •For machine learning engineers: Optimize complex models more efficiently by using multi-level evaluations with constraints on performance or resources.
- •For control system developers: Improve tuning of control parameters under strict operational limits by leveraging diverse quality simulations without requiring nested sampling.
Authors
Lucas Palazzolo, Mickaël Binois, Laëtitia Giraldi
Abstract
Many real-world optimization problems rely on expensive simulations or experiments, making the efficient use of available data essential. Multi-fidelity optimization of high-dimensional black-box functions subject to black-box constraints is increasingly relevant as the cost of objective evaluations continues to rise in applications such as machine learning, engineering, and control. To our knowledge, no existing method simultaneously addresses high-dimensionality, black-box constraints, an arbitrary number of fidelity levels, and non-nested sampling. In this work, we extend the Scalable Constrained Bayesian Optimization method to the multi-fidelity setting, resulting in the MF-SCBO method. The proposed approach is evaluated on standard benchmark functions as well as challenging problems. The experimental results demonstrate that MF-SCBO generally achieves better convergence than both the single-fidelity SCBO and the other multi-fidelity method considered in this high-dimensional and constrained settings.