Confidence-based Ranking with Adaptive Sampling for Noisy Black-Box Optimisation
Neural and Evolutionary Computing
Summary
The authors study optimization problems where the results are noisy and hard to predict, called noisy optimization problems. They focus on two ways to estimate how good a solution is: implicit averaging (using big groups with few repeats) and explicit averaging (repeating measurements to get better accuracy). They point out that implicit averaging may not work well when noise varies across the search space, which is common in real problems. To improve this, they create new test problems with varying noise and propose a new method that adapts sampling based on confidence, combining it with popular algorithms. Their approach performs better than current methods on various noisy problems.
Authors
Enrico Halim, Hemant Kumar Singh, Tapabrata Ray
Abstract
Real-world optimization problems often involve black-box functions and uncertainties in their evaluation, widely referred to as noisy optimization problems (NOPs). Evolutionary algorithms (EA), including Evolutionary Strategies (ES) and genetic algorithms (GA) have been commonly adopted to solve these problems in the contemporary literature. An ongoing challenge is the computational expense involved, given the number of evaluations required for good fitness estimation and ranking. Two fundamental methods commonly used for fitness estimation for NOPs are implicit averaging and explicit averaging. Explicit averaging uses resampling of solutions to improve the estimates, while implicit averaging typically uses a large population size with low resampling. Implicit averaging has been shown to have theoretical advantages for certain cases, which has motivated some recent approaches to use them. However, a recent study demonstrated that its performance is highly dependent on certain assumptions about the function, such as steepness and constant noise level, which may not apply for majority of the real world problems. Moreover, most existing algorithms have only considered homoscedastic noise, where the amplitude of variation is uniform across the entire search space, as opposed to more generic case of heteroscedastic noise. To address these issues, we introduce a set of heteroscedastic test problems and propose a novel confidence ranking method that employs a computationally efficient explicit averaging strategy with sampling budget adaptation. It is implemented within the Covariance Matrix Adaptation ES (CMA-ES) and GA frameworks to demonstrate its effectiveness and versatility. The resulting algorithm is evaluated on a range of problems with both homoscedastic and heteroscedastic noise, and it demonstrates superior performance compared to state-of-the-art approaches.