Exploration strategies improve final results in noisy optimization tasks
Exploration strategies for Gaussian process bandit optimization with inference regret guarantees
Machine Learning
Summary
Sometimes, when computers try to find the best solution to a problem with many options, they don’t do a great job picking the very best answer at the end, especially when their tests are noisy. The authors found ways to help by adding special steps that focus on exploring less-checked areas instead of only grabbing the best guess each time. This method helps ensure the final choice is closer to the true best solution. They tested this idea and found it improved performance in different tasks like tuning settings and controlling systems.
What this means in practice
- •For machine learning engineers: Improve hyperparameter tuning by integrating exploration steps to find better final model settings under noisy conditions.
- •For control system developers: Enhance controller optimization outcomes by scheduling deliberate exploration queries to discover higher quality configurations.
Authors
Hwanwoo Kim, Jungwoo Hong, Kyungjae Lee
Abstract
Existing cumulative regret guarantees for Bayesian optimization (BO) do not necessarily characterize the quality of the algorithm's final output. This gap is critical under noisy evaluations, where the goal is to return a single high-quality recommendation. To address it, we introduce efficient exploration strategies that can be incorporated into any BO procedure by occasionally replacing the acquisition query with an exploration query. To validate this approach, we bound the Gaussian process posterior standard deviation over the search space via the fill distance of the queries, which measures the largest region left unexplored. We then use this result to derive upper bounds on the inference regret, i.e., the gap between the global optimum and the function value at the algorithm's returned point. Empirically, these exploration strategies improve the final recommendations of existing BO algorithms on synthetic benchmarks, hyperparameter optimization, and a control task.