Bayesian optimization uses wait time to improve decision speed and quality
Work While They Sleep: Exploiting Evaluation Latency for Fully Bayesian Optimization
Machine LearningArtificial Intelligence
Summary
Black-box optimization is about finding the best solution when trying different options is slow and costly. Standard methods guess the next best option quickly but ignore uncertainty in their model settings. The authors created ELF-BO, which uses the waiting time during slow tests to better estimate model uncertainty without making decisions slower. This approach matches the quality of more thorough methods while keeping the decision time short. It makes fully Bayesian methods practical for real-world problems where evaluations take a long time.
What this means in practice
- •For machine learning engineers: Optimize hyperparameters of costly models by efficiently using evaluation wait time for better parameter uncertainty estimates.
- •For robotics system designers: Improve tuning of expensive robot control parameters by running Bayesian sampling during execution delays to speed up calibration.
Authors
Gustavo Sutter, Alejandro Comas-Leon, David Holzmüller, Hao Wang, Luis Ricardez-Sandoval, Pascal Poupart, Agustinus Kristiadi
Abstract
Black-box optimization problems are ubiquitous across science and engineering, often dealing with expensive objective functions. This objective latency has two consequences during optimization: (i) the objective evaluation dominates execution time, and (ii) sample-efficient algorithms are crucial to accelerate development and avoid wasting resources. Bayesian optimization (BO) methods are the \textit{de facto} choice of planners for suggesting the next point to try. Standard BO fits the surrogate model's hyperparameters with a point estimate. Alternatively, a fully Bayesian approach uses model averaging to account for uncertainty over the hyperparameters, leading to better uncertainty estimates---useful in the low-data regime that is pervasive in BO. However, it is often prohibitively expensive and thus rarely used. In this work, we propose ELF-BO, an algorithm that uses the objective evaluation latency to headstart the computation of the next suggestion, allowing for fully Bayesian optimization without incurring substantial decision-time costs. This is done by sampling from the hyperparameter posterior \emph{while} the objective is being evaluated, only requiring reweighting of the samples once the objective value is observed. Across synthetic functions and real-world applications, we show that ELF-BO matches the performance of fully Bayesian methods while only incurring decision latency on par with or better than standard BO. Thus, ELF-BO makes fully Bayesian optimization practical in real-world use cases.