Accelerating Evolutionary Strategy via Rao-Blackwellizing Realization of Uncertain Input
2026-08-03 • Machine Learning
Machine Learning
AI summaryⓘ
The authors study how to optimize problems where the input data is uncertain, rather than the function itself. They notice that previous methods ignore observed input information that could help improve optimization. By using a statistical technique called Rao-Blackwellization, they show the input data can reduce estimation noise in an Evolutionary Strategy approach. They develop a new method called Phenotype-Accelerated Evolutionary Strategy (PAES) that uses this idea and demonstrate it finds solutions faster on various test problems, including reinforcement learning tasks.
Optimization under Input UncertaintyEvolutionary StrategyRao-BlackwellizationGradient EstimatorPhenotype-Accelerated Evolutionary StrategyReinforcement LearningVariance ReductionMixture of ExpertsActuation NoiseContinuous Optimization
Authors
So Nakashima, Tetsuya J. Kobayashi
Abstract
We investigate Optimization under Input Uncertainty (OIU), in which the input to the objective function, rather than the objective function itself, is subject to uncertainty. OIU appears in manufacturing processes with production tolerance, control of physical systems with actuation noise, Mixture of Experts, and Reinforcement Learning (RL). Most of the existing approaches solve OIU by using the value of the objective function but discard the information of the realized input, even though the realized input is observable in various applications. The question here is whether the discarded information of the realized input is useful to accelerate the optimization process. We affirmatively answer this question for Evolutionary Strategy (ES) by theoretically showing that the information of the realized input can reduce the variance of the gradient estimator via Rao-Blackwellization. Using the Rao-Blackwellized gradient estimator, we propose Phenotype-Accelerated Evolutionary Strategy (PAES), which is a refinement of ES for OIU. Numerical experiments show that PAES converges faster than the usual ES from simple continuous optimization problems to RL benchmarks.