Deep reinforcement learning improves orchestration of evolutionary search algorithms
Learning to Orchestrate Evolutionary Search: Progression-Aware Deep Reinforcement Learning for Dynamic DE-CMA-ES Coordination in Optimization and Structural Model Updating
Neural and Evolutionary ComputingArtificial Intelligence
Summary
Solving complex problems where many variables interact is hard, especially when you need to explore many possible solutions efficiently. The authors created a smart system that learns to manage two powerful search methods, switching between exploring new possibilities and focusing on promising ones. This system uses a type of AI called deep reinforcement learning to decide how to balance these approaches dynamically as it searches. Their approach works better than existing methods on challenging optimization tasks including updating structural models used in engineering.
What this means in practice
- •For structural engineers: Optimize and update complex engineering models accurately by autonomously balancing different optimization techniques to better detect structural issues.
- •For machine learning engineers: Develop adaptive hybrid evolutionary algorithms that dynamically allocate resources between exploration and exploitation to improve high-dimensional optimization tasks.
Authors
Lechen Li, Rongye Shi, Wanhuan Zhou
Abstract
Solving high-dimensional structural model updating problems requires an algorithm capable of navigating complex, non-convex landscapes with correlated parameters. Existing hybrid evolutionary algorithms typically rely on static architectures or fixed switching rules, resulting in disjointed search phases. To address this, this study proposes a Deep Reinforcement Learning-governed dynamic DE-CMAES Orchestration (DRL-DCO) algorithm, in which a Deep Deterministic Policy Gradient (DDPG)-based actor-critic agent continuously governs the evolutionary process as a single, unified system rather than a mechanical concatenation of algorithms. Guided by a progression-aware state representation and a diversity-informed reward, the agent fluidly reallocates computational resources between the differencevector-based exploration of Differential Evolution (DE) and the covariance-guided exploitation of CMA-ES, while jointly regulating population size, elite preservation, and a restart mechanism to escape local optima. This allows DRL-DCO to autonomously transition between exploration-dominant, exploitation-dominant, and mixed-strategy regimes across generations. Beyond the training phase, the trained actor can operate in a supervision-free inference mode, where the internalized policy autonomously orchestrates DE and CMA-ES control from observed search states through forward inference alone, without critic evaluation or weight updates, enabling faster deployment while retaining full effectiveness. Validated on high-dimensional single-objective optimization benchmarks and the IASC-ASCE structural health monitoring benchmark, DRL-DCO achieves superior convergence accuracy and robustness compared to state-of-the-art adaptive and hybrid evolutionary algorithms, as well as single-operator DRL-governed baselines.