Reinforcement learning guides multi-objective design optimizations effectively

MOON-RL: Multi-Objective Optimization via Objective Space Navigation with Reinforcement Learning

Machine Learning

Summary

Many engineering problems involve deciding between several competing goals while making sure the design stays feasible. The paper presents MOON-RL, a new method that uses reinforcement learning to navigate through possible solutions by focusing search efforts based on user preferences. This approach adapts to each specific problem and learns how to improve designs toward desired trade-offs between objectives. The authors tested MOON-RL on real engineering tasks and found it competes well with traditional methods, especially when focusing on certain parts of the design space.

What this means in practice

  • For mechanical design engineers: Optimize actuator designs by directing search toward specific performance trade-offs with an adaptive reinforcement learning framework.
  • For industrial optimization teams: Integrate MOON-RL to focus computational resources on preferred regions of complex multi-objective constrained problems, improving search efficiency.

Authors

Andrii Artomov, Soheyl Massoudi, Jürg Schiffmann

Abstract

Many engineering design problems involve multiple conflicting objectives and strict feasibility constraints. They are typically solved with population-based optimizers that require many evaluations and offer limited online control over where the search is focused. We present MOON-RL, a reinforcement-learning-based optimization framework that formulates constrained multi-objective search as preference-directed navigation in the objective space. In MOON-RL, a lightweight PPO policy receives objective-space and constraint-related observations and acts as an instance-specific adaptive search process. It learns how to update the design so that the current solution advances along a user-specified trade-off direction, while a simple meta-algorithm periodically resets the policy, resamples starting designs from a continuously updated archive of non-dominated solutions, and adjusts step-size and penalty schedules. We evaluate MOON-RL on real-world actuator-design problems and additional convergence-hard constrained benchmarks, comparing against established evolutionary algorithms under matched evaluation budgets. In focused search scenarios, MOON-RL achieves competitive performance in the region of interest, and ablation studies show that both the RL-driven adaptive sampling process and the meta-algorithm contribute to this behaviour, suggesting that repurposing RL training as objective-space navigation is a viable building block for interactive, preference-guided design tools. Additional experiments show that the framework also accommodates adaptive non-RL samplers, while PPO remains a strong choice for fixed-target, preference-directed objective-space navigation.