Bridging Reinforcement Learning and Optimal Control via Feasible Action Mapping
2026-07-27 • Robotics
Robotics
AI summaryⓘ
The authors developed a new way to control systems that have strict safety and feasibility rules by combining two methods: Reinforcement Learning (RL) and Optimal Control (OC). Their method, called FAOC, ensures that the actions chosen by the learning algorithm always follow the system’s rules by converting these actions into safe commands based on the current state. This approach avoids the need for expert-designed action spaces and guarantees safety while still allowing the system to learn effectively. They tested FAOC on a robot playing table tennis and found it learns faster and performs better than other methods in simulations.
Reinforcement LearningOptimal ControlDynamical SystemsRecursive FeasibilitySafety ConstraintsOptimal Control ProblemAction SpaceRobot Motion PlanningSample EfficiencyClosed-Loop Performance
Authors
Stefan Richter, Alberto Giammarino, Guillem Torrente, Sam Blakeman, Peter Dürr
Abstract
Operating constrained dynamical systems requires controllers to efficiently solve complex tasks while enforcing recursive feasibility and safety constraints. To address these competing requirements, we present Feasible Action for Optimal Control (FAOC), a novel control framework integrating Reinforcement Learning (RL) and Optimal Control (OC). The key contribution is a computationally efficient, optimization-based mapping algorithm that transforms the RL agent's action from a static abstract set into a state-dependent feasible parameter set of the Optimal Control Problem (OCP), guaranteeing strict satisfaction of the dynamical system's constraints. Thus, FAOC effectively combines the predictable safety of OC with the flexibility of RL. In contrast to prior work, the abstract action space of the RL agent does not require expert or heuristic design, and the OCP formulation is not compromised by the inability of RL to guarantee feasibility. We apply our approach to real-time motion planning for robot table tennis, which encapsulates these challenges. Via simulated experiments, we show that FAOC outperforms state-of-the-art baselines in both sample efficiency and closed-loop performance.