Humanoid robot safety improved by separate learning of task and safety policies
ResSafe: Learning Safety Filtering with Residual Reinforcement Learning for Humanoids
Robotics
Summary
Controlling humanoid robots safely is hard because they have many moving parts and can easily fall. The authors propose splitting the robot’s control into two parts: one focuses on doing the task well, and the other fixes unsafe moves. This way, the robot can perform better while staying safe without needing complicated tuning. The second part acts like a safety filter that helps prevent falls or unstable actions.
What this means in practice
- •For robotics control engineers: Design control systems that maintain humanoid robot safety by separately learning task performance and safety corrections.
- •For industrial automation teams: Implement robust and safe humanoid robot motions in factories by applying separate safety-filtering policies to reduce falls and errors.
Authors
Gechen Qu, Tong Zhang, Bike Zhang, Yen-Jen Wang, Koushil Sreenath, Claire Tomlin, Jason Jangho Choi
Abstract
Safe control of humanoid robots remains challenging due to their high-dimensional dynamics, contact-rich interactions, and sensitivity to disturbances. Although reinforcement learning has enabled effective locomotion and motion tracking, learned policies can still generate unsafe actions that lead to instability or falls. In this work, we propose residual reinforcement learning as an implicit safety-filtering mechanism for safe humanoid control. Instead of relying on a single nominal policy to simultaneously balance performance, safety, and robustness, we decouple performance and safety. The nominal policy focuses solely on task performance, while a residual policy learns safety corrections. This decoupling leads to a better performance--safety Pareto trade-off and avoids the need for careful tuning of multiple competing reward terms within a single policy training. We show that the residual policy can act as an implicit safety filter.