Safety layer improves quadrotor control and speeds training
CALOS: Control-Affine Lyapunov On-manifold Safety Layer for Safe Deep Reinforcement Learning for Quadrotors
RoboticsArtificial Intelligence
Summary
Controlling quadrotor drones with AI can be tricky because the AI might try risky movements that break safety rules. The authors created CALOS, a safety system that automatically adjusts the commands to keep the drone’s tilt angles safe. This system works alongside existing AI control methods without changing them and runs fast enough to work during training and flying. Using CALOS makes the drones follow their paths more accurately, avoid unsafe angles completely, and learn quicker without losing quality.
What this means in practice
- •For drone developers: Integrate a runtime safety layer that ensures quadrotors maintain safe tilt angles while improving control accuracy and speeding up AI training.
- •For robotics simulation engineers: Use an efficient quadratic programming approach to enforce safety constraints in large-scale parallel quadrotor simulations during reinforcement learning.
Authors
Fabrizio Cesareo, Sebastiano Mengozzi, Nicola Mimmo, Andrea Acquaviva
Abstract
Deep Reinforcement Learning has demonstrated remarkable capability in quadrotor control, yet learned policies offer no guarantee of respecting safety constraints during training or deployment. We present CALOS (Control-Affine Lyapunov On-manifold Safety), a runtime safety layer that enforces attitude constraints on a quadrotor without modifying the underlying learning algorithm. CALOS formulates four tilt-angle inequalities and a Lyapunov descent condition as a single quadratic program whose solution is the minimum-norm correction to the nominal torque output of the policy. The quadratic program is solved exactly via active-set enumeration over the three-dimensional torque space, with a computational cost low enough to enforce constraints in real time across thousands of parallel simulation environments, as required by modern massively parallel Deep Reinforcement Learning training. Evaluated on trajectory-tracking tasks in NVIDIA Isaac Lab, CALOS reduces lateral tracking error by 55-60% relative to an unconstrained Proximal Policy Optimization baseline while achieving zero attitude-constraint violations on the training trajectory. By restricting exploration to safe regions of the state space, the safety layer also accelerates training convergence and improves data efficiency without producing suboptimal policies.