Safe quadrotor navigation policies trained with explicit constraints
Constrained Differentiable Policy Learning for Safe Quadrotor Navigation
Robotics
Summary
Flying drones safely around obstacles is tricky because they need to learn how to avoid crashes without losing performance. This paper shows a way to train drone controllers with clear rules that keep them safe, rather than guessing by penalties. The researchers create special math tools to help the drone learn to keep safe distances and focus on important safety problems during training. Their approach works well in tests and even on real drones without extra tuning.
What this means in practice
- •For drone software engineers: Train quadrotor control policies that guarantee explicit safety margins during obstacle avoidance using differentiable simulation.
- •For autonomous vehicle developers: Incorporate explicit safety constraints in real-time learning algorithms for safer navigation in complex, cluttered environments.
Authors
Yan Rui Tan, Praveen Krishnapur, Zeyu Feng, Han Bao, Yew-Soon Ong, Fan Shi
Abstract
Differentiable simulation enables efficient policy optimization through analytic gradients, yet safety is commonly incorporated as a weighted penalty, requiring manual tuning of the trade-off between task performance and constraint satisfaction. We instead formulate obstacle avoidance as an explicit constraint and train quadrotor policies using primal--dual gradient descent--ascent through a differentiable simulator. We introduce two key components to enable effective constrained training. First, we propose a bounded clearance-based constraint function that provides informative gradients around the safety boundary while preventing large violations from destabilizing primal--dual optimization. Second, we introduce log-mean-exp (LME) aggregation to smoothly emphasize critical per-step safety violations across a trajectory and training batch within a single constraint governed by one dual variable. Together, these components enable the desired safety level to be specified directly while balancing constraint satisfaction and task completion. We theoretically prove that the proposed optimization with differentiable simulation converges to feasible policy under mild assumptions. Experiments demonstrate improved safety--performance trade-offs over fixed-penalty and existing policy-learning baselines, as well as generalization to denser out-of-distribution environments. Finally, the learned policy transfers zero-shot to Gazebo/PX4 and a physical quadrotor without additional fine-tuning.