Quadrotors learn to fly fast through narrow gaps using visual data
Learning Vision-Based Agile Gap Traversal: Differentiable Simulation with a Warm-Started Critic
Robotics
Summary
Flying a small drone through tight spaces is hard because the drone must see and react quickly using camera images. The authors created a two-step training approach that first teaches a drone to navigate using detailed information, then trains a visual-only policy using simpler camera images. This method makes learning faster and more reliable, even when drone details change or the gaps are new shapes. They tested their approach in real life with drones successfully flying through narrow gaps using just simple visual input.
What this means in practice
- •For drone developers: Develop visual control software enabling drones to autonomously fly through narrow gaps using only onboard cameras.
- •For robotics engineers: Train agile robot navigation policies in simulation that transfer to real-world conditions efficiently with minimal retraining.
Authors
Nuthasith Gerdpratoom, Tianchen Sun, Yichao Gao, Lin Zhao
Abstract
Traversing narrow gaps is challenging for autonomous quadrotors, especially when control commands come directly from high-dimensional visual observations. Existing end-to-end methods often rely on behavior cloning or full-rollout backpropagation through time (BPTT) via differentiable simulation, which can limit policy performance or incur high training costs. We propose a two-stage reinforcement learning framework for more efficient ego-centric visuomotor gap-traversal policy training, leveraging quasi-analytical policy gradients (QPG) via differentiable simulation and critic warm-starting. The framework utilizes QPG to avoid backpropagation through visual rendering, reducing computation and memory costs while improving sample efficiency. In the first stage, an expert actor and critic are trained using privileged observations, including gap geometry. Unlike prior gap-traversal approaches, our training utilizing QPG does not require resetting the agent along optimized reference trajectories. In the second stage, a visual policy is trained using binary gap masks from two ego-centric cameras and low-dimensional observations, while its privileged critic is warm-started from the first stage. This substantially improves training efficiency and traversal success compared with cold-starting the critic or using full-rollout BPTT. Our framework does not require retraining the expert actor when system parameters change, enabling more efficient generalization across drone platforms than state-of-the-art visual gap-traversal methods based on action supervision. The learned visual policy also generalizes to gaps with unseen shapes. Extensive real-world experiments further demonstrate robust gap traversal using binary masks rendered online. Beyond gap traversal, the proposed framework is generic and can be extended to other visuomotor robot learning tasks.