Event camera based obstacle avoidance improves quadrotor flight safety
Learning Event-based Obstacle Avoidance via Differentiable Simulation
Robotics
Summary
Flying drones safely around obstacles is hard because regular cameras blur or miss details when moving fast or in low light. The authors developed a way to train drones using event cameras, which detect tiny changes in a scene very quickly without motion blur. They created a fast simulator to train drone controls directly from these event images without needing extra steps like estimating depth. Their method works well even in dark places or without special positioning data, helping drones avoid obstacles smoothly.
What this means in practice
- •For drone developers: Train drones to avoid obstacles in challenging environments using event camera inputs without requiring depth estimation or prior detailed training.
- •For robotics engineers: Implement real-time obstacle avoidance control for robots operating in low light or fast-motion scenarios using an event camera based policy.
Authors
Chenlu Shou, Yunfan Ren, Jiaxu Xing, Davide Scaramuzza
Abstract
Reliable obstacle avoidance is essential for safe quadrotor flight in unknown environments, yet conventional frame-based cameras remain challenged by motion blur, limited dynamic range, and latency under fast motion. Event cameras, with sub-millisecond latency, high dynamic range, and negligible motion blur, offer a promising alternative, but leveraging them for closed-loop control remains difficult. Existing approaches either adopt cascaded architectures that first estimate depth and then map it to control commands, introducing error propagation, or achieve end-to-end control through costly offline pretraining before online fine-tuning can even begin. We address these challenges with a CUDA-based event simulator that exploits GPU parallelism to render events efficiently within the training loop, integrated into a differentiable simulation pipeline that enables direct gradient backpropagation through the differentiable physics model, improving sample efficiency. This allows us to train an end-to-end policy that maps event frames directly to control commands, without intermediate depth estimation. We validate our approach through zero-shot real-world deployment on a quadrotor, achieving robust obstacle avoidance in both odometry-assisted and odometry-free settings, including under low-light conditions where conventional cameras typically fail.