Fast event simulation speeds up robot navigation learning
Rendering-Free Event Frame Simulation for Fast Reinforcement Learning of Autonomous Robot Navigation
Robotics
Summary
Training tiny robots to navigate around obstacles using special event cameras is usually very slow because simulation needs to render full images first. The authors developed a fast method that directly simulates the event camera output from simple geometry and motion without rendering images. This approach makes simulation over 400 times faster and allows robots to learn how to avoid obstacles in under 30 minutes. Their trained navigation policies worked well in both simulated and real-world obstacle courses without extra tuning.
What this means in practice
- •For robotics engineers: Train autonomous drones to avoid obstacles quickly using fast event-based simulation without full image rendering.
- •For embedded systems developers: Run efficient, onboard event-based navigation policies on microcontrollers for lightweight flying robots with no external processing.
- •For industrial inspection teams: Deploy agile drones for cluttered environment inspection using rapidly trained event-camera navigation policies that generalize well in real conditions.$Commercial implications: This enables selling inspection drone products with better autonomy and faster setup using event camera-based navigation.
Authors
Dequan Ou, Austin Lopez, Quentin Missinne, Christophe De Wagter, Guido C. H. E. de Croon
Abstract
Event cameras offer low latency, high dynamic range, and milliwatt-level power consumption, making them ideal for agile robots such as tiny drones navigating through cluttered environments. However, existing simulators generally render photorealistic images before converting them into events, making large-scale reinforcement learning prohibitively slow. We propose an analytic event simulator that exploits what event cameras fundamentally measure - moving contrast edges - to synthesize event streams directly from scene geometry and ego-motion without visual rendering. Our simulator generates a 64x64 event observation in 17 microseconds on a single GPU - over 400x faster than a state-of-the-art rendering pipeline - enabling end-to-end reinforcement learning for goal-directed obstacle avoidance to converge in under 30 minutes - three orders of magnitude faster than training on rendered events. Policies transfer zero-shot to physically based event simulations (ray-traced Isaac Sim). Our policy reaches 83% success in cluttered courses unseen during training (mid-difficulty, traversability probability approx. 0.57), at 2.4 m/s average goal velocity, and retains 70% in a simulated texture-dense indoor hall. Furthermore, we quantized the policy to a 1.2 MB int8 network running fully onboard a 56-gram Crazyflie equipped with a Prophesee GenX320 event camera and an OpenMV N6 microcontroller board. Without any fine-tuning, external processing, or positioning, the policy achieves an average success rate of 82% (5/5, 5/6, 4/6 flights) across three real-world obstacle courses of increasing difficulty. Code: https://github.com/tudelft/rendering-free-event-nav