Monocular visual planning enables fast drone flights without depth sensors
Learning Geometry-Aware Planning Representations for Monocular Flight with Cross-Modal Supervision
Robotics
Summary
Flying small drones fast and safely is hard because they need to see obstacles clearly, usually with special sensors like depth cameras. This work helps drones use just a regular camera to plan safe paths by learning from a system that does have depth information. The method trains the drone to understand important 3D features from regular images, helping it avoid crashes even at high speeds. The system was tested in simulated and real environments, showing it can fly up to 6.5 meters per second without needing extra fine-tuning.
What this means in practice
- •For drone manufacturers: Build drones that navigate at high speeds using only standard cameras, reducing sensor costs and weight while avoiding obstacles reliably.$Commercial implications: Enables production of cost-effective, lightweight drones with safe autonomous navigation, appealing for delivery and inspection markets.
- •For autonomous robotics developers: Incorporate monocular image-based planning modules trained with simulated RGB-D data to improve collision avoidance in aerial robots without extra sensors.
Authors
Wencan Feng, Bailing Tian, Yulin Hui, Zhiyu Li, Junjie Lu
Abstract
Existing high-speed navigation systems for micro aerial vehicles (MAVs) commonly rely on depth cameras, stereo vision, or LiDAR to obtain explicit geometric information, while monocular cameras offer a lighter and lower-cost alternative. However, monocular flight remains challenging because RGB images offer no direct metric cues and visual appearance varies considerably across environments. We propose a geometry-aware monocular visual planning framework that predicts trajectories from two image frames, their relative pose, and the MAV state. Without explicitly reconstructing depth, the proposed method complements the downstream planning objective with feature-level geometric supervision by transferring the planning-relevant representation learned by a privileged depth expert to the RGB planner through cross-modal distillation. The framework first trains a depth-based geometric expert, then distills its planning features into a monocular branch using paired RGB-D observations, and finally adapts the planner to the downstream trajectory planning objective. This design encourages the policy to learn planning-relevant geometric features, enabling collision avoidance. We further use a 3D Gaussian Splatting (3DGS) simulator to render visual observations for data collection, paired RGB-D supervision, and simulation evaluation. The planner trained in simulation is then deployed in real-world environments without real-world fine-tuning, achieving obstacle avoidance at speeds of up to 6.5 m/s.