Visual generative models improve robot navigation with scalable data
NavGen: Visual Generative Models as a Scalable Data Engine for Embodied 3D Navigation
RoboticsArtificial Intelligence
Summary
Robots need lots of good training data to learn how to navigate in 3D spaces, but getting this data can be expensive or unrealistic. The authors use advanced video-generation tools to create large, diverse simulated environments that look realistic to train navigation models. Their system, NavGen, produces hundreds of thousands of training examples by turning text descriptions into navigation videos. Models trained on this data perform better on navigation tasks and work well when tested on real drones flying in various environments.
What this means in practice
- •For drone operators: Train drone navigation systems with large-scale realistic simulated data to improve flight success rates in diverse environments.
- •For robotics software developers: Generate extensive indoor and outdoor navigation training datasets from text prompts to enhance robot spatial understanding without expensive real-world data collection.
Authors
Xijie Huang, Yongyang Wan, Chengbin Dong, Zimo Ding, Mo Zhu, Yijin Wang, Zhiyang Liu, Fei Gao, Yuze Wu, Xin Zhou
Abstract
General-purpose robot models increasingly rely on large and diverse datasets. For embodied 3D navigation, however, existing data sources face a fundamental trade-off: simulated data can be generated at scale but often suffer from the visual sim-to-real gap, whereas real-world flight data provide realistic observations but are costly to collect. This paper studies another direction: the use of high-fidelity visual generative models as scalable data engines for embodied 3D navigation. We introduce NavGen, a text-to-video data generation pipeline that produces diverse vision-language navigation (VLN) episodes across indoor and outdoor scenes. We also propose a style-diversification method that scales up long-tail data that are difficult and costly to collect. The resulting dataset contains approximately 400K navigation episodes. We evaluate our dataset against existing UAV navigation datasets across multiple metrics, and find that the model trained on our data generally improves with scale, outperforming those trained on existing datasets. To validate real-world transferability, we deploy the trained model in world-action-model paradigm to real-world flying experiments. The final model achieves a 75\% success rate across different navigation tasks and environments.