Vision language model controls lunar rover for safer autonomous docking
End-to-End Vision-Language-Action Policies for Autonomous Lunar Rover Docking
Robotics
Summary
Navigating the surface of the Moon is tricky because of harsh lighting and unexpected challenges with sensors. The authors created a new method that lets a lunar rover understand what it sees and directly decide how to move without relying on separate navigation parts. Their approach uses big vision-and-language models that already know a lot about the world to help the rover adapt to new situations. They tested this method in a detailed Moon simulator and found it works as well as traditional navigation methods, even when things go wrong. Additionally, the rover can decide how often to think about its moves, which helps it work safely and efficiently.
What this means in practice
- •For space robotics engineers: Use vision-language-action policies to improve autonomous docking under lunar surface conditions.
- •For autonomous vehicle developers: Implement adaptive inference scheduling to balance computational load and safety in challenging environments.
Authors
Sitan Huang, Grace Gao
Abstract
Surface navigation on the Moon is subject to extreme illumination contrast and deployment-time perceptual disturbances. While modular navigation pipelines have long supported planetary rover navigation, robustness is compromised by measurement errors that propagate through hand-engineered components. End-to-end learned policies are an alternative approach to navigation, which maps observations directly to control actions without intermediate components. Vision-language-action models, in particular, have driven significant breakthroughs in robotics and autonomous vehicle research. By leveraging internet-scale vision-language pretraining, these models can be exploited for their commonsense reasoning and visual understanding to provide robust generalization with limited task-specific data. However, their applicability and performance have yet to be characterized for lunar surface operations. In this study, we design a vision-language-action policy for autonomous lunar rover docking and characterize its performance relative to a modular navigation pipeline. We develop a scalable training framework that generates safety-aware expert demonstrations in a high-fidelity lunar simulation environment. In closed-loop evaluations, our policy performs competitively relative to the modular baseline under perceptual disturbances unseen during training. To address the tradeoff between compute consumption and closed-loop safety, the policy adapts its execution horizon by determining how far the rover may safely travel in open loop. Adaptive-horizon execution reduces inference frequency while limiting hazard-contact rates relative to fixed horizons in the evaluated scenarios.