VLN-AVP: Zero-Shot Vision-Language Navigation with Hybrid Long-Short-Term Memory for Autonomous Valet Parking
2026-07-20 • Robotics
Robotics
AI summaryⓘ
The authors developed a new method for autonomous valet parking (AVP) that doesn't need pre-made maps, which usually limit parking robots to known areas. They combined a bird's-eye-view perception system with vision-language models to understand parking lots better and follow natural language instructions. Their approach uses a special memory system to remember both recent visual information and past experiences for better navigation. They also created a large new dataset for underground parking scenarios to test their method. Tests in simulations and real cars showed their system works better than existing methods, making it more practical for real-world use.
Autonomous Valet Parking (AVP)Vision-Language Models (VLMs)Bird's-Eye-View (BEV)Zero-shot navigationHybrid memory systemTopological memoryVision-Language Navigation (VLN)Natural language instructionsAutonomous drivingBenchmark dataset
Authors
Yijian Li, Xiangru Mu, Changze Li, Hantian Shi, Jiyuan Cai, Jia Cai, Xiaoxue Liu, Yajing Sun, Ming Yang, Tong Qin
Abstract
Existing methods in Autonomous Valet Parking (AVP) typically rely on pre-built maps, which severely restricts their scalability to unseen environments and open-vocabulary targets. Inspired by the application of Vision-Language Models (VLMs) in Vision-Language Navigation (VLN) tasks, we propose VLN-AVP, a zero-shot navigation framework for AVP tasks. By combining the precise spatial perception of a Bird's-Eye-View (BEV) model with the general intelligence of VLMs, our framework 1) eliminates the dependency on pre-built maps, 2) interprets semantic environmental contexts in parking scenarios, and 3) enables intuitive navigation following natural language instructions. Specifically, we introduce a hybrid memory system: a short-term perception memory tracks semantic visual cues to address the limitations of VLM's single-frame reasoning in existing methods, while a long-term topological memory facilitates stable policy learning from past experiences. To bridge the gap in existing benchmarks, we also present the VLN-AVP dataset and benchmark. Featuring 10 high-fidelity parking scenes and over 1,000 navigation episodes, it has the largest number of garage scenes to date and is the first VLN benchmark for underground parking. Extensive experiments demonstrate that in simulation, our method achieves an over 25% improvement in success rate compared to VLN methods and an over 15% improvement compared to other autonomous driving methods. Furthermore, it attains a leading success rate in real-world vehicle experiments, proving its practical feasibility.