DH-VLM: Dual-Horizon Cooperative Latent Reasoning for Autonomous Driving
2026-08-10 • Robotics
Robotics
AI summaryⓘ
The authors present DH-VLM, a new method to improve how self-driving cars work together with smart infrastructure to make better driving decisions. Their system lets roadside units share useful, summarized information with cars to help them see beyond their limited sensors without needing too much computing power. They also created a special dataset to teach cars to understand scenes better and reason safely. Tests show their method makes driving plans more accurate and safer while using less communication and memory than previous approaches.
autonomous drivinglarge-scale language modelscooperative drivinglatent reasoninginfrastructure-vehicle communicationplanning horizonscene understandingsafety-aware reasoningmulti-layer hidden statesend-to-end learning
Authors
Ziyi Song, Chen Xia, Hang Yu, Sheng Zhou, Zhisheng Niu
Abstract
Large-scale language models for autonomous driving enable enhanced global understanding and long-horizon planning. However, when deployed in isolated vehicles, limited sensing range and occlusions restrict reliable decision-making, and the substantial computational and latency overhead makes on-board deployment impractical. Cooperative driving provides a potential solution by leveraging external agents for information exchange, but existing methods remain limited in semantic reasoning capability under practical constraints. To address these challenges, we propose DH-VLM, a dual-horizon cooperative latent reasoning framework that enables asymmetric semantic cooperation between the infrastructure and ego vehicle. The infrastructure aggregates multi-layer hidden states to form a global-reasoning horizon latent guidance, which is integrated into the ego model through an Infrastructure-Driven Latent Evolution mechanism for conditional latent refinement. This enables the ego vehicle to leverage long-range contextual understanding while preserving autonomous decision-making within its local planning horizon. Furthermore, we construct a cooperation-oriented question-answer (QA) dataset covering fundamental scene understanding and ego-personalized comprehension to support counterfactual and safety-aware reasoning. Extensive experiments demonstrate that DH-VLM achieves state-of-the-art planning performance, outperforming the previous state of the art by 14.6% in L2 error and 26.9% in collision rate. Compared with query-based end-to-end cooperative driving methods, our approach reduces the communication cost by 57.3% and GPU memory usage by 25.5%, while maintaining strong robustness against infrastructure guidance errors, providing a practical and robust paradigm for cooperative autonomous driving.