Latent expert chains improve autonomous driving by thinking densely acting sparsely
Think Densely, Act Sparsely: Latent Expert Cognitive Chains for Vision-Language-Action Autonomous Driving
Computer Vision and Pattern RecognitionRobotics
Summary
Autonomous driving needs a deep and detailed understanding of what’s around the car to make safe decisions. Existing systems often rely on limited instructions that don’t capture the full scene, causing them to miss important details. The authors propose a new method that mimics how humans think by creating a chain of expert knowledge layers inside the model to better understand visuals, meanings, and distances. This helps the system make smarter and safer driving decisions. Their tests show better results in seeing objects, predicting what will happen, and planning routes.
What this means in practice
- •For autonomous vehicle engineers: Enhance vehicle perception and decision-making by integrating dense latent expert cognition to improve safety and planning accuracy.
- •For robotics software developers: Use structured latent expert chains to improve robots’ understanding of complex environments through combined visual, semantic, and spatial cues.
Authors
Jie Wang, Guang Li, Zhijian Huang, Jinlong Li, Chenxu Dang, Hangjun Ye, Yahong Han, Long Chen
Abstract
Autonomous driving fundamentally demands dense understanding of visual, semantic, and geometric information, yet existing Vision-Language-Action (VLA) models are typically trained with sparse supervision such as language instructions or trajectory signals. This direct mapping from dense visual observations to sparse representations forces the model to compress or even discard critical scene information essential for safe driving. We argue that this limitation does not stem from insufficient model capacity, but from the lack of explicit mechanisms that encourage the formation of dense world representations in the latent space. In this spirit, we propose a novel paradigm for driving intelligence, Think Densely, Act Sparsely, and instantiate it with an efficient and interpretable framework, LECDrive (Latent Expert Cognitive Chains for Vision-Language-Action autonomous Driving). The core idea is to mimic the hierarchical perception process of humans by embedding a progressive chain of latent expert cognition within the VLA model. Concretely, we inject a compact set of task-specific latent tokens as carriers, enabling the model to internalize dense knowledge from foundational vision models, including visual representations (DINOv3), semantic structures (SAM3), and spatial geometry (DepthAnything3), while maintaining efficient inference. This interdependent cognitive chain follows a curriculum learning principle, where supervision is progressively structured from representation to semantics and geometry. Extensive experiments demonstrate that LECDrive consistently improves the performance of base VLMs across multiple autonomous driving benchmarks, covering key tasks such as object perception, state prediction, and trajectory planning. These findings suggest that constructing dense world models through structured latent expert chains is a promising direction for VLA-based autonomous driving.