DriveMind improves self-driving decisions by tracking multiple scene guesses
DriveMind: Mind-Evolving Belief Tracking for Closed-Loop Autonomous Driving
Artificial Intelligence
Summary
Self-driving cars often have to guess what’s happening around them, but they usually stick to one idea at a time. This can be risky when the situation is unclear, like at busy intersections. The paper’s authors created DriveMind, a system that keeps several possible ideas about the scene going and updates them as it sees new things. This helps the car make better decisions, especially in tricky situations with ambiguous or confusing traffic.
What this means in practice
- •For autonomous vehicle developers: Enhance self-driving car software to maintain multiple traffic scene hypotheses for safer and more reliable decision-making at ambiguous intersections.
- •For traffic simulation engineers: Use advanced belief tracking models to better simulate and analyze vehicle behaviors in complex traffic interactions like merging and emergency braking.
Authors
Yan Shi, Kehua Chen, Jiarui Qi, Yihong Tang, Haichen Liao, Yinhai Wang
Abstract
Recent multimodal large language models (MLLMs) can perceive a traffic scene, reason about it, and propose a driving action within a single model. However, existing MLLM-based drivers reason from scratch at each frame and commit to a single explanation, discarding earlier hypotheses about the scene. When the scene is ambiguous, the agent locks in on the wrong interpretation and acts on it before evidence accumulates. In this paper, we propose DriveMind, an MLLM-based driving framework that keeps several hypotheses about the scene alive at the same time and updates their weights as new frames arrive. Inspired by particle filtering, DriveMind propagates each hypothesis through a VLM, scores it against the current observation, and resamples to maintain diversity. On Bench2Drive, adding belief tracking to the same driving pipeline improves the Driving Score (DS) from 68.50 to 81.37 and the Success Rate (SR) from 41.36% to 54.09%. The largest gains occur on ambiguous scenarios such as unsignalized intersections (+20.0 DS) and oncoming-traffic interactions (+20.5 DS). Beyond the overall metrics, DriveMind also improves fine-grained interactive driving abilities such as merging, emergency braking, and give-way.