Vehicle dynamics benchmark measures realism in driving video models
VehDyn: A Driving World Model Benchmark for Vehicle Dynamics
Computer Vision and Pattern RecognitionArtificial IntelligenceEmerging TechnologiesRobotics
Summary
Visual models that predict future driving scenes often prioritize looking realistic but may not follow the true physics of how vehicles move. The authors offer a new benchmark, VehDyn, that uses realistic simulations with detailed vehicle physics and synchronized video to test if these models predict realistic vehicle motion. They test 12 popular video models and find that while many predict vehicle paths well, none fully capture accurate vehicle dynamics. VehDyn also shows that better visual quality does not always mean better physical realism in driving predictions.
What this means in practice
- •For autonomous vehicle developers: Evaluate and improve video-based driving prediction models by measuring how realistically they simulate vehicle movement dynamics.
- •For simulation engineers: Use the VehDyn dataset and evaluation framework to create or benchmark simulators that generate physically consistent and visually realistic vehicle trajectories.
Authors
Tianyi Wang, Wangsheng Du, Jiazhou Chen, Tianyi Zeng, Xiangyu Li, Jiseop Byeon, Yujin Wang, Yiming Xu, Yangyang Wang, Bingzhao Gao, Sikai Chen, Zhaomiao Guo, Junfeng Jiao, Christian Claudel, Alexandre Bayen
Abstract
Video world models are emerging as data engines, action planners, and generative simulators for autonomous driving, but existing benchmarks primarily assess visual fidelity and coarse physical plausibility, providing limited evidence on whether generated driving futures obey realistic vehicle kinematics and dynamics. This limitation is further compounded by the lack of datasets in which vehicle, road, maneuver, and speed conditions are independently controlled, and ground-truth vehicle states are recorded in synchrony with videos. We introduce VehDyn, a driving world model benchmark for vehicle dynamics. VehDyn is built on a CARLA-CarSim co-simulation platform where photorealistic rendering is coupled with a validated multi-body dynamics model, and it contains 10,080 configurations from a full factorial design over five vehicle types, four tire-road friction coefficients, three maneuvers, four target speeds, 14 scenes, and three illuminations, each paired with synchronized position, velocity, and attitude sequences. Built on this dataset, VehDyn introduces a hierarchical evaluation framework that measures trajectory alignment, kinematic consistency, and dynamic consistency, and benchmarks 12 state-of-the-art video world models. We further assess the video quality using two established protocols and correlate it with the VehDyn score. Trajectory-level metrics are nearly saturated, with ten of twelve models within 20\% of ground truth, while no model reaches 92\% of ground truth on dynamic consistency, and visual-quality metrics are only weakly correlated with vehicle-dynamics fidelity. DrivingWorld achieves the highest VehDyn score, followed by Cosmos 3 Nano and LTX-Video 2.5, and the VehDyn score agrees closely with human judgment. VehDyn provides a systematic foundation for developing driving world models that are physically consistent and visually realistic.