TriDrive forecasts driver actions vehicle moves and road conditions jointly
TriDrive: Joint Driver, Vehicle, and Road Modeling for Forecasting and Driver Monitoring
RoboticsComputer Vision and Pattern Recognition
Summary
It can be hard to predict how a driver’s movements, the vehicle’s behavior, and the road environment all influence each other while driving. The authors developed TriDrive, a system that looks at these three things together to better forecast what might happen next. This combined approach improved the accuracy of driver behavior predictions and helped issue timely warnings about driving risks. They tested TriDrive both on public datasets and in real cars, showing it can run fast enough for real-time safety alerts that drivers found helpful.
What this means in practice
- •For automotive safety engineers: Enhance in-car driver monitoring by jointly forecasting driver movements vehicle responses and road conditions to provide earlier and more accurate risk warnings.
- •For adas developers: Integrate joint driver vehicle road modeling into advanced driver-assistance systems to improve prediction of critical events like steering onset and collision risk.$Commercial implications: Enables improved safety features sold as driver assistance upgrades for vehicles using real-time joint forecasting.
Authors
Yuhang Wang, Jingxin Yang, Chuheng Wei, Yuechen Guo, Jinghan Xu, Zhao Han, Hao Zhou
Abstract
Predicting how drivers, vehicles, and road scenes interact and evolve together is central to driver monitoring. Prior work models in-cabin activity or traffic-conditioned driver motion in isolation, motivating joint driver, vehicle, and road modeling with real-time on-vehicle evaluation. We introduce TriDrive, to our knowledge the first unified framework that jointly forecasts driver kinematics, vehicle dynamics, and road demands through an automation-conditioned transition model. Modality-specific encoders (an anchored kinematic representation of the driver, causal CAN-bus dynamics, and frozen V-JEPA 2 road latents with structured road margins) are connected by directed residual connections through which driver and road context refine vehicle forecasts. We evaluate TriDrive on three downstream tasks. On the public AIDE benchmark, its kinematic encoder recipe sets a new full-set state of the art (SOTA) among published baselines (48.05 versus 71.47 All-MPJPE). On 197.2 hours of naturalistic BATON subset, directed connections and road margins raise assistance-engaged PR-AUC by 0.084 for steering onset and 0.286 for time-to-collision drops. For real-time use, we distill the road encoders and run TriDrive on a comma four with an external 8 GB GPU, where a lightweight current-state warning probe updates at 5 Hz with 177 ms p95 latency while the joint model forecasts concurrently. The probe is above an openpilot-based baseline on human-labeled manual-driving warnings (AUROC 0.725 versus 0.563), and in a paired on-road study 14 drivers rate its warnings as more appropriate (+1.79) and timely (+2.67) than those of openpilot's driver-monitoring system.