Summary
Predicting how drivers move their bodies can help understand their intentions and attention while driving, but past datasets mostly capture short clips or movements outside the vehicle. The authors created DriveMotion, a large dataset combining many sources to track detailed driver body movements over long periods. They designed tests that focus on predicting motion right before important driving maneuvers, where movements change more noticeably. Their results show that models trained on this data can better forecast driver motion than simple guesses and that combining different data sources improves performance. DriveMotion offers a shared resource and tools to support future research on predicting driver behavior continuously.
driver motion forecastingmulti-source benchmarknaturalistic driving databody keypointsCAN signalsmotion dynamicsbehavior predictionmachine learningin-cabin videoevaluation metrics
Authors
Yuhang Wang, Chuheng Wei, Jingxin Yang, Xishun Liao, Hao Zhou
Abstract
Driver motion can provide cues to ongoing behavior, attention, and near-term driving intent. However, most existing driver-centric datasets focus on recognizing predefined driver behaviors from short video clips, while human motion forecasting benchmarks largely target motion outside the vehicle. We introduce DriveMotion, a multi-source benchmark for continuous driver motion forecasting. DriveMotion contains 393 hours of 133-keypoint motion sequences at 10 Hz from 360 drivers, integrating naturalistic driving data, curated public in-cabin videos, and the AIDE dataset into a unified representation with per-joint validity masks and synchronized driving context. Naturalistic driving contains long periods of limited body movement, making uniformly sampled evaluation dominated by persistence and less sensitive to brief but behaviorally meaningful motion. To address this, we use dynamics-anchored evaluation, placing forecasting windows around vehicle maneuvers identified offline from CAN signals without providing CAN to the model at inference. Arm motion in pre-maneuver windows is 3.4x greater than in route-matched stable-driving controls. On these anchored windows, learned models reduce forecasting error over persistence by up to 15%, while maneuver-enriched training improves forecast-derived Part-State F1 by 44% over the zero-motion reference. Training on the full multi-source corpus further reduces forecasting error on held-out web drivers by 38% compared with BATON-only training. DriveMotion provides identity-disjoint splits, fixed evaluation subsets, and reference implementations for reproducible evaluation of continuous driver motion forecasting. The dataset and benchmark are available at https://huggingface.co/datasets/HenryYHW/DriveMotion