Vision language models improve adaptive driver alerting timing and accuracy
Observe Before You Alert: Adaptive Driver Alerting with Vision-Language Models
Computer Vision and Pattern RecognitionRobotics
Summary
Driver assistance systems that warn about dangers from dashcam videos usually decide only if there is risk, but not when to warn. The authors introduce VLAlert, which gives the system three choices: stay silent, keep watching, or alert the driver. This lets the system wait for clearer evidence before warning, making alerts more accurate and timely. They tested VLAlert on multiple real-world dashcam datasets and found it improves how well and when warnings are given compared to previous methods.
What this means in practice
- •For automotive safety engineers: Improve driver assistance systems by adaptively timing warnings based on clearer evidence from video inputs.
- •For vehicle software developers: Incorporate vision-language model based policies to reduce false alarms and missed alerts in driver alert systems.
Authors
Yuhang Wang, Lingyao Li, Hao Zhou
Abstract
Driver alerting from dashcam video requires sequential decision-making under partial observability: a system must decide not only whether a scene is risky, but also when the evidence is sufficient to warn. Most existing accident anticipation models output a binary risk score, leaving ambiguous scenes to be handled by thresholding. We propose VLAlert, a vision-language alerting framework that casts warning generation as a tri-action policy over SILENT, OBSERVE, and ALERT. The OBSERVE action acts as an internal evidence-gathering decision that delays uncertain warnings and changes the next observation window, creating a lightweight perception-action loop for adaptive alerting. VLAlert uses Qwen3-VL-4B as a safety-evidence generator and pools hidden states from structured belief spans to form compact representations for danger estimation and policy prediction. We evaluate VLAlert on VLAlert-Bench, a unified per-tick benchmark from four real-world dashcam alert datasets, and further test transfer to held-out naturalistic ADAS takeover clips. On VLAlert-Bench validation, VLAlert achieves the highest deployment-oriented utility among tested baselines, with DAUS 0.4878 compared with 0.4752 for Open-BADAS, and improves AUROC, AP_tick, F1_t, and balanced accuracy from 0.610, 0.176, 0.276, and 0.581 to 0.689, 0.195, 0.297, and 0.648, respectively. On 221 held-out ADAS-TO-Critic clips, VLAlert improves R@5s from 74.2% to 88.7% and F1 from 0.585 to 0.686. These results indicate that adaptive observation and safety-focused VLM representations provide measurable gains for driver-facing alert decisions.