Adaptive driver alert system improves safety by observing before warning
Observe Before You Alert: Adaptive Driver Alerting with Vision-Language Models
Computer Vision and Pattern RecognitionRobotics
Summary
Detecting potential risks from dashcam video is hard because a system must decide when to warn the driver, not just if a risk exists. The authors propose VLAlert, a system that can silently observe, gather more information, or alert the driver based on what it sees. This approach helps avoid false alarms by waiting for clearer signs of danger before warning. Their tests show that VLAlert outperforms existing methods in accurately and timely alerting drivers to hazards. This suggests smarter driver alerts can better balance safety and avoiding unnecessary warnings.
driver alertingdashcam videoaccident anticipationvision-language modelssequential decision-makingpartial observabilityQ-learningADASsafety-critical systemsperception-action loop
Authors
Yuhang Wang, Lingyao Li, Hao Zhou
Abstract
Driver alerting from dashcam video requires sequential decision-making under partial observability: a system must decide not only whether a scene is risky, but also when the evidence is sufficient to warn. Most existing accident anticipation models output a binary risk score, leaving ambiguous scenes to be handled by thresholding. We propose VLAlert, a vision-language alerting framework that casts warning generation as a tri-action policy over SILENT, OBSERVE, and ALERT. The OBSERVE action acts as an internal evidence-gathering decision that delays uncertain warnings and changes the next observation window, creating a lightweight perception-action loop for adaptive alerting. VLAlert uses Qwen3-VL-4B as a safety-evidence generator and pools hidden states from structured belief spans to form compact representations for danger estimation and policy prediction. We evaluate VLAlert on VLAlert-Bench, a unified per-tick benchmark from four real-world dashcam alert datasets, and further test transfer to held-out naturalistic ADAS takeover clips. On VLAlert-Bench validation, VLAlert achieves the highest deployment-oriented utility among tested baselines, with DAUS 0.4878 compared with 0.4752 for Open-BADAS, and improves AUROC, AP_tick, F1_t, and balanced accuracy from 0.610, 0.176, 0.276, and 0.581 to 0.689, 0.195, 0.297, and 0.648, respectively. On 221 held-out ADAS-TO-Critic clips, VLAlert improves R@5s from 74.2% to 88.7% and F1 from 0.585 to 0.686. These results indicate that adaptive observation and safety-focused VLM representations provide measurable gains for driver-facing alert decisions.