Driver emotion recognition improves with traffic context insight
Traffic-Context-Aware Driver Emotion Recognition with Expectation-Deviation-Driven Inference
Computer Vision and Pattern Recognition
Summary
Recognizing a driver's emotions can help improve interactions between people and cars. Most systems only look at the driver’s face inside the car, but this misses how things happening on the road affect feelings. The authors created a method that combines facial images with what’s going on in traffic, especially noting where drivers' expectations about other cars don’t match reality. This approach helped their system better guess driver emotions than previous methods on real and simulated data.
What this means in practice
- •For automotive safety teams: Integrate driver emotion insights with traffic context to enhance real-time driver monitoring systems for safer vehicle interaction.
- •For autonomous vehicle designers: Use combined driver emotional and traffic context data to improve human-vehicle interaction models in semi-autonomous driving environments.
Authors
Longhao Yan, Jinhao Liang, Kaidi Yang
Abstract
Driver emotion provides an important indication of how drivers perceive and evaluate ongoing traffic situations, making reliable emotion recognition an important capability for effective human-vehicle interaction. Yet, most existing driver emotion recognition approaches rely primarily on in-cabin images and overlook how surrounding traffic interactions shape driver emotional responses. To address this limitation, we propose a traffic-context-aware driver emotion recognition (TADER) framework that integrates in-cabin signals with out-of-cabin traffic scenarios, modeling driver emotions as arising from expectation deviation represented by the discrepancy between predicted and observed vehicle trajectories. The proposed framework comprises two key components: (1) a vision-based base recognizer that extracts spatiotemporal representations from in-cabin facial images to infer driver emotions, and (2) a traffic-context-aware fusion module that quantifies expectation deviation from external traffic scenarios and integrates it into the base recognizer via attention-based embedding fusion. To overcome the scarcity of paired in-cabin and out-of-cabin data, we introduce a transfer learning strategy that transfers a trajectory prediction model trained on large-scale naturalistic datasets to smaller paired datasets. Extensive experiments demonstrate that TADER consistently outperforms state-of-the-art methods across both naturalistic and simulated driving environments by incorporating traffic context information into emotion recognition, resulting in improvements of 5% in accuracy on the AIDE and manD datasets.