Transformer model predicts driver gaze on traffic objects with improved accuracy
TransGaze-Object: Transformer Based Driver Gaze Object Prediction Framework in Real Driving
Computer Vision and Pattern Recognition
Summary
Knowing where a driver is looking helps understand what they are paying attention to on the road. Previous methods guess gaze by pointing to a spot or zone, but this study predicts the exact object a driver is looking at, like a car or traffic light. The researchers created a new model using a transformer system that looks at the driver's face and the traffic scene to guess the gaze object. This method works better than older ways of linking gaze points to objects, reducing errors especially between objects and the background.
What this means in practice
- •For automotive safety engineers: Improve driver monitoring systems by directly predicting which traffic objects the driver is attending to for better awareness assessment.
- •For autonomous vehicle developers: Enhance human-vehicle interaction by integrating driver gaze object prediction to anticipate driver focus during mixed autonomy driving.
Authors
Pavan Kumar Sharma, Ayush Pande, Pranamesh Chakraborty
Abstract
Driver gaze provides information regarding driver visual attention and situational awareness to the surrounding traffic. Existing driver gaze estimation studies represent gaze in terms of gaze zone or gaze vector/point-of-gaze (PoG). However, object-level gaze information provides a more semantically meaningful representation of visual attention by identifying attended objects, such as vehicles, pedestrians, or traffic signals. In this study, we propose an end-to-end driver gaze object prediction framework, TransGaze-Object, Transformer-based Gaze Object prediction model. The proposed framework first extracts facial features, including face and iris-weighted eye features, along with trafficobject spatial features. A transformer based cross-attention mechanism is then used to compute similarity scores and attention weights for predicting the drivers gaze object. To train this model, we propose a benchmark driver gaze dataset, Urban Driving-Face Scene Gaze (UD-FSG), comprising synchronized driver-face and traffic-scene images, scene objects bounding boxes, and gaze labels in terms of 2D gaze coordinate and gaze object. The TransGaze-Object model achieves an overall accuracy of 60% for gaze-object prediction, compared to 51% accuracy obtained from associating the estimated Point-of-Gaze to traffic objects. The error analysis reveals that TransGaze-Object reduces confusion between traffic objects (predicted) and the background (ground-truth), achieving an error rate of 11.68%, a 49.7% relative reduction compared with 23.21% error obtained from PoG-based gaze-object association. Overall, the results demonstrate the effectiveness of directly predicting gaze objects from driver-face and traffic-scene information, rather than estimating an intermediate Point-of-Gaze and subsequently associating it with traffic objects.