Pedestrian crossing intentions predicted using attention-enhanced graph network
STC-MGNet: Spatial-Temporal-Channel Attention-Enhanced Multimodal Graph Network for Pedestrian Intention Prediction
Computer Vision and Pattern Recognition
Summary
Predicting when a pedestrian intends to cross the street helps make self-driving cars safer. The authors present a new method that looks carefully at body movements over time and combines information from videos and the car’s own motion sensors. Their model highlights important body parts and moments to understand the pedestrian's actions better. Tests on popular datasets show the method predicts crossing intentions more accurately than some existing approaches.
What this means in practice
- •For autonomous vehicle developers: Improve pedestrian crossing predictions by integrating body pose and vehicle motion data to enhance safety algorithms in driverless cars.$Commercial implications: Enables safer pedestrian detection and decision-making systems that can be sold as part of autonomous vehicle software packages.
- •For smart city planners: Use refined pedestrian intention models to design better traffic signal timings and pedestrian safety measures in urban environments.
Authors
Yuanzhe Li, Meng Wang
Abstract
Accurate prediction of pedestrian crossing intentions plays a critical role in enhancing the safety of pedestrian-vehicle interactions in autonomous driving. However, effectively capturing subtle spatiotemporal patterns in pedestrian pose sequences and integrating complementary visual and ego-vehicle motion information remain challenging. This paper proposes a spatial-temporal-channel attention-enhanced multimodal graph network (STC-MGNet) for pedestrian crossing intention prediction. STC-MGNet introduces a spatial-temporal-channel (STC) attention-enhanced pose graph encoder to learn discriminative representations from pedestrian pose sequences. Three parallel attention branches selectively emphasize informative skeletal joints, observation frames, and feature channels, with their outputs integrated through learnable fusion weights. To leverage complementary visual and ego-vehicle motion information, a multi-stage cross-modal channel modulation mechanism is introduced to adaptively refine pose representations through modality-specific channel gating at successive graph encoding stages. Experimental results demonstrate that STC-MGNet achieves accuracy rates of 70%, 87%, and 88% on the JAADbeh, JAADall, and PIE datasets, respectively, outperforming several representative baseline methods. Ablation experiments further highlight the complementary contributions of visual and ego-vehicle motion information to prediction performance.