Gait emotion recognition improved with fusion and timing robustness
Single-Stream Multi-Feature Fusion with Temporal Robustness for Gait Emotion Recognition
Computer Vision and Pattern Recognition
Summary
Recognizing emotions from the way people walk is hard because it needs lots of labeled data and current methods don’t work well on different kinds of data. The authors propose a new approach called SV-GCN that combines multiple features early and handles videos of different lengths and speeds better than before. Their method reduces sensitivity to the frame rate, meaning it works well even if the walking video is recorded at different speeds. Tests show it matches the best current results and generalizes better to varied data.
What this means in practice
- •For computer vision engineers: Develop more robust emotion recognition systems from walking patterns that handle videos with varying frame rates and lengths.
- •For security system designers: Create surveillance tools that infer emotional states from gait across heterogeneous video data with consistent accuracy.
Authors
Shirong Lyu, Silu Quan, Yixuan Ding, Chengpeng Wang
Abstract
3D skeleton-based gait emotion recognition faces high annotation costs, data scarcity, and poor generalization on heterogeneous data. This paper proposes SV-GCN, a single-stream multi-feature fusion framework with temporal invariance. We introduce intra-frame relative motion features to eliminate frame-rate sensitivity and embed heterogeneous cues at shallow layers, enabling early fusion without multi-stream complexity. For variable-length sequences, we design a global mask-guided valid-frame spatio-temporal graph convolution module, introducing frame-rate insensitivity for the first time in this domain. On the E-Gait dataset, our method achieves performance comparable to state-of-the-art while demonstrating strong generalization across varying sequence lengths and frame rates, offering a viable pathway for pre-training on large-scale skeleton-based action recognition datasets.