Gait recognition improves under occlusion using action detection experts
Occluded Gait Recognition with Mixture of Experts: An Action Detection Perspective
Computer Vision and Pattern Recognition
Summary
Gait recognition can struggle when parts of a person’s body are hidden or blocked in videos. The authors found that looking at how body movements change over time helps fill in these hidden parts. They treat walking sequences as a series of actions and use two sets of experts to detect and understand these actions better. They also created a new dataset with many examples of people being partially hidden to test their method. Their approach works better than others on several datasets.
What this means in practice
- •For security system developers: Improve identification of individuals in video surveillance where people are partially hidden or obscured by obstacles.$Commercial implications: Enables more reliable person identification products for security firms operating in crowded or obstructed environments.
- •For robotics engineering teams: Enhance robot perception to recognize human movement patterns despite occlusions in dynamic environments.
Authors
Panjian Huang, Yunjie Peng, Saihui Hou, Chunshui Cao, Xu Liu, Zhiqiang He, Yongzhen Huang
Abstract
Extensive occlusions in real-world scenarios pose challenges to gait recognition due to missing and noisy information, as well as body misalignment in position and scale. We argue that rich dynamic contextual information within a gait sequence inherently possesses occlusion-solving traits: 1) Adjacent frames with gait continuity allow holistic body regions to infer occluded body regions; 2) Gait cycles allow information integration between holistic actions and occluded actions. Therefore, we introduce an action detection perspective where a gait sequence is regarded as a composition of actions. To detect accurate actions under complex occlusion scenarios, we propose an Action Detection Based Mixture of Experts (GaitMoE), consisting of Mixture of Temporal Experts (MTE) and Mixture of Action Experts (MAE). MTE adaptively constructs action anchors by temporal experts and MAE adaptively constructs action proposals from action anchors by action experts. Especially, action detection as a proxy task with gait recognition is an end-to-end joint training only with ID labels. In addition, due to the lack of a unified occluded benchmark, we construct a pioneering Occluded Gait database (OccGait), containing rich occlusion scenarios and annotations of occlusion types. Extensive experiments on OccGait, OccCASIA-B,Gait3D and GREW demonstrate the superior performance of GaitMoE.OccGait is available at https://github.com/BNU-IVC/OccGait.