Summary
Recognizing people by the way they walk, called gait recognition, usually uses video images or simplified outlines. The authors point out that walking generates many different types of data like infrared, depth, radar, and more, which are rarely studied together. They created a big dataset called MMGait that combines all these types of data to compare and test gait recognition across different sensors. They also developed a system called OmniGait++ that can learn from and combine multiple sensor types at once, working well whether it uses one type of data or many. Their work shows it’s possible to have one system that adapts to different sensor combinations and still identify people accurately.
What this means in practice
- •For security system developers: Build identity verification systems that reliably recognize people using varied sensors like infrared or radar when video is unavailable or unreliable.$Commercial implications: Enables new multimodal biometric security products that work across lighting and visibility conditions by combining diverse sensor data into one system.
- •For robotics engineers: Equip robots with gait recognition that fuses multiple sensor types to identify humans robustly in diverse environments and sensor setups.
Authors
Saihui Hou, Chenye Wang, Qingyuan Cai, Aoqi Li, Yongzhen Huang
Abstract
Gait recognition is commonly studied using RGB videos or their derived silhouettes and poses. Yet human walking produces heterogeneous photometric, geometric, and motion cues that cannot be systematically examined with RGB-centered benchmarks. We present MMGait, a large-scale multi-sensor benchmark that brings visible, infrared, depth, LiDAR, and radar observations into sequence-level correspondence. It provides diverse modalities spanning appearance, contours, geometry, motion, and body structure. Under a shared impostor-augmented protocol, we evaluate single-modal recognition, cross-modal recognition via directed retrieval, and multi-modal recognition using task-specific experts. Across settings, modality rankings vary with probe conditions, cross-modal alignment remains difficult, and fusion often provides complementary gains. This analysis exposes a scalability problem: individual modalities, modality pairs, and fusion configurations are typically handled by separately trained experts. We formulate Omni-Modal Gait Recognition, which unifies single-modal, cross-modal, and multi-modal recognition within a shared identity space. OmniGait++ uses modality-specific front ends followed by a shared identity encoder to preserve modality-dependent cues while learning comparable identity descriptors. An anchor-guided fusion module aggregates modality subsets of varying size without frame-level synchronization. A jointly trained checkpoint covers all three recognition settings and accommodates modality subsets of different compositions and cardinalities. Experiments show OmniGait++ remains competitive with task-specific experts in many shared settings and extends to higher-cardinality fusion unavailable to fixed-pair models. The results establish MMGait as a common testbed for heterogeneous gait sensing and demonstrate the feasibility of unified recognition under varying modality availability.