Vocabulary guided gait recognition improves human walking identification
Vocabulary-Guided Gait Recognition
Computer Vision and Pattern Recognition
Summary
Gait recognition is the process of identifying people by how they walk, but current methods either focus on shapes or motion without clear human understanding. The authors introduce a new approach that links human walking actions to words using vision-language models, which helps computers better understand walking patterns. Their system, called α-Gait, connects language knowledge with walking data to find important walking features. Tests on multiple walking databases show this method can enhance gait recognition and offer new research ideas.
What this means in practice
- •For security system developers: Improve person identification systems by integrating vocabulary-guided gait features to enhance recognition accuracy in surveillance.
- •For health monitoring engineers: Develop improved gait analysis tools for health monitoring using vocabulary-informed gait features to detect movement abnormalities.
Authors
Panjian Huang, Saihui Hou, Chunshui Cao, Xu Liu, Yongzhen Huang
Abstract
What is a gait? Appearance-based gait networks consider a gait as the human shape and motion information from images. Model-based gait networks treat a gait as the human inherent structure from points. However, the considerations remain vague for humans to comprehend truly. In this work, we introduce a novel paradigm Vocabulary-Guided Gait Recognition, dubbed Gait-World, which attempts to explore gait concepts through human vocabularies with Vision-Language Models (VLMs). Although VLMs have achieved the remarkable progress in various vision tasks, the cognitive capability regarding gait modalities remains limited. The success element in Gait-World is the proper vocabulary prompt where this paradigm carefully selects gait cycle actions as Vocabulary Base, bridging the gait and vocabulary feature spaces and further promoting human understanding for the gait. How to extract gait features? Although previous gait networks have made significant progress, learning solely from gait modalities on limited gait databases makes it difficult to learn universal gait features for practicality. Therefore, we propose the first Gait-World model, dubbed α-Gait, which guides the gait network learning with vocabulary knowledge from VLMs. However, due to the heterogeneity of the modalities, directly integrating vocabulary and gait features is highly challenging as they reside in different embedding spaces. To address the issues, α-Gait designs Vocabulary Relation Mapper and Gait Fine grained Detector to map and establish vocabulary relations in the gait space for detecting corresponding gait features. Extensive experiments on CASIA-B, CCPG, SUSTech1K, Gait3D and GREW reveal the potential value and research directions of vocabulary information from VLMs in the gait field.