Blurring Modal Boundaries: A Unified Survey from Single- to Multi-Modal Person Re-ldentification
Computer Vision and Pattern Recognition
Summary
The authors review how person re-identification (ReID), which helps track people across different cameras, is moving from using only regular photos to combining different types of data like infrared, text, and sketches. They explain new tasks that use these diverse sources, including a new one that can see objects not directly visible (Non-Line-of-Sight). They also propose a new Transformer-based method to better match people using visible and infrared images by focusing on features common to both. Finally, they discuss datasets, challenges, and future research directions in this growing area.
Authors
Xiao Wang, Bing Wang, Bin Yang, Cuiqun Chen, Xin Xu, Mang Ye
Abstract
Person re-identification (ReID) serves as a critical component in intelligent surveillance systems, aiming to match identities across disjoint camera networks. While traditional methods primarily rely on single-modal RGB imagery, they are often constrained by environmental challenges such as low illumination and occlusion. To overcome these limitations, the field is rapidly evolving toward cross-modal and multi-modal paradigms. This survey presents a comprehensive overview of this transition, systematically reviewing key cross-modal tasks including visible-infrared (VI-ReID), text-image (TI-ReID), sketch-based (Sketch-ReID), and the emerging Non-Line-of-Sight (NLOS) ReID, which extends perception beyond direct visibility. Furthermore, we examine tri-spectral and multi-modal fusion ReID, discussing how complementary information from diverse sensors enhances robustness. Beyond summarizing datasets, challenges, and methodologies, we propose a Transformer-based baseline framework for visible-infrared ReID, designed to effectively capture modality-invariant features. Finally, based on the current landscape, we outline several promising directions for future research.