Textual complements improve visible-infrared person re-identification accuracy
Are Textual Residuals Truly Complementary? Learning Visual Complements under Semantic Superposition for Visible-Infrared Person Re-Identification
Computer Vision and Pattern Recognition
Summary
Identifying people across visible and infrared images is difficult because infrared images miss color and detailed appearance. The authors found that simply using the differences in text descriptions as visual clues can mix helpful and irrelevant details, making it unreliable. They developed a method to carefully select and adjust these text differences to better match what can be seen in both image types. Their approach improved accuracy in matching people in combined visible-infrared image datasets.
What this means in practice
- •For security system developers: Improve person identification accuracy in surveillance systems combining visible and infrared cameras by selectively integrating textual descriptions.$Commercial implications: Enables more reliable multispectral security products that better identify people in low-light or obscured conditions.
- •For multimodal ai engineers: Enhance cross-modal learning systems by applying selective text-based calibration to align different sensory inputs for better visual recognition.
Authors
Xiaoyang Li, Mingming Lu, Wei Zhou, Hao Li, Canhui Zhang
Abstract
Text-enhanced visible-infrared person re-identification (VI-ReID) exploits textual descriptions to compensate for appearance cues that are weak or unavailable under infrared imaging. Existing methods construct compensation from differences between modality-oriented textual representations, implicitly assuming that textual differences are visually complementary. We revisit this assumption through semantic superposition and show that dense textual residuals may entangle useful appearance variations with redundant, making direct residual fusion unreliable. We propose Superposition-Aware Residual Calibration, which transforms dense textual differences into selective and visually grounded complements. Low-Interference Sparse Residual Learning represents modality-oriented text embeddings as sparse activation strengths over a shared dictionary, selects positive strength differences, and suppresses both dictionary-level overlap and activation-weighted interference to reconstruct a compact, selectively controlled residual. Visual Complement and Counterfactual Calibration further calibrates the residual against identity-specific RGB-IR visual discrepancies, encouraging retained semantics to provide meaningful compensation while preserving the residual structure. Experiments on Tri-SYSU-MM01, Tri-RegDB, and Tri-LLCM show that SARC consistently improves the VI-ReID backbone and remains competitive with text-assisted methods. Mechanism analyses show that SARC produces more concentrated textual variations, reduces dictionary-level feature interference, and improves residual-visual-complement alignment for most samples. These results demonstrate that effective cross-modal compensation requires textual residuals to be selectively represented and explicitly calibrated, rather than directly treated as visual complements. Code and implementation details will be released upon publication.