Multi-modal object re-identification improved by token reduction and interaction
TRIM-ReID: Duplication-Aware Token Reduction and Modality-Aligned Interaction for Multi-Modal Object Re-Identification
Computer Vision and Pattern Recognition
Summary
Identifying objects in different types of images, like regular color photos, thermal images, and near-infrared views, is challenging because details can be lost or repeated during processing. The authors present a method called TRIM-ReID that carefully selects important parts of these images to keep diverse and detailed clues about the object's identity. Their approach also aligns and shares information across the different image types to improve recognition. Tests show their method recognized objects better than previous approaches.
What this means in practice
- •For security system developers: Use multi-modal imaging to reliably identify people or objects under varied lighting using compact and efficient token selection.$Commercial implications: Enables more accurate and efficient identification for surveillance and access control products by reducing noise and improving multi-sensor fusion.
- •For autonomous vehicle engineers: Improve object recognition reliability across visual, infrared, and thermal sensors to enhance vehicle perception systems in challenging environments.
Authors
Wanke Xia, Ruiding Zhu, Xingguo Xu, Zhengbo Zhang, Dongxia Liu, Yuan Jin, Taojie Zhu, Yiting Zhao, Yihang Ding
Abstract
Multi-modal object re-identification exploits complementary RGB, near-infrared (NIR), and thermal-infrared (TIR) observations to retrieve target objects. However, existing methods commonly employ visual encoders optimized for global image-text alignment and select tokens using learned importance scores. Such designs fail to preserve fine-grained identity cues or explicitly account for token redundancy, resulting in underrepresented local evidence and duplicated tokens that lead to noisy and costly cross-modal interaction. To address this gap, we propose TRIM-ReID, a compact framework that unifies dense feature extraction, intra-modal token reduction, and inter-modal aligned interaction. Specifically, semantically rich and spatially coherent patch features are extracted by Dense Identity Representation (DIR), which leverages DINOv3 to preserve fine-grained identity information. We then introduce Token Diversity Mining (TDM) to identify complementary local evidence and construct compact modality-specific token sets by suppressing repetitive patches while preserving informative diversity. Retained tokens are subsequently fused by Modal Relational Interaction (MRI) to enable effective information exchange across modalities, while a triangular alignment loss explicitly regularizes their joint relationships to maintain cross-modal semantic consistency under independent token selection. Extensive experiments on RGBNT201, RGBNT100, and MSVR310 demonstrate that TRIM-ReID achieves state-of-the-art performance.