Multi-modal object re-identification improved by orthogonal feature decoupling
ODSS-Net: Orthogonally Decoupled Shared-Specific Representations for Multi-Modal Object Re-Identification
Computer Vision and Pattern Recognition
Summary
Identifying the same object across different types of sensors (like visible and infrared cameras) is hard because the sensors see things differently. The authors developed ODSS-Net, which separates shared information that all sensors agree on from sensor-specific details that differ. This method also smartly combines these parts to better recognize objects across sensors. Tests show that this improves accuracy in matching objects captured by different sensors.
What this means in practice
- •For security system engineers: Enhance cross-sensor object tracking by improving object matching accuracy between visible and infrared camera feeds.
- •For autonomous vehicle developers: Improve identification of objects from diverse sensor inputs by better separating shared and sensor-specific features for reliable recognition.
Authors
Zhifeng Ma, Yuhong Feng, Chongyi Huang, Mingyang Zhou, Minwen Deng
Abstract
Multi-modal object re-identification (ReID) leverages complementary information from heterogeneous sensors. Cross-modal alignment and fusion remain challenging. Existing methods mainly reduce modality gaps via feature alignment and fusion. However, alignment may entangle modality-shared structural information with modality-specific cues, while conventional fusion may not fully exploit complementary information. To address these issues, we propose ODSS-Net, a framework for orthogonally decoupled shared-specific representation learning and adaptive fusion of complementary modality-specific features. First, a Shared Structural Feature Construction (SSFC) module suppresses modality-dependent appearance variations and extracts modality-shared structural information. Then, an Orthogonal Specific Feature Learning (OSFL) module aggregates identity-relevant local information and removes the shared-aligned component via geometric orthogonal projection. Finally, a Sample Adaptive Mixture of Experts (SAMoE) module uses the shared structure as routing context to adaptively refine and fuse modality-specific features. Experiments on three benchmarks demonstrate the effectiveness of ODSS-Net for multi-modal object ReID.