Visible infrared object detectors fooled by stealthy cross-modal attacks
Stealthy in Semantics, Antagonistic in Space: Attacking Visible-Infrared Object Detectors via Object-Level Misalignment
Computer Vision and Pattern Recognition
Summary
Visible-infrared object detectors help see things clearly in dark or bad weather by combining regular and infrared images. The authors found a way to trick these detectors by creating hidden patches that confuse the detectors across both types of images, without being obvious to people. Their method, called CamoShift, places camouflaged patches on objects and shifts infrared signals so the system fails to identify targets properly. This makes the attack hard to spot but very effective.
What this means in practice
- •For security teams: Test object detection systems' robustness against stealthy cross-modal physical attacks combining visual camouflage and infrared signal shifts.
- •For autonomous vehicle engineers: Improve and validate sensor fusion algorithms by experimenting with attacks that disrupt spatial correspondence between visible and infrared sensor data.
Authors
Yueqi Zhu, Qi Ming, Guo Cheng, Yongkang Zhang, Feiran Liu, Juan Fang, Jiahuan Zhou, Jiangmeng Li, Yuhan Zhang
Abstract
Visible-infrared object detectors are used for robust perception under challenging illumination and weather conditions. Current physical attacks apply conspicuous patches to spatially aligned target regions, which are noticeable to human observers. Meanwhile, most of these methods only perturb the appearance within the aligned region, without explicitly targeting the correspondence between modalities or the fusion process. In this paper, we propose CamoShift, an adversarial framework for visible-infrared object detection. By combining visual camouflage with object-level infrared shifting, CamoShift breaks cross-modal spatial alignment and disrupts fusion. Specifically, the Semantic Camouflage Module (SCM) generates a stealthy camouflaged patch that can be attached to the host object and maintains its effectiveness in the infrared branch through an RGB-IR adapter. The Object-level Spatial Decoupling Module (OSDM) shifts the infrared target evidence in a scale-aware manner, so as to break object-level correspondence and disrupt cross-modal fusion. Then, the Harmonic Adversarial loss (HarAdv loss) further balances attack strength and visual stealth during optimization. To the best of our knowledge, we are the first to target both visual stealthiness and attack success in visible-infrared object detection. Extensive experimental results show that CamoShift achieves a superior balance between attack effectiveness and visual stealth. Code and models will be available on GitHub.