Multispectral detector improves object recognition with rotation consistency

S$^3$E-Det: A Strict Spatial-Spectral Equivariant Detector for Multispectral Object Detection

Computer Vision and Pattern Recognition

Summary

Detecting objects using multispectral images is tricky because the detector must recognize materials by their color patterns and locate objects accurately even when the view changes. The authors introduce a new detector called S3E-Det that keeps the order of spectral data and ensures predictions stay consistent when images rotate. Their method uses special techniques to combine spectral information across image scales and aligns geometric features for better accuracy. They tested it on several datasets and found it detects objects well with fewer errors caused by rotation.

What this means in practice

  • For remote sensing teams: Improve detection of objects from aerial images by using a model that maintains accuracy under image rotation and integrates multispectral data effectively.
  • For drone surveillance operators: Enhance drone-based object recognition systems with a detector that reduces errors caused by changes in viewing angle and uses multispectral signals.

Authors

Peng Zhang, Tingfa Xu, Jinrui Sui, Yuyang Tian, Jianan Li

Abstract

Multispectral object detection requires detectors to exploit material-dependent spectral responses while maintaining consistent localization under geometric variations. Existing MOD detectors learn discriminative spectral representations, yet lack a unified mechanism that preserves spectral order, adapts spectral-context fusion across pyramid levels, and regulates geometric responses under rotation. To address these limitations, we propose S$^3$E-Det, a strict spatial-spectral equivariant detector that couples ordered spectral modeling with transformation-consistent prediction. Spectral Order-aware Sampling Calibration (SOSC) imposes local order constraints on input-derived spectral features through first-order variation, while Common-Differential Equivariant Fusion (CDEF) adapts spectral-context aggregation across pyramid levels. A compact Geometric Alignment Head (GAH) reconciles invariant and equivariant output properties within a unified prediction stage. Across MODA, HOD3K, and DroneVehicle, S$^3$E-Det combines high detection accuracy with a compact parameter footprint and reduced rotation-induced variation. Code is available at https://github.com/Riiluo/S3E-Det.