Fully Rotation-Equivariant Spectral-Spatial Learning for Multispectral Object Detection
2026-07-06 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors address problems in multispectral object detection, especially when objects are rotated or when combining color and shape information. They propose FressDet, a system that keeps track of colors continuously and handles rotations well, making it better at recognizing objects regardless of their orientation. Their method uses new techniques to adjust color information, decide which features to trust more, and predict object positions with rotation in mind. FressDet achieves strong results with fewer model parameters on multiple datasets, showing it works well and is efficient.
multispectral detectionrotation equivariancespectral-spatial fusionimplicit fieldpyramid levelsoriented object detectionquery-based resamplingfeature fusiondeep learningparameter efficiency
Authors
Peng Zhang, Tingfa Xu, Shuaihao Han, Jianan Li
Abstract
Existing multispectral detectors are limited by discrete spectral processing, a scale-dependent shift in the relative reliability of spectral and spatial cues across pyramid levels, and the lack of explicit rotation-equivariant geometric priors for arbitrarily oriented objects. To tackle these limitations, we propose FressDet, a fully rotation-equivariant spectral-spatial learning framework for multispectral object detection, capable of capturing the continuous, ordered nature of spectral structure and enabling reliable spectral-spatial fusion across pyramid levels under arbitrary in-plane rotations. FressDet integrates three complementary components. Spectral Implicit Warp (SpeIW) enables query-based spectral resampling via a coordinate-conditioned implicit field, yielding a monotone, order-preserving warp. Rotation-Equivariant Consistency Weighting (ReCoW) adaptively fuses spectral and spatial branches based on branch reliability, reinforcing informative cues while suppressing noise across pyramid levels. The oriented-aware head exploits group-indexed features to stably predict oriented objects without parameter replication. Taken together, FressDet learns more discriminative and robust spectral-spatial representations even under rotational perturbations. By achieving state-of-the-art performance with 93% fewer parameters on three public benchmarks, FressDet demonstrates its effectiveness and generalizability.