Preserving model geometry improves domain shift detection accuracy
GIFT: Representation Geometry Matters for Single-Domain Generalized Object Detection
Computer Vision and Pattern Recognition
Summary
Detecting objects works well when training and testing environments are similar, but often fails when conditions change. The authors found that fine-tuning vision models carelessly can mess up the internal geometry that helps recognize objects across different settings. They propose a method called GIFT that adjusts models while keeping this geometric structure intact, leading to better detection on new, unseen environments. Their experiments show notable improvements with minimal added complexity, highlighting the importance of preserving these internal relationships for robust detection.
What this means in practice
- •For autonomous vehicle developers: Improve robustness of object detection systems to diverse weather and city conditions without expensive retraining on new data.
- •For surveillance system engineers: Enhance detection accuracy in surveillance cameras operating under varying lighting and environmental changes using minimal model updates.
Authors
Yin Zhang, Yaoyue Zheng, Yongqiang Zhang, Bogdan Raducanu, Dan Li
Abstract
Single-Domain Generalized Object Detection (S-DGOD) aims to generalize from a single source domain to unseen domains under distribution shifts. Existing efforts mainly focus on simulating unseen domains or suppressing domain-specific factors. However, they largely overlook the intrinsic generalization capability embedded in Vision Foundation Models (VFMs). In this work, we show that unconstrained fine-tuning disrupts the geometric structure of pre-trained representations, which encodes domain-invariant semantics. Motivated by this, we propose Geometric-Invariance Fine-Tuning (GIFT), a geometric-aware fine-tuning method for VFMs. Specifically, we introduce a series of learnable Householder reflections to the pre-trained weights to encourage geometry-consistent adaptation under domain shift. In addition, we propose a lightweight channel-wise scaling to facilitate effective task-specific adaptation. Extensive experiments demonstrate that GIFT consistently outperforms state-of-the-art (SOTA) approaches on two S-DGOD benchmarks. Notably, by introducing fewer than 1% additional trainable parameters into the frozen backbone, the proposed method achieves improvements of +7.9% and +2.9% in mPC over the previous SOTA on the Cityscapes-C and Diverse Weather Dataset, respectively. This finding highlights that preserving representation geometry is crucial for robust cross-domain generalization, establishing GIFT as a principled and efficient solution to domain shift.