Transformer object detection improves in foggy weather with online adaptation
FogTTA: Online Test-Time Adaptation for Robust Transformer-based Object Detection in Foggy Weather
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Object detectors in self-driving cars usually work worse in foggy weather because they were trained mostly on clear images. The authors created FogTTA, a system that lets these detection models adjust themselves on the fly using new foggy images without needing to retrain from scratch. It works by having two versions of the model: one generates guesses about objects, and the other learns from those guesses after some checks. This helps the detection stay accurate and stable in foggy conditions while running fast enough for real-time use.
What this means in practice
- •For autonomous vehicle engineers: Improve real-time object detection accuracy for autonomous driving systems operating in foggy weather conditions by dynamically adapting the detection model during deployment.
- •For robotics system developers: Enhance robustness of robotic vision systems working outdoors by enabling on-the-fly adaptation to foggy environments without requiring retraining on new data.
Authors
Ali Abedi, Q. M. Jonathan Wu, Ning Zhang, Shiqi Tian
Abstract
Object detection models for autonomous driving commonly experience substantial performance drops when deployed under adverse weather due to the domain shift between training data and real-world operating conditions. This degradation is especially evident when models trained on clear-weather images encounter foggy environments with reduced visibility and contrast. To address this challenge, we introduce FogTTA, an online test-time adaptation framework designed to improve the robustness of Transformer-based object detectors in fog. Using RF-DETR as the underlying object detector, FogTTA enables real-time adaptation to the streaming target domain without requiring source data or retraining. The framework follows a teacher-student design, where the deployed model serves as the teacher and generates pseudo labels from weakly augmented target inputs. These predictions are subsequently refined through non-maximum suppression and confidence filtering. The student model then learns from strongly augmented target sample using the Varifocal loss to mitigate pseudo-label noise. The teacher is updated via exponential moving averaging to ensure stable and continuous adaptation. Experiments show that FogTTA outperforms prior baselines, delivering improved detection accuracy and stability while maintaining real-time performance.