Fisheye camera detection improved with minimal retraining steps
FUSEye: Training-Light Fisheye Detection with Overlapping Views and Zero-Initialized Adapters
Computer Vision and Pattern RecognitionRobotics
Summary
Detecting objects in images from fisheye cameras is hard because the curved lens distorts what the camera sees, especially near the edges. The authors propose FUSEye, a method that tweaks a popular object detector with only a small number of new parts and training steps to handle these distortions better. They improve detection accuracy significantly, even when using much less labeled training data. Their approach works across different detection models and keeps the original detector mostly unchanged.
What this means in practice
- •For mobile robot developers: Improve object detection from fisheye cameras on robots by reducing the need for large labeled datasets and heavy retraining.
- •For automotive perception engineers: Enhance surround-view fisheye camera detection with minimal retraining efforts, maintaining accuracy while saving computation.
Authors
Wenya Su, Kai Luo, Di Wen, Ruiping Liu, Yufan Chen, Junwei Zheng, Kunyu Peng, Kailun Yang
Abstract
Fisheye cameras give mobile robots a single-sensor, low-cost view of their surroundings, yet the COCO-pretrained detectors that practitioners routinely reuse fail on them: strong radial distortion warps local image structure, while boundary compression shrinks objects to near-invisible sizes. Full fine-tuning closes much of the gap but requires abundant fisheye labels and compute. We present FUSEye, a training-light framework that turns a frozen-backbone COCO-pretrained extra-large YOLO26 detector (YOLO26-x) into a fisheye detector. FUSEye adds roughly 227k new parameters while updating the inserted modules and the pretrained detection head. It addresses the transfer gap at three causally linked levels. At the input level, overlapping grid view generation and box remapping (GridViews) enlarge compressed boundary regions. At the feature level, zero-initialized residual adapters (Z-Adapters) correct distortion-induced feature misalignment. At the decision level, learned cross-projection agreement fusion (AgreeFusion) promotes low-confidence detections only when they are supported by consistent evidence across multiple views. On the WoodScape surround-view fisheye benchmark, FUSEye raises YOLO26-x from 0.148 to 0.266 mAP50 and retains 84.3% fully fine-tuned accuracy. Moreover, randomly using only 25% of the labeled training images, FUSEye achieves 0.2597 mAP50, retaining 97.6% of its full-label performance. FUSEye also consistently improves YOLOv8-11 detectors, showing that the recipe is architecture-agnostic. Source code will be available at https://github.com/Su-wenya/FUSEye.