Improving camera 3D object detection with robust sparse model averaging
Sparse Recovery Bird's-Eye View Model Soup for Robust Camera-Based 3D Object Detection
Computer Vision and Pattern RecognitionArtificial IntelligenceMachine Learning
Summary
Self-driving cars use cameras to see objects around them from a bird's-eye view, but this view can be affected by bad weather or noise, causing accidents. The authors developed a method called SR-BEVS that creates many slightly different simpler models by removing parts and then combines them into one stronger model. This new model works better at detecting objects accurately both in clear conditions and when the camera images are corrupted. They tested it on a popular driving dataset and found it improved accuracy and handled various disturbances like noise and weather changes well.
What this means in practice
- •For autonomous vehicle engineers: Enhance vehicle perception systems to maintain accurate 3D object detection in adverse weather and sensor noise using a pruning and model-averaging technique.
- •For robotics developers: Build more robust robot vision models that handle real-world camera distortions without needing extra data or changing existing model designs.
Authors
Jinwoo Chung, Sangho An, Hyunsung Park, Jangho Kim
Abstract
Camera-based bird's-eye view (BEV) perception methods such as Fast-BEV have demonstrated promising results with fast inference speed and competitive accuracy. However, their robustness to real-world corruptions remains insufficiently explored, posing risks for autonomous driving deployment where environmental variations are inevitable. In this paper, we propose Sparse Recovery Bird's-Eye View Model Soup (SR-BEVS), a sparse recovery learning approach that simultaneously improves clean accuracy and corruption robustness without requiring additional data or architectural modifications. Our method identifies pruning candidates based on layer-wise weight magnitudes, applies random pruning to create diverse sparse models, and retrains each model with a randomly sampled learning rate. By averaging these diverse recovered models into a single dense model, we achieve enhanced robustness against various corruptions while introducing zero inference overhead. We further provide a mode connectivity analysis demonstrating that our pruning-based diversification preserves the connected low-loss basin necessary for stable weight averaging. Experiments on the nuScenes dataset demonstrate up to 33.8% improvement in corruption robustness and a 1.8% gain in clean accuracy over the Fast-BEV baseline, with consistent improvements across sensor noise, weather distortions, and illumination changes. Our simple yet effective approach may be applicable to other BEV models.