Cooperative perception improves vehicle detection despite bad weather
Understanding and Repairing Cooperative Perception under Adverse Weather
Computer Vision and Pattern Recognition
Summary
Cooperative 3D vehicle detection can fail even when sensors still see all the points on an object, especially in bad weather. The authors show that changes outside the vehicles caused by weather disrupt how different sensor data combine, causing the system to miss vehicles. They propose a new method called Adverse-Weather Cooperative Repair (AWCR) that fixes these errors by filtering and correcting data. Tests on both real and simulated data show that AWCR meaningfully improves vehicle detection in foggy or rainy conditions.
What this means in practice
- •For autonomous vehicle engineers: Improve detection reliability in cooperative perception systems under rainy or foggy conditions using AWCR to restore sensor fusion benefits.
- •For roadside sensor operators: Enhance vehicle detection accuracy during adverse weather by applying local correction and quality selection on cooperative sensing data.
Authors
Mengxu Ma
Abstract
A cooperative 3D detector can miss a vehicle even when the LiDAR points on it remain unchanged. We study this failure under adverse weather as Weather-Induced Context Interference. To our knowledge, this is the first controlled study of how weather-related exterior changes remove cooperative benefits while preserving all points inside annotated objects. Controlled experiments on simulated and real roadside inputs show that exterior loss changes object representations before fusion and removes previously useful assistance. On real inputs, restoring object-neighborhood features recovers 30 of 31 induced failures; three equal-area exterior controls recover none. The failure condition also appears with a second fusion model. These findings motivate Adverse-Weather Cooperative Repair (AWCR), with separate implementations for the available supervision. Point Classifier filters added weather returns in simulation. On real inputs, Local Correction refines candidates and Quality Selection checks them against current measurements, without naturally paired clear/adverse-weather recordings. Candidate tracing shows that correction supplies accurate geometry for some targets that original candidates cannot localize at the evaluation threshold. On natural adverse-weather CoInfra development data, AWCR improves car bird's-eye-view (BEV) AP@0.7 by 4.66 points with V2X-ViT and 5.56 with AttFuse. On simulated V2X-R fog, it improves vehicle BEV AP@0.7 by 4.47 points at a fixed 4,096-candidate budget.