Frozen-host method repairs sensor features in corrupted camera and lidar data
Repair Before You Fuse: Frozen-Host Adaptation for Corrupted-but-Present Sensors
Robotics
Summary
Sometimes, sensors like cameras and lidar on cars or robots give noisy or corrupted data but still work together in a system. The authors present a way to fix parts of the features these sensors send to the fusion system without changing the main detector. Their method learns small corrections at the boundaries where sensor data combine to improve performance when sensors are corrupted but still present. This approach works without needing clean reference data or extra updates during use, and it shows better results on multiple test sets.
What this means in practice
- •For autonomous vehicle engineers: Improve robustness of existing camera-LiDAR detection systems to corrupted sensor inputs without retraining whole models.
- •For robotics system integrators: Add residual repair modules to fusion detectors to better handle noisy or partially corrupted sensor data in multi-sensor robots.
Authors
Gia-Huy Thai, Quang-Thinh Ly, Anh-Minh Phan, Tuan Dang
Abstract
Camera-LiDAR detectors can continue to consume unreliable features even when both sensors remain present, synchronized, and calibrated. We introduce \emph{Boundary Feature Repair} (BFR), a frozen-host adaptation framework that learns task-supervised residual corrections at modality interfaces the detector already consumes. BFR-C repairs each camera feature level read by fusion, whereas BFR-L aligns host-conditioned LiDAR candidates to a selected boundary and routes site-wise innovations relative to the frozen anchor. Their jointly trained composition is BFR-CL. Zero-initialized per-channel scales make every variant an exact detector-level identity before optimization; only the repair modules train, while the encoders, fusion consumer, router, detection head, and host normalization statistics remain fixed. At inference, BFR requires neither clean references, corruption metadata, temporal history, nor online updates. Across the complete 20-corruption, five-severity KITTI-C grid, BFR-C reduces RCE from $14.07$ to $11.92$ on MVX-Net and from $14.29$ to $11.00$ on Focals Conv-F relative to their reproduced frozen baselines. On the latter host, BFR-L raises AP$_{\mathrm{cor}}$ from $73.65$ to $74.48$, while BFR-CL reaches $77.01$ AP$_{\mathrm{cor}}$ and $10.46$ RCE with $86.02$ clean AP. On nuScenes-R, BFR-CL raises the reproduced MoME baseline's mAP robustness ratio from $80.1$ to $81.4$. These results establish boundary repair as a targeted retrofit for corrupted-but-present sensing without retraining the deployed detector.