Summary
Self-driving cars use 3D object detectors to understand their surroundings, but weather, location, and lighting changes can confuse these systems. The authors found that training these detectors together on different local data, while respecting privacy, can help but usually fails because a single global model does not fit all conditions well. They created FedCKA, which adapts parts of the detectors based on how similar each part is to a shared global model, allowing better customization for each environment. Testing on driving data showed FedCKA works better than existing methods, making vehicle perception more reliable across diverse conditions.
What this means in practice
- •For autonomous vehicle engineers: Build smarter 3D perception models that adapt to unique local conditions like weather or location without sharing raw data.
- •For smart city planners: Deploy varied traffic surveillance systems that collaboratively improve object detection while preserving privacy.
Abstract
Robust perception in intelligent vehicles demands 3D object detectors that remain dependable under domain shifts, such as changes in time of day, location, or weather. However, due to costly annotation and rare shifts, some environments lack sufficient data to train a standalone detector. Federated learning offers a privacy-preserving framework for collaborative model training, enabling clients to benefit from shared learning across diverse environments. Yet, this framework traditionally relies on a single global consensus model, which struggles to perform across heterogeneous local data distributions. Local conditions are better captured by adapting a subset of the model, but many personalization approaches rely on predefined layer partitions or fixed personalization ratios, thereby limiting adaptation to client-specific divergence. To reduce this rigidity, we propose FedCKA, a Centered Kernel Alignment (CKA)-based strategy that dynamically handles the personalization-globalization trade-off. Specifically, FedCKA computes layer-wise feature similarities between local client models and the global consensus model during training. By converting layer-wise similarity scores into client-specific aggregation masks, FedCKA selectively shares representation-consistent layers. Evaluation on a unified multi-domain benchmark based on nuScenes shows that FedCKA outperforms established federated baselines, including FedBN, FedRep, and FedSelect, improving average NDS by 7 percentage points over the strongest baseline. The findings offer both a comparative benchmark and a promising direction for robust federated 3D perception across shifts in location, weather, and illumination. Code is available at https://github.com/j-verhoog/FedCKA.