Asynchronous federated learning method boosts rare label impact in global models
FedRAW: Preserving Rare-Label Influence in Asynchronous Federated Learning
Machine LearningArtificial Intelligence
Summary
When multiple devices train a shared AI model without sharing raw data, some rare information might be overlooked, especially if those devices send updates less often. The authors found that this causes a hidden problem where rare data has little influence on the final model, even if the model seems accurate overall. They created FedRAW, a new way for the server to better balance incoming updates so that rare data influences the model more fairly without changing how devices train or communicate. This approach improves the model's accuracy on rare labels while keeping performance on common labels stable.
What this means in practice
- •For machine learning engineers: Improve federated learning models’ accuracy on underrepresented categories by adjusting server-side aggregation without modifying client devices.
- •For healthcare data teams: Enhance diagnostic models trained on distributed medical image data by preserving influence of rare disease labels in imbalanced datasets.
Authors
Prashant Bajpai, Divya Saxena, Philippe Lalanda, German Vega
Abstract
Asynchronous federated learning improves scalability by updating the global model from a server-side buffer of client updates as they arrive, rather than waiting for all selected clients to finish. While efficient, this arrival-driven aggregation can silently distort representation learning under heterogeneous participation. We identify silent rarity failure, a hidden failure mode in which clients holding rare labels contribute too weakly to the global model even though its overall accuracy appears largely unaffected. This failure arises from two coupled effects: rare-label clients may submit updates less frequently when they are slower or less available, creating participation bias; and once their updates enter the buffer, standard asynchronous aggregation assigns them no compensating influence, creating aggregation bias. We propose FedRAW, a fully server-side aggregation method that preserves rare-label influence without changing local training, client objectives, or communication protocols. FedRAW combines client-level update deduplication, which prevents frequently arriving clients from repeatedly dominating the update buffer, with rare-label-aware weighting, which increases the influence of clients carrying low-coverage labels. We formalize silent rarity failure through participation and aggregation bias, and show that FedRAW increases rare-label client influence over uniform aggregation while preserving convergence. Across EMNIST Balanced, CIFAR-10, HAM10000, and ISIC-2019, FedRAW improves rarelabel accuracy while preserving comparable global accuracy and adding negligible server-side computation.