Summary
Federated learning lets many devices help train a shared AI model without sending all their data to one place. However, when some devices have rare types of data, their experience can get lost if they update the model less often or are outnumbered by common data types. The authors identified this problem, called silent rarity failure, where rare data's effect on the overall model is too small despite good overall accuracy. To fix this, they created FedRAW, a way to give more weight to updates from devices with rare data on the server side, without changing how devices train or send updates. Their method better preserves the influence of rare data, improving predictions on those rare cases while keeping the overall model quality steady.
federated learningasynchronous aggregationrare labelsparticipation biasaggregation biasupdate deduplicationglobal modeldata heterogeneityserver-side weighting
Authors
Prashant Bajpai, Divya Saxena, Philippe Lalanda, German Vega
Abstract
Asynchronous federated learning improves scalability by updating the global model from a server-side buffer of client updates as they arrive, rather than waiting for all selected clients to finish. While efficient, this arrival-driven aggregation can silently distort representation learning under heterogeneous participation. We identify silent rarity failure, a hidden failure mode in which clients holding rare labels contribute too weakly to the global model even though its overall accuracy appears largely unaffected. This failure arises from two coupled effects: rare-label clients may submit updates less frequently when they are slower or less available, creating participation bias; and once their updates enter the buffer, standard asynchronous aggregation assigns them no compensating influence, creating aggregation bias. We propose FedRAW, a fully server-side aggregation method that preserves rare-label influence without changing local training, client objectives, or communication protocols. FedRAW combines client-level update deduplication, which prevents frequently arriving clients from repeatedly dominating the update buffer, with rare-label-aware weighting, which increases the influence of clients carrying low-coverage labels. We formalize silent rarity failure through participation and aggregation bias, and show that FedRAW increases rare-label client influence over uniform aggregation while preserving convergence. Across EMNIST Balanced, CIFAR-10, HAM10000, and ISIC-2019, FedRAW improves rarelabel accuracy while preserving comparable global accuracy and adding negligible server-side computation.