FedCC: Towards Addressing Label Distribution Skews in Distillation-Based Federated Learning
2026-08-24 • Machine Learning
Machine LearningArtificial Intelligence
AI summaryⓘ
The authors study federated learning, where many devices train a shared model without sharing their private data. They focus on a method where clients use an unlabeled public dataset to improve learning but face challenges due to uneven data distribution, causing biased predictions. To fix this, the authors propose FedCC, which lets clients mark uncertain samples as 'unknown' instead of forcing a guess, helping the server handle imbalance better. Their method improves accuracy, especially when clients have very limited types of data.
Federated LearningLabel Distribution SkewDistillation-based FLPseudo-labelsUnlabeled Public DatasetClient ModelsCalibrationBiasUnknown ClassAccuracy
Authors
Wenxuan Ye, Onur Ayan, Xueli An, Georg Carle
Abstract
Federated Learning (FL) enables distributed clients to collaboratively train models without sharing raw data, making it promising for leveraging massive devices in communication networks. In distillation-based FL, each client applies its local model on an unlabeled public dataset, and shares only prediction results with the server. While heterogeneous local data introduces label distribution skew, thus biasing client models toward majority classes and leading to potentially inaccurate predictions. The lack of ground-truth labels in the public dataset hampers the server's ability to calibrate predictions, which ultimately degrades overall performance. To address this, we propose FedCC, a simple and effective algorithm for mitigating client misclassification. Instead of being forced to classify and risking error propagation, clients are allowed to tag ambiguous samples as 'unknown'. This additional class, together with calibrated pseudo-labels on the public data, balances confidence in majority classes against uncertainty in under-represented ones. Extensive experiments demonstrate that FedCC significantly outperforms existing methods, especially under severe label skew. In the extreme scenario where each client holds samples from only one of ten classes, FedCC achieves 67.3% accuracy, while baselines collapse to near-random results.