Adaptive client scheduling improves federated learning diversity and fairness
Adaptive Determinantal Client Scheduling in Federated Learning
Machine LearningDistributed, Parallel, and Cluster Computing
Summary
Federated learning trains machine learning models across many devices without sharing raw data, but choosing which devices to use in each round is tricky. The authors found that simply picking high-quality devices ignores the benefits of having a diverse set of devices. They designed a method called Adaptive Determinantal Client Scheduling (ADCS) that smartly balances device quality and diversity using a mathematical tool called determinantal point processes. This approach helps the model learn better, especially for devices with different data and system capabilities.
What this means in practice
- •For machine learning engineers: Schedule training devices in federated learning systems to improve overall model fairness and robustness by balancing quality and diversity.
- •For mobile network operators: Manage client participation in federated learning on mobile devices with varying resources to enhance training effectiveness across heterogeneous networks.
Authors
Wen Xu, Ben Liang, Gary Boudreau, Hamza Sokun
Abstract
Scheduling clients for model training is critical in federated learning due to both data and system heterogeneity. Most previous works focus on the quality of the scheduled clients to achieve faster convergence, shorter wall-clock convergence time, or better average model performance. They rarely consider the diversity of clients, which is important to counter heterogeneity and improve performance for the worst-off clients. In this work, we advocate the use of determinantal point processes (DPPs) to model and enhance the diversity in client scheduling. We first design the kernel matrices of DPPs using gradient information and quality scores, which inherently enables a flexible quality-diversity trade-off. Applying fast MAP inference over DPPs, we propose Adaptive Determinantal Client Scheduling (ADCS) in FL. We further quantify the gradient approximation error of ADCS and develop convergence analysis for general biased client selection in FL with non-convex loss functions. We conduct comparative numerical experiments showing that ADCS outperforms state-of-the-art client scheduling algorithms, including both quality-based and diversity-based ones.