Faster federated learning with smarter client and data choices
Client and Training Data Selection for Computationally Efficient Synchronized Federated Learning
Machine LearningDistributed, Parallel, and Cluster Computing
Summary
Federated learning is a way to teach computers without sharing personal data. One problem is some users take longer to send updates, slowing down the whole learning. Another is that data stored by users can be very different, which makes learning tough. The authors show that picking which users and data to use together helps the learning process go faster and get better results. They tested this idea on a popular image dataset and found it works better than earlier methods.
What this means in practice
- •For mobile app developers: Reduce delays in updating shared machine learning models by selecting the best devices and data for faster training cycles.
- •For edge device engineers: Optimize federated learning on devices with varying compute power by jointly selecting contributing clients and their data to improve overall model accuracy and speed.
Tested on one dataset.
Authors
Muzaffer Citir, Hiroki Nishikawa, Sangyoung Park
Abstract
Federated learning (FL) is a promising paradigm of machine learning, which preserves user privacy by enabling learning without sharing raw data with a cloud server. Straggling clients have been a problem for FL as they introduce delays in aggregating the local models and hence, the convergence of the global model. Therefore, it is important to have a mechanism that ensures fast convergence of the global model as well as good FL participation rate. Another issue for the convergence of a model in FL is the non-independent and identically distributed (non-iid) data across the clients. Prior approaches based on probabilistic client selection do not work well under non-iid data especially when the number of clients is small. We show scenarios where such approaches fail and propose a joint client-training data selection algorithm for fast convergence of FL models. Our experiments on CIFAR-100 dataset show that convergence of the FL model can be significantly improved over prior works that can consider non-iid data and heterogeneous computation and higher model accuracy.