Split computing head networks personalized with federated learning
Unsupervised Head Network Personalization for Split Computing via Federated Learning
Machine LearningDistributed, Parallel, and Cluster ComputingNetworking and Internet Architecture
Summary
Running smart apps on phones can be slow and drain battery because phones can't do all the heavy work alone. The authors figured out how to split this work between phones and servers more efficiently by training a small piece of the app on each phone without needing lots of private examples. They used a teamwork method where phones learn together without sharing private data and made each phone's version special for it. This way, the system stays fast and private, using less data and less communication but still works almost as well as the original.
What this means in practice
- •For mobile app developers: Enable apps on phones to efficiently split AI tasks with servers by training lightweight models on-device using limited unlabeled private data.
- •For edge computing teams: Deploy personalized AI models at the network edge that respect user privacy and reduce communication costs when collaborating with central servers.
Authors
Yutaro Horikawa, Takayuki Nishio
Abstract
Split Computing (SC) is a distributed inference method designed for balancing computational load and reducing latency by partitioning a neural network into head and tail networks and deploying them across both a mobile device and a server, respectively. Due to the limited bandwidth and computational power of mobile devices, the head network is often replaced with a neural network that requires lower computational and communication resources. However, it is challenging to prepare a sufficient quantity and quality of labeled data on the mobile device side for training the replacement head network. Furthermore, the data prepared by mobile users often contains sensitive privacy information, making data aggregation difficult. In this paper, we propose a data-efficient head network training method for SC. This method utilizes federated learning and knowledge distillation to train the head network using a small amount of non-IID and unlabeled data from mobile devices, ensuring that the training datasets are not exposed. Moreover, by introducing personalization, we enhance the accuracy to levels comparable to or exceeding those of existing methods. The experimental evaluations demonstrate that, despite data constraints imposed only on our method, it achieves accuracy comparable to the baselines. Furthermore, it significantly reduces the communication requirements by 95% on mobile devices, with only a 3-5 percentage point drop in accuracy from the original model.