Privacy aligned personalized federated learning cuts communication needs
Privacy-Aligned Personalized Federated Learning with Compact Adaptation and Variable-Length Gaussian Communication
Machine Learning
Summary
Federated learning lets many devices train a shared model without sharing their private data, but keeping this training private can be tricky when updates are very detailed. The authors tackle this by sending a single private summary for each device, then letting the device adjust only a small part repeatedly. They also cleverly use a method that combines privacy protection and data compression, so devices send less data but stay private. Their approach works well on common image tasks and cuts communication needs by over two and a half times without losing accuracy.
What this means in practice
- •For mobile app developers: Enable privacy-protecting personalized updates in federated learning apps with lower uplink communication costs.
- •For iot device manufacturers: Implement efficient privacy-preserving adaptation on resource-constrained IoT devices with reduced data transmission.
Authors
Yilin Xu, Chun Hei Michael Shiu, Chih Wei Ling, Linqi Song
Abstract
Record-level differential privacy exposes a structural misalignment in personalized federated learning when client-specific variation is low-dimensional while training repeatedly releases high-dimensional updates. In this paper, we address this misalignment by releasing a private client context once and confining repeated adaptation to a fixed coefficient space. Beyond dimensionality reduction, the factorized generator induces an adaptive optimization geometry that reshapes noisy updates, and controlled ablations show that most of its private-training gain is retained by radial evolution. To further reduce the communication cost, we realize the Gaussian mechanism for coefficient updates directly through variable-length quantization with finite expected code length, so that the quantization error itself serves as the required privacy perturbation rather than extra distortion. Across MNIST and CIFAR-10, our design matches or outperforms full-model private adaptation across privacy budgets and client heterogeneity, while reducing protected uplink by a factor of 2.67 at \(\varepsilon=16\) on CIFAR-10 with comparable future-client accuracy.