Federated learning framework protects data from gradient attacks
A Novel Federated Learning Framework for Defending Against Gradient Inversion Attacks
Machine Learning
Summary
Federated learning allows multiple people to teach a computer model together without sharing their private data. However, hackers can sometimes guess personal data from information shared during this process. The authors developed a new method called HyperFL++ that changes how the model is shared so that private data is harder to extract. They use special techniques to keep this method efficient while still keeping models accurate. Their experiments show that this approach keeps data more private without making the model worse.
What this means in practice
- •For machine learning engineers: Use HyperFL++ to build federated learning systems that better protect client data from gradient inversion attacks without sacrificing model accuracy.
- •For cloud service providers: Incorporate HyperFL++ techniques to offer more secure federated learning services that reduce risks of private data leaks during model training.$Commercial implications: Enables secure federated learning offerings to enterprise clients concerned about data privacy.
Authors
Pengxin Guo, Ziying Huang, Shuang Zeng, Xiaodan Zhang, Weihong Ren, Yuyin Zhou, Liangqiong Qu
Abstract
Federated Learning (FL) offers a paradigm for collaborative machine learning model training among clients, preserving data privacy by avoiding direct data sharing. However, the gradient information exchanged in FL has been shown to be vulnerable to Gradient Inversion Attacks (GIA), which can reconstruct sensitive user data. To counter these threats, various privacy-enhancing technologies like Secure Multi-party Computing (SMC), Homomorphic Encryption (HE), and Differential Privacy (DP) have been integrated into FL. While effective, these methods often introduce significant trade-offs between privacy and model utility. By revisiting the key to privacy exposure in FL under GIA, which lies in the frequent sharing of model gradients that contain private data, we take a new perspective by designing a novel privacy-preserving FL framework that effectively ``breaks the direct connection'' between the shared parameters and the local private data to defend against GIA. Our proposed framework, HyperFL++, leverages hypernetworks to generate local model parameters, where only the hypernetwork parameters are aggregated by the server. To ensure practicality, HyperFL++ incorporates a Row-wise Kronecker Decomposition (RKD) to mitigate the memory and communication overhead from the hypernetwork. Comprehensive theoretical analysis and extensive experiments validate our method's effectiveness in protecting data privacy while maintaining high model utility. Our code is available at https://github.com/Pengxin-Guo/HyperFL/tree/hyperfl%2B%2B.