Efficient decentralized learning reduces communication and protects privacy
SPADE-DFL: Communication-Efficient Decentralized Federated Learning via Derivative-Free Linearized ADMM
Machine LearningNetworking and Internet Architecture
Summary
Training AI models can be slow and costly when devices have to talk to each other too often. The authors developed a new method called SPADE-DFL that lets devices work more on their own before sharing updates, which cuts down the communication needed without slowing learning progress. This method also keeps sensitive information private while the devices exchange data. Tests show it learns better than other similar systems on different classification tasks.
What this means in practice
- •For mobile app developers: Build AI models across many devices with less network data and ensure user data privacy without slowing down training.
- •For edge computing engineers: Deploy decentralized learning on distributed devices with lower communication costs and preserved data privacy.
Authors
Mengli Wei, Mengkai Zhu, Jiawen Chen, Wenwu Yu, Duxin Che
Abstract
Reducing communication in derivative-free decentralized learning requires controlling the disagreement accumulated over multiple local updates. This paper develops SPADE-DFL, a primal--dual method that allows the number of local function-value updates between neighbor exchanges to grow with the computation budget while preserving the nonprivate convergence order. For smooth nonconvex objectives under uniform query-moment bounds, the prescribed nonprivate schedule achieves a time-averaged stationarity and consensus bound of $\mathcal{O}(T^{-1/3})$ using only $Θ(T^{2/3})$ communication rounds, where $T$ is the number of local updates per client. For private training, the accumulated data-dependent increment is isolated from the graph correction, allowing one protected state per client and round to generate all outgoing messages. We prove client-level differential privacy for the full interactive transcript and quantify the resulting optimization error over a finite horizon. Experiments on four classification tasks show that SPADE-DFL achieves higher mean test accuracy than existing decentralized learning methods.