Edge network method cuts communication and speeds training with limited devices
Windowed and Quantized Group-Based ADMM for Distributed Optimization in Heterogeneous Edge Networks
Distributed, Parallel, and Cluster Computing
Summary
Training machine learning models over many devices that have different speeds and limited communication is hard. The authors created a method called WQ-GADMM that groups devices based on their speeds and only activates some groups each round. They also reduce communication by sending model information in smaller bits. They tested it on common image datasets and showed it keeps accuracy while using less communication and finishing faster.
What this means in practice
- •For edge network engineers: Coordinate training of machine learning models on diverse edge devices with limited communication capacity to reduce network load and training delay.
- •For mobile app developers: Implement efficient model updates for apps that rely on on-device learning across users with varying device capabilities to save data and improve responsiveness.
Authors
Gaiguo Wei, Qingying Zhang, Heqiang Wang, Yu Zhang, Xiaoxiong Zhong
Abstract
Distributed optimization in edge networks is constrained by heterogeneous client computing capabilities and limited communication resources. We propose the Windowed and Quantized Group-Based Alternating Direction Method of Multipliers (WQ-GADMM) to coordinate group updates under limited activation capacity and reduce communication costs. Clients are grouped by estimated computation time. Each window activates a limited number of groups per round, and the cloud updates the global model after all groups have updated once. The method quantizes both downlink and uplink model exchanges to reduce communication costs and allows bounded model staleness and inexact proximal local updates. For smooth nonconvex objectives, we establish an average squared Karush-Kuhn-Tucker residual bound under the stated assumptions and parameter conditions. The bound consists of a term that decreases with the iteration count and a quantization-dependent error term. Experiments on MNIST and CIFAR-10 show that 12-bit communication reduces communication volume and simulated wall-clock time while maintaining test accuracy comparable to full precision. The 12-bit configuration also maintains complete group coverage and achieves shorter mean group inter-completion gaps than the evaluated baselines.