Parallel optimization method speeds up learning in edge networks
P-GADMM: Parallel Group-Based ADMM for Asynchronous Optimization in Heterogeneous Edge Networks
Distributed, Parallel, and Cluster Computing
Summary
In networks where many devices work together to learn from data, slower devices can hold up the whole process. The authors created a new method called P-GADMM that groups devices based on their speed and data size, so faster groups can update more quickly without waiting for the slowest ones. This method uses a smart way to combine results with some delay allowed, keeping the learning accurate while saving time. Tests showed it finishes training faster than other methods with similar results.
What this means in practice
- •For edge network operators: Improve distributed learning tasks by managing device groups based on compute power to reduce training delays.
- •For cloud platform engineers: Implement asynchronous aggregation schemes that allow fast groups to update models without waiting for slow groups.
Authors
Gaiguo Wei, Qingying Zhang, Heqiang Wang, Yu Zhang, Xiaoxiong Zhong
Abstract
The Alternating Direction Method of Multipliers (ADMM) is widely used for distributed optimization, but its synchronous implementation can suffer from efficiency loss in heterogeneous edge networks, where fast clients or groups need to wait for slower ones before global updates can be completed. Existing group-based ADMM methods reduce communication overhead through grouping, but their grouping rules usually focus on data similarity or network topology and do not explicitly account for computation heterogeneity. To address this issue, this paper proposes Parallel Group-Based ADMM (P-GADMM) for distributed optimization in heterogeneous edge networks. P-GADMM forms computation-aware edge groups according to client computational capabilities and local data sizes, which reduces training-speed variation within each group. It further combines edge-level aggregation with bounded asynchronous coordination at the cloud, allowing active groups to participate in global updates without waiting for slower groups while controlling stale group information through a delay threshold. For strongly convex group objectives, we establish convergence guarantees for an idealized form of P-GADMM under bounded group-level staleness, showing a time-averaged convergence behavior up to a staleness-induced asymptotic error neighborhood. Experiments show that P-GADMM reduces wall-clock training time compared with representative baselines while maintaining comparable final accuracy.