Consensus approach improves learning across devices with different big data
Consensus-based Decentralized Distributed Swarm Learning with Heterogeneous Big Data
Distributed, Parallel, and Cluster Computing
Summary
Making smart decisions on many devices at the edge of a network is hard because the data they see can be very different and the network can be complex. The authors created a new way for these devices to work together without sharing all their raw data, using ideas from how groups of birds or fish move together. Their approach helps the devices share knowledge and learn better models faster by balancing local experience with input from neighbors. Tests show this method works well even when data is very diverse and the learning tasks are difficult.
What this means in practice
- •For edge network engineers: Coordinate machine learning across many edge devices without central servers despite varying local data and network limits.
- •For iot system developers: Build robust decentralized AI models on Internet of Things devices that must learn collaboratively but have different types and amounts of data.
Authors
Zhuoyu Yao, Dong Yang, Yue Wang, Songyang Zhang, Yingshu Li, Zhi Tian, Zhipeng Cai
Abstract
Artificial intelligence increasingly relies on large-scale, distributed, and heterogeneous data collected by edge devices. However, the practice of edge intelligence remains challenging due to non-convex objectives, data heterogeneity, and complex wireless network topology. To address these issues, this paper proposes a consensus-based decentralized distributed swarm learning (CD-DSL) framework for wireless edge networks. Our CD-DSL integrates consensus optimization with particle swarm optimization (PSO), by reaching the model consensus among neighboring devices while leveraging the PSO exploration and exploitation. The consensus mechanism supports decentralized coordination without raw-data exchange, while PSO-inspired updates utilize historical and neighbor-shared experience to enhance exploration for non-convex optimization, improve robustness to data heterogeneity, and accelerate convergence. We further develop an adaptive neighbor-mixing strategy that learns performance-aware consensus weights, improving decentralized collaboration among heterogeneous edge devices. Theoretical analysis establishes that CD-DSL maintains participant consistency and achieves non-ergodic convergence to a neighborhood of a stationary point under non-convex objectives. Experimental results show that CD-DSL can mitigate the performance degeneration of existing decentralized baselines caused by heterogeneous data.