Optimal peer choice method reduces communication in federated learning
SAGE: Optimal-Stopping Peer Selection for Decentralised Federated Learning
Distributed, Parallel, and Cluster ComputingMachine Learning
Summary
In some types of machine learning, many devices work together without a central boss, sharing updates with random partners. Picking the best partners is tricky and can waste effort if you always probe a fixed number. The authors propose SAGE, a method that decides when enough proof exists to pick a partner and when to stop probing without a costly process. This method avoids picking worse partners than random choice, reduces communication, and works well across different tests.
What this means in practice
- •For distributed machine learning teams: Efficiently select collaborator devices in peer-to-peer learning to reduce communication while maintaining model quality.
- •For network protocol engineers: Design distributed protocols that minimize message exchange by stopping early when further probing is not cost-effective.
Authors
Ke Xiao, Qiyuan Wang, Christos Anagnostopoulos
Abstract
Decentralised federated learning replaces server aggregation with peer-to-peer model exchange, making collaborator selection a local decision under uncertainty. Fixed probe budgets waste effort on easy choices yet fall short when peers are hard to distinguish. We propose SAGE (Sequential Anchor-Gated Exchange), an optimal-stopping peer selector under a one-model-bearing-exchange budget. A receiver scores candidate neighbours on receiver-owned anchor evidence and selects once an advantage is certified. It continues probing only while further evidence repays its cost, and otherwise falls back to random gossip. We show that the stopping problem admits an optimal rule attained at a finite stage, and that the anchor schedule is order-optimal in the peer-risk gap and the confidence level. We further show that the selector never returns a peer worse than random gossip with high probability, and prove that no such guarantee holds for selectors that commit without a certificate. A separability threshold follows, below which no probing budget improves on gossip. Experiments span two image benchmarks, two graph families and three heterogeneity levels. Selectors that always act on their evidence lose to gossip in every configuration tested. SAGE-OS matches gossip on 75.5% less evidence than a fixed budget, at half the communication overhead of two published selectors. The operative decision is not which peer to rank first, but whether the evidence justifies ranking at all.