Decentralised learning improves with lag-aware model weighting
Discount Yourself: Self-inclusive Freshness-weighted Gossip for Decentralised Learning under Client Outages
Machine LearningDistributed, Parallel, and Cluster Computing
Summary
Devices like smartphones often train personalized AI models by sharing with nearby devices instead of a central server, but when a device goes offline, its own model can become outdated. The authors identify that existing methods don’t properly adjust for how stale a device’s own model is after an outage. They propose new rules that treat the returning device’s model like others, measure staleness as inherited delay rather than just age, and strongly reduce the influence of outdated models. Their approach improves accuracy on image tasks and handles real device outages better than previous methods.
What this means in practice
- •For mobile app developers: Improve AI model updates on devices that frequently go offline by better handling of outdated models during training.
- •For edge computing engineers: Enhance decentralised learning systems in networks with intermittent connectivity by applying lag-informed model aggregation rules.
Authors
Qiyuan Wang, Ke Xiao, Christos Anagnostopoulos
Abstract
Decentralised federated learning (DFL) trains personalised models on devices that average with their neighbours rather than with a central server. Real devices, however, are often offline for hours or days, and a client returning from an outage can recover only by aggregating the models its neighbourhood holds. We observe that the stalest of these models is usually the client's own, a case that existing staleness-aware rules, which discount only the models a client receives, do not address. We formalise aggregation after an outage as estimating the state of a fully available network and derive three principles. First, the receiver must be discounted like any neighbour. Every rule that weights it at least as much as each neighbour has an error that grows without bound with the outage, whereas discounting all candidates by relative staleness keeps the error bounded. Second, staleness must be measured as inherited lag rather than age, since local training resets a model's age but not the lag it absorbed from stale neighbours. We introduce the effective lag, which tracks this quantity at the cost of one scalar per message. Third, the discount must be steep, because absorbed lag persists in every later round. These principles define Self-inclusive Freshness-weighted Gossip (SFG), a family of drop-in aggregation rules with the communication and computation of decentralised federated averaging, whose guarantees hold for the stale receiver itself. On three image benchmarks and eight outage processes, including real smartphone availability traces, the lag-based members SFG-L and SFG-Cut consistently outperform decentralised federated averaging and recent age-of-information rules, and ablations show that each principle is necessary. The analysis also explains why tuned limited merging from gossip learning is effective.