Decentralized learning boosts predictions by sharing expertise locally
Learning to Refer: Inference-Time Collaboration in Decentralized Federated Learning
Machine Learning
Summary
When many computers (clients) learn together without a central boss, they often only use their own knowledge. The authors show a way for these computers to ask their neighbors for help during predictions, moving requests step-by-step through trusted connections. This collaboration lets them combine their unique strengths without needing central control. Their method improves accuracy, especially when the network connections between clients are few.
What this means in practice
- •For medical data teams: Enable medical institutions with limited connectivity to share model insights locally for better disease prediction without central data pooling.
- •For edge computing engineers: Improve edge device predictions by routing queries efficiently through neighboring devices, boosting accuracy while respecting connectivity limits.
Authors
Yuheng Ma, Chendi Wang, Qiong Zhang
Abstract
In decentralized federated learning, heterogeneous clients may develop complementary expertise, yet each query is typically evaluated only by the client where it arrives. To exploit this distributed expertise without global coordination, we formulate inference-time collaboration as budgeted local expert referral, a graph-constrained routing process in which a query can move only between neighboring clients and only visited clients contribute to its final prediction. EdgeDRM implements this framework by learning a density-based routing potential for each client: differences between neighboring potentials provide the comparisons used to select successive referrals, while local task models generate predictions that are aggregated along the resulting route. Both training and inference are decentralized, using only neighbor-local information without a central router or graph-wide client scoring. We establish estimation guarantees for the learned comparisons and characterize how local improvement and graph structure govern prior-weighted density gains from referral. Experiments on benchmark and real-world medical data show that EdgeDRM outperforms graph-matched decentralized baselines in most settings, with the clearest benefits under sparse connectivity.