Performance Evaluation of Social Learning
2026-06-08 • Multiagent Systems
Multiagent SystemsInformation Theory
AI summaryⓘ
The authors study how groups of agents share information to decide which hypothesis is correct based on streaming data. They show that a common way to measure how fast wrong ideas disappear (the rejection rate) can be misleading. Instead, they analyze the chance that agents make mistakes in a scenario with two possible models and normally distributed data. Their formula reveals that both the network connections between agents and prior knowledge affect decision accuracy, causing a lasting difference compared to a centralized approach. This means decentralized groups may never be perfectly as accurate as a single agent with all the data.
Social LearningDecentralized Decision-MakingHypothesis TestingBelief UpdatingRejection RateError ProbabilityBayesian ProbabilityNetwork ConnectivityGaussian ModelsPrior Information
Authors
Felice Scala, Marco Carpentiero, Vincenzo Matta, Ali H. Sayed
Abstract
Social Learning is a decentralized decision-making paradigm in which spatially dispersed agents collect streaming observations regulated by one of a finite number of models (the hypotheses). The agents are interested in assigning probability scores (the beliefs) to the possible hypotheses. To this end, the agents exchange their beliefs according to a certain communication graph. It has been shown that, under reasonable conditions on the identifiability of the decision model and the network connectivity, each agent ultimately places all the belief mass on the true hypothesis governing the data. However, several questions remain unanswered regarding the evaluation of the social learning performance. One recently adopted performance metric is the rejection rate, i.e., the rate at which the beliefs about the erroneous hypotheses vanish. One contribution of this work is to establish that the rejection rate leads to several paradoxes, which make it unsuitable as a valid performance measure. We then focus on studying the error probability measure. For a binary Gaussian problem, we derive an analytical formula characterizing the ratio between the individual agents' probabilities and the optimal Bayesian probability. The formula shows that this ratio is expressed by the product of two terms quantifying the effect of the network connectivity and the role of the prior information. As a result, an irreducible gap emerges between the decentralized and the centralized error probabilities, which is agent-dependent and does not disappear asymptotically.