Accuracy alone fails to ensure teamwork success in AI systems
Smart Alone, Blind Together: Why Accuracy Fails to CertifyTheory of Mind in Multi-Agent Teams
Artificial IntelligenceComputation and LanguageMachine LearningMultiagent Systems
Summary
When AI systems try to work together by guessing what their teammates will do, just being accurate in these guesses isn’t enough to guarantee good teamwork. The authors show that mistakes aren’t independent and can either help or hurt teamwork depending on how errors overlap or combine. They tested this with different games and AI models, finding that accuracy can both approve harmful teamwork or reject helpful teamwork. Instead, they suggest a new method called the Blind-Spot Gate to better predict whether a team of AI agents will work well without needing the system to be fully tested in action.
What this means in practice
- •For ai system designers: Assess multi-agent teamwork modules more reliably by evaluating error patterns and dependencies rather than accuracy alone to avoid harmful deployments.
- •For game ai developers: Improve AI coordination in team-based games like task assignment and Hanabi by applying the Blind-Spot Gate to certify teamwork performance before deployment.
Authors
Will Beaumaster, Paul Schrater
Abstract
Coordinators that act on what they infer about teammates are usually certified by the accuracy of those inferences. We argue that accuracy is the wrong certificate, for two reasons. First, the accuracy needed to act depends on what a wrong action costs, so the bar belongs to the deployment, not the module. Second, a module's errors are rarely independent, and dependence has a sign: positively associated errors help where their damage overlaps and hurt where it compounds, so an accuracy threshold fails in a predictable direction, and once wrong beliefs interact and can do harm, no rule based on accuracy is right for every module. We test these predictions in task assignment, Level-Based Foraging and Hanabi, with injected errors, trained and Bayesian modules and five language models, and in human judgments. Accuracy fails in both directions, approving harmful language-model deployments in Hanabi, where shared errors compound, and rejecting beneficial ones in task assignment, where costly errors are rare. In our tasks it also buys far less redundancy than independence predicts: from GPT-5.4-mini to Claude Opus 5.5, error on teammates the models have evidence about falls from 32% to 11%, but the chance that all three observers misread one falls only from 16% to 9% (independence: 3.4% to 0.15%). We propose certifying the deployment, with a Blind-Spot Gate that prices error costs, pairwise interactions and measured dependence without deploying the module. On held-out data from the cells it was developed on, it makes no significant false approval in 20 splits (accuracy: 3.2 per split), and in preregistered tests on modules it was not developed on, it recovers what accuracy rejects (seven classifiers: regret 0.004 against 0.479) and catches what accuracy approves (a Bayesian module: significant false approvals in 17 of 20 splits for accuracy, 1 for the gate).