Cooperation and coordination improve AI object recognition with limited messages
The Tower of Babel in artificial intelligence: who should speak, and when it pays to listen
Machine LearningMultiagent Systems
Summary
Sometimes two AI systems that each recognize parts of an object can still fail when working together. The authors study how these systems can send short, efficient messages to a single receiver that identifies the object class. They find that designing messages together leads to much better accuracy than designing them separately, especially when communication is limited. Teams work best when each system shares unique evidence, not overlapping information. The effectiveness depends on the specific pair of systems, so who should communicate must be tested in pairs.
What this means in practice
- •For sensor network engineers: Improve multi-sensor object detection by designing communication protocols that maximize distinct shared information within limited bandwidth.
- •For mobile device developers: Enable devices with limited communication capacity to collaborate on object recognition tasks more accurately by coordinating message content.
Authors
Mohammad Salahshour
Abstract
Two machines can each recognize an object well and still fail together. We study classifiers that see different parts of one object and send short messages to a receiver that identifies its class. Coordination asks how to design messages together; cooperation asks whether the team outperforms one sender given the whole budget. On three digit datasets, messages designed together beat messages designed apart by 18.07 to 22.68 percentage points on average with two-bit messages, with gains in all 24 configurations. A fixed agreement, you report the row and I the column, recovers about half this gain, with only the receiver learned. Senders that draw the same distinctions waste a narrow vocabulary, and senders whose distinctions cross multiply it. A team has no more possible messages than one sender with the same budget, so a partner must bring evidence the first lacks. A solvable model shows individual statistics alone cannot determine whether a partnership helps. On MNIST digits, six shared bits remove 49.54 percent of the strongest tested solo's errors on average. Paired measurements correctly predicted most sensor teams would reduce average accuracy on new participants. Who should speak is a property of the pair, and must be measured in pairs.