Small language models improve answers by talking to each other
Reinforcement Learning of Communication in a Mesh of Small Language Models
Machine Learning
Summary
Usually, language models try to get better answers by making many guesses and choosing the most common one, but this isn’t always helpful. The authors developed TalkMesh, where several small language models communicate by sharing their best guesses and hints to improve overall decisions. Each model decides how confident it is and can revise its answer based on hints from others, working together without a single leader. This method lets a few small models perform as well as many guesses from one model, even when some try to mislead others.
What this means in practice
- •For software engineers: Improve accuracy in AI decision systems by enabling small language model agents to communicate and revise their answers without centralized control.
- •For traffic management teams: Enhance automated traffic signal control by sharing information between agents to improve coordinated decisions when individual views are limited.
Authors
Mehmet Kerem Turkcan
Abstract
Language models gain accuracy from more compute at test time, but majority voting over independent samples saturates: as samples grow, the vote converges to the model's most frequent answer. Communication can add what sampling cannot: an agent that solves a problem can pass the key step to the others. We present TalkMesh, a decentralized mesh of small language model agents that learns when and what to communicate. Each agent samples a proposal and scores it with a trained confidence head. The most confident agent broadcasts a hint; agents below a confidence threshold revise, keeping each revision that outscores its proposal. Gossip consensus approximates the vote weighted by confidence without a coordinator. A talk policy, trained with group relative policy optimization on the change in correctness after revision, writes hints and revisions. With three agents, which together generate at most six outputs, the mesh reaches the accuracy of majority voting over 32 samples with each of three models. Trained with at most 8 agents and evaluated with 32, it raises accuracy from 0.568 under self-consistency to 0.705 (Qwen3.5-0.8B, GSM8K) and from 0.492 to 0.722 (SmolLM3-3B, MATH-500). When 4 of 8 agents collude on a wrong answer with fabricated confidence and poisoned hints, majority vote accuracy falls to 0.000 (Qwen3.5-0.8B, GSM8K). A defended mesh, whose agents rescore solutions with their own confidence heads, retains 0.507. Across reasoning, embodied coordination, and traffic signal control, messages improve a decision when the acting agent cannot observe the information it requires and another agent can send it.