Belief conditioned method improves decentralized large language model teamwork
Belief-Conditioned Multi-Agent Actor Critic for Decentralized LLM Collaboration
Artificial IntelligenceMachine LearningMultiagent Systems
Summary
Working together with large language models (LLMs) can be tough when each model acts independently and communicates little with others. This paper presents a new training method called BCMAAC that helps each model keep track of beliefs about the shared situation, improving how they guess useful information from limited messages. The authors tested BCMAAC on tasks like playing Minecraft, searching the web, and using tools, where it led to better teamwork and higher success, especially in complex, multi-step tasks. This shows that teaching models to maintain and use 'beliefs' about their context helps coordination without constant communication.
What this means in practice
- •For game developers: Train independent AI agents to collaborate better in complex game environments like Minecraft through belief-based coordination.
- •For software engineers: Improve tool-using chatbots and web search agents by enabling decentralized models to infer shared context and collaborate efficiently.
Authors
Tianle Chen, Shuo Liu, Christopher Amato
Abstract
Recent work has explored optimizing LLM collaboration through Multi-Agent Reinforcement Learning (MARL), but many approaches still rely on predefined workflows, limiting efficiency and scalability. Decentralized LLM collaboration is challenging because agents execute independently with limited communication, share a joint objective, and must infer useful information from communication traces. In this paper, we propose BCMAAC, a Belief-Conditioned Multi-Agent Actor-Critic method that maintains beliefs over contextual information and uses them to facilitate training. Experiments on Minecraft game-playing, web searching and tool-using tasks show that BCMAAC improves task performance and collaborative behavior compared to naive actor-critic baselines. The improvements are particularly pronounced in multi-turn tasks with richer contextual information, where belief modeling provides a clear advantage.