Constrained multi-agent learning improves cost prediction efficiency
Constrained Multi-Agent Reinforcement Learning in Imagination
Artificial Intelligence
Summary
Making decisions with multiple agents working together can be tricky when there are limits or costs to consider. The authors show that using a model to imagine possible outcomes helps learn better strategies while respecting these limits. Their method, called CA-MAMBA, uses far fewer real interactions and achieves better rewards compared to other standard methods. They also tested how accurately the model estimates costs and found it is not the biggest challenge, meaning other improvements can focus elsewhere.
What this means in practice
- •For robotics engineers: Develop multi-robot coordination strategies that efficiently respect operational cost limits using imagination-based learning.
- •For game ai developers: Create smarter multi-agent game characters that learn to optimize rewards while managing constraints with fewer real interactions.
Authors
Raz Shmueli, Shahaf S. Shperberg
Abstract
Predicting cost in a multi-agent world model moves most constrained policy updates into imagination: the stronger model-free baseline needs $10$ to $20$ times CA-MAMBA's interactions on the median SMAC configuration, and CA-MAMBA earns $2.8$ to $12$ times the baselines' MAMuJoCo reward at equal budget. In controlled tests on one task per benchmark, pricing the model's cost advantages at parity more than halves a cost-blind learner's cost and meets a tight continuous-control limit the learned multiplier misses, suggesting the model's cost estimates are not the bottleneck.