Agents share cost control to improve group outcomes in multi-agent systems
Who Bears the Burden? Learning Responsibility for Shared Constraints in Multi-Agent Reinforcement Learning
Machine LearningArtificial IntelligenceComputer Science and Game TheoryMultiagent Systems
Summary
When several agents share a limited resource or budget, deciding who should be responsible for keeping costs within limits can be tricky. The authors introduce a way for the system to learn how much each agent should bear the penalty for overspending during training. This method helps groups of agents work together better without changing their goals or rules. Their approach improved overall performance and kept costs under control in various simulated environments with many agents.
What this means in practice
- •For traffic system planners: Manage city-wide energy use among multiple control agents to improve overall efficiency while respecting energy budgets.
- •For fleet management teams: Coordinate autonomous vehicles to share driving cost constraints more fairly and achieve better fleet-wide performance.
Tested on simulated data.
Authors
Xiaoyang Cao, Jingqi Li, Zhe Fu, Alexandre M. Bayen
Abstract
When multiple agents share a cost budget, a common Lagrange multiplier can enforce the aggregate constraint but does not determine how its penalty should be allocated across agents. Uniform penalties ignore heterogeneity in the rewards agents sacrifice, while agent-specific multipliers may still rely on the same aggregate cost signal. We introduce Lagrangian Responsibility Allocation (LiRA), which learns each agent's share of a common multiplier by optimizing social welfare over a finite training horizon. The multiplier enforces the aggregate budget, while responsibility shares redistribute its influence without modifying the original rewards or constraints. For convex games under standard regularity conditions, varying these shares induces a smooth family of normalized generalized Nash equilibria in which active constraints remain at their budgets while welfare varies. To optimize responsibility before convergence, we derive a welfare gradient that accounts for both learning updates and the induced change in data distribution. Across CityLearn, MABIM, Harvest, and MetaDrive, spanning 3 to 400 agents, LiRA improves average social welfare by up to 29% over uniform and agent-specific multiplier baselines. Grid and driving costs remain within budget, inventory violations decrease, and Harvest makes more effective use of available budget.