Framework enables safe learning in cooperative multi-agent tasks
Learning Dynamic Games for Cooperative Iterative Tasks
Robotics
Summary
Cooperative tasks where many agents interact can be tricky because each agent's decisions affect the others. The authors present a learning method that uses past successful team actions to guide future moves, making sure all agents keep improving or at least do not get worse. Their method includes rules that agents agree on to decide which new solutions to try. It works with complex situations where the game is nonlinear and players have shared limits to respect.
What this means in practice
- •For autonomous vehicle teams: Coordinate multi-vehicle routes that improve over time while respecting shared safety limits.
- •For industrial robot operators: Enable groups of robots to iteratively learn cooperative tasks with guaranteed safety and performance non-degradation.
Authors
Mark Pustilnik, Francesco Borrelli
Abstract
We consider cooperative and iterative tasks involving multiple agents whose interactions are modeled as a dynamic game with shared constraints. We propose a general learning framework applicable to nonlinear and nonconvex dynamic games that uses previously executed feasible trajectories to construct a sampled joint safe set and player-specific cost-to-go functions. At each time step, the agents jointly select a previously sampled terminal state and a Generalized Nash equilibrium used to reach it. A new solution is adopted only if it satisfies a cooperative acceptance rule agreed upon by the players. The framework accommodates individual non-degradation requirements and selection criteria such as weighted costs and Nash bargaining. We show that the proposed framework guarantees recursive feasibility and characterize conditions under which each player's cost is non-increasing across iterations.