Team members learn to coordinate decisions with delayed shared information
A Decentralized Partially Observable Team Decision Methodology with Delayed Information Sharing
Machine Learning
Summary
When people or agents have to make decisions together but only see partial information and get updates that arrive late, it becomes very tricky to act as a team. The authors studied this problem when the system rules are unknown and complicated but have simple underlying patterns. They showed that each team member can learn a good plan on their own, using just what they observe and the delayed messages from others. This approach means teams can coordinate well without a central leader or pre-shared knowledge, and with guarantees on how much learning it takes.
What this means in practice
- •For autonomous vehicle teams: coordinate driving decisions despite delayed communication and unknown road dynamics using decentralized learning.
- •For industrial robot operators: enable multiple robots to cooperate in complex tasks when they share observations with delay and lack full model knowledge.
Authors
Xiaoxing Ren, Thomas Parisini, Andreas A. Malikopoulos
Abstract
We study decentralized partially observable team decision problems with low-rank latent dynamics and unknown system models. The proposed framework combines team-theoretic equivalence with low-rank model representations to address cooperative decision-making in partially observable Markov decision processes without prior knowledge of the transition model. Each team member makes decisions based on local private information and delayed common information shared across the team. Using only this available information, each member learns an approximate low-rank Markov decision process and applies least-squares value iteration to compute its policy. This yields a fully decentralized learning and planning algorithm that requires neither a centralized coordinator nor centralized training. We show that the resulting member-side solutions approximate the centralized team solution: despite partial observability, unknown dynamics, and delayed common information, each member recovers the corresponding component of an approximate team-optimal policy. We further establish finite-sample performance guarantees and derive a corresponding sample-complexity bound for the proposed algorithm.