Hierarchical agent improves navigation by adapting to opponents
Hierarchical Belief Modeling for Zero-Shot Opponent Adaptation in Partially Observable Multi-Agent Navigation
Artificial IntelligenceMultiagent Systems
Summary
Playing strategy games is hard when you cannot see everything and your opponent changes their play style. The authors created HORIZON, a smart agent that uses layers of thinking to both quickly react and slowly learn about hidden game details and opponents. It remembers important information and explores to learn more, then uses this knowledge to choose the best moves. Tests show HORIZON beats other agents that don’t adapt as well or use simpler memory techniques.
What this means in practice
- •For game ai developers: Create agents that quickly adapt to unknown opponents and changing environments for complex strategy games with hidden information.
- •For robotics engineers: Build robots that navigate partially observable spaces while inferring hidden environmental features and adapting to dynamic conditions.
Authors
Kowei Shih, Lu Cheng, Zeyu Wang, Yeyun Xu, Kejian Tong
Abstract
Lux AI Season 3 requires agents to act under partial observability, randomized episode level dynamics, and a best of five match structure that rewards both tactical execution and fast adaptation. We present HORIZON, a hierarchical agent that combines symmetry aware spatial perception, dual memory belief tracking, relic centric graph attention, information gain driven exploration, and an opponent conditioned policy mixture. HORIZON separates short horizon control from cross match meta reasoning, while auxiliary belief and world model objectives stabilize learning. Trained with PPO in a large scale JAX simulator, the resulting agent explicitly infers hidden game parameters and opponent style. Experiments show consistent gains in match win rate, episode win rate, adaptation gain, and league rating over strong recurrent and feed forward baselines.