Multi timescale model cuts error in game decision making
MHWA: Multi-timescale Hierarchical World-Action Model
Artificial Intelligence
Summary
Making computer programs that plan ahead can be hard because small mistakes add up over time, leading to bad decisions. The authors found a way to reduce these mistakes by building a model that looks at the game in steps that move at different speeds, updating some parts less often. Their model also switches between different simpler models based on context, which helps it stay accurate and flexible. Tests showed this method performed better than similar models using less data and computing power.
What this means in practice
- •For game ai developers: Improve decision making in simulated environments by reducing error drift with hierarchical multi-timescale models using less compute.
- •For robotics control engineers: Design controllers that predict and plan actions more reliably over time by applying hierarchical models that adjust context update rates dynamically.
Authors
Pengcheng Pan, Guoqing Ma, Yuhan Zhang, Yang Chen, Yichen Liu, Ziheng Li, Shan Yu
Abstract
Autoregressive world models have emerged as a powerful paradigm for sample efficient decision making, yet their open loop rollouts often exhibit increasing drift that leads to error accumulation even within a relatively small planning horizon. Hierarchical temporal abstraction is a natural candidate to mitigate such drift by reducing the effective prediction horizon; however, existing hierarchical designs can introduce added structural and conceptual complexity, sometimes making them difficult to control and limiting gains over flat baselines. To this end, we introduce MHWA (Multi-timescale Hierarchical World-Action model), which employs a strided high level context channel to reduce global context update frequency, effectively curbing the propagation of compounding drift. Crucially, via context aware routing, the gating network dynamically reconfigures a low level mixture of experts by using both global context and current observations, thereby enabling flexible high-low coordination, helping alleviate interference commonly observed in static hierarchies. Empirically, using RLiable's seed-by-game aggregation, MHWA (77M parameters) achieves an IQM human normalized score (HNS) of 0.884 across three training seeds on a 14-game Atari subset using 10% subsampled offline data, outperforming a matched JOWA-70M baseline (0.529) and achieving numerically comparable performance to JOWA-150M while using nearly half the parameters and 70.88% fewer training FLOPs per sequence.