Evolving world models improve large language agents over time
EvoBelief: Falsifiable World Models for Self-Improving LLM Agents
Artificial Intelligence
Summary
Large language model (LLM) agents often learn to act better from their experiences, but they can struggle if their understanding of how the world works is wrong or incomplete. The authors introduce EvoBelief, a system that helps these agents build and update testable beliefs about their environment as they interact with it. By forming hypotheses about how things should work and checking them with experiments, the agents can revise or discard incorrect beliefs to improve decisions. This approach lets agents get better at tasks without changing their base language model, showing strong results in various simulated environments.
What this means in practice
- •For robotics developers: Enable robots to refine their understanding of changing environments by testing and updating hypotheses about how things work.
- •For game ai programmers: Improve AI agents’ adaptability to unexpected game dynamics by revising outdated rules based on player interactions and environmental feedback.
Authors
Janet Wang, Jiang Guo, Yi Xiang, Jiaqing Yuan, Mingwen Dong, Hideo Kobayashi, Shuaichen Chang, Chandan K. Reddy, Jiarong Jiang
Abstract
Self-evolving LLM agents typically improve how they act through accumulated experience, reflection, or skills. Yet even procedurally capable agents can fail when their beliefs about environment dynamics are incomplete, incorrect, or outdated. We study a complementary axis of self-evolution: evolving those beliefs during deployment. Such beliefs can support multiple tasks, but incorrect ones can also propagate across them. We introduce EvoBelief, a self-evolving world model that represents environment knowledge as falsifiable hypotheses. Unexpected outcomes and decision-relevant unknowns trigger hypothesis formation. Later observations and controlled experiments provide independent validation before a rule can guide predictions or plans within its supported scope. Counterexamples narrow or retire rules as the environment changes. The language model remains frozen, and each rule remains linked to its evidence and the decisions it changes. Across four benchmarks spanning static dynamics, hidden drift, and scientific discovery, EvoBelief improves task performance and reuses learned rules across tasks. On ALFWorld without actor-side transition memory, it improves success over the matched Frozen World Model by 23.9% and 31.3% with Claude Haiku 4.5 and GPT-5.4, respectively. On MiniGrid, it revises stale rules and recovers after unannounced dynamics changes. In DiscoveryWorld's Reactor Lab, it discovers and validates quantitative laws, completing 83.3% of Normal-tier and 58.3% of Challenge-tier instances, versus 0% for each evaluated baseline. These results demonstrate self-improvement through testable, revisable environment knowledge rather than model-parameter updates.