Bridging the Agent-World Gap: Text World Models for LLM-based Agents
2026-06-08 • Computation and Language
Computation and Language
AI summaryⓘ
The authors discuss how language-based AI agents often just react to text without understanding how the environment changes over time. They introduce text world models, which predict what happens next in a text-based setting when an agent takes an action, helping with planning and learning. The paper reviews how these models are defined, built, used, and evaluated, aiming to organize current knowledge and identify future challenges. Their work helps clarify how to make AI agents better at handling interactive text environments.
Large language modelsText world modelsInteractive textual environmentsState representationAgent planningExperience synthesisModel evaluationTransition modelsEnvironment grounding
Authors
Yixia Li, Hongru Wang, Peng Lai, Zhiwen Ruan, He Zhu, Youxin Zhu, Ganlong Zhao, Minda Hu, Yun Chen, Sibei Yang, Peng Li, Jeff Z. Pan, Jia Pan, Guanhua Chen, Yang Liu, Guanbin Li
Abstract
Large language model (LLM)-based agents are increasingly used in interactive textual environments, from web navigation and code editing to tool use and long-horizon dialogue. Yet many remain largely reactive, mapping observations to actions without an explicit model of how these environments are structured and evolve. This motivates text world models (TWMs): transition models over textual states that, given a state and a candidate action, predict the resulting webpage, terminal output, API response, or user reply, thereby supporting planning, efficient learning, and principled evaluation. We systematically review text world models for LLM-based agents, organized around a formal framework and the agent lifecycle: (1) Foundations, defining text world models and characterizing them by state representation and grounding domain; (2) Construction, taxonomizing LLM-as-WM and code-as-WM paradigms and reviewing methods for building them; (3) Application, examining how world models support agents at training time through experience synthesis and at inference time through planning, verification, and adaptation; and (4) Evaluation, covering both evaluation of the world model itself and its use as an evaluation environment for agents. We aim to consolidate this rapidly developing area, clarify its design space, and highlight open challenges for future research.