Mental World Modeling
2026-07-29 • Computation and Language
Computation and Language
AI summaryⓘ
The authors propose a new way to model the world called Mental World Modeling (MWM) that includes not just physical information but also what people believe, want, or think. They argue that understanding these hidden mental states is important to predict human actions accurately. They created a system called MENTIS that follows this idea and tested it on stories with text, images, and videos. Their experiments show that tracking mental states helps predict decisions better than just looking at the physical scene. They also identify current challenges in this type of modeling for future work.
world modelsmental statepredictive modelinghuman decision makinglarge language models (LLMs)partial observationstate parsingaction decompositionvalue evaluationsimulation
Authors
Hao Fei, Yiran Zhao
Abstract
World models enable a predictive substrate for planning and action, yet existing formulations merely answer a physical question: what/where it is, and how will it evolve. Human behavior, however, is driven by hidden mental state (what a person believes, wants, intends, feels, and considers socially permissible), so a model that tracks the physical scene but not what each agent knows and believes about it predicts the wrong action for the right-looking scene. We formulate Mental World Modeling (MWM), a generic theoretical framework that makes mental variables core components of a world model rather than posthoc rationales: MWM aintains a coupled physical-mental world state, renders a target-specific partial observation, and simulates how candidate actions jointly update both components. We instantiate the framework in MENTIS, a training-free and fully inspectable baseline that decomposes the process into state parsing, target-observation generation, action decomposition, coupled physical and mental transition, and branch-level value evaluation. On a manually constructed, quality-controlled dataset of situated decision scenarios spanning text, image, and sounding-video stories, experiments with 8 modern LLM-based world models demonstrate that explicitly modeling the mental state is essential for predicting human decisions. Deeper analyses further expose the bottlenecks of current mental world modeling. We expect MWM as a next stage of world modeling, from simulating physical scenes to simulating the minds that act in them.