Language models struggle to use beliefs in task decisions
From Belief Inference to Task Action: A Multi-Level Evaluation of Theory of Mind on Verifiable Tasks
Computation and Language
Summary
Working well together means understanding what others know and believe, then acting based on that. Many tests check if AI models can guess others’ thoughts, but guessing right isn’t the same as using those guesses to make good choices. The paper’s authors made tasks where AI must not only figure out others’ beliefs but also act correctly afterward. They found models often know what others believe but still make plans that don't fully match those beliefs, showing a big gap between understanding and doing.
What this means in practice
- •For ai system designers: Improve AI assistants to better use inferred beliefs when planning tasks in collaborative settings with differing user knowledge.
- •For chatbot developers: Design chatbots that recognize and adapt to users’ knowledge gaps, enhancing collaborative communication and decision-making accuracy.
Authors
Zhenshuai Yin, Yongxin Guo, Mingda Zhang, Zhongze Luo, Haoyue Liu, Lianmin Zhang, Xiaoying Tang
Abstract
Effective collaboration requires language models to understand what others know and believe, and to act accordingly. Models that assume others share their information may omit necessary communication and undermine coordination. Many theory-of-mind (ToM) benchmarks assess mental-state inference through story questions, but correct answers do not establish that models can use these judgments in subsequent tasks. We introduce a multi-level evaluation built on existing, verifiable tasks. We extend household collaboration and calendar scheduling with asymmetric information while retaining their original constraints and validators. Independent interfaces assess world understanding, information access, beliefs, decisions, and execution. Controls supply personal evidence or correct beliefs to distinguish state recovery from state use. Across six model configurations, these abilities diverge. On matched scheduling cases, Claude recovers all agent beliefs correctly but produces subjectively optimal plans in only 67.9% of cases. In household collaboration, supplying beliefs raises Qwen-27B success from 55.0% to 75.0%, yet Qwen-4B answers every fixed supplied-belief source question correctly while completing only 20.0% of the corresponding continuation condition. These results show why ToM evaluation must connect perspective inference to independently verified decisions and task consequences.