Benchmark evaluates building code based 3d worlds with physics
ExeWorldBench: Benchmarking Code-Based Executable 3D World Modeling
Computer Vision and Pattern Recognition
Summary
It is hard to teach computers to not just draw 3D worlds, but also to build working models that behave like the real world. The authors created ExeWorldBench, a benchmark to test how well computers understand and build 3D worlds that follow physical rules and cause and effect. It has three tasks, getting harder from building simple mechanisms to creating full systems and worlds from goals. They tested current models and found many challenges remain in making computers build physically correct and causal 3D worlds.
What this means in practice
- •For game developers: Create game environments with physically accurate and interactable 3D mechanisms that respond realistically to player actions.
- •For robotics programmers: Build and test virtual robot environments where components behave according to physical and causal principles, improving simulation fidelity.
Authors
Xinge Peng, Zhen Fang, Ruibing Wang, Yongxuan Lv, Haodong Li, Guohui Zhang, Xiaoxiao Ma, Yiting Lu, Lin Chen, Zehui Che, Feng Zhao
Abstract
Coding agents are increasingly evolving beyond reproducing what a world looks like toward constructing executable worlds that capture how a world works. This shift calls for executable world modeling, which integrates two core capabilities: world understanding, reasoning about the underlying structural, mechanical, and causal principles of a world, and executable realization, translating that understanding into a code-based system whose behavior emerges from physical interactions. However, existing benchmarks provide only fragmented evaluations centered on physical reasoning, visual fidelity, or behavioral correctness, leaving the integration of these capabilities insufficiently assessed. To bridge this gap, we introduce ExeWorldBench, to our knowledge the first benchmark to comprehensively evaluate world understanding and executable realization through code-based 3D world construction. ExeWorldBench comprises three progressively challenging tasks: Mechanism Executability, which evaluates the realization of individual mechanisms from visual observations; Compositional Executability, which extends to multi-stage systems with spatial, mechanical, and causal dependencies; and Intent-Driven Executable World Synthesis, which requires constructing executable worlds from high-level goals and physical requirements. Extensive experiments on frontier models reveal substantial limitations in executable world modeling, offering insights into the development of more physically grounded and causally coherent world construction systems.