GameDirector separates gameplay rules from rendering for flexible gaming
GameDirector: Decoupling Gameplay Logic from Rendering for Player-Configurable Game World Models
Computer Vision and Pattern Recognition
Summary
Today’s video game models try to learn how game worlds behave all at once, including how things look and how they act, but this often misses important game rules like health or combat. The authors present GameDirector, a new system that splits the game’s rules and logic from the visuals. It manages the game state and enforces rules like a traditional game engine, then tells a separate visual model how to show what’s happening. This method helps players customize and control the game more like developers do, while keeping the game behaving correctly.
What this means in practice
- •For video game developers: Create games where mechanics and visuals are controlled separately, enabling player customization with reliable game rule enforcement.
- •For interactive media creators: Use separate logic and rendering systems to design flexible interactive experiences that maintain consistent user-driven state changes.
Authors
Zijun Lin, Zhiyang Deng, Yuzhe Wu, Bihan Wen, Yeying Jin
Abstract
Recent game world models support realistic visual simulation and interactive gameplay based on player inputs. However, they typically learn environment dynamics from pixel-level supervision, jointly modeling perception, memory, state transitions, and rendering within a single end-to-end framework. While this design enables open-ended, action-controllable generation, it still falls short of delivering a complete gameplay experience. Games are governed by explicit mechanics, such as health deduction, skill activation, combat rules, and termination conditions. These mechanics depend on precise and consistent state transitions that generative models alone cannot reliably enforce. In contrast, game engines can guarantee such mechanics through hard-coded rules, but provide limited flexibility for player-driven creation. To bridge these paradigms, we introduce GameDirector, the first agentic framework that decouples rule-based gameplay logic from visual rendering. Given player-defined configurations, the framework acts as an intelligent director that interprets visual observations, updates game states, tactically controls NPCs, and enforces gameplay rules. It then translates these decisions into text prompts that guide the video world model to render the resulting gameplay. This separation allows players to configure characters, states, and rules much like a game developer while preserving coherent game mechanics. Experiments on three games, using data collected by our automated gameplay agent, show that GameDirector achieves accurate state tracking, reliable rule following, and improves boss action quality by more than 39.9% over various end-to-end game world model settings. Overall, by externalizing player-controllable game logic, GameDirector establishes a middle ground between hard-coded simulation and generative modeling, enabling more flexible and closed-loop gameplay experiences.