Summary
Comparing game AI systems is hard because small differences in how the game is represented can change the decisions made during play. The authors created CoToan, a clear set of rules and a representation contract that links game descriptions to a popular AI framework, focusing on Vietnamese Mathematical Chess. They show that including details like how actions change states, remaining time, and how winning is decided is essential for fair and repeatable AI testing. Leaving out these details can cause mistakes where different situations look the same or where action chances differ. Their work highlights these details as crucial parts of the game setup, not just small coding choices.
What this means in practice
- •For game ai developers: Ensure consistent game state representation and action semantics for reliable AI evaluation across implementations.
- •For software quality engineers: Use executable representation contracts to detect subtle inconsistencies in complex software systems with state-dependent rules.
Abstract
Game AI results are comparable only when different implementations expose the same decision problem. Yet seemingly minor representation choices can silently change that problem by altering action probabilities or collapsing states with different outcomes. We study this issue through CoToan, an executable specification and representation contract for Vietnamese Mathematical Chess connecting the Game Description Language (GDL) with OpenSpiel. The game provides a useful case because a single capture may have multiple arithmetic justifications, while outcomes can depend on scoring rules and a finite playing horizon. CoToan therefore represents actions by their state transitions rather than by arithmetic witnesses, and treats time and victory configuration as part of task identity. Cross-representation tests show agreement in legal actions, successor states, and terminal outcomes across hundreds of sampled and constructed positions, complemented by exhaustive checks of all 32 operator masks and 512 opposing-material subsets. Controlled diagnostics further show that uniform witness-level sampling alters endpoint-action probabilities, while omitting remaining time or victory settings can merge states with different terminal behavior. These results show that action semantics, time, and victory configuration are not merely implementation details. They are part of the experimental task itself and must be represented explicitly for reproducible game AI.