Summary
Solving puzzles where the rules are hidden helps test whether AI can figure out new rules by trying things out. The authors created a puzzle system called WITNESS to see how well current AI language models and reinforcement learning methods can learn and use these hidden rules. They found that even the best models only solved about a quarter of the hardest puzzles, and knowing the rules in advance made a big difference for some models but hardly helped others. Training with reinforcement learning improved AI's ability to learn and apply hidden rules, showing promise for tackling real-world problems where rules are unknown. This test setup also helps compare different AI methods fairly.
What this means in practice
- •For ai system builders: Improve AI agents for discovering and applying unknown environment rules by using reinforcement learning on structured puzzle benchmarks.
- •For game developers: Create interactive puzzle games that dynamically test a player or AI’s ability to infer and utilize hidden rules for challenge design.$Commercial implications: Enables development of advanced puzzle games with adaptive difficulty based on AI-driven rule learning capabilities.
Authors
Guanghan Ning, Ping Liu, Linyi Li, Huangjie Zheng, Arjun Neervannan, Huu Nguyen, Michael Sklar, Deniz Zorlu, Nicolai Ouporov
Abstract
Automated science needs agents that can work out the rules of an unfamiliar environment by interacting with it. Interactive rule-discovery puzzles offer a controlled setting for studying this ability: an agent infers hidden rules through experimentation and uses what it has inferred to reach a stated goal. We ask what limits current language models on these puzzles and whether reinforcement learning (RL) improves performance on rules held out from training. To study both, we introduce WITNESS, a 2D grid-based puzzle environment with ground-truth ASCII observations and controlled access to rules. An agentic pipeline generates games for WitnessGym, the RL training suite, and WitnessBench, comprising public validation and private test games. The validation set separately tests new compositions of trained rule primitives and primitives absent from training. Under a shared harness, the best of 18 frontier proprietary and open-weight models solves only 24\% of private test level slots, with scores sensitive to the observation interface and agent configuration. Providing ground-truth rules raises Opus-5's validation RHAE-L5 (relative human action efficiency over the first five levels) from 59.9 to 97.8, whereas a 27B open-weight model gains only 2.1 points and remains limited even with the rules provided. RL on WitnessGym raises the 27B model's private test RHAE-L5 from 2.1 to 5.4 and yields a mean gain of 4.1 points on four external discovery benchmarks. Together, these results point to rule acquisition as a major difficulty for frontier models like Opus-5 while smaller models further struggle on rule-based execution, and indicate that RL on hidden-rule puzzles transfers to broader rules and real-world tasks beyond training. Benchmark is available at: https://witnessbench.ai