AI models separate prediction accuracy from true mechanism understanding
Can AI Scientists Change Their Minds? Prior-Evidence Conflict in Synthetic Universes
Artificial Intelligence
Summary
This paper asks if AI can tell the difference between recognizing a pattern and truly understanding the rule behind it. The authors created synthetic worlds paired with slightly altered versions to test AI laws two ways: whether they predict future events well and whether they match the true underlying cause. They found that sometimes AI predictions succeed without matching the real cause, and sometimes AI finds the real cause but fails at predicting accurately. This shows that just because AI models predict well doesn’t mean they truly understand the science behind it.
What this means in practice
- •For machine learning engineers: Design evaluation tests that separately check if AI models predict accurately and if they recover true generating mechanisms in simulations.
- •For software verification teams: Develop testing suites that distinguish between correct predictions and deep causal understanding for AI systems used in critical forecasting.
Authors
Kargi Chauhan
Abstract
Can a scientific agent distinguish a law it inferred from evidence from one it merely recognizes? We introduce Synthetic Universes, a controlled benchmark that pairs canonical famous worlds with matched twisted twins governed by nearby noncanonical mechanisms. We evaluate each reported law twice: by executing it on held-out continuations and transfer settings, and by independently checking whether it recovers the generating mechanism. In the current checkpoint of a pre-specified 60-cell study, 22 trials were graded and one additional run ended in infrastructure failure. Among 20 twin trials, 8 pass predictive verification while 5 recover the generator. The dissociation is bidirectional: six parsable outputs predict successfully while missing the mechanism, whereas three recover the mechanism but fail predictive rollout. Drag exhibits the first pattern (5/5 predictive pass, 1/5 mechanism recovery); Gravity exhibits the second (1/5 predictive pass, 4/5 mechanism recovery). Because matched famous controls, the corrected identifiability sweep, and the Evidence Ladder remain incomplete, we do not claim a confirmatory causal prior-conflict effect. Instead, the completed runs establish a narrower verification result: predictive adequacy and mechanism recovery are distinct scientific claims and require distinct tests.