Discovery system improves experiments by changing theory mechanisms

Learning to Outgrow a Theory: Experimental Discovery Beyond the Initial Hypothesis Space

Artificial Intelligence

Summary

Scientific discovery often relies on testing ideas within a fixed set of theories, which can lead to problems if all theories miss an important part. The authors present a method that can suggest changes to the theory itself and design experiments to test if those changes are needed. Their system decides when to update the theory only after enough evidence shows the current theories don't fit. Tested on many simulated dynamic systems, their approach recovered correct theories more often and with fewer experiments than previous methods. It also showed it could detect when the available building blocks for theories were not enough.

What this means in practice

Authors

SiYuan Ma, Albert Gao, Chunzheng Zhu, Xin Yan, Wenlong Zhang, Wenxin Zhang, Luqi Gong, Tianlin Li, Qixin Zhang

Abstract

Scientific discovery systems typically optimize experiments within a fixed hypothesis space. This creates a failure mode when all available candidates omit the same missing mechanism: candidate disagreement can collapse even while the model class is systematically wrong. We formulate experimental model-class revision, in which a discovery policy jointly proposes a structural edit and a diagnostic experiment that tests whether that edit is necessary. The method couples a class-level distinguishability objective, in which one shared parameterization must explain all selected experiments, with anytime-valid sequential evidence that triggers structural revision only after the current class is rejected. On 400 held-out controlled dynamical environments, the joint policy reaches 89.5% exact recovery with a budget of 32 real experiments, improving the strongest matched baseline by 10.0 percentage points while requiring fewer executed experiments and candidate fits. The learned revision-experiment pairing transfers across unseen mechanism combinations, held-out but expressible primitives, parameter extrapolation, and shifted experiment costs; when the true mechanism is outside the edit grammar, it detects library insufficiency in 88% of cases with a 5.5% false-support rate. Revision gains also transfer to ODEBench and ODEBase model-library tasks, as well as DiscoverPhysics worlds. These results support a view of scientific discovery in which deciding what mechanisms a theory should make expressible and where to collect evidence are treated as a single sequential decision problem.