AI discovery improves when asking the right questions early
Asking the right question is key to AI discovery in a chain of hypotheses
Artificial Intelligence
Summary
AI systems often try to find new discoveries by testing where their current guesses disagree the most. The authors found that this approach can miss important but narrow details that early data did not show. Two advanced AI models tested missed these hidden details more than if tests were done randomly. Instead, success came when experiments asked early questions focused on the unmeasured gaps, not just where models disagreed. This shows that asking the right questions matters more than simply testing uncertainty for AI discovery.
What this means in practice
- •For ai research teams: Design more effective AI discovery processes by focusing early experiments on unmeasured gaps rather than only on current model disagreements.
- •For industrial automation engineers: Improve anomaly detection in automated systems by targeting test data collection in regions lacking prior measurements instead of uncertain areas alone.
Authors
Hu Cang
Abstract
Artificial intelligence (AI) scientists pursue discovery through a chain of hypotheses, fitting candidate models to existing observations and often choosing experiments where those models disagree most. Yet when early data miss a narrow feature, model selection favors simple curves that agree across the unmea- sured gap, so disagreement-guided search has no reason to look where the new phenomenon lies. When we tested two frontier AI scientists, Claude Sonnet 5.5 and GPT-6.1-Sol, across 32 closed-loop trials with a narrow peak omitted from the starting data, both sampled inside the peak region less often than random search. Surprisingly, the agent that followed model disagreement more closely missed the peak in all 32 trials, whereas discovery succeeded when experiments probed the unmeasured gap early and followed up across the anomaly. Asking the right question, rather than testing only where current models disagree, is therefore key to AI discovery in a chain of hypotheses.