AI discovery improves when asking the right questions early

Asking the right question is key to AI discovery in a chain of hypotheses

Artificial Intelligence

Summary

AI systems often try to find new discoveries by testing where their current guesses disagree the most. The authors found that this approach can miss important but narrow details that early data did not show. Two advanced AI models tested missed these hidden details more than if tests were done randomly. Instead, success came when experiments asked early questions focused on the unmeasured gaps, not just where models disagreed. This shows that asking the right questions matters more than simply testing uncertainty for AI discovery.

What this means in practice

  • For ai research teams: Design more effective AI discovery processes by focusing early experiments on unmeasured gaps rather than only on current model disagreements.
  • For industrial automation engineers: Improve anomaly detection in automated systems by targeting test data collection in regions lacking prior measurements instead of uncertain areas alone.

Authors

Hu Cang

Abstract

Artificial intelligence (AI) scientists pursue discovery through a chain of hypotheses, fitting candidate models to existing observations and often choosing experiments where those models disagree most. Yet when early data miss a narrow feature, model selection favors simple curves that agree across the unmea- sured gap, so disagreement-guided search has no reason to look where the new phenomenon lies. When we tested two frontier AI scientists, Claude Sonnet 5.5 and GPT-6.1-Sol, across 32 closed-loop trials with a narrow peak omitted from the starting data, both sampled inside the peak region less often than random search. Surprisingly, the agent that followed model disagreement more closely missed the peak in all 32 trials, whereas discovery succeeded when experiments probed the unmeasured gap early and followed up across the anomaly. Asking the right question, rather than testing only where current models disagree, is therefore key to AI discovery in a chain of hypotheses.