Ai models fix problems but often fail to find the cause
Treating the Symptom, Missing the Cause: Anatomy of a Scientific Reasoning Failure
Artificial Intelligence
Summary
Sometimes AI systems can fix a problem without understanding what caused it. The authors studied this by testing AI on made-up biological systems where the AI had to find a hidden fault and fix it. They found that while AI models often fixed the problem, they struggled to correctly identify the actual cause behind it. This happens even when evidence clearly points to the cause, because the AI has trouble predicting experiment outcomes needed to tell causes apart. The study shows the importance of checking not just whether AI fixes issues but also how well it reasons about causes.
What this means in practice
- •For biotech developers: Improve AI tools that help design experiments by focusing on validating causal inferences, not just effective interventions.
- •For medical device designers: Develop diagnostic AI systems that better interpret experimental test results to accurately identify root causes of health issues.
Authors
Anna Gogleva, Thomas Charman, Martin Balla, Karl Gemayel
Abstract
Scientific agents can find interventions that work without understanding why they do. We study this failure mode in controlled synthetic regulatory systems, where agents must experimentally identify a hidden fault and rescue the resulting phenotype. Across seven LLMs, every model restores the healthy phenotypes more often than it identifies the fault. Claude Opus 4.6 and Sonnet 4.6 rescue every organism while diagnosing only 55% and 35% of them. This failure is not simply due to insufficient exploration or unavailable evidence. In most cases, the cause is experimentally identifiable, and agents encounter evidence that distinguishes competing causes in 91% of relevant sessions, and still frequently misdiagnose the system. Targeted probes reveal where experimental reasoning breaks down. Models reliably interpret decisive evidence and revise hypotheses when the causal comparison is explicit, and can locate that evidence within a full experimental record. However, they struggle to predict the outcome of the experiment under each competing cause, a necessary step in using the observed result to distinguish between them. Agents can therefore gather the right evidence and still infer the wrong cause. As AI systems take on a growing role in synthesising scientific evidence, these findings highlight the need to evaluate how they use evidence to reach causal conclusions, alongside whether their interventions succeed.