Language models struggle to judge scientific evidence but improve with training
Do Language Models Know What Compelling Scientific Evidence Looks Like?
Machine LearningArtificial IntelligenceComputation and Language
Summary
It can be hard for computers to tell how strong scientific evidence is when studies have different levels of reliability and relevance. This paper created a big test using biology experiments to see if language models can predict how likely a scientific claim is true based on multiple results. At first, the models were often too cautious or unsure and didn’t do much better than guessing. But after special training to better understand uncertainty and make predictions, their skills improved a lot and they could pick experiments more efficiently. The authors also studied how these models update their beliefs and compare to expert scientists.
What this means in practice
- •For biotech research developers: Use calibrated language models to prioritize biological experiments and improve discovery efficiency in areas like cancer biology.
- •For clinical trial designers: Incorporate language models trained on evidence weighting to better predict which hypotheses are promising based on multiple study results.
Authors
Conor Igoe, Gabriel Grand, Zane Koch, Albert Bou, Eli Moss, Michaela M. Hinks, Samuel G. Rodriques, Andrew White, Siddharth Narayanan
Abstract
Expert scientists excel at weighing empirical findings from multiple sources that vary in both statistical reliability and epistemic relevance. To what extent are language models, which are increasingly driving autonomous discovery systems, capable of such evidence deliberation? We introduce EDB-Bio, a scientific reasoning benchmark curated from real-world data in three biological domains: drug-perturbation transcriptomics, CRISPR screens, and human genome-wide association studies (GWAS). Each of EDB-Bio's 96k episodes presents a sequence of experimental results from published studies and asks the model to predict the probability that a target hypothesis is true. Out-of-the-box, open-weight and frontier models alike demonstrate systematic underconfidence and attain Brier scores comparable to a trivial base-rate forecaster. Post-training against strictly proper scoring rules significantly improves both calibration and discrimination. This calibration pays off downstream: in a sequential experiment selection task, our Ibis0 model is 2.5 times more efficient than the best frontier model. Throughout, quantifying evidence weights as Bayes factors reveals how models update their beliefs over time, whether they agree with expert biologists on relevance, and where they exhibit inconsistencies and biases. Our work extends calibration training and uncertainty analysis to a realistic scientific forecasting setting with applications to human cancer biology. We release all code, data, and trained models.