Ontology-grounded benchmarks measure large language model reasoning
Ontology-Grounded, Reasoner-Verified Benchmarks for Evaluating LLM Reasoning in Scientific AI
Artificial IntelligenceMachine Learning
Summary
Large language models are used to answer tricky science questions, but they sometimes get the facts wrong. The authors developed a way to automatically create multiple-choice tests that are based on detailed background knowledge, so answers can be checked logically. They showed that these tests are clear, challenging, and can spot specific types of reasoning errors in different scientific areas. When tested, the language models did better than guessing, but still struggled, showing these tests can help improve how well they reason scientifically.
What this means in practice
- •For biomedical data teams: Use ontology-grounded benchmarks to assess and improve language models’ reasoning accuracy on biomedical question answering tasks.
- •For materials informatics engineers: Employ the generated benchmarks to diagnose specific reasoning failures in LLMs during materials science knowledge analysis.
Authors
Nishtha N. Vaidya, Stephan Grimm, Thomas Hubauer, Thomas A. Runkler
Abstract
Large language models (LLMs) increasingly underpin scientific AI applications that reason over structured knowledge, from biomedical question answering to materials informatics. However, their logical reasoning often falls short, producing factual inaccuracies unacceptable in these settings. Reliable evaluation remains challenging: manual dataset construction scales poorly, and LLM-based generation risks embedding the very flaws it aims to measure. High-quality benchmarks must ground both correct and incorrect labelled examples in explicit background knowledge, formally verifiable by a standard reasoner. We propose a pipeline that automatically generates ontology-grounded multiple-choice question (MCQ) benchmarks from any sufficiently axiomatised OWL 2 ontology, with correct answers grounded in the ontology by design. Distractors are generated by perturbing the right-hand-side class expressions of class definition axioms, and their incorrectness is formally verified by an OWL reasoner via entailment checks. We evaluate the pipeline on three ontologies: Pizza (small, academic), PMDco (complex, materials science), and DOID (large, biomedical), generating 112, 2,491, and 15,216 MCQs respectively. Distractors span four semantic categories from class unsatisfiability to weakened subsumptions, enabling diagnostic evaluation of specific reasoning failures. Items meet natural language quality standards: mean LLM judge scores of 4.02, 4.36, and 3.36 out of 5 confirm fluency, and correct-answer-to-distractor similarity above 0.8 shows that wrong options cannot be dismissed on surface form alone. Six LLMs evaluated zero-shot achieve 41.1-76.8% accuracy, well above the 25% random-guessing baseline, indicating the benchmarks are challenging and discriminative. This work is a step towards more reliable benchmarks for assessing logical reasoning in scientific AI.