Life sciences questions evaluated with graph structure and new scores
LG-Bench: A Graph-Structured Evaluation Benchmark for Life Sciences
Artificial Intelligence
Summary
Most tests for knowledge in life sciences treat questions as unrelated, which can miss connections between ideas. The authors created a large set of over 10,000 questions linked by how their meanings relate, forming a big connected network. They designed new ways to measure how well AI models understand science by looking at how consistent their answers are in connected groups of questions and where they struggle. This approach helps see patterns in how AI organizes knowledge, rather than just tallying right or wrong answers.
What this means in practice
- •For life sciences ai developers: Compare and improve AI models by measuring consistency and error patterns in connected life science concepts.
- •For healthcare ai teams: Diagnose where medical language models fail by analyzing errors in related question clusters to target improvements.
Authors
Lu Sun, Xiangyi Zhang, Xiangyang Zhu, Kaiyuan Ji, Zijian Chen, Qiang Hu, Zicheng Zhang, Yuan Tian, Guangtao Zhai
Abstract
Traditional evaluation benchmarks reduce inherently interconnected scientific knowledge in life sciences into flat lists of questions, disregarding the underlying topological structure of the knowledge. We introduce, the first graph-structured benchmark for life sciences, featuring over 10,000 high-quality multiple-choice questions across medicine, biology, and chemistry. Our approach constructs a weighted evaluation graph using bidirectional matching and semantic similarity algorithms, where nodes represent questions and edge weights capture their semantic relationships. Leveraging this graph topology, we design two novel evaluation metrics. The Global Coherence Score (GCS) measures a model's consistency within semantically related neighborhoods, while Knowledge Balance Score (KBS) analyzes how model errors are distributed across the graph to reveal conceptual blind spots. LG-Bench facilitates fine-grained comparison of LLMs by surfacing differences in conceptual coherence and patterns of knowledge organization across models. Our framework shifts the evaluation paradigm from flat accuracy metrics to structure-aware analysis, offering a new lens for diagnosing and improving LLM performance in the life sciences domain.