Summary
Scientific papers often include complex diagrams that show their methods visually, which helps readers understand the research quickly. However, current AI models that read images and text together struggle with these dense scientific figures because they miss detailed parts and the flow of information. The authors created a new test called SciFigBench with thousands of real scientific method figures and questions to measure how well AI can understand them. They also made a tool, SciFigCue, that helps AI models follow arrows and relationships in these figures to improve reasoning, without needing to retrain the models. Their experiments show that even strong AI models still find visual relationships challenging, but SciFigCue helps them do better.
What this means in practice
- •For scientific software developers: Integrate SciFigBench to improve AI-assisted analysis of scientific method diagrams for enhanced paper understanding tools.
- •For document processing engineers: Use SciFigCue to boost visual relation detection in complex figures without retraining models, enhancing automated document interpretation.
Authors
Lijia Wu, Zheng Li, Yuxuan Liu, Jingwen Zhang, Penghai Zhao, Xiang Li
Abstract
Scientific method figures often provide researchers with a clear visual entry point into a paper's methodology and core contributions, yet their dense layouts and complex information flows are difficult for Vision Language Models (VLMs) to interpret. Existing benchmarks predominantly focus on coarse-grained tasks, such as context-based retrieval or holistic comprehension, overlooking the fine-grained content and inter-module relations within complex methodological diagrams. To address these overlooked aspects, we introduce Scientific Figure Benchmark (SciFigBench), a benchmark for evaluating core capabilities required for scientific method figure understanding. Specifically, it targets three capabilities: dense content perception, visual relation comprehension, and information-flow tracing. For this purpose, we curate 4,352 scientific method figures from real-world research papers and construct 40,670 QA pairs to operationalize these capabilities as measurable question-answering tasks. All annotations are generated by dedicated pipelines and then manually verified. We further propose SciFigCue, a training-free visual prompting module that externalizes arrow-induced dependencies as visual relational evidence for method-figure reasoning. Experiments across open-source and proprietary VLMs show that even strong models remain substantially limited in visual relation comprehension, while SciFigCue improves performance across several tasks without parameter updates. The code and related resources are available at https://github.com/LeNcyer/SciFigBench.