AI reconstructs editable code from scientific figures for better access
SciFigure2Code: An AI-Reconstructed Benchmark for Scientific Figure-to-Code
Computer Vision and Pattern Recognition
Summary
Scientific figures in research papers show important information, but the original data or code used to make them is usually not shared, making it hard to understand or reuse them. The authors created SciFigure2Code, a system that uses AI to generate Python code that recreates how these figures look and are arranged, without trying to find the exact original data. They built a large collection of reviewed examples and tested several AI models, finding that some parts like axes and legends are still difficult for AI to recreate perfectly. This work helps make scientific figures easier to explore and edit by providing a way to turn images into reusable code.
scientific figurebenchmarkpresentation recoveryPython programmingCodex agentsvisual hierarchyimage-to-codezero-shot modelscaption-assisted reconstructionplan-then-code prompting
Authors
Wentao Li, Yibo Wu, Yizhe Chen, Ruixuan Chen, Jiangjie Qiu, Yijun Li, Zhao Leyi, Xiaonan Wang
Abstract
Scientific figures are the interface through which research claims are inspected and reused, but final published panels rarely expose the data or plotting code that produced them. Recovering this hidden provenance from pixels is therefore underdetermined. We introduce SciFigure2Code, an AI-reconstructed benchmark that instead evaluates presentation recovery: generating editable Python programs that preserve how a scientific panel is arranged and read. Role-specialized Codex agents generate, execute, visually refine, and audit silver-standard presentation programs that capture geometry, visual hierarchy, encodings, annotations, and typography without claiming to recover original measurements or author source code. This reconstruction-and-audit protocol turns final published panels into auditable reference packages; the resulting resource contains 6,740 reviewed panels and SciFigureBench, a balanced 337-panel test set across 31 chart subtypes, five domains, and three complexity levels. Across 14 zero-shot models in image-only and caption-assisted settings, execution, multi-component layouts, axes, legends, and scientific labels remain weak. Claude Opus 4.7 achieves the highest image-only Overall score, Claude Opus 4.6 leads caption-assisted reconstruction, and two-stage plan-then-code prompting improves Overall for all four tested models. SciFigure2Code provides an auditable testbed for agents that construct editable, visually faithful scientific figure presentations.