Summary
Scientific method diagrams show how ideas or computations flow using entities and connections, but making these diagrams better can be tricky. The authors created a system called SIVIA-RSI that learns from previous diagram improvements to help make better first drafts for new papers. They tested this system on papers about language processing and found the learned skills improved diagram accuracy a bit but did not always work consistently across topics. They also found some challenges arise because instructions and diagrams sometimes leave parts unclear. Their work helps show why checking diagrams closely and connecting them to the original text is important for improving reusable diagram skills.
What this means in practice
- •For technical writers: Use source-linked critiques to adapt and improve recurring diagramming skills for drafting technical figures in research papers.
- •For software documentation teams: Refine and reuse diagram templates based on previous edits to enhance accuracy in documentation workflows for complex software processes.
Authors
Feng Yuan, Yifan Gao, Haoyue Li, Xin Gao
Abstract
Scientific method diagrams express computations through entities, dependencies, and conditional routes. Although generated figures can be improved through repeated editing, it is less clear whether experience from one paper improves the first figure of another. We present \sys, a framework for source-grounded adaptation of reusable diagramming skills, and study transfer through a complete-candidate evaluation. The framework links critiques to source passages, proposes bounded edits to a persistent skill library, and separates candidate competition from skill acceptance. We evaluate the original skill and four learned candidates on two NLP and language-agent papers, with two fresh generations per condition. The strongest candidate attains 87.50\% required-relation accuracy compared with 83.33\% for the original, while candidate behavior differs across papers. Local improvements on a separate development paper and automatic selector preferences do not establish consistent transfer. Tracing all 22 non-correct relation judgments to their production prompts reveals both incomplete conditional specifications and ambiguities despite explicit instructions. All ten planning diagrams leave an already-terminal selected leaf's route unclear; none of their prompts explicitly binds that route. Our findings show why evaluating reusable diagram skills requires source-grounded relation assessment, complete candidate coverage, and inspection of both prompts and images. We provide all twenty transfer outputs, skill snapshots, assessment records, and executable analyses.