AI summaryⓘ
The authors studied how to improve the process of making scientific diagrams based on paper content through multiple rounds of feedback. They found that people usually want changes after seeing an initial version and that refining diagrams leads to better satisfaction. To help with this, they created a benchmark called MTPaperBananaBench with many examples of diagrams and user requests, along with a program that simulates user feedback. They identified two common problems in current methods: the quality of diagrams getting worse over time and forgetting earlier improvements. To fix this, the authors designed PaperBanana-Interact, a system that improves diagrams step-by-step using critiques, which performed better than previous approaches.
scientific diagramsmulti-turn generationuser simulationbenchmark datasetdiagram refinementquality driftfeature forgettinginteractive systemsnatural language feedback
Authors
Xueqing Wu, Ashwin Balasubramanian, Bingxuan Li, Dawei Zhu, Kai-Wei Chang, Yale Song, Yiwen Song, Rui Meng, Tomas Pfister, Nanyun Peng
Abstract
Recent efforts have aimed to automate scientific diagram generation from paper content (Lin et al., 2026; Zhu et al., 2026a). However, fully satisfying an author's visual and communicative preferences in a single turn is challenging: in our formative user study (N = 14), all participants requested further revisions after viewing an initial draft, and 86% of them rated the refined diagrams as more satisfactory. Despite the clear demand, the multi-turn workflow remains largely underexplored. To bridge this gap, we present MTPaperBananaBench, a benchmark for multi-turn diagram generation containing 292 images annotated with 3,518 user requirements. To reduce expensive human studies and enable scalable benchmarking, we construct a user simulator that, at each turn, identifies unsatisfied requirements and converts k of them into natural language feedback. Evaluating both requirement satisfaction and overall diagram quality reveals two key failure modes shared across baseline multiturn systems: (1) quality drift, where diagram quality progressively declines over turns, and (2) forgetting, where previously implemented features are lost in subsequent turns. To address these issues, we introduce PaperBanana-Interact, a multi-agent system that refines diagrams via an internal critique-and-refine loop. PaperBanana-Interact consistently improves rather than degrades diagram quality across turns, outperforming baselines by 11.9-18.6 points in quality score and reducing forgetting by 3.7-6.2 points.