Visual artifact editing benchmark compares pixels code and agents
Pixels or Code? Benchmarking Instruction-Guided Visual Artifact Operation
Computer Vision and Pattern Recognition
Summary
Editing images like charts or webpages is tricky because changes in one part can affect other parts. The authors introduce a new benchmark called VAO-Bench that tests different ways to edit these images, including using pixel-based tools, code generation, and agent systems. They found that code-based methods are better for handling dependent changes, while pixel editors run faster and look good. The benchmark helps understand how well each method manages direct edits, updates, and keeps everything looking right.
What this means in practice
- •For graphic designers: Evaluate tools for efficiently editing complex visuals like charts and webpages without breaking related elements.
- •For software developers: Develop more reliable visual editing software by testing how well different methods handle dependent changes in graphics.
Authors
Shun Guo, Shuizhou Shi, Boao Wang, Wei Xiong, Siyu Sun, Jiangtong Li, Jie Li
Abstract
Visual artifacts such as charts, technical diagrams, webpages, and document pages underpin scientific, commercial, and engineering workflows. Editing these artifacts requires direct modification, dependency-aware updates, preservation of unaffected content, and compliance with domain constraints. We define this capability as Visual Artifact Operation (VAO). Prior benchmarks test parts of this problem, including cascading updates in charts. We study direct edits, dependent updates, and preservation across visual domains and pixel, code, and agent paradigms under one image-only protocol. We introduce Visual Artifact Operation Bench (VAO-Bench), comprising 1,573 manually screened tasks across nine artifact categories and three difficulty levels. VAO-Bench uses executable artifacts and deterministic rendering to trace reference transformations and annotate direct edits, dependent updates, and preservation constraints. All systems receive the same source image and instruction; we evaluate pixel-space editors, code-generating multimodal systems, and tool-using agents on direct-edit accuracy, dependency-aware update, content preservation, domain validity, and visual quality. Experiments show that executable-code systems better support dependent updates, while pixel-space editors offer faster execution and strong visual quality. Explicitly stating dependencies improves both paradigms, identifying implicit-update inference as a central bottleneck. VAO-Bench provides a reproducible foundation for dependable visual artifact editing across generation paradigms.