Summary
Creating images and videos from computer programs can be tricky because the final picture might not match the instructions exactly. The authors of this paper identified this gap between what a visual program should do and what it actually produces, calling it the Program-to-Visual (P2V) gap. They built MaLiang-Harness, a system that helps create visuals by letting machines construct, check, and improve code step-by-step, keeping track of changes and verifying results as they go. They tested this system with multiple powerful language models and found that their approach helps ensure visuals match what the code intends, revealing important insights about how these models work with visual tasks.
What this means in practice
- •For visual effects teams: Create more reliable image and video content by generating and refining visual programs with consistent verification and revision support.
- •For interactive media developers: Develop tools that allow iterative programming and inspection of visuals, improving control over computer-generated imagery.
Authors
Haoyu Zhao, Zihao Zhang, Xudong Wang, Jiaxi Gu, Zuxuan Wu, Yu-Gang Jiang, Shuicheng Yan
Abstract
Executable programs offer explicit control over how images and videos are constructed, but generating runnable code is only the beginning of visual creation. A program can execute correctly while violating the requested composition, appearance, or motion. We define this discrepancy as the Program-to-Visual (P2V) gap and introduce MaLiang-Harness, a unified framework for organizing MLLM-driven visual generation into a persistent process of construction, inspection, and revision. Its central design is to make the evolving visual program, its construction history, and its verification share a common revision reference. We define the Persistent Executable Generation (PEG) state as preserving programs and task context. Traceable Generation Process (TGP) connects edits to rendered evidence, and Revision-aware Editing and Verification (REV) supports restoration and checks the current revision before completion. Together, these mechanisms coordinate planning, execution, and visual feedback across rendering backends. We evaluate 11 powerful closed-source MLLMs on MaLiang-IBench and four on MaLiang-VBench, measuring generation success, visual quality, and computational cost. GPT-6-Astra achieves 100% generation success on both benchmarks, with 96.0% of image tasks and 76.9% of video tasks meeting all quality thresholds. The comparison also reveals a mismatch between general capability scores and visual generation performance, with similarly scored models differing substantially in their ability to satisfy visual requirements. MaLiang-Harness provides a systematic basis for studying how MLLMs translate executable code into visual outcomes, exposing both the potential of programmable generation and the limitations of general benchmarks as predictors of this ability. The project is available at https://github.com/gulucaptain/MaLiang-Harness.