Agent driven systems automate complex computer graphics coding tasks
Advances in Agentic Computer Graphics
Computer Vision and Pattern Recognition
Summary
Creating computer graphics often involves writing a lot of detailed code. The authors highlight how smart software agents, powered by large language models, can now write and improve graphics code by themselves through repeated feedback. They created a new way to describe these agent systems to better understand ongoing research. They also built a tool that helps researchers analyze many papers quickly and found key challenges in using these agents for graphics. This includes how to make agents work together, collaborate with humans, plan long tasks, and evaluate performance.
What this means in practice
- •For graphics software developers: Create more automated tools that generate and improve graphics code with less human input by using agentic systems.
- •For user interface designers: Design new interfaces that enable smoother collaboration between human users and intelligent coding agents for graphics workflows.
A survey. It maps existing work.
Authors
Hengyu Liu, Jihyeon Je, Ruiqi Wang, Mingxuan Wu, Haven Feng, Qixing Huang, Georgios Pavlakos, Hao Zhang, Jiajun Wu, Maneesh Agrawala, Leonidas Guibas, R. Kenny Jones, Shangzhe Wu, Guandao Yang
Abstract
Computer graphics is built on code. Large language model (LLM) agents can now generate graphics code and iteratively refine it through various feedback. This capability opens the door to a potentially new paradigm of computer graphics, with graphics algorithms increasingly built around agentic systems to automate progressively complex tasks. Over the past years, research on agentic graphics has expanded rapidly across a diverse range of tasks. This rapid growth makes it increasingly difficult for manual literature reviews to keep pace with new developments. At the same time, similar ideas are often described using different terminology across these tasks, further complicating efforts to develop a systematic understanding of the field. In this report, we address these challenges through two contributions. First, we introduce a unified taxonomy for characterizing agentic systems across diverse computer graphics tasks. Second, we develop an agentic literature system that can assist human researchers in analyzing existing and newly published papers at scale. Building on these contributions, we analyze emerging research trends and discuss open challenges in coordinating agents and tools across graphics tasks, designing interfaces and representations for human-agent collaboration, sustaining long-horizon planning and refinement, and reliably evaluating interactive graphics systems.