Model decodes graphic design elements in order to improve detection

Detect Anything in Graphic Design: Element-Level Rewards for Autoregressive Detection

Computer Vision and Pattern RecognitionMachine Learning

Summary

Graphic designs like posters are made of many layers stacked on top of each other, but current detection tools treat these layers like a jumbled set. The authors created a model, DAD, that looks at elements in the order they appear, using information from lower layers to detect higher ones better. Their system can also predict the full shape of each element, even if part of it is hidden behind others. They also developed a new training method that gives detailed feedback for each element to help the model learn better. They tested their approach on a huge dataset of graphic designs and found it works as well as humans for detecting covered elements.

What this means in practice

  • For graphic design software developers: Improve software that automatically separates design components by detecting layered elements in correct order and handling occlusions accurately.$Commercial implications: Enables advanced design editing features in commercial graphic design tools by reliably identifying and isolating design elements.
  • For advertising automation teams: Automate analysis of layered advertisement images by decomposing them into their component graphical elements for content optimization.

Authors

Jiangning Zhu, Bowen Li, Shenyu Qiao, Yima Gu, Zhao Zhang, Yuhui Yuan, Shixia Liu

Abstract

Graphic designs, such as posters, advertisements, and infographics, are an important medium for communicating information and shaping understanding. Unlike natural images, they consist of layered elements with explicit compositional order. However, existing object detection models treat these elements as an unordered set, leaving compositional order unexploited. To address this limitation, we present Detect Anything in Graphic Design (DAD), a model that formulates graphic design detection as compositional deconstruction. It decodes elements in compositional order, using lower-layer elements to better detect higher-layer ones. The key feature of DAD is amodal detection, which predicts the full bounding box of each element, including regions occluded by elements placed above it. Building on this formulation, we propose Element Relative Policy Optimization (EleRPO), which extends GRPO from sequence-level supervision to element-level optimization. EleRPO provides fine-grained training signals that capture how each detected element contributes to overall detection quality, and works synergistically with compositional order to improve detection performance. To support training and evaluation, we build a dataset of 10 million graphic designs. Experiments show that DAD outperforms all baselines and achieves human-level performance in amodal detection, supporting effective image-to-layer decomposition. EleRPO consistently improves over GRPO across nine detection benchmarks.