Model generates images and layouts together for better design templates
Learning Interaction between Image and Layout Priors for Joint Image-Layout Generation in Design Templates
Computer Vision and Pattern RecognitionArtificial IntelligenceGraphics
Summary
Creating graphic design templates usually means making a background image and placing items on top one after another. The authors found this sequential method misses how backgrounds and layouts affect each other. They built a new model that creates the background and layout at the same time, letting them interact during the process. This approach keeps the realistic look of designs and lets users guide the results without retraining the model.
What this means in practice
- •For graphic designers: Automatically generate harmonious design templates from text input that combine images and layouts in one step for faster creative workflows.
- •For advertising agencies: Create consistent ad templates with well-integrated background images and layouts that better match campaign requirements on demand.$Commercial implications: Enables selling customized, harmonious ad template generators that speed up creative production for marketers.
Authors
Shirong Yang, Bo Yang, Ying Cao
Abstract
In this paper, we address the problem of graphic design template creation, which generates a background image and a layout of foreground elements over the background to form a harmonious composition from an input text. Prior work on graphic design generation mostly adopts a sequential paradigm, where design elements are generated sequentially. We argue that such a sequential scheme falls short of faithfully capturing the dependency between the background and layout (and thus the joint image-layout distribution), which limits the quality of generated design templates. To overcome this limitation, we propose a model, InterIL, which jointly generates the two modalities, background image and layout, in a single generative process. The novel design of our joint model connects the backbones of pretrained image and layout diffusion models with a learnable communication module to explicitly model bidirectional image-layout interaction. During training, the image and layout backbones are frozen to maintain and leverage the vast pretrained single-modality prior knowledge, while only the communication module is updated, so that the model can focus on learning image-layout interaction and thereby better capture the joint image-layout distribution for improved composition harmony. Our model has no design-specific inductive bias, which allows it to better preserve the original characteristics of realistic designs. We further introduce a test-time guidance strategy to enable users to impose their specific preferences on generated results. Our experiments show that, compared with prior approaches, our model can generate significantly better results in terms of image, layout and image-layout harmonization, producing outputs closer to real samples. We also demonstrate the flexibility of our model in enforcing user preferences at inference without retraining.