Framework maps interactive visual controls for image generators
A Design Space for Visual Interfaces for Generative Image Models
Human-Computer Interaction
Summary
It can be hard to compare different ways people use interactive tools to change computer-generated pictures because each tool looks very different. The authors studied 51 different systems and made a simple model with three parts that describe how these tools work together: what the user sees and does, what the computer model can change, and how user actions turn into model changes. This framework helps find common patterns and gaps in how these tools are designed. They also built a tool to explore these systems so others can learn from and improve them.
What this means in practice
- •For ui designers: Access a shared framework to compare and combine controls in generative image tools for better user experiences.
- •For ai developers: Use the design space to guide new interaction features that improve how users can manipulate image generation models.
A survey. It maps existing work.
Authors
Susie S. Y. Li, Mingwei S. G. Li, Remco Chang
Abstract
Interactive visual interfaces have become an important means of controlling generative image models, enabling users to manipulate generation through prompts, direct manipulation, and a range of interactions. However, existing techniques are typically presented as independent systems, making it difficult to understand how they relate, compare their interaction mechanisms, or identify opportunities for new interface designs. We introduce a design space for interactive visual interfaces for generative image models derived from 51 research systems and practitioner tools. The framework decomposes each system into three complementary components: the user interface (U), the controllable model objects (Z), and the mapping function ($φ$) that translates user interaction into model operations. This decomposition provides a common representation for analyzing heterogeneous interaction techniques across model families, revealing recurring design patterns and underexplored regions of the design space. We further present an interactive corpus explorer that support comparative analysis, and discuss usage scenarios for both educational settings and HCI/AI practitioners identifying research and design opportunities.