PolyGandr lets users steer AI models by navigating hidden spaces
PolyGandr: Steering Generative Models via Spatial Navigation through Latent Space
Human-Computer Interaction
Summary
Generative AI models create content like pictures or designs mostly through text instructions, which can be hard for detailed edits or exploring options. The authors present PolyGandr, a visual tool that lets users explore and control AI creations by moving around in an abstract space representing design variations. This method helps users start from one point and steer creations in directions they choose, adjusting details on the fly without rewriting text prompts. The authors tested PolyGandr on face images, chair designs, and engineering shapes, showing it fits well with current AI workflows and supports both broad exploration and precise edits.
What this means in practice
- •For graphic designers: Create and fine-tune face images or product designs interactively by navigating visual model representations without complex text prompts.
- •For mechanical engineers: Manipulate and explore engineering shapes like kirigami structures directly through a visual interface to enhance design iterations.
Authors
Mingwei S. G. Li, Susie S. Y. Li, Isabel Moreira de Oliveira, Ryan P. Adams, Remco Chang
Abstract
Generative models support content creation across text, images, videos, and design disciplines, yet their interfaces rely primarily on natural language. While effective for specifying high-level intent, language can be inefficient and imprecise for fine-grained control, iterative refinement, and open-ended exploration. We present PolyGandr, a visual interface for exploring relationships between latent representations and generated outputs, complementing natural-language model steering. PolyGandr formulates generative control as high-dimensional navigation, enabling users to define starting points, steering directions, and step sizes through a unified map-and-mixing-palette interface. Combining global exploration with local refinement supports both broad design-space navigation and fine-grained output manipulation without repeatedly translating visual intent into prompts. We evaluate PolyGandr across three use cases: StyleGAN2 face generation, Stable Diffusion chair design, and rotational kirigami engineering. A formative study indicates that PolyGandr complements existing workflows by supporting real-time editing while simultaneously providing an overview and means to explore the broader design space.