Users and large language models co-design flexible spatial layout concepts
Co-DSL: Human-LLM Co-design of Reusable Spatial Concepts for Layout Generation
GraphicsArtificial IntelligenceComputer Vision and Pattern Recognition
Summary
Designing complex layouts, like building floor plans or graphic arrangements, often requires reusable spatial rules. The authors introduce Co-DSL, a way for people to work together with large language models (LLMs) to create and refine personalized layout concepts. Users describe a concept, the LLM creates a starting version, and users give quick feedback to improve it. Later, the LLM uses these concepts to automatically generate new layouts that follow the user’s request, helping to easily explore many options without starting from scratch each time.
What this means in practice
- •For architectural design teams: Create and customize spatial rules collaboratively with AI to generate building layouts that meet specific client needs automatically.
- •For graphics software developers: Integrate AI-assisted reusable layout concepts into design tools to help users explore multiple graphic arrangements without manual reconfiguration.
Authors
R. Kenny Jones, Jiajun Wu, Maneesh Agrawala
Abstract
We present Co-DSL, a framework in which users and LLMs co-design reusable spatial concepts for layout generation. In the first stage, our system allows users to create a library of personalized operations. To add a concept, a user provides a description, which the LLM converts into an initial implementation. The user can then observe the concept's execution behavior over example scenarios and provide lightweight feedback to the LLM, which revises the concept's implementation logic. In the second stage, given a library, an LLM can author a layout program that meets a new layout request from a user. Executing a layout program instantiates a concept-graph, a declarative composition of library concepts. In this way, we require the LLM only to specify which concepts should hold, while we use a solver to search for a layout that satisfies them. Across five spatial layout domains, we find that Co-DSL libraries let LLMs capture layout patterns that are challenging for prior fixed-operation systems, and demonstrate that our concept-graph representation is needed for LLMs to reliably compose concepts together. We further find that Co-DSL lets LLMs author generative layout programs that expose high-level parametric controls users can manipulate to explore a distribution of related layouts without re-prompting.