Vision language model improves editable svg creation with semantic structure
Compositional SVG Generation via VLM-Driven Hierarchical Semantic Parsing
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Creating editable vector graphics can be tricky because current methods produce disconnected shapes that are hard to modify individually. The authors propose a new approach where a vision-language model breaks down images into meaningful parts arranged in a hierarchy, making each part complete and easy to edit. They also introduce a new benchmark to measure how well these parts match human understanding and how easy they are to work with. Their method produces more accurate and user-friendly vector graphics than earlier techniques.
What this means in practice
- •For graphic designers: Generate complex vector images with clear, editable parts that match semantic concepts for easier customization and refinement.
- •For ui developers: Produce structured scalable vector graphics with components that can be programmatically accessed and edited based on their semantic meaning.
Authors
Sehwan Park, Taehoon Kim, Geonhee Han, Dohyun Kim, Seung Wook Kim, Paul Hongsuck Seo
Abstract
While Vision-Language Models (VLMs) excel at visual reasoning, generating structured, editable Scalable Vector Graphics (SVG) remains a fundamental challenge. Existing pipelines predominantly yield flat, semantically agnostic collections of paths, where editing a single object requires manually identifying its constituent paths. To address this, we propose a VLM-driven agentic framework for semantic compositional SVG generation. Our pipeline recursively parses visual scenes into semantic and geometric hierarchies via top-down decomposition, visual grounding, and prompt-driven amodal occlusion recovery, ensuring each component is geometrically complete. Furthermore, we introduce the Semantic SVG Benchmark with human-annotated semantic groups and novel sub-component metrics (Semantic Recall/Precision, PERE) to explicitly evaluate structural compositionality and functional editability. Experiments show that our natively predicted structures surpass the upper bounds of existing flat-generation methods in both grouping quality and editability, while maintaining state-of-the-art visual fidelity.