VFIG: Vectorizing Complex Figures in SVG with Vision-Language Models
2026-03-25 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionArtificial Intelligence
AI summaryⓘ
The authors created VFIG, a set of AI models that can turn images of complex diagrams into editable SVG vector files, which are easier to modify and scale. Since existing datasets were small and simple, they built a large collection called VFIG-DATA with 66,000 figure-SVG pairs from real and generated diagrams. They trained their models using a two-step process that first learns basic shapes and then improves the overall layout and details. They also made a new test suite, VFIG-BENCH, to better measure how well these models keep the structure of diagrams. Their approach matches top models like GPT-5.2 in quality while being open source.
Scalable Vector Graphics (SVG)vectorizationVision-Language Models (VLM)dataset curationsupervised fine-tuningreinforcement learningdiagram reconstructionstructural integrity metricscoarse-to-fine trainingvector graphic primitives
Authors
Qijia He, Xunmei Liu, Hammaad Memon, Ziang Li, Zixian Ma, Jaemin Cho, Jason Ren, Daniel S Weld, Ranjay Krishna
Abstract
Scalable Vector Graphics (SVG) are an essential format for technical illustration and digital design, offering precise resolution independence and flexible semantic editability. In practice, however, original vector source files are frequently lost or inaccessible, leaving only "flat" rasterized versions (e.g., PNG or JPEG) that are difficult to modify or scale. Manually reconstructing these figures is a prohibitively labor-intensive process, requiring specialized expertise to recover the original geometric intent. To bridge this gap, we propose VFIG, a family of Vision-Language Models trained for complex and high-fidelity figure-to-SVG conversion. While this task is inherently data-driven, existing datasets are typically small-scale and lack the complexity of professional diagrams. We address this by introducing VFIG-DATA, a large-scale dataset of 66K high-quality figure-SVG pairs, curated from a diverse mix of real-world paper figures and procedurally generated diagrams. Recognizing that SVGs are composed of recurring primitives and hierarchical local structures, we introduce a coarse-to-fine training curriculum that begins with supervised fine-tuning (SFT) to learn atomic primitives and transitions to reinforcement learning (RL) refinement to optimize global diagram fidelity, layout consistency, and topological edge cases. Finally, we introduce VFIG-BENCH, a comprehensive evaluation suite with novel metrics designed to measure the structural integrity of complex figures. VFIG achieves state-of-the-art performance among open-source models and performs on par with GPT-5.2, achieving a VLM-Judge score of 0.829 on VFIG-BENCH.