SVGEval: A Vision-Grounded Framework for Perceptual-Quality Benchmarking and Evaluation in Text-to-SVG Generation
2026-08-03 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors created SVGEval, a new way to judge the quality of SVG images made by AI models that considers how the images actually look, not just the code behind them. They found that many models do well when checking meaning and how nice an image looks but have trouble with the shapes and layout. The authors also made a tool that explains why an SVG is good or bad by looking at different parts and giving reasons. Their work helps improve how we test and make AI-generated SVG images by focusing on both visuals and the details that matter.
SVG (Scalable Vector Graphics)Multimodal modelsVisual renderingSemantic alignmentGeometry assessmentSpatial compositionHuman evaluationExplainable AIQuality scoringRaster-image metrics
Authors
Yiming Wang, Ye Chen, Hanqi Chen, Bingbing Ni
Abstract
Multimodal large models are increasingly used to generate scalable vector graphics (SVG), but reliable evaluation remains underexplored. Existing protocols are often code-centric or borrow raster-image metrics after rendering SVGs, which fail to reflect human perception and overlook SVG-specific qualities such as geometry and spatial composition. We introduce SVGEval, a vision-grounded multimodal benchmark for human-aligned SVG quality assessment. SVGEval explicitly incorporates visual renderings to evaluate whether models can judge the rendered outcome rather than only inspect SVG code, and provides high-quality annotations obtained via multi-round human labeling with expert refinement. Systematic evaluations across representative multimodal models reveal a clear gap: models perform relatively well on semantic alignment and aesthetics, yet struggle on geometry- and layout-related judgments. Building on SVGEval, we train an explainable SVG quality scorer that outputs multi-aspect scores with textual rationales. Ablations show that explicit visual grounding and reasoning supervision are crucial, especially for spatial and geometric assessment. SVGEval offers a reliable testbed and practical scorer for evaluating and improving SVG generation in the era of multimodal models.