Multimodal benchmark reveals challenges in AI geometry reasoning
TriGeoBench: A Multimodal Mathematical Geometry Benchmark with Figures and TikZ
Artificial IntelligenceDatabases
Summary
AI models that read both text and images still struggle to solve geometry problems involving shapes and figures. The authors created a large test set from Chinese school math exams, including pictures and code describing the shapes logically. Testing current AI showed they often miss important details and don't fully understand the diagrams. This work offers a new way to measure and improve how AI handles math problems with pictures.
What this means in practice
- •For math software developers: Use the benchmark to improve AI tools that interpret geometry problems with diagrams in educational software.
- •For ai model trainers: Evaluate and enhance vision-language models’ ability to understand geometric figures beyond simple image patterns.
Authors
Jing Wang, Yitong Yang, Hao Dou, Yaomin Wu, Shuting He, Tingfei Shen, Jia Wu, Jiaqi Li, Yinglin Wang
Abstract
Multimodal Large Language Models show strong vision-language capabilities, yet their geometry-centric reasoning remains insufficiently understood. Existing mathematical benchmarks typically rely on rasterized figures and cover limited geometric knowledge areas and response formats, making it difficult to assess whether models reason over geometric structures or exploit superficial visual patterns. We introduce TriGeoBench, a geometry-centric multimodal benchmark constructed from Chinese middle- and high-school mathematics examinations. TriGeoBench contains 7,000 question-answer pairs and 12,235 geometric figures with TikZ code, spanning plane geometry, solid geometry, analytic geometry, function, and vector geometry across diverse response formats. Each instance forms a {question-answer pair, image, TikZ} triplet: question-answer pairs are stored in LaTeX, images as RGB figures, and TikZ preserves geometric topology, symbolic annotations, and relational structure. We evaluate representative open- and closed-source MLLMs through image-to-TikZ and multimodal reasoning probes. Results reveal persistent fine-grained structure and image-text alignment challenges. Oracle experiments with human-curated reference TikZ suggest that accurate executable representations provide complementary geometric cues; they do not evaluate an end-to-end conversion pipeline. TriGeoBench offers a testbed for structure-aware mathematical reasoning.