GeoMathCode: Understanding Interleaved Math-Code Reasoning for Geometry Problem Solving
2026-05-25 • Computation and Language
Computation and Language
AI summaryⓘ
The authors studied how to improve large language models in solving geometry problems by making the problem-solving steps clearer and more like how humans think. They introduced GeoMathCode, which uses program-like visual outputs to show intermediate steps in reasoning. Their experiments found that the model's reasoning and code generation processes can be separated internally, and training the model more carefully helps organize this reasoning better. They also discovered that the code structures contain richer mathematical information than just pictures.
mathematical reasoningmultimodal large language modelsgeometry problemsprogrammatic representationslatent spacesupervised fine-tuninghierarchical syntactic structuressymbolic manipulationvisual constructionsmulti-step reasoning
Authors
Yingji Zhang, Yong Dai, André Freitas
Abstract
Mathematical reasoning is a hallmark of human intelligence, requiring logical deduction, symbolic manipulation, and abstract thinking. Recent multimodal large language models (MLLMs) have demonstrated strong performance on geometry problems through multi-step reasoning. To better emulate human problem-solving, intermediate steps can incorporate auxiliary visual constructions, such as additional lines or points, which improve geometric interpretation and educational clarity. In this work, we introduce the GeoMathCode, where programmatic representations serve as intermediate visual outputs. We further conduct an in-depth analysis of the underlying reasoning geometry. Experimental results show that reasoning and code generation steps can be disentangled in the latent space, while supervised fine-tuning (SFT) makes the reasoning manifold more structured and informative. Moreover, hierarchical syntactic code structures emerge as disentangled latent subspaces, and contain more mathematical symbolic information than visual representations.