CircuitReason-1k: Benchmarking Long-Horizon Visual-to-Symbolic Reasoning inElectrical Circuits
2026-08-10 • Artificial Intelligence
Artificial IntelligenceComputer Vision and Pattern Recognition
AI summaryⓘ
The authors created a dataset called CircuitReason1K to test how well AI models can understand electrical circuit diagrams and solve related problems by turning images into correct symbolic answers. The dataset includes 1,000 real textbook problems with detailed solutions, carefully linking questions, images, and answers. They tested several AI models and found the best scored about 85% accuracy but struggled with more complex, step-by-step problems, especially in understanding circuit connections and physical rules. This benchmark helps measure how well AI can handle detailed reasoning involving both visuals and physics.
electrical circuitsvisual reasoningsymbolic reasoningmultimodal modelsbenchmark datasettopologyphysical conventionslarge language modelsproblem-solvinggrounding
Authors
Xinqi Yang, Kang An, Tengyue Wang, Zhongyu Yang, Chenxu Du, Yuanchi Zhu, Hebao Zhu, Ziliang Wang, Faqiang Qian, Yunli Yang, Qibing Ren
Abstract
Electrical circuit analysis requires more than recognizing components in an image. A solver must ground symbols and labels, recover latent topology, select a physical model, formulate coupled equations, propagate intermediate quantities, and preserve units, signs, directions, and phase conventions. We introduce \benchmark, a benchmark of 1,000 authentic textbook problems for evaluating this complete long-horizon visual-to-symbolic reasoning process. Each problem pairs one or more circuit diagrams with a self-contained question, a typed or semantically specified answer, and a reference worked solution. An evidence-first construction pipeline aligns questions, figures, and solutions, while a reasoning-oriented taxonomy organizes problems by circuit type and dependency depth. Evaluation combines conservative typed scoring with identity-blinded multi-model semantic consensus, retaining every problem in the denominator. Across three commercial chatbot systems and six open-source multimodal large language models, the highest-scoring system reaches 84.8\% accuracy. However, performance consistently deteriorates on long-horizon problems, and qualitative analysis exposes persistent failures in topology-to-target binding, physical conventions, and late-stage output propagation. \benchmark{} provides a focused testbed for measuring whether multimodal models can transform technical visual evidence into sustained, physically valid symbolic reasoning. Code are available at GitHub - CircuitReason/CircuitReason1K.