Multimodal models struggle to link physics diagram parts to roles accurately
PhysAlign: A Benchmark for Evidence-Grounded Role Alignment in Multimodal Physics Reasoning
Artificial Intelligence
Summary
Understanding physics diagrams can be tricky because even if a model sees the right numbers or symbols, it might connect them to the wrong part of the problem. The researchers created PhysAlign, a test to check if AI can match diagram elements to their correct physical meanings. They found that many AI systems recognize parts correctly but still often mix up their physical roles, leading to mistakes. This shows that just seeing the diagram well is not enough; AI also needs to understand what it means physically.
What this means in practice
- •For educational technology developers: Improve automated physics tutors by ensuring AI correctly links diagram details to physics concepts for clearer problem explanations.
- •For computer vision engineers: Enhance multimodal AI systems to better ground visual physics evidence, reducing errors in interpreting technical diagrams.
Authors
Kecheng Liang, Haoyang Liu, Zexin Chen, Zirong Liu, Weixing Chen, Qiufeng Wang, Yang Liu, Liang Lin
Abstract
A key challenge in physics diagram understanding is correctly associating visual information with the physical entities, relations, and conditions it describes. Even when a value, symbol, or other local element is accurately recognized, assigning it to the wrong entity or scope can distort the underlying physical premise and lead to incorrect reasoning. To systematically study this challenge, we introduce \textbf{PhysAlign}, a benchmark designed to assess whether multimodal models correctly associate information recognized from physics diagrams with its intended physical role. By disentangling visual recognition from physical-role assignment through localized probes and controlled variants, PhysAlign isolates correspondence errors from recognition failures. It contains 3,341 human-validated probes spanning 986 physics problems, enabling systematic evaluation of visual recognition and physical-role correspondence at scale. We further introduce five complementary evaluation metrics, including CAcc, GAcc, and JAcc, which provide a comprehensive assessment of models' ability to recognize diagram content, establish correct physical correspondences, and solve the underlying physics problem. Across our evaluated multimodal models, PhysAlign reveals a consistent gap between local visual recognition and physical-role grounding. Even when the queried content is correctly recognized, the conditional correspondence error rate remains 13.8\% for GPT-6-Astra and rises to about 50.6\% for InternVL3.5-8B. These findings indicate that strong perception alone does not ensure reliable physical interpretation, exposing a distinct grounding bottleneck that is largely hidden by answer-level accuracy and highlighting the need for future models to better align recognized visual evidence with its physical meaning.