MOF-Verify improves verification of metal-organic framework hypotheses
MOF-VERIFY: A Failure-Aware Agentic Harness for MOF Hypothesis Verification
Artificial IntelligenceComputational Engineering, Finance, and Science
Summary
Metal-organic frameworks (MOFs) are complex materials with tricky details spread across many sources, making it hard to check scientific ideas about them. The authors created a set of tests to figure out where AI models make mistakes when verifying MOF-related hypotheses, such as recognizing structures or checking conditions. Using these insights, they built MOF-Verify, a system that carefully checks different potential problems before giving a final answer. This system was tested on various AI models and showed better performance than simpler methods.
What this means in practice
- •For materials engineers: Verify complex MOF synthesis hypotheses by combining literature and computational evidence with higher accuracy than existing AI tools.
- •For chemical data integrators: Detect and handle conflicting or insufficient MOF data from heterogeneous sources to improve database consistency and reliability.
Authors
Donghyun Lee, Taehoon Lee, Geonhee Ahn, Jieun Kim, Jihyun Park, Suyeon Cho, Yoona Kim, Chaerim Shin, Hoi Ri Moon, Jonggeol Na, Sukho Hong, Jihwan Oh, Soo Kyung Kim
Abstract
Large language models are increasingly used as reasoning components in AI-driven materials Co-Scientists, yet the reliability of the resulting verification pipeline remains unclear. Metal-organic frameworks (MOFs) provide a particularly challenging setting because structures may appear under different identifiers, synthesis outcomes depend strongly on experimental conditions, evidence is distributed across heterogeneous sources, and some hypotheses require computation rather than literature alone. We introduce a diagnostic benchmark with four task families covering structural grounding, synthesis-condition verification, evidence-sufficiency verification, and MLIP-based computational verification. T-MOF-1-3 are evaluated under closed-book, retrieval-enabled, and oracle-evidence settings to localize failures in knowledge access, evidence acquisition, and reasoning, while T-MOF-4 separately evaluates computational verification. Guided by these diagnosed failure modes, we develop MOF-Verify, a failure-aware agentic harness that targets structural, literature, evidence-sufficiency, and computational bottlenecks before producing a final verdict. Across multiple backbone LLMs, MOF-Verify substantially improves hypothesis-verification performance over direct inference and retrieval-based baselines. Benchmark datasets are released at https://github.com/IMMS-Ewha/MOF-Verify-Benchmark.