MatQuery enables natural language querying across multiple materials databases
MatQuery: Execution-Grounded Natural-Language Querying over Heterogeneous Materials Databases
Databases
Summary
Materials scientists often struggle to search different materials databases because each one uses different formats and terms. The authors created MatQuery, a system that lets users ask questions in natural language and translates them into queries across several databases. The system checks answers carefully and combines results, helping compare data from different sources. Early tests show promise, but also reveal challenges in fully answering complex questions and comparing data accurately.
What this means in practice
- •For materials data engineers: Integrate and query multiple materials science databases with a single natural language interface to streamline data retrieval.
- •For scientific software developers: Build tools that translate user's natural language requests into executable queries across heterogeneous scientific datasets.
Authors
Chen Jason Zhang, Qi Chen
Abstract
Materials scientists increasingly combine information from several public databases, each with its own interface, schema, units and calculation conventions. MatQuery is a prototype and experimental infrastructure for studying how a natural-language request can be turned into queries over such databases and how the returned values can be interpreted. This report describes what has been built and the preliminary results obtained so far. The prototype holds frozen, self-verifying local copies of six public materials databases (about 10,000 records each), a catalog of declared fields derived from them, a database-independent query representation with an adapter for each database, and an eight-step pipeline in which a language model proposes and deterministic program steps check, execute and assemble. A benchmark generator produces questions whose reference queries are verified by execution on the frozen copies and whose wording is checked by back-translation and review. An evaluation harness runs direct prompting, CHESS and Alpha-SQL on the same frozen basis. Preliminary experiments on an 800-question development set show that direct generation with three demonstrations reaches 82.4% strict execution accuracy (56.5% without demonstrations), that a low-cost scorer selects the exact set of databases for 85.0% of complete requests with well-calibrated probabilities, and that band gaps differ by more than 0.5 eV in 31.6% of composition-matched cross-database clusters. Two pilots show the limits of the current approach: a probe-then-ask policy gave answers rated valid by both reviewers in only 11 of 50 sessions, and only 115 of 300 sampled record pairs could be compared by structure. The complete pipeline is in smoke testing and has not yet been evaluated at scale.