Natural language queries improved by clear knowledge graph vocabulary
Natural Language Knowledge Graph Query Execution: Leveraging Controlled Semantics in the LLM Context Window
DatabasesArtificial Intelligence
Summary
It can be hard for AI to understand database questions when the words used are unclear or informal. The authors show that using a precise vocabulary with defined meanings helps the AI write the right database queries from natural language. They built a system called NLKGQ that uses a formal ontology to teach the AI how data is structured. This approach works better across different datasets and questions, making it easier to get accurate answers from complex knowledge graphs.
What this means in practice
- •For database developers: Generate precise database queries in SPARQL directly from natural language using machine-readable ontologies to define data terms clearly.
- •For data integration teams: Use wrapper ontologies to replace unclear database vocabularies, enabling natural language querying over multiple diverse databases with consistent term meanings.
Authors
Blake G. Fitch
Abstract
Large Language Model (LLM) applications often transfer domain concepts into the model's context informally, through prompt prose, schema dumps, and examples. We show that for database queries, data model concepts pass to LLMs more effectively through representations whose vocabulary terms carry declared, machine-readable semantics (controlled semantics). NLKGQ is a working system and reusable framework that does this for data modeled in a knowledge graph. A formal OWL ontology serves as the transfer mechanism, concentrating the meaning of the data into semantically precise tokens the model can use directly. In a single LLM call, NLKGQ places in the context a system prompt instructing on SPARQL, the complete domain OWL ontology, and a domain-specific prompt addition, together with the user's natural language query. The model then generates the SPARQL query directly, zero-shot. Where the native vocabulary of an existing database or federation of databases is opaque, a wrapper ontology substitutes clean terms and a runtime rewriter restores the native forms. Evaluating on DBLP-QuAD 2.0 showed that its scores depend on the graph snapshot, the endpoint used, and the wording of its machine-generated questions, so we propose DBLP-QuAD 3.1, which maintains the intent of 2.0 while making reference results deterministic, revising reference SPARQL where needed, and rewriting the natural language questions, with a frontier model, to state each reference query's intent clearly and completely. We evaluate on the DBLP-QuAD 2.0 benchmark (57.6% Match under deterministic re-scoring), DBLP-QuAD 3.1 (89.9% Match on 1,000 questions), SemOpenAlex (98% Match against a published baseline's 86% on the identical test set), and neuroimaging metadata (100%).