Ontologies partly catch and partly miss errors in LLM-based graph answers
What Does an Ontology Catch? Curated, LLM-Generated and Data-Derived Ontologies as Checkers of LLM Agents over Knowledge Graphs
Artificial IntelligenceDatabases
Summary
When using language models to answer questions on large knowledge graphs, people often rely on ontologies—formal descriptions of how data is structured—to check if answers make sense. The authors studied different kinds of ontologies: those carefully created by experts, those automatically generated by language models, and those derived from the data itself. They found that no ontology type is perfect: expert ontologies can wrongly flag many good answers, while others miss errors. Combining ontologies or giving feedback to the language model can help, but errors and false flags remain common.
What this means in practice
- •For knowledge graph engineers: Improve error detection strategies by comparing curated, data-derived, and LLM-generated ontologies used as checkers over large graphs.
- •For ai system builders: Design more reliable language model question-answering agents by understanding the tradeoffs in ontology-based answer validation and feedback approaches.
Authors
Michal Orzechowski, Rund Tawfiq, Tomasz Kosciolek
Abstract
LLM agents that answer questions over Web knowledge graphs are increasingly wrapped in checks derived from the graph's ontology, on the assumption that a formal schema makes a reliable judge. Yet a checker can be wrong as well: it can miss errors, and it can flag correct answers and push the agent into "fixing" them. Whether a curated ontology is worth its cost as such a judge, and how it compares with an ontology an LLM writes or a schema mined from the data, is unknown. We make the checker itself the object of study. Across curated, LLM-generated and data-derived ontologies, used as prompt context, as a checker, or both, and across open-weight and proprietary agents on public graphs and a private graph of nearly $10^{9}$ triples, we account for what ontology-derived checks catch and what they wrongly flag. We show how each source trades caught errors against false alarms, how a curated schema used as published can contradict its own data, which errors structural checks miss, and whether combining sources or giving the agent the checker's specific feedback pays off. On UniProt, the curated ontology flags more correct answers than either an ontology derived from the data or one an LLM derives from a summary of the data with masked identifiers. On real agent answers, its flags are only marginally more precise than random flags.