QRAKEN improves question-based access to knowledge graphs using guided patterns
Natural Language Questions as an Interface for Knowledge Graphs: QRAKEN Graph Distillation and Semantic Self-Healing
Artificial IntelligenceComputation and Language
Summary
Accessing information stored in knowledge graphs using natural language questions is hard because AI models often misunderstand unfamiliar data. The paper presents QRAKEN, a method that helps AI create better database queries by learning common patterns in the data without needing special training or schema knowledge. This approach improves accuracy significantly compared to previous methods. It also flags and fixes mistakes during query creation, making interaction with knowledge graphs more reliable.
What this means in practice
- •For enterprise software developers: Build more accurate natural language query interfaces for internal knowledge graphs by integrating QRAKEN’s pattern-guided query generation and iterative validation.
- •For knowledge management teams: Use QRAKEN’s graph distillation and self-healing approach to improve retrieving correct data from RDF graphs without needing costly manual schema annotations.
- •For digital assistants developers: Create conversational agents that generate more reliable semantic queries on enterprise data by adopting QRAKEN’s empirical pattern guidance and self-correction methods.$Commercial implications: Enables companies to sell smarter AI assistants with improved accuracy on custom data sources, reducing failures in semantic query tasks.
Tested on one dataset.
Authors
Remo Grillo, Lukas Klic, Giovanni Colavizza
Abstract
Natural-language access to RDF knowledge graphs is a core Semantic Web ambition. Large language models (LLMs) have advanced Text-to-SPARQL, yet on unfamiliar graphs they often generate valid queries that misrepresent the populated data model. QRAKEN is a training-free, ontology-agnostic neurosymbolic pipeline grounding generation in empirical graph evidence rather than schema expectations. An offline distiller produces TTQL, a compact description of populated multi-hop patterns, conditional frequencies and path-conditioned literal examples, plus a class-property co-occurrence matrix. Online, TTQL guides the LLM, while deterministic syntax, vocabulary and data-model checks provide diagnostics for iterative refinement. On CK25 (First International Text2SPARQL Challenge), under matched-condition recomputation on a QLever snapshot, QRAKEN achieves strict F1 of 0.643 $\pm$ 0.026 with GPT-4.1 mini and 0.652 $\pm$ 0.012 with GPT-5.4: relative gains of 30% and 32% over the strongest recomputed participant, outperforming systems using the same base model family. Ablations identify TTQL patterns as the dominant driver (+0.31 strict F1 over a shape-only baseline); the refinement loop provides a cheap safety net, rejecting triple patterns unsupported by the co-occurrence matrix. Compared with auto-derived SHACL, TTQL yields 64% higher strict F1, supporting the value of empirical patterns beyond schema exposure. With two local 35B 4-bit open-weight models at zero marginal cost, the same pipeline matches the strongest recomputed participant, and TTQL advantages over shape-only and SHACL baselines persist. Results on a single, relatively small benchmark provide an initial empirical signal; monolithic TTQL injection on very open cross-domain graphs remains the main limitation.