NeSy-RAG: Neuro-Symbolic RAG for Explainable Question Answering
2026-08-06 • Computation and Language
Computation and LanguageSymbolic Computation
AI summaryⓘ
The authors present NeSy-RAG, a system that improves how large language models answer questions by linking their reasoning steps clearly to the evidence they use from texts. They convert pieces of text into Prolog rules that represent logical claims and combine these with user information to answer queries transparently. The system also detects when important user details are missing and asks follow-up questions to fill those gaps. Tested on a benchmark without extra training, their approach gave more accurate answers than a similar standard method.
Retrieval-augmented generationLarge language modelsNeuro-symbolic reasoningPrologPredicate logicKnowledge-gap detectionExplainabilityShARC benchmarkNatural language-code embeddingsQuestion answering
Authors
Jonas Gann, Michael Gertz
Abstract
Retrieval-augmented generation (RAG) improves question answering by grounding large language models (LLMs) in external knowledge such as text corpora. However, its reasoning process remains largely opaque: intermediate reasoning steps are difficult to verify and cannot be reliably attributed to specific evidence. Moreover, missing user-specific context is rarely detected systematically, often leading to incomplete or incorrect output. We propose NeSy-RAG, a modular neuro-symbolic RAG framework that synthesizes attributable Prolog modules from retrieved text chunks. For each chunk, the system generates semantically meaningful predicates that encode Boolean claims, which may depend on user facts. Using joint natural language-code embeddings, predicates are retrieved and composed into Prolog queries. To address incomplete user context, we introduce a symbolic knowledge-gap detection mechanism that identifies missing user facts whose truth values affect the query outcome and automatically triggers follow-up interactions. Executing the resulting Prolog queries yields deterministic answers together with transparent execution traces that link each reasoning step to its originating source. On the ShARC benchmark, without domain-specific training, NeSy-RAG achieves 61.1% accuracy, outperforming a same-model RAG baseline that achieves 42.8% accuracy.