Unify graph models and large language models for better reasoning and retrieval
UniGFM-RAG: Unify Graph Foundation Models and Large Language Models for End-to-End Retrieval and Reasoning
Artificial Intelligence
Summary
Sometimes computers must find and combine many pieces of information to answer complicated questions, but they find this hard. The paper’s authors build a system that links graph-based models, which understand connections between facts, with language models that generate answers. Their method retrieves relevant information by reasoning over knowledge graphs and then uses that to help the language model give better answers. They also created a way to train the system efficiently without needing lots of manual labeling. Their approach works well on several tests and different types of questions.
What this means in practice
- •For ai product developers: Build systems that combine graph reasoning with large language models to answer complicated questions more accurately and efficiently.$Commercial implications: Enables creation of advanced question answering and retrieval products by tightly integrating graph models with large language models.
- •For enterprise knowledge management teams: Improve retrieval of complex, multi-source information by using end-to-end graph reasoning combined with language model generation to deliver better insights.
Authors
Zicheng Zhao, Linhao Luo, Zongji Yu, Junnan Dong, Yanchao Tan, Shirui Pan, Chen Gong
Abstract
Large Language Models (LLMs) have shown remarkable capabilities across tasks, but they still struggle with complex reasoning that requires integrating scattered knowledge. Graph Retrieval-Augmented Generation (GraphRAG) addresses this challenge by organizing fragment knowledge as graphs, while recent Graph Foundation Models (GFMs) further improve GraphRAG with graph-native retrieval and reasoning capabilities. However, existing GraphRAG methods usually adopt decoupled retrieve-then-generate pipelines, which create objective misalignment between retrieval and reasoning, and training GFM retrievers typically depends on expensive human annotations. To address these limitations, we propose \ourmethod, a unified end-to-end GraphRAG framework that tightly couples a GFM retriever with an LLM generator. Given a query, the GFM retriever first reasons over the graph to score and retrieve relevant document nodes. These node embeddings are then passed through a fully differentiable graph bridge, which projects them into soft-weighted graph prefixes according to retrieval scores and combines them with raw document texts to form graph-augmented prompts. In this way, structural information is injected directly into LLM generation, while the final answer loss can be back-propagated through the bridge to jointly optimize retrieval and reasoning. To stabilize training and reduce reliance on costly annotations, we further introduce a semantic-guided distillation module that provides auxiliary supervision using pseudo-labels derived from query-document semantic similarities. Extensive experiments on three multi-hop QA datasets and three out-of-domain GraphRAG benchmarks show that \ourmethod consistently outperforms strong baseline methods and generalizes well, which demonstrates the effectiveness of end-to-end GraphRAG with tightly integrated GFM retrieval and LLM reasoning.