Ladder boosts multi step question answering speed and accuracy
LADDER: Graph-Guided Diffusion Language Models for Efficient Multi-Hop Reasoning
Artificial Intelligence
Summary
Answering questions that require connecting several facts is difficult and slow for current language models. The authors created LADDER, a new method that speeds up this process by letting the model check its progress more efficiently using graphs. LADDER predicts important pieces of the answer early, allowing it to focus on the right information and finish faster. Tests show that LADDER answers more questions correctly and cuts the time needed to reason by about four times.
What this means in practice
- •For customer support teams: Provide faster and more accurate answers for complex multi-step customer queries using graph-guided language models.
- •For data analytics engineers: Integrate efficient multi-hop reasoning to extract insights from structured data more quickly in large-scale analytics pipelines.
Authors
Senlei Zhang, Linhao Luo, Qian-Wen Zhang, Siyu An, Junnan Dong, Shuhao Zhang, Xing Sun
Abstract
Graph Retrieval-Augmented Generation (GraphRAG) has remarkably enhanced large language models on complex reasoning by leveraging structured entity topologies. However, existing frameworks heavily rely on standard autoregressive language models where the nature of inherent sequential generation severely hinders overall inference efficiency. Inspired by Diffusion Language Models (DLMs) that offer massive parallelism via continuous refine-in-parallel decoding, we aim to accelerate GraphRAG in the discrete space. However, it remains non-trivial for two challenges. First, partially denoised drafts are highly dynamic and uncertain, making dynamic graph grounding non-trivial. Second, raw denoising states are inherently noisy and unstable, making synchronous graph retrieval and multi-hop aggregation computationally prohibitive. To this end, we present LADDER, a novel framework that bridges diffusion language modeling with GraphRAG through graph-guided parallel decoding. Specifically, (i) we propose an event-driven self-clocking retrieval, inspired by our key insight that 88% of target entities emerge early in the partially denoised state, leading final commitment by an average of 5.7-9.6 steps. This mechanism dynamically triggers graph retrieval only when the set of graph-linkable entities expands, yielding an asynchronous self-clocking policy that bypasses learned gates or heuristic thresholds. (ii) An incomplete-query graph propagation module is designed to process the newly emerging entity queries using a specialized graph foundation model, continuously aggregating multi-hop evidence to sharpen parallel predictions and accelerate overall decoding convergence. Extensive experiments on three challenging multi-hop QA benchmarks show that LADDER raises average exact match from 39.6% to 45.2% while achieving a 4.1x latency reduction.