TimelyRAG improves question answering on changing documents with overlapping updates
TimelyRAG: Semantic-Temporal Hybrid Retrieval for Time-Critical Question Answering in Overlapping-Evolving Documents
Information Retrieval
Summary
Information like laws and policies often change over time by updating existing documents rather than replacing them entirely. This makes it hard for computer systems to find the right and most recent version to answer questions accurately. The authors developed TimelyRAG, a method that considers both the meaning and timing of updates to better match questions with the correct document version. They also created a benchmark called TimelyQABench to test these challenges in regulation-heavy texts, showing that accounting for document timing greatly improves question answering accuracy.
What this means in practice
- •For legal technology teams: Enhance automated legal question answering by aligning queries with correctly updated versions of regulations and policies.$Commercial implications: Enables legal tech companies to build tools offering more reliable, up-to-date regulation answers, improving compliance services.
- •For policy analysis teams: Support accurate retrieval of relevant policy documents when amendments partially overlap older versions for timely decision-making.
Authors
Youngeun Nam, Joeun Kim, Hwanjun Song, Susik Yoon, Jae-Gil Lee, Byung Suk Lee
Abstract
Although large language models (LLMs) and retrieval-augmented generation (RAG) have advanced open-domain question answering (QA), they remain unreliable when documents evolve through amendments. Existing time-sensitive retrieval methods address only the disjoint-evolving environment, where each update is an independent snapshot. However, laws, policies, and regulations often operate in overlapping-evolving environments, where amendments override earlier clauses while preserving most content, creating strong semantic overlap across versions. We propose TimelyRAG, a retriever-agnostic framework that incorporates temporal distance into ranking to align queries with version-appropriate documents. We also introduce TimelyQABench, the first benchmark for regulation-heavy domains with overlapping-evolving challenges. Experiments show consistent gains, up to +28.6% in nDCG@10, highlighting the importance of temporal reasoning for reliable QA over evolving documents. All resources are available at https://github.com/kaist-dmlab/TimelyRAG.