Improving large language models for time-aware knowledge reasoning
Temporal Evidence-Parameter Co-Alignment for LLM Reasoning over Temporal Knowledge Graphs
Computation and LanguageArtificial Intelligence
Summary
Large language models struggle to understand events that happen over time because their memory is fixed and doesn’t change as real-world facts evolve. The authors propose a new technique, TEPCA, that helps these models align better with knowledge that changes over time by selecting relevant time-based information and adjusting how the model processes this information depending on when it happened. This approach improves how the models reason about sequences of events and relations that vary with time. They tested their method on benchmarks and it performed better than existing techniques.
What this means in practice
- •For knowledge graph engineers: Enhance temporal reasoning capabilities in knowledge graph systems to improve time-sensitive query answering and inference.
- •For nlp developers: Integrate dynamic, time-aware fine-tuning methods to boost language model performance on tasks involving evolving real-world facts like event timelines.
Authors
Jiapu Wang
Abstract
Large Language Models (LLMs) often struggle with temporally grounded reasoning because their static parametric memory is misaligned with the dynamic evolution of real-world knowledge. Although RAG and Parameter-Efficient Fine-Tuning (PEFT) provide practical ways to enhance LLMs, they remain insufficient for temporal reasoning. Existing retrieval methods typically prioritize semantic relevance while overlooking temporal validity, event order, and evolutionary dependencies, resulting in evidence that is relevant in isolation but temporally fragmented. Meanwhile, conventional PEFT methods usually employ fixed and input-agnostic low-rank updates, making it difficult to capture time-dependent entity semantics and relation drift. To address these limitations, we propose Temporal Evidence-Parameter Co-Alignment (TEPCA), a framework for LLM reasoning over TKGs. TEPCA aligns LLM reasoning with temporal knowledge at both the evidence and parameter levels. At the evidence level, it introduces time-constrained Dynamic Relevance-Guided Path (DRGP) pruning to preserve globally coherent temporal paths, together with Mutual-Information (MI)-based pruning to retain query-informative local facts under temporal constraints. At the parameter level, it integrates Dynamic-Gated LoRA (DG-LoRA), a timestamp-conditioned low-rank adaptation mechanism that dynamically modulates the effective adaptation subspace according to temporal context. This design enables LLMs to better capture fine-grained entity evolution and time-dependent relation semantics while preserving parameter efficiency. Experimental results on two TKG benchmarks demonstrate that TEPCA achieves state-of-the-art performance. Additional evaluations of DG-LoRA on eight commonsense QA datasets further verify its general input-adaptive capacity, scalability across LLM backbones, and interpretability.