Concept lifecycle model improves knowledge reliability and timing in language models
Concept-Grounded Attention: A Controlled Evaluation of Graph-Injected Attention, Temporal Versioning, and Epistemic Status
Artificial Intelligence
Summary
Language models often struggle to track facts that change over time and to tell if information is proven or just guessed. The authors created a way to represent concepts as evolving entities with clear time stamps and reliability labels. They also designed attention methods to help models focus on these concept graphs but found that simply adding graph structure does not always improve answers. Making the timing and trustworthiness of information clear does, significantly improving accuracy and reducing false claims in tests.
What this means in practice
- •For language model engineers: Build question-answering systems that better handle concepts evolving over time and distinguish proven facts from inferences using concept lifecycle modeling.
- •For information retrieval developers: Enhance retrieval-based systems to reduce unsupported assertions by integrating epistemic status protocols into knowledge representations.
Authors
Sachin Dev Duggal, Pradyumna Swarnalatha Ramanna, Alexandros Vassiliades
Abstract
Knowledge-intensive language-model systems typically represent external knowledge as text chunks or static graphs, with limited support for concept evolution, point-in-time reasoning, and distinctions between validated and inferred knowledge. We introduce the Concept Lifecycle Model (CLM), which represents concepts as persistent, graph-grounded, temporally versioned entities with explicit provenance and epistemic status, and Concept-Grounded Attention (CGA), which injects concept-graph structure into transformer computation through graph-biased self-attention (Form A) and gated cross-attention over concept nodes (Form B). We evaluate the framework in controlled settings using disabled-mechanism baselines. On 200 MuSiQue and HotpotQA questions with retrieval fixed, concept-graph retrieval recovers explicit multi-hop paths but does not improve evidence recall. Form A appears to steer attention, with 2.76 times more attention on gold than distractor concepts, but the same ratio occurs when Form A is disabled; the learned bias is negligible and no answers change. An identity-preserving Form B improves F1 from 0.188 to 0.221, but control concepts yield 0.213, indicating that most of the gain reflects added capacity. On LongMemEval, explicit temporal representation improves answer accuracy by 13 to 25 points across all tested generators, up to 122B parameters, while simplified CLM version resolution performs similarly to dated serialization because concept identity is not established reliably. On a synthetic source-independence task, protocol-derived epistemic status reduces unsupported assertions from 28% to 0.1% in a fine-tuned small model and from 19-68% to 0-5% in 72-122B models. Overall, the results support making temporal validity and epistemic status explicit, while showing that graph-attention diagnostics are not informative without disabled-mechanism controls.