GRATE: Temporal Extensions for Inductive KG Foundation Models via Gated Rotary Attention
2026-07-11 • Artificial Intelligence
Artificial Intelligence
AI summaryⓘ
The authors address the challenge of making temporal knowledge graph models that can work well with new, unseen data involving different entities, relations, and times. They introduce GRATE, a method that encodes time by adjusting messages based on when events happened relative to each other, without adding extra learnable parts. GRATE can be combined with existing knowledge graph models while keeping their ability to generalize. To test this, the authors created new benchmarks with completely different vocabularies for training and testing. Their results show that a model using GRATE performs better in predicting and understanding temporal data across different datasets.
Knowledge GraphTemporal Knowledge GraphInductive TransferEntity RepresentationRelation EncodingRotary AttentionNBFNetTemporal EncodingBenchmark DatasetForecasting
Authors
Jiaxin Pan, Osama Mohammed, Daniel Hernández, Steffen Staab
Abstract
Knowledge graph foundation models such as Ultra and Trix achieve strong inductive transfer by learning relation-graph representations that generalise to unseen entities and relations. Extending this transferability to temporal knowledge graphs (TKGs) remains challenging: existing temporal models tie their parameters to dataset-specific entities, relations, or timestamps and are not designed to transfer to TKGs with disjoint vocabularies. We propose GRATE (Gated Rotary Attention for Temporal Encoding), an entity-side message function that adds no learnable parameters and encodes time through relative time differences by rotating each edge message according to its time gap to the query and applying a query-conditioned gate to select temporally relevant signals. GRATE integrates into NBFNet-style KG foundation models while preserving structural transferability. Existing TKG benchmarks evaluate within shared train/test vocabularies and cannot directly test cross-dataset temporal transfer; we therefore construct GDELTIndT and WIKIIndT, inductive transfer benchmark suites with disjoint entities, relations, and timestamps spanning both interpolation and extrapolation. Across these benchmarks and held-out forecasting datasets, a single jointly pretrained GRATE checkpoint improves over the static base model in most settings.