Temporal encoding improves relational graph learning accuracy by up to 3 percent
Temporal Heterogeneous Graph Pretraining for Relational Deep Learning
Machine Learning
Summary
Predicting information from complex databases is hard because the relationships and the timing of data records are important but tricky to model. The authors show that using two different ways to represent time—how old a record is and the exact intervals between records—helps improve learning in graph-based models. They created a method that first learns neighborhood patterns and then refines predictions using historical and future relationships. Testing on multiple datasets showed this approach performs better than training without these time techniques or pretraining.
What this means in practice
- •For database developers: Enhance prediction accuracy in relational databases by incorporating two distinct temporal signals in graph-based deep learning models.
- •For financial risk analysts: Improve classification and regression models for time-sensitive financial data using temporal encodings that respect record age and intervals.
Authors
Yixin Peng, Er Jin, Diego Collarana, Stefan Decker
Abstract
Relational deep learning models database rows and foreign-key links as a heterogeneous graph for prediction from record attributes and relational context. These graphs contain two distinct temporal signals: record age changes with the prediction cutoff, while intervals between observed records remain fixed. Prior work often treats time as a single signal or studies temporal representation and pretraining separately. We investigate how explicitly encoding both signals affects temporal pretraining for downstream tasks. Our framework combines Multi-scale Time Encoding, which captures record age using learnable time scales and type-specific projections, with Rotary Time Encoding, which represents signed inter-record intervals through rotary transformations during graph propagation. We pair these encodings with three self-supervised objectives: historical relation recovery, horizon-aware future relation activity prediction, and temporal subgraph contrast. All inputs respect their observation cutoffs. Pretraining proceeds in two stages: subgraph contrast first learns neighborhood representations, followed by refinement through either relation recovery or future activity prediction. We evaluate on five RelBench datasets across 11 classification and regression tasks using heterogeneous GNN and graph Transformer backbones. With both encodings, the best evaluated staged schedules improve over supervised training with the same encodings by 3.02% and 1.06% on the two backbones, respectively, and over controls without pretraining or either encoding by 3.24% and 2.37%.