Dynamic graph learning improved by simple low dimensional predictors
From Complex Histories to Simple Predictors: Predictive Dimension in Dynamic Graph Learning
Machine LearningArtificial Intelligence
Summary
Predicting future connections or interactions in continuously changing networks can be complicated because it usually requires looking at long histories. The authors find that only a few key aspects of the history really matter for making accurate predictions. They define a way to measure how many of these key predictive factors are needed, called the intrinsic predictive dimension. Using this insight, they build a much simpler and smaller model that predicts future interactions as well as much larger ones but with far fewer parameters.
What this means in practice
- •For recommendation system engineers: Build recommendation systems that predict future user-item interactions with models tuned to the intrinsic predictive complexity, reducing model size drastically.
- •For network monitoring teams: Implement lightweight dynamic graph models for real-time prediction of network events based on minimal necessary predictive information.
Authors
Hantong Feng, Duxin Chen, Wenwu Yu
Abstract
Existing continuous-time graph learning methods rely heavily on complex neural architectures to encode ever-expanding interaction histories. However, the future conditional distribution induced by complex histories can often be characterized by a low-dimensional predictive state manifold, implying that only a few predictive degrees of freedom are essential for future interactions. We therefore argue that continuous-time graph learning should identify low-dimensional states sufficient for future prediction rather than reconstruct the full history. To make this perspective measurable, we define and independently estimate the intrinsic predictive dimension, which quantifies the effective predictive degrees of freedom required by a given task. Specifically, we remove the history-independent marginal from the history-conditioned future distribution, decompose the resulting predictive residual operator, and estimate the minimum effective rank needed to preserve the predictive power of the full history. Experiments across datasets show that model performance saturation is largely determined by the intrinsic predictive dimension. Based on this finding, we propose PreDyG, a lightweight model whose capacity is matched to the intrinsic predictive dimension. PreDyG achieves superior predictive performance with only 0.07\% of the parameters of a representative complex baseline.