Benchmark reveals differences in tracking structure and meaning in evolving graphs
TTGBench: Benchmarking Topological Evolution and Semantic Drift in Text-attributed Temporal Graphs
Artificial Intelligence
Summary
Understanding how networks change over time is important, especially when both their connections and meanings evolve. The authors introduce TTGBench, a set of tests using six real-world datasets that measure how well computer models follow these changes in structure and meaning together. They found that current methods tend to specialize: some are good at predicting connections but not meanings, while others understand meanings but not structure. This work highlights the need for better models that can handle both aspects simultaneously.
temporal graph learningstructural evolutionsemantic drifttemporal node classificationtemporal link predictiontemporal graph neural networkslarge language modelsmulti-label classificationdynamic systemsbenchmarking
Authors
Longfei Ma, Zemin Liu, Fei Wu
Abstract
Temporal graph learning models the evolution of dynamic systems, where both structural interactions and semantic states change over time. However, existing benchmarks primarily emphasize structural evolution via temporal link prediction (TLP), while support for semantic evolution remains limited. Although temporal node classification (TNC) is sometimes included, it is typically restricted to simplistic binary settings that fail to capture realistic semantic drift. Moreover, commonly used datasets exhibit high link repetition, leading to inflated performance estimates and obscuring true model capability. To address these limitations, we introduce \textbf{TTGBench}, a new benchmark that jointly evaluates structural and semantic evolution. TTGBench comprises six real-world, text-rich datasets characterized by \emph{Dual Volatility}, enabling rigorous and fair evaluation of existing models. Notably, it is the first benchmark to support both multi-class and multi-label TNC, filling a critical gap in evaluating temporal semantic drift. We conduct a comprehensive evaluation of 17 state-of-the-art methods across Temporal Graph Neural Networks (TGNNs) and Large Language Model (LLM)-based paradigms. The results reveal a clear \emph{capability divide} between the two paradigms: TGNN-based methods excel at structural prediction but fail at semantic tracking, whereas LLM-based predictors show the opposite trend. Through in-depth analysis, we uncover their fundamental limitations and provide insights for developing more comprehensive temporal graph models.