Benchmark compares models tracking evolving network structure and meaning
TTGBench: Benchmarking Topological Evolution and Semantic Drift in Text-attributed Temporal Graphs
Artificial Intelligence
Summary
Tracking how connections and meanings change over time in networks is important but challenging. The authors point out that past tests mostly focus on how network links change and overlook changes in what those connections mean. They created TTGBench, a set of real-world datasets designed to test both structural and meaning changes together. They tested 17 advanced models and found some are good at predicting link changes, while others track meaning shifts better. Their findings reveal gaps and help guide future improvements in these models.
What this means in practice
- •For data science teams: Evaluate and select models that jointly predict evolving network connections and changing node meanings using a balanced real-world dataset.
- •For social network analysts: Measure and track both structural and semantic changes in social media interaction graphs for better understanding of evolving communities.
Authors
Longfei Ma, Zemin Liu, Fei Wu
Abstract
Temporal graph learning models the evolution of dynamic systems, where both structural interactions and semantic states change over time. However, existing benchmarks primarily emphasize structural evolution via temporal link prediction (TLP), while support for semantic evolution remains limited. Although temporal node classification (TNC) is sometimes included, it is typically restricted to simplistic binary settings that fail to capture realistic semantic drift. Moreover, commonly used datasets exhibit high link repetition, leading to inflated performance estimates and obscuring true model capability. To address these limitations, we introduce \textbf{TTGBench}, a new benchmark that jointly evaluates structural and semantic evolution. TTGBench comprises six real-world, text-rich datasets characterized by \emph{Dual Volatility}, enabling rigorous and fair evaluation of existing models. Notably, it is the first benchmark to support both multi-class and multi-label TNC, filling a critical gap in evaluating temporal semantic drift. We conduct a comprehensive evaluation of 17 state-of-the-art methods across Temporal Graph Neural Networks (TGNNs) and Large Language Model (LLM)-based paradigms. The results reveal a clear \emph{capability divide} between the two paradigms: TGNN-based methods excel at structural prediction but fail at semantic tracking, whereas LLM-based predictors show the opposite trend. Through in-depth analysis, we uncover their fundamental limitations and provide insights for developing more comprehensive temporal graph models.