TemporalGraphLLM combines language models and temporal graphs for better analysis
TemporalGraphLLM: Temporal Graph Neural Networks with Large Language Models for Dynamic Text-Attributed Graphs
Machine Learning
Summary
Dynamic graphs where connections and text change over time are hard to analyze well. TemporalGraphLLM combines two powerful tools: temporal graph neural networks, which understand evolving connections, and large language models, which understand text deeply. By merging these, the authors create a system that better predicts relationships and generates text based on these changing graphs. Their experiments show this combined approach works better than previous methods.
What this means in practice
- •For social media analysts: Improve detection of evolving relationships and influence in social networks by combining text and connection changes over time.
- •For knowledge graph engineers: Enhance dynamic knowledge graph reasoning and update predictions by integrating temporal graph methods with advanced text understanding.
Authors
Moran Beladev, Or Eitan, Gilad Katz, Lior Rokach
Abstract
Dynamic text-attributed graphs (DTAGs), where nodes, edges, and textual attributes evolve over time, are crucial in applications such as social networks, citation graphs, and knowledge graphs. However, existing approaches struggle to jointly model the temporal evolution of graph structures and the semantic richness of textual attributes. While Temporal Graph Neural Networks (TGNNs) capture evolving node relationships, they often lack contextual text reasoning. Conversely, Large Language Models (LLMs) excel in textual understanding but struggle with structured graph reasoning in temporal settings. To bridge this gap, we propose TemporalGraphLLM, a novel framework that can integrate any temporal GNN with an LLM for enhanced reasoning in DTAGs. Our approach fine-tunes LLMs using graph-time-aware instruction tuning and novel temporal GNNs injection to replace dedicated added tokens with graph embeddings. TemporalGraphLLM effectively leverages pretrained TGNNs within an LLM framework to achieve state-of-the-art performance on edge classification, link prediction, and edge-based text generation tasks. Extensive evaluation on real-world dynamic graph datasets demonstrates state-of-the-art performance. Our findings highlight the synergistic potential of LLMs and TGNNs, opening new directions for learning on evolving graphs.