Simple methods approach complex models for relational deep learning
Revisiting Simple Baselines for Relational Deep Learning
Machine Learning
Summary
Relational deep learning helps predict things from connected data, like in databases. The authors looked at how simpler methods, involving basic spreading of information across related data points, perform compared to more complex models. They found that simple techniques using neighborhood information and smart label spreading can nearly match the results of advanced graph neural networks. However, organizing the connections between the things being predicted remains a challenge. This work shows simpler approaches can be strong baselines for relational learning tasks.
What this means in practice
- •For database engineers: Use simple feature propagation combined with basic neural networks for predictive tasks over relational databases without complex models.
- •For knowledge graph developers: Improve label prediction by constructing meta-graphs for label propagation on temporal heterogeneous graphs.
Authors
Viet The Nguyen, Vijay Prakash Dwivedi, Gunther Gust
Abstract
Relational deep learning has emerged as a powerful paradigm for learning end-to-end predictive models over relational databases by representing them as temporal heterogeneous graphs. Recent advances based on graph neural networks and Transformers have achieved strong performance across diverse domains. However, the effectiveness of simpler relational baselines remains underexplored, making it difficult to assess how much architectural complexity is needed to achieve strong predictive performance. Motivated by the observation that predictive signals arise from target attributes, relational attributes, and historical labels, we revisit how effectively these signals can be exploited using simple propagation methods. We adapt fixed low-pass feature propagation to relational entity graphs and train an MLP for prediction. The resulting predictions are further refined using Correct-and-Smooth, a graph-based label propagation algorithm that incorporates historical labels over task-specific meta-graphs connecting target entities. Across classification and regression tasks on RelBench, an MLP trained on neighborhood-filtered features outperforms one using only target-entity features on most tasks and recovers a substantial portion of the performance of heterogeneous GNNs. Correct-and-Smooth provides further gains from historical labels, although applying it requires a separately constructed meta-graph because connections among target entities are not directly specified by the relational entity graph. Together, our studies revisit the effectiveness of established propagation methods as simple RDL baselines while identifying canonical meta-graph construction as a key challenge for effective label propagation.