Graph neural networks speed up knowledge graph learning with less memory
Scalable GNN-based Knowledge Graph Representation Learning with Efficient Message Passing
Machine LearningArtificial Intelligence
Summary
Large knowledge graphs help computers understand relationships between things, but teaching computers to learn from these graphs often takes a lot of time and memory. The authors found a way to make this learning process faster and use less memory without losing accuracy by improving how information is passed between nodes in the graph. They extended an existing math technique to handle more complex operations efficiently. This means computer programs can analyze big knowledge graphs more quickly and affordably.
What this means in practice
- •For data engineers: Reduce runtime and memory needs when running graph neural networks on large knowledge graphs for data integration tasks.
- •For machine learning engineers: Implement more efficient message passing algorithms in graph neural networks to accelerate training on complex relational data.
Authors
Huu Tan Mai, Cuong Xuan Chu, Heiko Paulheim, Daria Stepanova
Abstract
Graph neural networks (GNNs) excel at representation learning on Knowledge Graphs (KGs), achieving stateof-the-art performance on tasks like link prediction or entity classification. However, their high computational complexity, inherent to their user-defined message passing (MP) algorithm, still prohibits their widespread adoption, especially for large KGs. Current efforts to mitigate the scalability bottlenecks of GNNs on KGs, such as subgraph sampling, are often task- and model-specific, and do not reliably guarantee lossless (if applicable) runtime/space reductions. To address this, we extend Relational Sparse Matrix Multiplication (RSPMM), originally designed to losslessly lower the space complexity of composition-based MP with pointwise composition functions, to support more expressive functions (e.g., 2x2 block-diagonal matrix multiplication, Givens rotation, circular correlation). Our method delivers significant task-independent reductions in runtime and space for current GNNs on KGs and facilitates efficient re-implementations of GNNs that maintain near state-of-the-art performance on challenging KG tasks, for a fraction of computational costs.