Graph neural network improves node classification using shared experts
PreGS: A Parameter-Transfer-Based Multi-Expert Graph Neural Network for Node Classification
Machine Learning
Summary
Node classification is the task of labeling parts of a network, like identifying groups in social networks or categories in knowledge graphs. The authors found that using just one way to gather information from a network may miss important details. They developed PreGS, a model that shares learned parts from one network type to multiple others, creating a stronger overall system without extra training. Their improved version PreGSv2 adds ways to weigh and combine different information sources better. Tests on various networks show their approach is effective and stable.
What this means in practice
- •For data engineers: Improve automated labeling of nodes in large graphs by integrating multiple pretrained graph models without retraining all components from scratch.
- •For social network analysts: Use multi-source graph features to enhance detection and classification of user groups or topics in social media graphs with more stable results.
Authors
Zhicong Cai, Yinglong Zhang, Xiaoying Hong, Xuewen Xia, Xing Xu
Abstract
Graph neural networks have achieved strong performance in node classification by aggregating information from graph neighborhoods. However, a single aggregation mechanism may be insufficient to capture diverse structural patterns across graph datasets. Moreover, independently training multiple structural branches can introduce substantial overhead without necessarily producing stable node representations. To address these issues, this paper proposes PreGS, a parameter-transfer-based multi-expert graph neural network framework. PreGS first pretrains a multi-head graph attention network (GAT) and transfers the linear transformation weights of its first-layer attention heads to multiple GraphSAGE experts. The transferred experts are frozen and used as complementary structural branches. The fused raw node features, GAT head representations, and GraphSAGE expert representations are fed into a multilayer perceptron (MLP), whose output is further fused with the pretrained GAT logits. Based on PreGS, we further develop PreGSv2, which introduces source-level weighting and a structural gating mechanism for adaptive multi-source feature integration. Experiments on eight public graph datasets show that PreGS and PreGSv2 achieve competitive performance against representative graph neural network baselines. Ablation, parameter-transfer, sensitivity, aggregator, visualization, and training-time analyses further validate the effectiveness and stability of the proposed framework. The code and datasets are available at https://github.com/LH-Czc/PreGS.