Bayesian updates improve node classification on changing graphs
Online Bayesian Node Classification on Inductive Graphs under Distribution Shift
Machine Learning
Summary
Classifying nodes (like people or items) in networks is hard when new nodes arrive and the data changes over time. The authors improve this by combining a fixed graph neural network with a Bayesian model for the last classification layer that updates its knowledge as new data comes in. This method better handles uncertainty and changing data compared to standard approaches. They tested their approach on several benchmarks and found it to be more accurate and better calibrated than competing methods.
What this means in practice
- •For network engineers: Maintain accurate node classifications in communication networks despite evolving traffic patterns and network changes by updating last-layer models online.
- •For cybersecurity teams: Improve detection of anomalous or malicious nodes in dynamic network graphs by quantifying uncertainty and adapting to distribution shifts in real time.
Authors
Jinwen Xu, Gonzalo Mateos Buckstein, Qin Lu
Abstract
On evolving graphs, node classifiers must satisfy two key requirements: inductive generalization to newly arriving nodes under distribution shift and calibrated uncertainty for safety-sensitive applications. Standard graph neural networks (GNNs) are typically trained once and address neither requirement. We adapt the Bayesian last-layer (BLL) model by placing random last-layer parameters on top of a deterministic GNN encoder for uncertainty quantification. The categorical softmax likelihood required for classification breaks Gaussian conjugacy, so neither the training posterior nor the test-time streaming update has a closed-form solution. To address both challenges, we introduce a variational Bayesian last-layer (VBLL) objective that jointly trains the encoder and an approximate last-layer posterior by maximizing an evidence lower bound with a Monte Carlo expected log-likelihood. At test time, we freeze the encoder and apply an online Laplace update to the last-layer posterior. This update corresponds to a power-prior Bayesian model with exponential forgetting and a Kullback-Leibler anchor to the training posterior. Across five node-classification benchmarks under distribution shift, online GVBLL is the only method to achieve the best accuracy and negative log-likelihood on every dataset. It improves accuracy by up to 17 percentage points on Cora and 14 percentage points on ogbn-arxiv over the strongest non-GVBLL baseline, while remaining competitive in calibration with MC Dropout, Deep Ensembles, Temperature Scaling, and Gaussian-process classifiers.