Graph neural networks improve confidence calibration with local structure
HoTS: Homophily-Aware Temperature Scaling for Graph Neural Network Calibration
Machine Learning
Summary
When computers try to guess labels for things connected in a network, they also give confidence scores. These scores need to be accurate to know when to trust them or ask for human help. The authors found that knowing how similar a node’s neighbors are (called homophily) helps better adjust these confidence scores. They created a new method called HoTS that uses local network info to make these confidence scores more accurate across many tests.
What this means in practice
- •For machine learning engineers: Improve confidence scores in graph-based classification models, enabling safer automated decisions and better human review triggers.
- •For network security analysts: Enhance trustworthiness of node risk scores in network graphs to prioritize investigation of uncertain or risky nodes.
Authors
Inwoo Tae, Yoontae Hwang, Yongjae Lee
Abstract
For graph node classification, calibrated class probabilities are needed when confidence scores, usually the maximum predicted class probability, are used to rank predictions, defer uncertain nodes to human review, or control risk. Existing post-hoc calibrators either apply one global temperature or use graph-aware modules without a principled structural form. We study how local graph structure should enter node-level calibration. Our first results show that a logit-only temperature rule is insufficient when nodes with identical logits but different local homophily require different optimal temperatures. We then analyze a population-concentration contextual stochastic block model with Gaussian features and a one-layer linear GCN. Under equidistant class means, the Bayes posterior over class-template scores is a temperature-scaled softmax whose inverse-temperature is governed by a homophily-dependent signal strength. In the positive-signal homophilic regime, the resulting temperature decreases approximately inversely with normalized local homophily. This law motivates Homophily-aware Temperature Scaling (HoTS), a simple post-hoc calibrator that assigns each node a positive scalar temperature from entropy-based logit concentration and estimated local homophily. HoTS has three temperature parameters, preserves the predicted class, and learns the strength of the structural correction from calibration data. Across 18 node-classification benchmarks, two GNN backbones, and eight calibration baselines, HoTS achieves the best mean Expected Calibration Error (ECE) of 4.79%, the best average rank, and the most reliable confidence ranking in selective classification. Code is available at https://github.com/inu0104/HoTS.