Heterophily-Aware Adaptive Knowledge Distillation for Hypergraph Neural Networks
2026-06-08 • Machine Learning
Machine Learning
AI summaryⓘ
The authors study how to make smaller, faster models that learn from bigger hypergraph neural networks (HNNs) without losing much accuracy. They find that the big models struggle more on nodes connected by very different types of links, so not all parts of the big model are equally reliable. To fix this, the authors create HADES, which adjusts how much the small model learns from the big one by checking how mixed the node connections are. Tests on real data show that their method helps the small models work better, sometimes even outperforming the big models while running much faster.
Hypergraph Neural NetworksKnowledge DistillationHeterophilyNode ConnectivityTeacher-Student ModelsModel CompressionInference SpeedDistillation ObjectiveGraph LearningSemantic Diversity
Authors
Joohee Cho, David Yoon Suk Kang, Yunyong Ko
Abstract
Hypergraph knowledge distillation aims to retain the predictive performance of a hypergraph neural network (HNN) teacher while reducing inference costs through a lightweight student model. In this work, we observe that HNNs exhibit substantially lower prediction performance on heterophilic nodes connected through semantically diverse hyperedges, indicating that the reliability of teacher knowledge varies across nodes. Motivated by this observation, we propose HADES, a heterophily-aware adaptive distillation method for hypergraph neural networks. HADES quantifies node heterophily and leverages it as an estimate of teacher reliability to modulate the transfer of teacher knowledge during distillation. Experimental results on real-world hypergraphs demonstrate that HADES consistently improves student performance across different HNN teachers and distillation objectives. In many cases, the resulting student models surpass the predictive performance of their teachers while achieving up to 12.3 times faster inference.