Prior-guided attention improves hypergraph neural network performance
PANDA: Prior-guided Attentional Dual-path Architecture
Machine Learning
Summary
Hypergraph neural networks are a type of AI that helps computers understand complex connections involving many items at once, beyond simple pairs. The authors found that existing methods sometimes give misleading importance to connections, especially when relationships are mixed or unusual. They propose a new architecture called PANDA that uses both prior knowledge about connections and flexible attention mechanisms, allowing each part to contribute without competing unfairly. Testing PANDA with different types of hypergraph networks across multiple real-world datasets showed consistent improvements in accuracy.
What this means in practice
- •For data scientists: Improve predictive models in complex relational data by incorporating prior-guided attention in hypergraph neural networks for better accuracy in heterogeneous contexts.
- •For knowledge graph engineers: Enhance graph representation learning on directed and undirected knowledge graphs by integrating dual-path attentional methods that preserve diverse sender influences.
Authors
Rishabh Jain, Federico Belotti, Stefano Coniglio, Pietro Lio, Stefano Fiorini
Abstract
Hypergraph Neural Networks (HNNs), designed to model higher-order relations beyond pairwise interactions, typically rely either on fixed structural weights or on normalised attention. Fixed weights can propagate misleading information in heterophilic settings, while normalisation makes attention coefficients competitive, in the sense that, in a message-passing framework, increasing the weight of one sender must reduce the weight of others in the same aggregation set. Our main contribution is the introduction of PANDA (Prior-guided AttentioNal Dual-path Architecture), a plug-in for two-stage message passing HNNs applicable to both undirected and directed settings. The main path interpolates between learned receiver-normalised attention and structural incidence priors, while a non-competitive (not normalised) auxiliary path lets each sender contribute its own transformed information. We also show that, with fixed structural coefficients, PANDA recovers a certain complex-valued convolution operator adopted in previous works. Across four HNNs spanning spatial and spectral methods in both directed and undirected settings, plugging PANDA in yields an average improvement of 5.18 percentage points across eight real-world benchmark datasets. The code is available at https://github.com/jainris/panda.