Attention infused flow model enhances graph structured data learning
$α$-Graph: Attention-Infused Normalizing Flow Approach to Tractable Graph Modeling
Machine LearningComputer Vision and Pattern Recognition
Summary
Graphs are a way to show complex relationships, but current methods struggle to clearly capture those connections. The authors present a new approach called α-Graph that uses a special attention-based mathematical process to model graph data more explicitly and understandably. Their model works with an invertible attention mechanism, making it easier to analyze and train. They also extend it to condition on additional factors, improving its ability to understand complex graph correlations. Tests on common benchmarks show their method performs better than existing ones.
What this means in practice
- •For data science teams: Model complex relationships in network or relational data with clearer and more interpretable techniques than prior graph neural methods.
- •For machine learning engineers: Implement stable and efficient graph generative models that capture intricate dependencies using attention-based normalizing flows.
Authors
Thanh-Dat Truong, Sarah Alharbi, Susan Gauch, Xinghui Zhao, Marios Savvides, Khoa Luu
Abstract
Graph modeling, a crucial task for representing complex relationships in graph-structured data, has achieved significant success in recent years. However, current graph modeling methods rely on traditional Graph Neural Networks and pre-training approaches to implicitly learn the underlying relational structure of graph data. Thus, these prior methods cannot capture the complex graph structure and correlations among inputs. In this paper, we introduce a novel Attention-based Normalizing Flow-based Approach\footnote{Our implementation and models will be released publicly for research reproducibility.} (ANFA or $α$) that provides an explicit, interpretable, and tractable Graph Modeling ($α$-Graph). In particular, we propose a new Unconditional Graph Normalizing Flow with an Invertible Attention Mechanism to capture the complex relational structure of graph data. To further enhance the expressiveness of the model, we introduce Conditional Graph Normalizing Flow with Learnable Queries that enables efficient modeling of correlations in graph-structured data. We show that our Conditional Graph Normalizing Flows behave similarly to Unconditional Graph Normalizing Flows, enhancing expressiveness while maintaining training stability and efficiency. Our experimental results on three benchmarks will illustrate the effectiveness and the state-of-the-art (SoTA) performance of the proposed $α$-Graph method.