Attention based flow model improves graph data understanding and generation
$α$-Graph: Attention-Infused Normalizing Flow Approach to Tractable Graph Modeling
Machine LearningComputer Vision and Pattern Recognition
Summary
Modeling graphs is important for understanding relationships in complex data like social networks or molecules. Existing methods often use neural networks that guess these structures without fully capturing their complexity. The authors propose a new approach called α-Graph that uses a special flow model combined with attention to explicitly and clearly represent graph relationships. Their method also includes a way to handle different conditions in the data while staying efficient and stable during training. Experiments show that this approach performs very well compared to previous techniques.
Graph modelingGraph neural networksNormalizing flowAttention mechanismConditional modelingInvertible transformationsRelational structureTraining stabilityExpressivenessGraph structured data
Authors
Thanh-Dat Truong, Sarah Alharbi, Susan Gauch, Xinghui Zhao, Marios Savvides, Khoa Luu
Abstract
Graph modeling, a crucial task for representing complex relationships in graph-structured data, has achieved significant success in recent years. However, current graph modeling methods rely on traditional Graph Neural Networks and pre-training approaches to implicitly learn the underlying relational structure of graph data. Thus, these prior methods cannot capture the complex graph structure and correlations among inputs. In this paper, we introduce a novel Attention-based Normalizing Flow-based Approach\footnote{Our implementation and models will be released publicly for research reproducibility.} (ANFA or $α$) that provides an explicit, interpretable, and tractable Graph Modeling ($α$-Graph). In particular, we propose a new Unconditional Graph Normalizing Flow with an Invertible Attention Mechanism to capture the complex relational structure of graph data. To further enhance the expressiveness of the model, we introduce Conditional Graph Normalizing Flow with Learnable Queries that enables efficient modeling of correlations in graph-structured data. We show that our Conditional Graph Normalizing Flows behave similarly to Unconditional Graph Normalizing Flows, enhancing expressiveness while maintaining training stability and efficiency. Our experimental results on three benchmarks will illustrate the effectiveness and the state-of-the-art (SoTA) performance of the proposed $α$-Graph method.