Machine learning finds layered meanings in graphs with texts
Uncovering Semantic Hierarchies in Text-Attributed Graphs via Variational EM-based LLM-GNN Synergy
Machine Learning
Summary
Many real-world ideas are organized in layers, from broad categories to specific details, but most existing methods ignore these layers when analyzing text connected in networks. The authors propose a method called SHiFT that combines large language models (LLMs) and graph neural networks (GNNs) to discover and organize these layers automatically. SHiFT uses a two-step approach where LLMs first build a semantic tree of concepts and then GNNs learn to represent these concepts aligned with the tree structure. This approach uncovers clear and understandable hierarchies in data and performs well in experiments.
What this means in practice
- •For knowledge graph engineers: Construct clearer hierarchical categories in complex text-linked graphs to improve organization and retrieval of structured knowledge.
- •For content management teams: Automatically build understandable taxonomies that help organize large volumes of documents by layered topics and concepts.
Authors
Yunhui Liu, Xudong Jin, Qizhuo Xie, Chunhui Zhao, Xiao Luo, Tao Zheng, Bin Chong, Tieke He
Abstract
While hierarchical structures are intrinsic to organizing human knowledge, current representation learning on text-attributed graphs predominantly operates on flat semantic spaces, overlooking the rich coarse-to-fine granularity inherent in real-world data. To bridge this gap, we propose SHiFT, which synergizes the reasoning power of LLMs with the structural representation capability of GNNs within an Expectation-Maximization (EM) paradigm. In the E-step, we harness LLMs as expert taxonomists to induce an explicit semantic tree, introducing a lightweight Mapping & Diagnosis strategy that reduces inference costs by surgically updating the hierarchy only when anomalies are detected. In the M-step, we align the GNN encoder with this induced hierarchy from complementary perspectives, including clustering partition, topological skeleton, and semantic concepts. Theoretical analysis interprets SHiFT as a generalized Variational EM algorithm. Extensive experiments demonstrate that SHiFT not only achieves state-of-the-art performance but also uncovers high-quality, human-readable taxonomies. Our code can be found at https://github.com/Cloudy1225/SHiFT.