Graph tokenizer uses curved math to transfer structure knowledge
Riemannian Graph Tokenizer for Structural Knowledge Transfer
Machine Learning
Summary
Graphs are structures made of points and connections, and understanding their shapes is tricky because they don't fit well with regular flat math. This paper introduces a new way to convert these graphs into tokens using curved geometry, which helps computers understand and share graph patterns from one area to another. The authors create special sets of shapes like trees and cycles as a vocabulary and use a math method called Riemannian geometry to handle these shapes properly. Their approach improves the ability of models to transfer knowledge between different graph types.
What this means in practice
- •For machine learning engineers: Improve training of models that work with graph data by enabling better knowledge transfer between different graph types.
- •For network analysts: Analyze and interpret complex network structures more effectively by converting network features into standardized tokens across systems.
Authors
Qimin Zhou, Haibo Liu, Yujie Wang, Li Sun, Chuan Shi
Abstract
Foundation models are at the forefront of artificial intelligence. A tokenizer, converting the raw input into discrete representations that the model can understand, plays an important role to the success of foundation models. Unlike the text tokenizer that is well studied in large language models, graph tokenizer is still at its early stage, facing the challenges of tackling the non-Euclidean structures and capturing the structural semantics. How to design a graph tokenizer for structural knowledge transfer? To this end, we propose a Riemannian Graph Tokenizer (RGT) that bridges the structural knowledge and quantized representations to support cross-domain structural knowledge transfer. The connection is established by Riemannian geometry. Specifically, we first define the geometric vocabulary (trees, cycles and sequences), which captures fundamental structural patterns and reflects the intrinsic geometry of graph. Second, we construct a Riemannian quantizer with Riemannian Straight-Through Estimator to tokenise graph structures across multiple domains into discrete tokens. To ensure consistency and transferability across diverse geometric spaces, RGT further incorporates a geometry-aligned decoder that projects manifold-specific tokens into a unified tangent space. The theoretical analysis and geometric interpretations are provided to support the effectiveness of our proposed method. Extensive experiments across diverse datasets demonstrate that RGT significantly enhances structural knowledge transferability across graph domains.