Chimaera blends graph experts to boost cross-task graph learning
Chimaera: A Mixture-of-Graph-Experts Architecture for Cross-Task and Cross-Dataset Graph Learning
Machine Learning
Summary
Graphs are complicated data structures that are hard to analyze because their shapes and connections vary a lot. The authors created Chimaera, a system that combines different graph learning tools and models, including simple and complex ones, to better understand graphs across different tasks and datasets. Chimaera uses both large and small language models to create effective data features, and it can work on different types of problems involving nodes, links, or whole graphs. Their tests show that Chimaera performs well and can learn from a few examples to apply knowledge to new tasks and datasets.
What this means in practice
- •For machine learning engineers: Combine multiple graph models efficiently to improve prediction accuracy across varying graph-based problems and datasets.
- •For natural language processing teams: Use both large and small language models to generate diverse graph embeddings that enhance graph learning tasks involving text attributes.
Authors
Jonathan Frank, David Richerby, Ansgar Scherp
Abstract
Designing foundation models for graphs is challenging due to the irregular structure of graphs and the different sizes and characteristics of embeddings. Chimaera integrates mixture-of-experts with graph foundation models (GFM). It integrates different GFM architectures, such as graph prompts and linear GNN models. Large language models are used to generate embeddings, and experts can be trained and combined following different strategies, GFMs, embeddings, etc. Furthermore, Chimaera extends existing linear GNNs to support link-level and graph-level tasks in addition to node-level tasks. Empirical analyses are performed on same-task and cross-task experiments with node, link, and graph classification tasks using six benchmark text-attributed graph datasets. The experiments demonstrate the effectiveness of Chimaera and its capabilities for transfer across tasks and datasets. Further insights include the need to use both large and small language models to generate embeddings for the experts, a strong cross-task transferability of simple but effective linear GNNs, and using few samples only to provide strong results.