Chimaera mixes graph models to improve learning across tasks and datasets
Chimaera: A Mixture-of-Graph-Experts Architecture for Cross-Task and Cross-Dataset Graph Learning
Machine Learning
Summary
Graphs are tricky for AI because they have complex shapes and different kinds of information. The authors created Chimaera, a system that combines different types of graph models and uses language models to help understand the data. This system can work with tasks involving individual points, links, or whole graphs and performs well when learning new tasks or using new datasets. They found that using both big and small language models helps, and simpler graph models can transfer knowledge effectively across tasks even with little training data. Overall, Chimaera shows promise in making graph learning more flexible and effective.
graph foundation modelsmixture-of-expertsgraph neural networks (GNN)large language modelsnode classificationlink predictiongraph classificationtransfer learningtext-attributed graphs
Authors
Jonathan Frank, David Richerby, Ansgar Scherp
Abstract
Designing foundation models for graphs is challenging due to the irregular structure of graphs and the different sizes and characteristics of embeddings. Chimaera integrates mixture-of-experts with graph foundation models (GFM). It integrates different GFM architectures, such as graph prompts and linear GNN models. Large language models are used to generate embeddings, and experts can be trained and combined following different strategies, GFMs, embeddings, etc. Furthermore, Chimaera extends existing linear GNNs to support link-level and graph-level tasks in addition to node-level tasks. Empirical analyses are performed on same-task and cross-task experiments with node, link, and graph classification tasks using six benchmark text-attributed graph datasets. The experiments demonstrate the effectiveness of Chimaera and its capabilities for transfer across tasks and datasets. Further insights include the need to use both large and small language models to generate embeddings for the experts, a strong cross-task transferability of simple but effective linear GNNs, and using few samples only to provide strong results.