Merit speeds up knowledge graph models for faster reasoning tasks
MERIT: Scaling KG Foundation Models with Relation Transformers
Machine Learning
Summary
Predicting how things are connected in knowledge graphs can be slow when models get more complex. The authors propose MERIT, which uses a new transformer-based approach to understand relationships more efficiently. MERIT keeps prediction quality high while making the process much faster. When adapted for complex questions, MERITQuery improves results without extra cost.
What this means in practice
- •For knowledge graph engineers: Use MERIT to accelerate inference in large-scale knowledge graph prediction applications while maintaining accuracy.
- •For question answering system developers: Integrate MERITQuery to improve complex query answering over knowledge graphs with efficient training and runtime cost.
Authors
Cosimo Gregucci, Daniel Hernandez, Antonio Vergari, Steffen Staab
Abstract
Knowledge graph (KG) foundation models (KGFMs) predict links on unseen graphs zero-shot. KGFMs have been pioneered by ULTRA which builds a relation graph from co-occurrences that match motif templates, and passes messages over it to obtain relation representations. More recent KGFMs such as MOTIF and TRIX increase the size of this relation graph, and FLOCK replaces message-passing with probabilistic random-walk ensembles. However, larger relation graphs mean more messages, and more walks mean more sequences to encode, so they improve performance at higher training and inference cost. This cost hinders their use in more complex reasoning tasks, such as complex query answering. To reverse this trend, we design MERIT, the first KGFM with a transformer relation encoder: every relation name attends to every other, and the motifs enter as a soft prior rather than as edges. Over 56 benchmarks, MERIT matches the zero-shot MRR of SoTA KGFMs while being 6x faster than TRIX and over 100x faster than FLOCK at inference. Fine-tuned for complex query answering like UltraQuery, MERITQuery improves over it at matched cost.