RADAR improves knowledge graph learning by smart negative sampling
How Many and How Much? Contrastive Learning with Negative Samples for Knowledge Graph Embedding
Machine Learning
Summary
Knowledge graphs are like huge maps of facts, and computers learn to understand them by comparing true facts to false ones called negative samples. This paper looks at two questions: how many negative samples to use per true fact, and how strongly each negative sample should affect learning. The authors introduce RADAR, which smartly decides how many negatives to use based on the chance of mistakenly including true facts as negatives, and adjusts their importance by looking at how the negative samples are spread out. Their experiments show that RADAR boosts the learning quality and greatly reduces wrong negative samples compared to usual methods.
What this means in practice
- •For web developers: Improve AI-powered web apps that use knowledge graphs by providing more accurate embedded representations with fewer misleading negative samples.
- •For enterprise data engineers: Enhance knowledge graph based AI systems by dynamically allocating and weighting negative samples, thus improving embedding training at scale.
Authors
Sooho Moon, Yunyong Ko
Abstract
Knowledge graph embedding (KGE), a fundamental technique for knowledge-centric Web and KG-augmented AI applications, is commonly trained through contrastive learning with negative samples to learn discriminative representations. Despite its empirical success, two crucial questions regarding negative samples remain underexplored: (Q1) how many negative samples should be used for each positive sample and (Q2) how much should each negative sample contribute to learning? To answer these questions, we propose RADAR, a novel contrastive learning framework for KGE models with two effective strategies, respectively: (1) Risk-aware negative budget allocation quantifies false-negative sampling risk by modeling an open-world KG as a stochastic partial observation of its closed-world counterpart and adaptively assigns a negative budget to each positive sample based on its risk; (2) Distribution-calibrated negative weighting assigns negative weights by calibrating the weighting strength for each positive based on two complementary characteristics of the negative-score distribution: dispersion and concentration. Extensive experiments on 3 real-world KGs with 4 KGE models reveal that RADAR achieves the best KGE performance on average across all model--KG combinations and reduces sampled false negatives by over 90% compared with conventional fixed-budget methods.