Decision models improve entity matching with faster and cheaper results
Entity Matching using Decision Models
Computation and LanguageArtificial Intelligence
Summary
The problem is figuring out if two records talk about the same real-world thing, like matching people or products in databases. The authors study decision models, a kind of AI that gives probabilities for answers instead of writing out text, to see if they can match entities well without special training. They find that decision models do almost as well as a powerful text-generating AI and sometimes better than methods trained on similar tasks. These models are faster, cheaper, and often don’t need extra tuning to work well. Combining decision models with text-generating AI on tricky cases improves matching even more, beating older best scores on some tests.
What this means in practice
- •For database administrators: Speed up matching of duplicate records in large databases by using decision models that don’t need task-specific training data.
- •For e-commerce platform engineers: Improve product catalog matching efficiency and accuracy by adopting decision models with calibrated confidence scores and fallback generative AI for uncertain matches.
Authors
Aaron Steiner, Ralph Peeters, Christian Bizer
Abstract
Entity matching is the task of deciding whether two records describe the same real-world entity. Current entity matching methods rely on pre-trained language models (PLMs) which require task specific fine-tuning or generative large language models (LLMs) which do not require task-specific fine-tuning but are slow and expensive at matching large record sets. Decision models are a new class of LLM-based models that return a probability for each predefined answer option rather than generate text. This paper investigates decision models as an alternative to generative LLMs and fine-tuned PLMs for entity matching. We evaluate six decision models in a zero-shot setting across eight benchmarks and compare their performance with that of the generative LLM GPT-6 Luna and the PLM-based matcher Ditto. Our experiments show that, without task-specific training data, the four strongest decision models achieve mean F1 scores close to that of GPT-6 Luna, with the best model slightly surpassing it. This model also outperforms Ditto on six of the eight benchmarks. All decision models are faster than GPT-6 Luna, and most are less expensive. For three of the four strongest decision models, the predicted match probabilities are well calibrated without requiring any task-specific recalibration. A model cascade that routes the least confident record pairs to an ensemble of generative LLMs improves the mean F1 scores of the four strongest decision models by 1.9-4.0 percentage points. Using the best decision model and its best-performing prompt for each benchmark, the cascade surpasses the highest published F1 scores from literature on four of the eight benchmarks.