Inductive method creates concept embeddings from language patterns
Inductive Learning of Concept Embeddings from Cross-Linguistic Colexification Patterns with Textual Attributes
Computation and Language
Summary
Languages often share related meanings across words, and these connections help understand how ideas relate worldwide. The authors developed a way to create word-like vector representations, called concept embeddings, from patterns where different languages use the same word for related ideas. Their method can create embeddings even for new concepts by using short descriptions, which was not possible before. This approach helps capture semantic similarities across many languages and concepts.
What this means in practice
- •For language technology developers: Generate semantic vector representations for new or rare concepts to improve multilingual natural language processing tools across diverse languages.
- •For lexicographers and language data curators: Embed new dictionary entries or word senses into semantic networks using brief concept definitions without requiring extensive language data.
Authors
Arne Rubehn
Abstract
The advent of distributional semantics as a framework for computational semantics marked a paradigm shift in the nature of representations, by which meanings are represented as dense vectors commonly known as word embeddings. Computational approaches to historical or typological language comparison, on the other hand, still mostly rely on discrete representations, while the potential of embedding-based methods remains largely untapped in this context. Recent studies have proposed concept embeddings as a language-independent, onomasiological counterpart to word embeddings, based on networks of cross-linguistic colexification patterns. While the resulting concept embeddings reflect meaningful semantic properties and relations, they suffer from a lack of scalability, since they rely on transductive embedding approaches which only learn embeddings for nodes present in the training data. In this study, we present an inductive framework for learning concept embeddings, combining the structural properties of full and partial colexification networks with semantic information from short concept descriptions. This framework allows the embedding of unseen concepts based on a short definition. We show that the resulting concept embeddings correlate with various cross-linguistic measures of similarity or relatedness, while generalizing well to new concepts that were absent from the training data.