Clifford variational autoencoder improves symbolic data embeddings
Learning Holographic Reduced Representations with Clifford Variational Autoencoders
Machine LearningArtificial IntelligenceNeural and Evolutionary Computing
Summary
Embedding complex data like images into symbolic forms useful for reasoning is a challenge. This paper presents Clifford-VAE, a machine learning model that learns to map data into a specific high-dimensional space called a Clifford torus. The authors show that their model makes embeddings that work well for classification and outperform similar methods in symbolic reasoning tests. This approach offers a new way to connect raw sensory data with symbolic processes used for reasoning.
What this means in practice
- •For machine learning engineers: Generate symbolic vector embeddings from raw data that improve semi-supervised classification and symbolic reasoning benchmarks.
- •For robotics software teams: Embed sensor data into symbolic formats that support reasoning and manipulation of structured knowledge in robots.
Authors
Mohamed Malek Abid, P. Michael Furlong
Abstract
Vector Symbolic Algebras project data structures into a hyperdimensional vector space through the application of their vector algebras to randomly generated atomic vector symbols and fractional power encodings of real-valued data. Embedding unstructured data remains an open question. We present \textit{Clifford-VAE}, a variational autoencoder that learns to project data onto a Clifford torus in arbitrary dimensions. Experiments using the MNIST, FashionMNIST, and CIFAR-10 datasets demonstrate that Clifford-VAE produces representations that are competitive with those produced by Gaussian and Hyperspherical VAEs for semi-supervised classification tasks while outperforming Gaussian and Hyperspherical counterparts in the VSA benchmark tests of self-binding and unbinding, role-filler recovery, and bundle capacity. Clifford-VAE provides a principled technique for grounding perceptual data into a symbolic reasoning framework, providing a new approach to a long-standing problem in the VSA literature.