Latent space tweaks speed up neural network search and ranking
Latent space configuration for neural network finetuning, fast ranking, and embedding dimensionality reduction
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Finding the most similar items in a big set of neural network outputs is often slow. The authors show a way to adjust how these outputs are arranged to make finding matches much faster without losing accuracy. They improve a common model with a new method that reshapes these outputs and speeds up searching by up to three times on GPUs. They also show that shrinking the size of these outputs can make the search even quicker and sometimes better for ranking. This can help any system relying on comparing many neural network-generated features.
What this means in practice
- •For computer vision teams: Speed up searching for similar faces or objects in large image databases by reconfiguring and reducing neural network embeddings.
- •For recommendation engine developers: Improve the speed of finding the top matches from large item catalogs by configuring latent spaces for faster ranking.
Authors
Nikita Gabdullin, Anton Raskovalov
Abstract
Finding K most similar elements for a query in a large gallery of neural network (NN) embeddings is essential in various areas of computer science and industrial applications of artificial intelligence. This paper proposes search algorithms for structured latent spaces (LS) of NNs which allow to speed up top K item retrieval owing to O(1) complexity small candidate gallery subset search. ArcFace-pretrained IResNet50 models are finetuned using Latent Space Configuration (LSC) methodology to obtain new distribution of embeddings with desired geometric properties. Applying the proposed algorithms to embeddings obtained from finetuned models leads to several times speed-up over baseline exhaustive search. Specifically, we report up to 3 times speed-up on GPU and 40% speed-up on CPU compared to the baseline method GPU speed. These results are obtained for the original and reduced numbers of LS dimensions. The results show that LS dimensionality reduction allows to further accelerate the search and even improve rank-N metrics.