Regularization shapes performance and efficiency in linear recommendation models
On the Regularization Landscape for the Linear Recommendation Models
Artificial Intelligence
Summary
Different recommendation algorithms based on deep learning techniques often perform similarly despite using different methods. The authors found that many of the best linear recommendation models work by adding a specific type of mathematical penalty called a regularizer. Some use a nuclear-norm regularizer that creates simpler, lower-rank solutions but with limited flexibility. Others use a Frobenius-norm regularizer, which can represent more complex solutions but need more complicated computations. The authors also propose new methods that combine the benefits of both approaches for better performance and efficiency.
What this means in practice
- •For data scientists: Build recommendation systems that achieve higher accuracy and efficiency by using low-rank regularization techniques with closed-form solutions.
- •For business intelligence teams: Improve personalized product recommendations by deploying more expressive and computationally efficient linear models derived from new regularization frameworks.
Authors
Dong Li, Zhenming Liu, Ruoming Jin, Hao Zhou, Zhi Liu, Jing Gao, Bin Ren
Abstract
Recently, a wide range of recommendation algorithms inspired by deep learning techniques have emerged as the performance leaders on several standard recommendation benchmarks. While these algorithms were built on different DL techniques (e.g., dropouts, autoencoder), they have similar performance and even similar cost functions. This paper studies whether the models' comparable performance are sheer coincidence, or they can be unified under a single framework. We find that all linear performance leaders effectively add only a nuclear-norm based regularizer, or a Frobenius-norm based regularizer. The former ones possess a (surprising) rigid structure that limits the models' predictive power but their solutions are low rank and have closed form. The latter ones are more expressive and more efficient for recommendation but their solutions are either full-rank or require executing hard-to-tune numeric procedures such as ADMM. Along this line of finding, we further propose two low-rank, closed-form solutions, derived from carefully generalizing Frobenius-norm based regularizers. The new solutions get the best of both nuclear-norm and Frobenius-norm world.