Better recommendation models made by combining simple math tricks
On the Regularization Landscape for the Linear Recommendation Models
Artificial Intelligence
Summary
Many recommendation systems use deep learning ideas but end up performing similarly. The paper finds that top linear models actually use only two main types of regularizers, special math techniques that shape the model. One type produces simpler, low-rank solutions that are easy to calculate but less flexible. The other type is more flexible but harder to solve. By combining the strengths of these two, the authors create new, easy-to-use solutions that work well and keep models simple.
Recommendation systemsDeep learningRegularizationNuclear normFrobenius normLow-rank approximationClosed-form solutionADMM (Alternating Direction Method of Multipliers)Matrix factorization
Authors
Dong Li, Zhenming Liu, Ruoming Jin, Hao Zhou, Zhi Liu, Jing Gao, Bin Ren
Abstract
Recently, a wide range of recommendation algorithms inspired by deep learning techniques have emerged as the performance leaders on several standard recommendation benchmarks. While these algorithms were built on different DL techniques (e.g., dropouts, autoencoder), they have similar performance and even similar cost functions. This paper studies whether the models' comparable performance are sheer coincidence, or they can be unified under a single framework. We find that all linear performance leaders effectively add only a nuclear-norm based regularizer, or a Frobenius-norm based regularizer. The former ones possess a (surprising) rigid structure that limits the models' predictive power but their solutions are low rank and have closed form. The latter ones are more expressive and more efficient for recommendation but their solutions are either full-rank or require executing hard-to-tune numeric procedures such as ADMM. Along this line of finding, we further propose two low-rank, closed-form solutions, derived from carefully generalizing Frobenius-norm based regularizers. The new solutions get the best of both nuclear-norm and Frobenius-norm world.