* Matrix Factorization and Recommendation Systems

Transcription

1 Matrix Factorization and Recommendation Systems Originally presented at HLF Workshop on Matrix Factorization with Loren Anderson (University of Minnesota Twin Cities) on 25 th September, th March, 2018 Datalys

2 Recommendation is must have in e-commerce 2 Movies, products, etc. Preventing information overload Explicit (rating) vs. implicit (click) feedback

3 Recommendation = Prediction of rating 3 Item 1 Item 2 Item 3 Item 4 Item 5 Bob ? User User User User

4 Recommendation = Prediction of rating 4 Item 1 Item 2 Item 3 Item 4 Item 5 Bob 3 4? User User User User In real systems, matrix is really big and sparse Dataset from Netflix Price: 500,000 users 17,000 movies 100 million (1%) ratings are known

5 Recommender systems - Classification 5 Recommender Systems Content-based Collaborative filtering Hybrid Neighborhoodbased Model-based State of the art Item-based User-based Rule-based Latent factors Matrix Factorization Neural networks

6 Matrix factorization Formalization 6 Goal is to map users and items into latent factor f-dimensional space m n ratings matrix R is approximately factorized into an m f matrix P and an n f matrix Q R PQ T Users Items r 11 r 1n r m1 r mn Users Factors p 11 p 1f p m1 p mf Factors Items q 11 q 1n q f1 q fn

7 Matrix factorization Formalization 7 Goal is to map users and items into latent factor f-dimensional space m n ratings matrix R is approximately factorized into an m f matrix P and an n f matrix Q R PQ T Users Items r 11 r 1n r m1 r mn Users Factors p 11 p 1f p m1 p mf Factors Items q 11 q 1n q f1 q fn Each item i is associated with a vector q i (item factor) q i measures affinity of the item i towards those f factors Each user u is associated with a vector p u (user factor) p u measures interest of the user in these f factors

8 Matrix factorization Formalization 8 Goal is to map users and items into latent factor f-dimensional space m n ratings matrix R is approximately factorized into an m f matrix P and an n f matrix Q R PQ T Users Items r 11 r 1n r m1 r mn Users Factors p 11 p 1f p m1 p mf Factors Items q 11 q 1n q f1 q fn Each item i is associated with a vector q i (item factor) q i measures affinity of the item i towards those f factors Each user u is associated with a vector p u (user factor) p u measures interest of the user in these f factors Rating r ui of user u on item i can be approximated as a dot product r ui = q i T p u

9 Matrix factorization Basic model 9 The main challenge is computing of user and item factors Incomplete and sparse rating matrix R (conventional SVD is not applicable) Overfitting when only few ratings are available

10 Matrix factorization Basic model 10 The main challenge is computing of user and item factors Incomplete and sparse rating matrix R (conventional SVD is not applicable) Overfitting when only few ratings are available Solution 1: amputation of missing values Memory inefficient Introduce bias

11 Matrix factorization Basic model 11 The main challenge is computing of user and item factors Incomplete and sparse rating matrix R (conventional SVD is not applicable) Overfitting when only few ratings are available Solution 2: modeling only known ratings + minimize error e ui = (r ui r ui ) min (r ui q T i p u ) 2 +λ( q 2 i + p 2 u ) q,p (u,i) R Parameter λ controls the extent of regularization Optimization problem -> learning algorithms Stochastic gradient descent Alternating least squares

12 Matrix factorization Differences in models 12 Constraints imposed on P and Q Unconstrained MF Orthogonality of the latent vectors (SVD) Non-negativity of the latent vectors (NMF)

13 Matrix factorization Differences in models 13 Constraints imposed on P and Q Unconstrained MF Orthogonality of the latent vectors (SVD) Non-negativity of the latent vectors (NMF) Objective function Minimize squared (Frobenius) norm Maximizing the likelihood estimation

14 Matrix factorization Differences in models 14 Constraints imposed on P and Q Unconstrained MF Orthogonality of the latent vectors (SVD) Non-negativity of the latent vectors (NMF) Objective function Minimize squared (Frobenius) norm Maximizing the likelihood estimation Additional information used in modeling of ratings User and item bias (SVD) Implicit feedback (SVD++) Temporal dynamics (TimeSVD++)

15 Improvement of basic model for real systems

16 Incorporating user and item bias 16 Real systems usually exhibit some systematical tendencies Some users tend to give higher ratings Some items are widely rated better r ui = μ + b i + b u + q i T p u μ denotes overall average rating b i represents item bias b u represents user bias

17 Incorporating implicit feedback 17 Real systems must tackle with cold-start problem Users usually provide very few explicit ratings Implicit feedback can provide some insights into user preferences r ui = μ + b i + b u + q i T (p u + R(u) 1/2 σ jεr(u) y j ) R(u) 1/2 stabilizes variance across the range of observed R(u) y j additional item factor reflecting implicit feedback

18 Incorporating temporal dynamics 18 Real systems must tackle with temporal shifts Item popularity changes over time User baseline rating changes over time User preferences change over time r ui = μ + b i (t ui ) + b u (t ui ) + q T i (p u (t ui ) + R(u) 1/2 jεr(u) y j t ui denotes the time of rating r ui

19 Performance comparison on Netflix Price dataset 19 Koren, Y., Bell, R., & Volinsky, C. (2009). Matrix factorization techniques for recommender systems. Computer, 42(8),

20 * Advanced models

21 Advanced models 21 Probabilistic matrix factorization (PMF) Constrained PMF Mnih, Andriy, and Ruslan R. Salakhutdinov. "Probabilistic matrix factorization." Advances in neural information processing systems Bayesian PMF using Markov Chain Monte Carlo (MCMC) Salakhutdinov, Ruslan, and Andriy Mnih. "Bayesian probabilistic matrix factorization using Markov chain Monte Carlo." Proceedings of the 25th international conference on Machine learning. ACM, Local Low-Rank Matrix Approximation (LLORMA) Lee, J., Kim, S., Lebanon, G., & Singer, Y. (2013). Local Low-Rank Matrix Approximation, 28.

22 Conclusion 22 Matrix factorization = state of the art within collaborative filtering recommenders High scalability Can be applied also at large datasets Possibility to easily integrate various additional aspects of data User and item biases Implicit feedback Temporal dynamics

23 Literature and sources 23 Aggarwal, Charu C Recommender Systems - The Textbook. Communications of the ACM. Vol. 40. Cham: Springer International Publishing. Ricci, Francesco, Lior Rokach, Bracha Shapira, and Paul B. Kantor, eds Recommender Systems Handbook. 2nd Edition. Boston, MA: Springer US. Koren, Y., Bell, R., & Volinsky, C. (2009). Matrix factorization techniques for recommender systems. Computer, 42(8), Lee, J., Kim, S., Lebanon, G., & Singer, Y. (2013). Local Low-Rank Matrix Approximation, 28. Retrieved from Lei Guo Matrix Factorization Techniques For Recommender Systems, Retrieved from Mnih, Andriy, and Ruslan R. Salakhutdinov. "Probabilistic matrix factorization." Advances in neural information processing systems Salakhutdinov, Ruslan, and Andriy Mnih. "Bayesian probabilistic matrix factorization using Markov chain Monte Carlo." Proceedings of the 25th international conference on Machine learning. ACM, 2008.