Dynamic Poisson Factorization

Similar documents
Content-based Recommendation

An Overview of Edward: A Probabilistic Programming System. Dustin Tran Columbia University

* Matrix Factorization and Recommendation Systems

Scaling Neighbourhood Methods

Mixed Membership Matrix Factorization

Recurrent Latent Variable Networks for Session-Based Recommendation

Mixed Membership Matrix Factorization

Large-scale Ordinal Collaborative Filtering

A Gradient-based Adaptive Learning Framework for Efficient Personal Recommendation

SQL-Rank: A Listwise Approach to Collaborative Ranking

Restricted Boltzmann Machines for Collaborative Filtering

Scalable Bayesian Matrix Factorization

Causal Inference and Recommendation Systems. David M. Blei Departments of Computer Science and Statistics Data Science Institute Columbia University

arxiv: v2 [cs.ir] 14 May 2018

Decoupled Collaborative Ranking

Probabilistic Matrix Factorization

Modeling User Exposure in Recommendation

Collaborative topic models: motivations cont

Practical Bayesian Optimization of Machine Learning. Learning Algorithms

13: Variational inference II

STA 4273H: Statistical Machine Learning

Bayesian Learning and Inference in Recurrent Switching Linear Dynamical Systems

Matrix Factorization Techniques for Recommender Systems

Recommender Systems EE448, Big Data Mining, Lecture 10. Weinan Zhang Shanghai Jiao Tong University

Matrix Factorization Techniques For Recommender Systems. Collaborative Filtering

Time-aware Collaborative Topic Regression: Towards Higher Relevance in Textual Item Recommendation

Collaborative Filtering. Radek Pelánek

A Randomized Approach for Crowdsourcing in the Presence of Multiple Views

Review: Probabilistic Matrix Factorization. Probabilistic Matrix Factorization (PMF)

Sparse Stochastic Inference for Latent Dirichlet Allocation

arxiv: v2 [stat.ml] 16 Aug 2016

Generative Models for Discrete Data

arxiv: v2 [stat.ml] 4 Feb 2016

Clustering based tensor decomposition

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 13: SEQUENTIAL DATA

Context-aware factorization methods for implicit feedback based recommendation problems

TFMAP: Optimizing MAP for Top-N Context-aware Recommendation

Collaborative Filtering on Ordinal User Feedback

Bayesian Nonparametric Poisson Factorization for Recommendation Systems

Andriy Mnih and Ruslan Salakhutdinov

Bayesian Networks BY: MOHAMAD ALSABBAGH

Locally Private Bayesian Inference for Count Models

VCMC: Variational Consensus Monte Carlo

Approximate Bayesian inference

Factor Modeling for Advertisement Targeting

Probabilistic Canonical Tensor Decomposition for Predicting User Preference

Bayesian nonparametric models for bipartite graphs

A Bayesian Latent Variable Model of User Preferences with Item Context

Matrix and Tensor Factorization from a Machine Learning Perspective

STA 4273H: Statistical Machine Learning

Scalable Recommendation with Hierarchical Poisson Factorization

Why Aren t You Using Probabilistic Programming? Dustin Tran Columbia University

Recurrent Coevolutionary Latent Feature Processes for Continuous-Time Recommendation

20: Gaussian Processes

Gaussian Process Factorization Machines for Context-aware Recommendations

Black-box α-divergence Minimization

Neural Networks and Machine Learning research at the Laboratory of Computer and Information Science, Helsinki University of Technology

ECE521 Lecture 19 HMM cont. Inference in HMM

Gaussian Processes. Le Song. Machine Learning II: Advanced Topics CSE 8803ML, Spring 2012

Lecture: Gaussian Process Regression. STAT 6474 Instructor: Hongxiao Zhu

Ranking-Oriented Evaluation Metrics

Linear Dynamical Systems

Probability and Information Theory. Sargur N. Srihari

Probabilistic Low-Rank Matrix Completion with Adaptive Spectral Regularization Algorithms

Temporal Collaborative Filtering with Bayesian Probabilistic Tensor Factorization

From Mating Pool Distributions to Model Overfitting

Downloaded 09/30/17 to Redistribution subject to SIAM license or copyright; see

STA414/2104 Statistical Methods for Machine Learning II

Collaborative Filtering via Different Preference Structures

STA 4273H: Sta-s-cal Machine Learning

NetBox: A Probabilistic Method for Analyzing Market Basket Data

A Novel Click Model and Its Applications to Online Advertising

CS249: ADVANCED DATA MINING

Density Propagation for Continuous Temporal Chains Generative and Discriminative Models

Bayesian Semi-supervised Learning with Deep Generative Models

Machine Learning for Data Science (CS4786) Lecture 24

Introduction to Logistic Regression

Pattern Recognition and Machine Learning

Bayesian Low-Rank Determinantal Point Processes

Maximum Margin Matrix Factorization for Collaborative Ranking

Natural Gradients via the Variational Predictive Distribution

Offline Evaluation of Ranking Policies with Click Models

Optimizing Factorization Machines for Top-N Context-Aware Recommendations

Large-scale Collaborative Ranking in Near-Linear Time

Online Advertising is Big Business

STA 4273H: Statistical Machine Learning

Personalized Social Recommendations Accurate or Private

Ranking and Filtering

CS Lecture 18. Topic Models and LDA

Matrix Factorization and Collaborative Filtering

Probabilistic Collaborative Representation Learning for Personalized Item Recommendation

Introduction to Machine Learning Fall 2017 Note 5. 1 Overview. 2 Metric

A Bayesian Treatment of Social Links in Recommender Systems ; CU-CS

arxiv: v2 [cs.ir] 4 Jun 2018

CS 188: Artificial Intelligence Fall 2011

Variational Scoring of Graphical Model Structures

A Modified PMF Model Incorporating Implicit Item Associations

Coupled Hidden Markov Models: Computational Challenges

Data Mining Techniques

Bayesian leave-one-out cross-validation for large data sets

Transcription:

Dynamic Poisson Factorization Laurent Charlin Joint work with: Rajesh Ranganath, James McInerney, David M. Blei McGill & Columbia University Presented at RecSys 2015

2

Click Data for a paper 3.5 Item 4663: The Google Similarity Distance Access frequency 3.0 2.5 2.0 1.5 The Google Similarity effect arxiv: cs/0412098 1.0 2005 2006 2007 2008 2009 2010 2011 2012 Year

arxiv evolves over time Click data evolves over time Popularity of papers change over time Users interests change over time The same is also true of other datasets 4

Recommendations should evolve over time

3.5 Raw Data Item 4663: The Google Similarity Distance Modeled Data (e.g., Matrix Factorization) 0.14 Access frequency 3.0 2.5 2.0 1.5 The Google Similarity effect arxiv: cs/0412098 0.12 0.10 0.08 0.06 0.04 DDtD 6tructures GrDSh Theory ComSresseG 6eQsiQg 4uDQtum CryStogrDShy 0.02 1.0 2005 2006 2007 2008 2009 2010 2011 2012 0.00 6

3.5 Raw Data Item 4663: The Google Similarity Distance Modeled Data (e.g., Matrix Factorization) 0.14 Access frequency 3.0 2.5 2.0 1.5 The Google Similarity effect arxiv: cs/0412098 0.12 0.10 0.08 0.06 0.04 DDtD 6tructures GrDSh Theory ComSresseG 6eQsiQg 4uDQtum CryStogrDShy 0.02 1.0 2005 2006 2007 2008 2009 2010 2011 2012 0.00 Time-aware Modeled Data Factor expression 6 2005 2006 2007 2008 2009 2010 2011 2012

Objectives Understand click data over time Better predictions of future user interests Explore how (large) datasets evolve over time Motivated by the arxiv Study other datasets 7

Technical approach Dynamic model of user preferences Poisson matrix factorization [Gopalan et al. UAI 15] User and item latent factors evolve over time Develop inference procedure Fit this model using data Explore fits 8

Data (e.g., clicks) Recommendations Generative Model a, b u y M Learning & Inference Exploration (latent factors) c, d v N Factor expression 2005 2006 2007 2008 2009 2010 2011 2012 9

Background 10

Gaussian Matrix Factorization [Salakhutdinov et al. 15] a, b u y M c, d v N [wikipedia] 11

Gaussian Matrix Factorization [Salakhutdinov et al. 15] a, b u Poisson Matrix Factorization (PF) [Gopalan et al. 15] y M c, d v N [wikipedia] 11 [wikipedia]

Properties of PF Suitable for implicit and explicit data Model can be fit to massive datasets 12

9 User 755 8 Click frequency 7 6 5 4 3 2 2005 2006 2007 2008 2009 2010 2011 2012 13

Dynamic Poisson factorization 14

Users preferences evolve over time Item popularity evolves over time Ratings are not exchangeable over time 15

Dynamic Matrix Factorization (DPF) 16

Dynamic Matrix Factorization (DPF) y 0 y 1... y T 16

Dynamic Matrix Factorization (DPF) u 0 u 1... u T y 0 y 1... y T M v 0 v 1... v T N 16

Dynamic Matrix Factorization (DPF) a, b u 0 u 1... u T y 0 y 1... y T M c, d v 0 v 1... v T N Modeling: Smooth evolution through time 16

Smooth Evolution through time Gaussian time series model Exponentiate to obtain positive Poisson rates 17

Local and Global factors Two levels of modeling: Local factors: model evolution through time Global factors: model constant preferences wrt time µū, ū ū µ u, u u 0 u 1... u T y 0 y 1... y T M µ v, v v 0 v 1... v T v N µ v, v 18

Local and Global factors Two levels of modeling: Local factors: model evolution through time Global factors: model constant preferences wrt time µū, ū ū µ u, u u 0 u 1... u T y 0 y 1... y T M µ v, v v 0 v 1... v T v N µ v, v 18

Local and Global factors Two levels of modeling: Local factors: model evolution through time Global factors: model constant preferences wrt time µū, ū ū µ u, u u 0 u 1... u T y 0 y 1... y T M µ v, v v 0 v 1... v T v N µ v, v 18

A 3 timeslice Dynamic PF µū, ū ū µ u, u u 0 u 1 u 2 y 0 y 1 y 2 M µ v, v v 0 v 1 v 2 v N µ v, v

Inference Variational inference algorithm Mean-field (factorized) approximation Non-conjugacy between and Use an additional approximation See the paper for details on the algorithm Scales linearly in time, we provide a parallel implementation 20

Empirical results 21

Explicit data Augmented matrix factorization [Koren 10, Li 11, Gultekin 14] Tensor factorization [Karatzoglou 10] Bayesian Tensor Probabilistic Factorization [Xiong 10] Implicit data HMM [Sahoo 12] Gamma process MF [Acharya 15] DPF (us): User and item dynamicity Implicit and explicit Scalable 22

Experimental setup Test on multiple time steps Train data: clicks up to time slice t Predict held-out clicks at time t+1 Metrics for implicit data Ranking-based (NDCG, Recall, MAR, MRR) 23

Netflix-time Recall@50 MAR MRR NDCG dpf 0.170 640 0.027 0.294 BPTF [Xiong 10] 0.148 668 0.020 0.277 PF-all [Gopalan 15] 0.145 691 0.021 0.280 PF-last [Gopalan 15] 0.065 807 0.019 0.268 7.5K users, 3.6K items, 2M non-zero ratings, binarized 3 months per time slice 24

Netflix Recall@50 MAR MRR NDCG dpf 0.156 1605 0.021 0.358 PF-all [Gopalan 15] 0.120 1807 0.015 0.338 PF-last [Gopalan 15] 0.138 1635 0.018 0.351 225K users, 14K items, 6.9M non-zero ratings, binarized 6 months per time slice 25

arxiv Recall@50 MAR MRR NDCG dpf 0.035 21822 0.0062 0.186 PF-all [Gopalan 15] 0.032 22402 0.0056 0.182 PF-last [Gopalan 15] 0.023 25616 0.0040 0.168 75K users, 5K items, 1.3M clicks 18 months per time slice 26

Usage Data DPF Factors 3.5 The Google Similarity effect arxiv: cs/0412098 Item 4663: The Google Similarity Distance Factor expression 3.0 Access frequency 2.5 2.0 1.5 1.0 2005 2006 2007 2008 2009 2010 2011 2012 GOobDO fdctor exsressioq 1.4 1.2 1.0 0.8 0.6 0.4 0.2 2005 2006 2007 2008 2009 2010 2011 2012 DPF Global Factors GrDSh Theory 4uDQtuP & &ops. &opsoexity DDtD 6tructures & ProbDbiOity GeQerDO 0Dth 2StiPizDtioQ 0.0 27

Paper Selected at Random 600000 500000 FaFtor expression 400000 300000 200000 100000 0 2005 2006 2007 2008 2009 2010 2011 2012

600000 500000 FaFtor expression 400000 300000 200000 100000 0 2005 2006 2007 2008 2009 2010 2011 2012

600000 500000 FaFtor expression 400000 300000 200000 100000 0 2005 2006 2007 2008 2009 2010 2011 2012 7 7he entrrpy frrmuld frr the 5icci flrw Dnd its germetric DpplicDtiRns (mdth/0211159) 6 5 Access fretuency 4 3 2 1 2003 2004 2005 2006 2007 2008 DDte 2009 2010 2011 2012 2013

600000 500000 FaFtor expression 400000 300000 200000 100000 0 2005 2006 2007 2008 2009 2010 2011 2012 7 7he entrrpy frrmuld frr the 5icci flrw Dnd its germetric DpplicDtiRns (mdth/0211159) 6 5 Access fretuency 4 3 2 1 2003 2004 2005 2006 2007 2008 DDte 2009 2010 2011 2012 2013

600000 500000 FaFtor expression 400000 300000 200000 100000 0 2005 2006 2007 2008 2009 2010 2011 2012 7 Perelman 6 declines the 5 Fields Medal Access fretuency 4 3 7he entrrpy frrmuld frr the 5icci flrw Dnd its germetric DpplicDtiRns (mdth/0211159) 2 1 2003 2004 2005 2006 2007 2008 DDte 2009 2010 2011 2012 2013

600000 500000 FaFtor expression 400000 300000 200000 100000 0 2005 2006 2007 2008 2009 2010 2011 2012 7 Perelman 6 declines the 5 Fields Medal Access fretuency 4 3 7he entrrpy frrmuld frr the 5icci flrw Dnd its germetric DpplicDtiRns (mdth/0211159) Perelman rejects the Millenium Prize 2 1 2003 2004 2005 2006 2007 2008 DDte 2009 2010 2011 2012 2013

Usage Data DPF Factors 9 User 755 8 Click frequency 7 6 5 4 3 Factor expression 2 2005 2006 2007 2008 2009 2010 2011 2012 2005 2006 2007 2008 2009 2010 2011 2012 30

Fields evolve over time 1.0 0.8 0.6 0.4 0.2 0.0 Computability Classes Physics & Society Quantum Information Theory Quantum Logic Quantum Algorithms General Interest CS Graph Theory Information Theory Message Passing Algorithms Mathematical Statistics Information Theory Computational Geometry Probability & Statistics for Graphs Monte Carlo & Simulation Community Detection Quantum Computing Machine Learning & Information Theory Computational Complexity Condensed Matter & Computing Network Analysis 04/05 10/05 04/06 10/06 04/07 10/07 04/08 10/08 04/09 10/09 04/10 10/10 04/11 10/11 03/12 31

Fields evolve over time 1.0 0.8 0.6 0.4 0.2 0.0 Computability Classes Physics & Society Quantum Information Theory Quantum Logic Quantum Algorithms General Interest CS Graph Theory Information Theory Message Passing Algorithms Mathematical Statistics Information Theory Computational Geometry Probability & Statistics for Graphs Monte Carlo & Simulation Community Detection Quantum Computing Machine Learning & Information Theory Computational Complexity Condensed Matter & Computing Network Analysis 04/05 10/05 04/06 10/06 04/07 10/07 04/08 10/08 04/09 10/09 04/10 10/10 04/11 10/11 03/12 31

Conclusions Dynamic Poisson factorization (DPF) 9 User 755 8 Click frequency 7 6 5 4 Factor expression 3 2 2005 2006 2007 2008 2009 2010 2011 2012 2005 2006 2007 2008 2009 2010 2011 2012 Implicit data, scalable Open source implementation is available: github.com/blei-lab/ Future work: levels of granularities, continuous time 32

Thank you! Dynamic Poisson factorization (DPF) 9 User 755 8 Click frequency 7 6 5 4 Factor expression 3 2 2005 2006 2007 2008 2009 2010 2011 2012 2005 2006 2007 2008 2009 2010 2011 2012 Implicit data, scalable Open source implementation is available: github.com/blei-lab/ Future work: levels of granularities, continuous time 32

2024 © sciencedocbox.com Privacy Policy | Terms of Service | Feedback