FAQs. Fitting h θ (x)

Transcription

1 10/15/ FALL 2018 W9.A /15/ FALL 2018 W9.A.1 FAs How to iprove the ter project proposal? Brainstor with your teaates Make an appointent with e (eeting ust include all of the tea ebers) PART 1. LARGE SCALE DATA ANALYTICS 4. RECOMMENDATION SYSTEMS Coputer Science, Colorado State University 10/15/ FALL 2018 W9.A.2 Today s topics 10/15/ FALL 2018 W9.A.3 Fitting h θ (x) Linear Regression: Running with MapReduce Recoendation Systes Overview Background: Data Siilarity h θ(x) J(θ 0,θ 1) θ x x x x 10/15/ FALL 2018 W9.A.4 10/15/ FALL 2018 W9.A.5 Gradient descent for Linear Regression Repeat until convergence { } θ 0 :=θ 0 α 1 (x(i) ) y (i) ) θ 1 :=θ 1 α 1 (x (i) ) y (i) )x (i) (for j=0 and j=1) Update θ0 and θ1 siultaneously Part 1. Large Scale Data Analytics 3. Predictive Analysis Linear Regression: Running with MapReduce 1

2 10/15/ FALL 2018 W9.A.6 Batch Gradient Descent Batch Each step of gradient descent uses all of the training exaple θ 0 :=θ 0 α 1 θ 1 :=θ 1 α 1 x (i) 10/15/ FALL 2018 W9.A.7 Running with MapReduce For the saple size 1,000 (=1,000) Batch gradient descent: θ 0 :=θ 0 α 1 (x(i) ) y (i) ) θ 1 :=θ 1 α 1 x (i) Using 4 achines 10/15/ FALL 2018 W9.A.8 10/15/ FALL 2018 W9.A.9 For θ 0 Step 1. 4 input splits (x (1),y(1)),,((x(250),y250)) (x (251),y(251)),,((x(500),y500)) (x (501),y(501)),,((x(750),y750)) (x (751),y(751)),,((x(1000),y1000)) Step 2. Calculate tep1 ~ 4 tep1= Step 3. Calculate final results θ 0 :=θ 0 α 1 (tep1+ tep2 + tep3+ tep4) 1, tep2 = i= tep3 = i= tep4 = i=751 For θ 1 Step 1. 4 input splits (x (1),y(1)),,((x(250),y250)) (x (251),y(251)),,((x(500),y500)) (x (501),y(501)),,((x(750),y750)) (x (751),y(751)),,((x(1000),y1000)) Step 2. Calculate tep1 ~ 4 tep1= tep2 = tep3 = tep4 = Step 3. Calculate final results θ 1 :=θ 1 α 1 (tep1+ tep2 + tep3+ tep4) 1, i= i= i=751 x (i) x (i) x (i) x (i) 10/15/ FALL 2018 W9.A.10 10/15/ FALL 2018 W9.A.11 This aterial is built based on Large Scale Data Analytics 4. Recoendation Systes Yifan Hu, Yehuda Koren, and Chris Volinsky Collaborative Filtering for Iplicit Feedback Datasets. In Proceedings of the 2008 Eighth IEEE International Conference on Data Mining (ICDM '08). IEEE Coputer Society, Washington, DC, USA, DOI= Sandy Ryza, Uri Laserson, Sean Owen, and Josh Wills, Advanced Analytics with Spark, O Reilly,

3 10/15/ FALL 2018 W9.A.12 What percentage of the top 10,000 titles in any online edia store (Netflix, itunes, Aazon, or any other) will rent or sell at least once a onth? 10/15/ FALL 2018 W9.A.13 The long tail phenoenon [1/2] Distribution of nubers with a portion that has a large nuber of occurrences far fro the head or central part of the distribution The vertical axis represents popularity The ites are ordered on the horizontal axis according to their popularity The long-tail phenoenon forces online institutions to recoend ites to individual users Erik Brynjolfsson, Yu (Jeffrey) Hu, and Duncan Siester Goodbye Pareto Principle, Hello Long Tail: The Effect of Search Costs on the Concentration of Product Sales. Manage. Sci. 57, 8 (August 2011), DOI= 10/15/ FALL 2018 W9.A.14 The long tail phenoenon [2/2] Touching the Void, Joe Sipson, /15/ FALL 2018 W9.A.15 Recoendation systes Seek to predict the rating or preference that a user would give to an ite Into Thin Air: A Personal Account of the Mt. Everest Disaster, Jon Krakauer, /15/ FALL 2018 W9.A.16 Applications of Recoendation Systes Product recoendations Aazon or siilar online vendors Movie recoendations Netflix offers its custoers recoendations of ovies they ight like News articles News services have attepted to identify articles of interest to readers based on the articles that they have read in the past Blogs, YouTube 10/15/ FALL 2018 W9.A.17 Netflix Prize The Netflix Prize challenge concerned recoender systes for ovies (October, 2006) Netflix released a training set consisting of data fro alost 500,000 custoers and their ratings on 18,000 ovies. More than 100 illion ratings The task was to use these data to build a odel to predict ratings for a hold-out set of 3 illion ratings 3

4 10/15/ FALL 2018 W9.A.18 10/15/ FALL 2018 W9.A.19 Jaccard Coefficient (a. without description) Large Scale Data Analytics 4. Recoendation Systes Background: Data Siilarity Copare two sets P and with the following forula: StringJaccard(P,) = P P Measures the fraction of the data that is shared between P and Copared to all data available in the union of these two sets. What are P and? Set of tokens fro Strings Coplete description about data (candidates) 10/15/ FALL 2018 W9.A.20 Exaple 10/15/ FALL 2018 W9.A.21 Jaccard Coefficient (b. with description) StringJaccard S(c1) = {Thoas, Sean, Connery} S(c2) = {Sir, Sean, Connery} What is the string Jaccard coefficient between c1 and c2? Given two candidates, the Jaccard coefficient of two candidates c1 and c2 is given by, DescriptionJaccard(c1,c2) = OD(c1) OD(c2) OD(c1) OD(c2) StringJaccard(P,) = P P = 2 / 4 = /15/ FALL 2018 W9.A.22 Exaple 10/15/ FALL 2018 W9.A.23 uestion Now, specify the parts of a person s nae as title, firstnae, iddlenae, and lastnae DescriptionJaccard and StringJaccard have the sae value. Is this always true? OD(c1) = {(firstnae, Thoas),(iddlenae, Sean), (lastnae, Connery)} OD(c2) = {(title, Sir),(iddlenae, Sean),(lastnae, Connery)} DescriptionJaccard(c1, c2) = 2/4 4

5 10/15/ FALL 2018 W9.A.24 Exaple What if Sean would have been put in the firstnae/iddlenae attribute? OD(c1) = {(iddlenae, Thoas), (firstnae, Sean), (lastnae, Connery)} OD(c2) = {(title, Sir),(iddlenae, Sean),(lastnae, Connery)} DescriptionJaccard(c1, c2) = 1/5 10/15/ FALL 2018 W9.A.25 Deficiencies of the Jaccard Siilarity Soe attribute is ore descriptive Title is less descriptive than firstnae and lastnae Very sensitive to typographical errors in single tokens Shean Conery and Sean Connery have a siilarity of zero. 10/15/ FALL 2018 W9.A.26 Cosine siilarity Given two n-diensional vectors V and W, the cosine siilarity coputes the cosine of the angle α between these two vectors as V W CosineSiilarity(V,W) = cos(α) = V W Where V is the length of the vector V = [a,b,c..] coputed as a 2 + b 2 + c /15/ FALL 2018 W9.A.27 Cosine siilarity - Continued The vectors V and W Tokens in a string Descriptions of a candidate The d diensions of these vectors correspond to all d distinct tokens in a set of strings. Denoted as D For a large database, d ay be large V and W have high diensionality d 10/15/ FALL 2018 W9.A.28 Weight of Token Vector contains a weight for each of the d distinct tokens How to easure the weight? Measuring frequency Ter frequency inverse docuent frequency (tf-idf) 10/15/ FALL 2018 W9.A.29 Ter frequency Nuber of ties that ter t occurs in the docuent d Raw ter frequency: ft,d Boolean frequencies tf(t,d) is 1 if t occurs in d and 0 otherwise Logarithically scaled frequency: tf (t, d) =1+ log f t,d 0 if ft,d is zero 5

6 10/15/ FALL 2018 W9.A.30 Exaple: What is the raw tf Aerican,? 10/15/ FALL 2018 W9.A.31 Exaple: What is the raw tf Aerican,? 0 Nae Aerican Autoobile Aerican Copany Farers Mutual of Aerican Life Safeway Group Aerican Autoobile Copany Farers 0 Nae Aerican Autoobile Aerican Copany Farers Mutual of Aerican Life Safeway Group tf Aerican, = 1 Aerican Autoobile Copany Farers 10/15/ FALL 2018 W9.A.32 Inverse docuent frequency Assigns higher weights to tokens that occurs less frequently in the scope of all candidate descriptions N idf (t, D) = log 10 {d D : t d} Where, N is the total nuber of docuents in the corpus Nuber of docuents where the ter t appears (i.e. tf(t,d) 0) 10/15/ FALL 2018 W9.A.33 Exaple: What is the idf Aerican, D? (D: this corpus) Nae Aerican Autoobile Aerican Aerican Copany Autoobile Farers Copany Mutual of Aerican Life Farers Safeway Group 0 10/15/ FALL 2018 W9.A.34 Exaple: What is the idf Aerican, D? 10/15/ FALL 2018 W9.A.35 tf-idf weighting Nae Aerican Autoobile Aerican Copany Farers Aerican Autoobile The product of its tf weight and its idf weight W t,d = (1+log 10 tf t,d ) log 10 (N /df t ) For the total nuber of docuents, N If the nubers of ters are different in the docuents, you should noralize tft,d to ( tft,d / (nuber of words in d) ) 0 Mutual of Aerican Life Safeway Group idf Aerican, D= log(10/3) Copany Farers Best known weighting schee in inforation retrieval Note: the - in tf-idf is a hyphen, not a inus sign! Alternative naes: tf.idf, tf x idf Increases with the nuber of occurrences within a docuent Increases with the rarity of the ter in the collection 6

7 10/15/ FALL 2018 W9.A.36 Exaple: What is the tf-idf Aerican,,D and tf-idf Aerican,,D? 10/15/ FALL 2018 W9.A.37 Exaple: What is the tf-idf Aerican,,D and tf-idf Aerican,,D? 0 Nae Aerican Autoobile Aerican Copany Farers Mutual of Aerican Life Safeway Group Aerican Autoobile Copany Farers 0 Nae Aerican Autoobile Aerican Copany Farers Mutual of Aerican Life Safeway Group Aerican Autoobile Copany Farers tf-idf Aerican,,D= (1+log(1/4)) x log(10/3) tf-idf Aerican,,D= (1+log(1/6)) x log(10/3) 10/15/ FALL 2018 W9.A.38 10/15/ FALL 2018 W9.A.39 Exaple continued Copute the siilarity between the two strings s1= Farers, s2 = Ter vector = (, Aerican, Autoobile,,,, Farers,,.) s1= Farers = (0, 0, 0, 0, 0, 1, 1, 0 ) s2= = (0, 0, 0, 0, 0, 1, 0, 1, ) Next step: find the weights Exaple continued Nae c1 c2 Aerican Autoobile c3 Aerican Copany c4 Farers c5 c6 c7 c8 Mutual of Aerica Life c9 Safeway Group c10 s1= Farers = (0, 0, 0, 0, 0, 1, 1, 0 ) s2= = (0, 0, 0, 0, 0, 1, 0, 1, ) V W 0 0 Aerica 0 0 Autoobil 0 0 e Copany 0 0 Farers /15/ FALL 2018 W9.A.40 10/15/ FALL 2018 W9.A.41 Exaple continued Copute the siilarity between the two strings s1= Farers, s2 = Six of the candidates contain the token. idf, D = log10 (10/6) = 0.78 idf Farers, D = log10 (10/1) = 1 idf, D = log10 (10/1) =1 tf-idf Farers, c4 = (1+log10 1/2) x log10 (10/1) = 0.7 tf-idf, c4 = (1+log101/2) x log10 (10/6) 0.55 tf-idf, c7 = (1+log10 1/2) x log10 (10/1) = 0.7 tf-idf, c7 = (1+log101/2) x log10 (10/6) 0.55 Exaple continued c1 c2 c3 c4 c5 c6 c7 c8 c9 c10 Nae Aerican Autoobile Aerican Copany Farers Mutual of Aerica Life Safeway Group s1= Farers = (0, 0, 0, 0, 0, 0.55, 0.7, 0 ) s2= = (0, 0, 0, 0, 0, 0.55, 0, 0.7, ) V W 0 0 Aerica 0 0 Autoobil e Copany 0 0 Farers

8 10/15/ FALL 2018 W9.A.42 10/15/ FALL 2018 W9.A.43 Exaple continued s1= Farers = (0, 0, 0, 0, 0, 0.55, 0.7, 0 ) s2= = (0, 0, 0, 0, 0, 0.55, 0, 0.7, ) Copute the siilarity between the two strings s1=farers, s2 = V W CosSiilarity(V,W ) = cos(α) = V W = Large Scale Data Analytics 4. Recoendation Systes Collaborative Filtering What is the Jaccard siilarity for the sae case? 10/15/ FALL 2018 W9.A.44 Collaborative filtering Focus on the siilarity of the user ratings for ites Users are siilar if their vectors are close according to soe distance easure E.g. Jaccard or cosine distance Collaborative filtering The process of identifying siilar users and recoending what siilar users like 10/15/ FALL 2018 W9.A.45 Measuring siilarity How to easure siilarity of users or ites fro their rows or coluns in the utility atrix? Jaccard Siilarity for A and B: 1/5 Jaccard Siilarity for A and C: 2/4 For user A, user C ight have siilar opinion than user B Can user C provide a prediction for A? HP1 HP2 HP3 TW SW1 SW2 SW3 A B C D /15/ FALL 2018 W9.A.46 Cosine siilarity (1/2) We can treat blanks as a 0 values The cosine of the angle between A and B is = HP1 HP2 HP3 TW SW1 SW2 SW3 A B C D /15/ FALL 2018 W9.A.47 Cosine siilarity (2/2) We can treat blanks as 0 values The cosine of the angle between A and C is = A is slightly closer to B than to C HP1 HP2 HP3 TW SW1 SW2 SW3 A B C D 3 3 8

9 10/15/ FALL 2018 W9.A.48 Noralizing ratings (1/2) What if we noralize ratings by subtracting fro each rating the average rating of that user? Soe rating (very low) will turn into negative nubers If we take the cosine distance, the opposite views of the ovies will have vectors in alost opposite directions It can be as far apart as possible 9