Foundations of Machine Learning Ranking. Mehryar Mohri Courant Institute and Google Research

Transcription

1 Foundations of Machine Learning Ranking Mehryar Mohri Courant Institute and Google Research

2 Motivation Very large data sets: too large to display or process. limited resources, need priorities. ranking more desirable than classification. Applications: search engines, information extraction. decision making, auctions, fraud detection. Can we learn to predict ranking accurately? 2

3 Related Problem Rank aggregation: given n candidates and k voters each giving a ranking of the candidates, find ordering as close as possible to these. closeness measured in number of pairwise misrankings. k =4 problem NP-hard even for (Dwork et al., 2001). 3

4 Score-based ranking This Talk Preference-based ranking 4

5 Score-Based Setting Single stage: learning algorithm receives labeled sample of pairwise preferences; returns scoring function h: U R. Drawbacks: h induces a linear ordering for full set U. does not match a query-based scenario. Advantages: efficient algorithms. good theory: VC bounds, margin bounds, stability bounds (FISS 03, RCMS 05, AN 05, AGHHR 05, CMR 07). 5

6 with Score-Based Ranking Training data: sample of i.i.d. labeled pairs drawn from U U according to some distribution D, S = (x 1,x 1,y 1 ),...,(x m,x m,y m ) U U { 1, 0, +1}, y i = Problem: find hypothesis h:u R in H with small generalization error R(h) = +1 if x i > pref x i 0 if x i = pref x i or no information 1 if x i < pref x i. Pr (x,x 0 ) D h (f(x, x 0 ) 6= 0) ^ (f(x, x 0 ) h(x 0 ) h(x) apple 0) i. 6

7 Empirical error: R(h) = 1 m m i=1 Notes 1 (yi =0) (y i (h(x i ) h(x i )) 0). The relation x R x f(x, x )=1may be nontransitive (needs not even be anti-symmetric). Problem different from classification. 7

8 Distributional Assumptions Distribution over points: labels for pairs. dependency issue. points (literature). squared number of examples O(m 2 ). Distribution over pairs: m m pairs. label for each pair received. independence assumption. same (linear) number of examples. 8

9 Confidence Margin in Ranking Labels assumed to be in {+1, 1}. Empirical margin loss for ranking: for > 0, br (h) = 1 m br (h) apple 1 m mx i=1 mx i=1 y i h(x 0 i) h(x i ) 1 yi [h(x 0 i ) h(x i)]apple.. 9

10 Marginal Rademacher Complexities Distributions: D 1 marginal distribution with respect to the first element of the pairs; D 2 marginal distribution with respect to second element of the pairs. Samples: S 1 = (x 1,y 1 ),...,(x m,y m ) S 2 = (x 0 1,y 1 ),...,(x 0 m,y m ). Marginal Rademacher complexities: R D 1 m (H) =E[ b R S1 (H)] R D 2 m (H) =E[ b R S2 (H)]. 10

11 Ranking Margin Bound (Boyd, Cortes, MM, and Radovanovich 2012; MM, Rostamizadeh, and Talwalkar, 2012) Theorem: let H be a family of real-valued functions. Fix >0, then, for any >0, with probability at least 1 over the choice of a sample of size m, the following holds for all h H: R(h) R (h)+ 2 R D 1 m (H)+R D 2 m (H) + log 1 2m. 11

12 Ranking with SVMs Optimization problem: application of SVMs. min w, Decision function: 1 2 w 2 + C m i=1 subject to: y i w (x i ) (x i ) 1 i i 0, i [1,m]. h: x w (x)+b. i see for example (Joachims, 2002) 12

13 Notes The algorithm coincides with SVMs using feature mapping (x, x ) (x, x )= (x ) (x). Can be used with kernels: K 0 ((x i,x 0 i), (x j,x 0 j)) = (x i,x 0 i) (x j,x 0 j) = K(x i,x j )+K(x 0 i,x 0 j) K(x 0 i,x j ) K(x i,x 0 j). Algorithm directly based on margin bound. 13

14 Boosting for Ranking Use weak ranking algorithm and create stronger ranking algorithm. Ensemble method: combine base rankers returned by weak ranking algorithm. Finding simple relatively accurate base rankers often not hard. How should base rankers be combined? 14

15 CD RankBoost (Freund et al., 2003; Rudin et al., 2005) H {0, 1} X. 0 t + + t + t =1, s t (h) = Pr sgn f(x, x )(h(x ) h(x)) = s. (x,x ) D t RankBoost(S =((x 1,x 1,y 1 )...,(x m,x m,y m ))) 1 for i 1 to m do 1 2 D 1 (x i,x i ) m 3 for t 1 to T do 4 h t base ranker in H with smallest t + t = E i Dt y i h t (x i ) h t (x i ) 1 5 t 2 log + t t 0 6 Z t t +2[ + t t ] 1 2 normalization factor 7 for i 1 to m do 8 D t+1 (x i,x i ) 9 T 10 return T T t=1 th t D t (x i,x i )exp ty i h t (x i ) h t (x i ) Z t 15

16 Distributions D t originally uniform. Notes over pairs of sample points: at each round, the weight of a misclassified example is increased. observation:, since D t+1 (x, x )= e y[ t (x ) t (x)] S Q t s=1 Z s D t+1 (x, x )= D t(x, x )e y t[h t (x ) h t (x)] Z t = 1 S weight assigned to base classifier h t : t directly depends on the accuracy of at round t. h t e y P t s=1 s[h s (x ) h s (x)] t s=1 Z s. 16

17 Coordinate Descent RankBoost Objective Function: convex and differentiable. F ( )= e y[ T (x ) T (x)] = exp y (x,x,y) S (x,x,y) S t=1 T t[h t (x ) h t (x)]. e x 0 1pairwise loss 17

18 Direction: unit vector with e t e t =argmin t df ( + e t ). d =0 Since F ( + e, t )= e y P T s=1 s[h s (x ) h s (x)] e y [h t(x ) h t (x)] df ( + e t ) = d =0 = (x,x,y) (x,x,y) (x,x,y) S S S y[h t (x ) h t (x)] exp y y[h t (x ) = [ + t t ] m T s=1 Z s. Thus, direction corresponding to base classifier selected by the algorithm. T s=1 h t (x)]d T +1 (x, x ) m s[h s (x ) T s=1 Z s h s (x)] 18

19 Step size: obtained via df ( + e t ) d (x,x,y) (x,x,y) (x,x,y) S S S =0 y[h t (x ) h t (x)] exp y y[h t (x ) T s=1 h t (x)]d T +1 (x, x ) m s[h s (x ) h s (x)] e y[h t(x ) h t (x)] T s=1 Z s e y[h t(x ) y[h t (x ) h t (x)]d T +1 (x, x )e y[h t(x ) h t (x)] [ + t e t e ]=0 =0 h t (x)] =0 =0 = 1 2 log + t t. Thus, step size matches base classifier weight used in algorithm. 19

20 Bipartite Ranking Training data: sample of negative points drawn according to S =(x 1,...,x m ) U. sample of positive points drawn according to S + =(x 1,...,x m ) U. Problem: find hypothesis h: U! R in H with small generalization error D D + R D (h) = Pr h(x )<h(x). x D,x D + 20

21 Notes Connection between AdaBoost and RankBoost (Cortes & MM, 04; Rudin et al., 05). if constant base ranker used. relationship between objective functions. More efficient algorithm in this special case (Freund et al., 2003). Bipartite ranking results typically reported in terms of AUC. 21

22 AdaBoost and CD RankBoost Objective functions: comparison. F Ada ( )= exp ( y i f(x i )) x i S S + = exp (+f(x i )) + exp ( f(x i )) x i S + x i S = F ( )+F + ( ). F Rank ( )= exp [f(x j ) f(x i )] (i,j) S S + = exp (+f(x i )) exp ( f(x i )) (i,j) S S + = F ( )F + ( ). 22

23 AdaBoost and CD RankBoost Property: AdaBoost (non-separable case). constant base learner h=1 equal contribution of positive and negative points (in the limit). consequence: AdaBoost asymptotically achieves optimum of CD RankBoost objective. Observations: if F + ( )=F ( ), d(f Rank )=F + d(f )+F d(f + ) = F + d(f )+d(f + ) = F + d(f Ada ). (Rudin et al., 2005) 23

24 with Bipartite RankBoost - Efficiency Decomposition of distribution: for (x, x ) (S,S + ), Thus, D(x, x )=D (x)d + (x ). D t+1 (x, x )= D t(x, x )e t[h t (x ) h t (x)] Z t = D t, (x)e t h t (x) Z t, D t,+ (x )e tht(x ) Z t,+, Z t, = x S D t, (x)e t h t (x) Z t,+ = x S + D t,+ (x )e tht(x ). 24

25 ROC Curve (Egan, 1975) Definition: the receiver operating characteristic (ROC) curve is a plot of the true positive rate (TP) vs. false positive rate (FP). TP: % positive points correctly labeled positive. FP: % negative points incorrectly labeled positive. 1 True positive rate h(x 3 ) h(x 14 ) h(x 5 ) h(x 1 ) h(x 23 ) sorted scores False positive rate 25

26 Area under the ROC Curve (AUC) (Hanley and McNeil, 1982) Definition: the AUC is the area under the ROC curve. Measure of ranking quality. 1 True positive rate AUC Equivalently, AUC(h) = 1 mm m i=1 =1 R(h) False positive rate m j=1 1 h(xj )>h(x i ) = Pr x x b D b D+ [h(x ) >h(x)] 26

27 Score-based ranking This Talk Preference-based ranking 27

28 Preference-Based Setting Definitions: U: universe, full set of objects. V : finite query subset to rank, V U. : target ranking for V (random variable). Two stages: can be viewed as a reduction. learn preference function h: U U [0, 1]. given V, use h to determine ranking of V. Running-time: measured in terms of calls to h. 28

29 Preference-Based Ranking Problem Training data: pairs to D: subsets ranked by different labelers. (V, ) sampled i.i.d. according (V 1, 1 ), (V 2, 2 ),...,(V m, m) V i U. learn classifier preference function h: U U [0, 1]. Problem: for any query set V U, use h to return ranking h,v close to target with small average error R(h, )= E (V, ) D [L( h,v, )]. 29

30 Preference Function h(u, v) close to when u preferred to, close to otherwise. For the analysis, h(u, v) {0, 1}. Assumed pairwise consistent: 1 v 0 h(u, v)+h(v,u) =1. May be non-transitive, e.g., we may have h(u, v) =h(v, w) =h(w, u) =1. Output of classifier or black-box. 30

31 (for fixed (V, )) Preference loss: L(h, )= Ranking loss: L(, )= Loss Functions 2 n(n 1) u =v h(u, v) (v, u). 2 n(n 1) u =v (u, v) (v, u). 31

32 Preference regret: (Weak) Regret R class (h) = E [L(h V, )] E min V, V h E V [L( h, )]. Ranking regret: R rank(a) = E [L(A s(v ), )] E min V,,s V S(V ) E V [L(, )]. 32

33 Deterministic Algorithm Stage one: standard classification. Learn preference function h: U U [0, 1]. Stage two: sort-by-degree using comparison function h. sort by number of points ranked below. quadratic time complexity O(n 2 ). (Balcan et al., 07) 33

34 Randomized Algorithm (Ailon & MM, 08) Stage one: standard classification. Learn preference function h: U U [0, 1]. Stage two: randomized QuickSort (Hoare, 61) using as comparison function. comparison function non-transitive unlike textbook description. but, time complexity shown to be O(n log n) in general. h 34

35 Randomized QS v h(v, u)=1 h(u, v)=1 left recursion u random pivot right recursion 35

36 Deterministic Algo. - Bipartite Case (V = V + V ) (Balcan et al., 07) Bounds: for deterministic sort-by-degree algorithm expected loss: E V, regret: [L(A(V ), )] 2 E V, [L(h, )]. R rank (A(V )) Time complexity:. ( V 2 ) 2 R class (h). 36

37 Randomized Algo. - Bipartite Case (V = V + V ) (Ailon & MM, 08) Bounds: for randomized QuickSort (Hoare, 61). expected loss (equality): regret: E V,,s [L(Qh s (V ), )] = E [L(h, )]. V, Time complexity: full set: O(n log n). top k: R rank (Q h s ( )) R class (h). O(n + k log k). 37

38 Proof Ideas p uv QuickSort decomposition: w {u,v} Bipartite property: p uvw h(u, w)h(w, v)+h(v, w)h(w, u) u (u, v)+ (v,w)+ (w,u) = =1. (v, u)+ (w, v)+ (u, w). v w 38

39 Lower Bound Theorem: for any deterministic algorithm A, there is a bipartite distribution for which R rank (A) thus, factor of 2 = Proof: take simple case that h induces a cycle. up to symmetry, A returns best in deterministic case. randomization necessary for better bound. u, v, w w, v, u. or 2 R class (h). U =V ={u, v, w} v and assume u h w 39

40 Lower Bound If A returns, then choose as: u, v, w u u, v w - + If A returns w, v, u, then choose as: v h L[h, ]= 1 3 ; L[A, ]= 2 3. w w, v u

41 Guarantees - General Case Loss bound for QuickSort: E V,,s [L(Qh s (V ), )] 2 E [L(h, )]. V, Comparison with optimal ranking (see (CSS 99)): E s [L(Q h s (V ), optimal)] E s [L(h, Q h s (V ))] 2 L(h, optimal) 3 L(h, optimal), where optimal = argmin L(h, ). 41

42 Generalization: Weight Function (u, v) = (u, v) ( (u), (v)). Properties: needed for all previous results to hold, symmetry: (i, j) = (j, i) for all i, j. monotonicity: (i, j), (j, k) (i, k) for i<j<k. triangle inequality: (i, j) (i, k)+ (k, j) for all triplets i, j, k. 42

43 Weight Function - Examples Kemeny: w(i, j) =1, i, j. Top-k: Bipartite: w(i, j) = w(i, j) = 1 if i k or j k; 0 otherwise. 1 if i k and j>k; 0 otherwise. k-partite: can be defined similarly. 43

44 (Strong) Regret Definitions Ranking regret: R rank (A) = E [L(A s(v ), )] V,,s Preference regret: min E V, [L( V, )]. R class (h) = E V, [L(h V, )] min h E V, [L( h V, )]. All previous regret results hold if for V 1,V 2 {u, v}, E V1 [ (u, v)] = E V2 [ (u, v)] for all u, v (pairwise independence on irrelevant alternatives). 44

45 References Agarwal, S., Graepel, T., Herbrich, R., Har-Peled, S., & Roth, D. (2005). Generalization bounds for the area under the roc curve. JMLR 6, Agarwal, S., and Niyogi, P. (2005). Stability and generalization of bipartite ranking algorithms. COLT (pp ). Nir Ailon and Mehryar Mohri. An efficient reduction of ranking to classification. In Proceedings of COLT Helsinki, Finland, July Omnipress. Balcan, M.-F., Bansal, N., Beygelzimer, A., Coppersmith, D., Langford, J., and Sorkin, G. B. (2007). Robust reductions from ranking to classification. In Proceedings of COLT (pp ). Springer. Stephen Boyd, Corinna Cortes, Mehryar Mohri, and Ana Radovanovic (2012). Accuracy at the top. In NIPS Cohen, W. W., Schapire, R. E., and Singer, Y. (1999). Learning to order things. J. Artif. Intell. Res. (JAIR), 10, Cossock, D., and Zhang, T. (2006). Subset ranking using regression. COLT (pp ). Mehryar Mohri - Courant Seminar 45 Courant Institute, NYU

46 References Corinna Cortes and Mehryar Mohri. AUC Optimization vs. Error Rate Minimization. In Advances in Neural Information Processing Systems (NIPS 2003), MIT Press. Cortes, C., Mohri, M., and Rastogi, A. (2007a). An Alternative Ranking Problem for Search Engines. Proceedings of WEA 2007 (pp. 1 21). Rome, Italy: Springer. Corinna Cortes and Mehryar Mohri. Confidence Intervals for the Area under the ROC Curve. In Advances in Neural Information Processing Systems (NIPS 2004), MIT Press. Crammer, K., and Singer, Y. (2001). Pranking with ranking. Proceedings of NIPS 2001, December 3-8, 2001, Vancouver, British Columbia, Canada] (pp ). MIT Press. Cynthia Dwork, Ravi Kumar, Moni Naor, and D. Sivakumar. Rank Aggregation Methods for the Web. WWW 10, ACM Press. J. P. Egan. Signal Detection Theory and ROC Analysis. Academic Press, Yoav Freund, Raj Iyer, Robert E. Schapire and Yoram Singer. An efficient boosting algorithm for combining preferences. JMLR 4: , Mehryar Mohri - Courant Seminar 46 Courant Institute, NYU

47 References J. A. Hanley and B. J. McNeil. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, Ralf Herbrich, Thore Graepel, and Klaus Obermayer. Large margin rank boundaries for ordinal regression. Advances in Large Margin Classifiers, s , Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. Foundations of Machine Learning. The MIT Press Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd. The PageRank citation ranking: Bringing order to the Web, Stanford Digital Library Technologies Project, Lehmann, E. L. (1975). Nonparametrics: Statistical methods based on ranks. San Francisco, California: Holden-Day. Cynthia Rudin, Corinna Cortes, Mehryar Mohri, and Robert E. Schapire. Margin-Based Ranking Meets Boosting in the Middle. In Proceedings of The 18th Annual Conference on Computational Learning Theory (COLT 2005), s 63-78, Thorsten Joachims. Optimizing search engines using clickthrough data. Proceedings of the 8th ACM SIGKDD, s , Mehryar Mohri - Courant Seminar 47 Courant Institute, NYU