Online Ranking by Projecting

Transcription

1 LETTER Communicaed by Edwad Haingon Online Ranking by Pojecing Koby Camme Yoam Singe School of Compue Science and Engineeing, Hebew Univesiy, Jeusalem 91904, Isael We discuss he poblem of anking insances. In ou famewok, each insance is associaed wih a ank o a aing, which is an inege in 1 o k. Ou goal is o find a ank-pedicion ule ha assigns each insance a ank ha is as close as possible o he insance s ue ank. We discuss a goup of closely elaed online algoihms, analyze hei pefomance in he misake-bound model, and pove hei coecness. We descibe wo ses of expeimens, wih synheic daa and wih he EachMovie daa se fo collaboaive fileing. In he expeimens we pefomed, ou algoihms oupefom online algoihms fo egession and classificaion applied o anking. 1 Inoducion The anking poblem we discuss in his aicle shaes common popeies wih boh classificaion and egession poblems. As in classificaion poblems, he goal is o assign one of k possible labels o a new insance. Simila o egession poblems, he se of k labels is sucued, as hee is a oal ode elaion beween he labels. We efe o he labels as anks and wihou loss of genealiy assume ha he anks consiue he se {1, 2,...,k}. Seings in which i is naual o ank o ae insances ahe han classify ae common in asks such as infomaion eieval and collaboaive fileing. We use he lae as ou unning example. In collaboaive fileing, he goal is o pedic a use s aing on new iems such as books o movies given he use s pas aings of he simila iems. The goal is o deemine whehe a movie fan will like a new movie and o wha degee, which is expessed as a ank. An example fo possible aings migh be, un-o-see, vey-good, good, only-if-you-mus, and do-no-bohe. While he diffeen aings cay meaningful semanics, fom a leaning-heoeic poin of view, we model he aings as a oally odeed se (whose size is five in he example above). The inees in odeing o anking of objecs is by no means new and is sill he souce of ongoing eseach in many fields, such as mahemaical economics, social science, and compue science. Fo an oveview of ank- Neual Compuaion 17, (2005) c 2004 Massachuses Insiue of Technology

2 146 K. Camme and Y. Singe ing poblems fom a leaning-heoeic poin of view see Cohen, Schapie, and Singe (1999). One of he main esuls undescoed in his aicle is a complexiy gap beween classificaion leaning and anking leaning. To sidesep he inheen inacabiliy poblems of anking leaning, seveal appoaches have been suggesed. One possible appoach is o cas a anking poblem as a egession poblem. Anohe is o educe a oal ode ino a se of pefeences ove pais (Feund, Iye, Schapie, & Singe, 2003; Hebich, Gaepel, & Obemaye, 2000). The fis case imposes a meic on he se of anking ules ha migh no be ealisic, while he second appoach is ime-consuming since i equies inceasing he sample size fom n o O(n 2 ). In his lee, we conside an alenaive appoach ha diecly mainains a oally odeed se via pojecions. Ou saing poin is simila o ha of Hebich e al. (2000) in he sense ha we pojec each insance ino he eal numbes. Howeve, ou wok hen deviaes and opeaes diecly on ankings by associaing each anking wih disinc subineval of he eal numbes and adaping he suppo of each subineval while leaning. The aicle is oganized as follows. In he nex secion, we descibe a simple and efficien online algoihm ha manipulaes concuenly he diecion ono which we pojec he insances and he division ino subinevals. In secion 3, we pove he coecness of he algoihm and analyze is pefomance in he misake-bound model on vaious assumpions. Secion 4 conains he descipion and analysis of a nom-opimized vesion of he basic anking algoihm. We hen shif ou aenion o a muliplicaive algoihm ha is descibed and analyzed in secion 5. We povide empiical validaion of he meis of he vaious anking algoihms we devise in secion 6. This secion descibes expeimens compaing he anking algoihms o online classificaion and egession algoihms. Finally, we conclude wih a bief discussion and menion a few open poblems. Befoe moving on o he coe of he aicle, we poin o a few closely elaed aicles. Fis, a peliminay vesion of his lee appeaed a Neual Infomaion Pocessing Sysems, 2001, unde he ile Panking wih Ranking (Camme & Singe, 2001a). This aicle exends he confeence vesion in numeous diecions by poviding complee analysis as well as hee new algoihms ha wee no discussed in he confeence vesion. Second, in a ecen wok by Shashua and Levin (2002), an SVM-based algoihm fo insance anking was descibed. The algoihms of Shashua and Levin shae numeous popeies wih he algoihms pesened in his aicle. Howeve, i is designed fo bach seings, while ou focus is on online algoihms. Las, we would like o noe ha Haingon (2003) demonsaed empiically an impoved genealizaion pefomance of ou algoihm using an aveaging echnique, while peseving he ode of he hesholds. 2 The Basic PRank Algoihm This lee focuses on online algoihms fo anking insances. We ae given a sequence ( x 1, y 1 ),...,( x, y ),...of insance-ank pais. Fo conceeness,

3 Online Ranking by Pojecing 147 we assume ha each insance x is in R n, and is coesponding ank y is an elemen fom finie se Y wih a oal ode elaion. We assume wihou loss of genealiy ha Y ={1, 2,...,k} wih > as he ode elaion. The oal ode ove he se Y induces a paial ode ove he insances in he following naual sense. We say ha x is pefeed ove x s if y > y s.we also say ha x and x s ae no compaable if neihe y > y s no y < y s.we denoe his case simply as y = y s. Noe ha he induced paial ode is of a unique fom in ha he insances fom k equivalence classes ha ae oally odeed. 1 We sess ha alhough he elemens of Y ae denoed by ineges, we do no use he fac ha he ineges also belong o a meic space. We assume only ha he se Y is discee and is elemens ae oally odeed. A anking ule H is a mapping fom he insance space o he se of ank values, H: R n Y. The family of anking ules we discuss in his aicle employs a veco w R n and a se of k hesholds b 1 b k 1 b k =. Fo convenience, we denoe by b = (b 1,...,b k 1 ) he veco of hesholds, excluding b k, which is fixed o. Given a new insance x, he anking ule fis compues he inne poduc beween w and x. The pediced ank is hen defined o be he index of he fis (smalles) heshold b fo which w x < b. Such a anking ule divides he space ino paallel, equally anked egions: all he insances ha saisfy b 1 < w x < b ae assigned he same ank. Fomally, given a anking ule defined by w and b, he pediced ank of an insance x is H( x) = min {1,...,k} { : w x b < 0}. Noe ha he above minimum is always well defined since we se b k =. The analysis ha we use in his lee is based on he misake-bound model fo online leaning. The algoihms we descibe wok in ounds. On ound, he leaning algoihm ges an insance x. Given x, he algoihm oupus a ank, ŷ = min { : w x b < 0}. I hen eceives he coec ank y and updaes is anking ule by modifying w and b. We say ha ou algoihm made a anking misake if ŷ y and wish o make he pediced ank as close (in a sense descibed lae in his aicle) as possible o he ue ank. Fomally, he goal of he leaning algoihm is o minimize he anking loss, which is defined o be he numbe of hesholds beween he ue ank and he pediced ank. Using he epesenaion of anks as ineges in {1,...,k}, he anking loss afe T ounds is equal o he cumulaive diffeence beween he pediced ank values and ue ank values, T =1 ŷ y. The algoihm we descibe updaes is anking ule only on ounds on which he pediced ank value was incoec. Such algoihms ae called consevaive. We now descibe he updae ule of he algoihm, which is moivaed by he pecepon algoihm fo classificaion; hence, we call i he PRank algoihm, shohand fo pecepon anking. Fo simpliciy, we omi he index of he ound when efeing o an inpu insance ank pai ( x, y) and he anking ule w and b. Since b 1 b 2 b k 1 b k, he pediced 1 Fo a discussion of his ype of paial odes see Kemeny and Snell (1962).

4 148 K. Camme and Y. Singe ank is coec if w x > b fo = 1,...,y 1and w x < b fo = y,...,k 1. We epesen he above inequaliies by expanding he ank y ino k 1 viual vaiables y 1,...,y k 1. We se y =+1fo he case w x > b and y = 1fo w x < b. Pu anohe way, a ank value y induces he veco (y 1,...,y k 1 ) = (+1,...,+1, 1,..., 1) whee he maximal index fo which y =+1isy 1. Thus, he pedicion of a anking ule is coec if y ( w x b )>0fo all. If he algoihm makes a misake by anking x as ŷ insead of y, hen hee is a leas one heshold, indexed, fo which he value of w x is on he wong side of b, ha is, y ( w x b ) 0. To coec he misake, we need o move he values of w x and b owad each ohe. We do so by modifying only he values of he b s fo which y ( w x b ) 0 and eplace hem wih b y. We also eplace he value of w wih w + ( y ) x whee he sum is aken ove he indices fo which hee was a pedicion eo, ha is, y ( w x b ) 0. An illusaion of he updae ule is given in Figue 1. In he example, we used he se Y ={1,...,5}. (Noe ha b 5 = is omied fom all he plos in Figue 1.) The coec ank of he insance is y = 4, and hus he value of w x should fall in he fouh ineval, beween b 3 and b 4. Howeve, in he illusaion, he value of w x fell below b 1 and he pediced ank is ŷ = 1. The heshold values b 1, b 2, and b 3 ae a souce of he eo since he value of b 1, b 2, b 3 is highe han w x. To compensae fo he misake, he algoihm deceases b 1, b 2, and b 3 by a uni value and eplaces hem wih b 1 1, b 2 1, and b 3 1. I also modifies w o be w + 3 x since y = 3. :y ( w x b ) 0 Thus, he inne poduc w x inceases by 3 x 2. This updae is illusaed a he middle plo of Figue 1. The pedicion ule afe he updae is illusaed on he igh-hand side of Figue 1. Noe ha afe he updae, he pediced ank of x is ŷ = 3, which is close o he ue ank y = 4. The pseudocode of algoihm is given in Figue 2. To conclude his secion, we noe ha PRank can be saighfowadly combined wih Mece kenels (Vapnik, 1998) and voing echniques (Feund & Schapie, 1999) ofen used fo impoving he pefomance of magin classifies in bach and online seings (Cisianini & Shawe-Taylo, 2000). To do so, we assume access o an inne-poduc space X equipped wih a kenel opeao K: X X R. We now need o eplace any innepoduc opeaion he algoihm pefoms wih an implici inne-poduc opeaion defined via he kenel opeao. We hus keep inne-poduc opeaos ahe han explici vecos. Fo insance, he updae of PRank becomes K( w +1, ) K( w, ) + ( τ ) K( x, ),

5 Online Ranking by Pojecing 149 Pediced ank Coec ineval Coec ineval Updaed pediced ank Coec ineval Figue 1: An Illusaion of he updae ule. The anking ule pedics a ank of ŷ = 1 insead of y = 4 (op). The updae deceases he hesholds b 1, b 2, b 3 by one uni and eplaces w wih w + 3 x (cene), yielding ha he pediced ank of x afe he updae is ŷ = 3 (boom). and hus he inne poduc w T x becomes 1 τ K( xs, x ). s=1 3 Analysis Befoe we pove he misake bound of he algoihm, we fis need o show ha i mainains a consisen hypohesis. Tha is, we need o show ha PRank peseves he coec ode of he hesholds; ohewise, i migh be impossible o induce a ank pedicion ule. We pove ha he consisency of hesholds is mainained by showing inducively ha fo any anking ule ha can be deived by he algoihm along is un, ( w 1, b 1 ),...,( w T+1, b T+1 ) he se of inequaliies b b k 1 hold fo all. Clealy, since he iniializaion of he hesholds is such ha b 1 1 b1 2 b1 k 1, hen i suffices

6 150 K. Camme and Y. Singe Iniialize: Se w 1 = 0, b 1 1,...,b1 k 1 = 0, b1 k = Loop: Fo = 1, 2,...,T Receive a new insance x R n Pedic: ŷ = min { : w x b < 0} {1,...,k} Receive a new ank-value y If ŷ y updae w (ohewise se w +1 = w, : b +1 = b ): 1. Fo = 1,...,k 1: Ify Then y = 1 Else y = 1 2. Fo = 1,...,k 1: If( w x b )y 0 Then τ = y Else τ = 0 3. Updae: w +1 w + ( τ ) x Fo = 1,...,k 1: b +1 b τ Oupu: H( x) = min {1,...,k} { : w T+1 x b T+1 < 0}. Figue 2: The PRank algoihm. o show ha he claim holds inducively. Fo simpliciy of he poof below, le us wie he updae ule of PRank in an alenaive fom. Le [[π]] be 1 if he pedicae π holds and 0 ohewise. We now ewie he value of τ (fom Figue 2) as τ = y [[( w x b )y 0]]. Noe also ha he values of b ae ineges fo all and since fo all, we iniialize b1 = 0 and b +1 b { 1, 0, +1}. Lemma 1 (ode pesevaion). Le w and b be he cuen anking ule, whee b 1 b k 1, and le ( x, y ) be an insance ank pai fed o PRank on ound. Denoe by w +1 and b +1 he esuling anking ule afe he updae of PRank. Then b +1 1 b +1 k 1. Poof. In ode o show ha PRank peseves a nondeceasing ode of he hesholds, we use he definiion of he algoihm fo y. We define y =+1 fo < y and y = 1fo y. To pove ha b b+1, we ewie he heshold updae as b +1 = b y [[( w x b )y 0]]. (3.1)

7 Online Ranking by Pojecing 151 In ode o pove ha b +1 ha +1 b+1 fo all feasible, i is sufficien o show b +1 b y +1 [[( w x b +1 )y +1 0]] y [[( w x b )y 0]].(3.2) Since by ou inducive assumpion b +1 b and b, b +1 Z, we ge ha he value of b +1 b on he lef-hand side of equaion 3.2 is a nonnegaive inege. Recall also ha y = 1ify > and y = 1ohewise, and heefoe y +1 y. We now need o analyze wo cases. We fis conside he case whee y +1 y, which implies ha y +1 = 1, y = +1. In his case, he igh-hand side of equaion 3.2 is a mos zeo, and he claim ivially holds. The second case is when y +1 = y. Hee we ge ha he value of he igh-hand side of equaion 3.2 canno exceed 1. If b +1 > b, hen since boh values ae ineges, he bound of 1 on he igh-hand side of equaion 3.2 implies ha he value of b canno exceed he value of b +1. We ae hus lef wih he case whee b = b +1 and y +1 = y. Fo his case, we have ha he ems y +1 [[( w x b +1 )y +1 < 0]] and y [[( w x b )y < 0]] aain he same value. Theefoe, he igh-hand side of equaion 3.2 is zeo and b +1 = b +1. This complees he poof. +1 We now un o he misake-bound analysis of he algoihm. In ode o simplify he analysis of he algoihm, we inoduce he following noaion. Given a hypeplane w and a se of k 1 hesholds b, we denoe by v R n+k 1 he veco ha is a concaenaion of w and b, ha is, v = ( w, b). Fo beviy, we efe o he veco v as a anking ule. Given wo vecos v = ( w, b ) and v = ( w, b), we have v v = w w + b b and v 2 = w 2 + b 2. Noe ha a veco v induces a paial ode of he vecos R n. Theoem 1 (misake bound). Le ( x 1, y 1 ),...,( x T, y T ) be an inpu sequence o PRank whee x R n and y {1,...,k}. Denoe by R 2 = max x 2.If hee exiss a anking ule v = ( w, b ) wih b 1 b k 1 of a uni nom ha classifies he enie sequence coecly wih magin γ = min, {( w x b )y } > 0, he anking loss of he algoihm T =1 ŷ y is a mos (k 1) R2 + 1 γ 2. Poof. Le us examine an example ( x, y ) ha he algoihm eceived on ound. By definiion, he algoihm anked he example using he anking ule v, which is composed of w and he se of hesholds b. Similaly, v +1 is he anking ule ( w +1, b +1 ) afe ound. Theefoe, w +1 = w + ( τ ) x and b +1 = b τ fo = 1, 2,...,k 1. Le us denoe by n = ŷ y he

8 152 K. Camme and Y. Singe diffeence beween he ue ank and he pediced ank. Since τ is zeo fo all he indices such ha sign( w x b )y > 0, hen i is saighfowad o veify ha n = τ. Noe ha if hee was no a anking misake on ound, hen τ = 0 fo = 1,...,k 1, and hus also n = 0. To pove he heoem, we bound n fom above by bounding v 2 fom above and below. Fis, we deive a lowe bound on v 2 by bounding v v +1. Subsiuing he values of w +1 and b +1, we ge ha k 1 v v +1 = v v + τ ( w x b ). (3.3) =1 We fuhe bound he em on he igh-hand side by consideing wo cases coesponding o whehe τ is posiive o zeo. Using he definiion of τ fom he pseudocode in Figue 2, we fis examine he case whee ( w x b )y 0 and heefoe τ = y. Fom he assumpion ha v anks he examples coecly wih a magin of a leas γ, we ge ha τ ( w x b ) γ. The second case is when ( w x b )y > 0. In his case, we have τ = 0 and hus τ ( w x b ) = 0. Combining he wo cases and summing now ove, wege k 1 k 1 τ ( w x b ) τ γ = n γ. (3.4) =1 =1 Combining equaions 3.3 and 3.4, we ge ha v v +1 v v +n γ. Unfolding he sum, we ge ha afe T ounds, he pojecion of he veco v T+1 on v saisfies v v T+1 n γ = γ n. (3.5) Recall ha Cauchy-Schwaz inequaliy implies ha v T+1 2 v 2 ( v T+1 v ) 2. Using equaion 3.5 wih Cauchy-Schwaz inequaliy and he assumpion ha v is of a uni nom, we ge he following lowe bound: ( ) 2 v T+1 2 n γ 2. (3.6) We nex bound he nom of v T+1 fom above. As befoe, assume ha an example ( x, y ) was anked using he anking ule v, and denoe by v +1

9 Online Ranking by Pojecing 153 he anking ule a he end of ound. We now expand he values of w +1 and b +1 whose sum is he nom of v +1 and ge v +1 2 = w 2 + b τ ( w x b ) + ( τ ) 2 x 2 + (τ )2. Since τ { 1, 0, +1}, we have ha ( τ )2 (n ) 2 and (τ )2 = n, and we heefoe ge v +1 2 v τ ( w x b ) + (n ) 2 x 2 + n. (3.7) We fuhe develop he second em using he updae ule of he algoihm and ge ( w x b ) = [[( w x b )y 0]] ( ( w x b ) )y 0. (3.8) τ Plugging equaion 3.8 ino equaion 3.7 and using he bound x 2 R 2, we ge ha v +1 2 v 2 + (n ) 2 R 2 + n. Thus, he anking ule we obain afe T ounds of he algoihm is uppe bounded by v T+1 2 R 2 (n ) 2 + n. (3.9) Combining he lowe bound v T+1 2 ( n ) 2 γ 2 wih he uppe bound of equaion 3.9, we have ha ( ) 2 n γ 2 v T+1 2 R 2 (n ) 2 + n. Dividing he above equaions by γ 2 n, we finally ge n R2 [ (n ) 2 ]/[ n ] + 1 γ 2. (3.10) Since by definiion, n is a mos k 1, i implies ha (n ) 2 n (k 1) = (k 1) n.

10 154 K. Camme and Y. Singe Plugging his inequaliy ino equaion 3.10, we ge he desied bound, T ŷ y = =1 T =1 n (k 1)R2 + 1 γ 2 (k 1) R2 + 1 γ 2. We now un ou aenion o he insepaable case in which hee does no exis a posiive magin value γ. To analyze he insepaable case, we use an analysis echnique suggesed by Feund and Schapie (1999). (This poof echnique was fis infomally given by Vapnik, 1998.) To pove a misake bound fo he insepaable case, each example ( x, y ) is augmened wih a slack vaiable, denoed d. Infomally, fo each ime sep, he vaiable d designaes how much he magin assumpion is violaed by he example. Fomally, we obain he following misake bound fo he insepaable case: Theoem 2 (misake bound fo insepaable case). Le ( x 1, y 1 ),...,( x T, y T ) be an inpu sequence fo PRank whee x R n and y {1,...,k}. Denoe by R 2 = max x 2. Le v = ( w, b ) be a anking ule of a uni nom wih b 1 b k 1. Le γ>0and define d = max{0,γ min{( w x b )y }}. (3.11) Denoe by D 2 = (d ) 2. Then he anking loss of he algoihm is bounded above by T =1 ŷ y (k 1) (D + R 2 + 1) 2 γ 2. The poof of he heoem is based on he poof echnique fo heoem 1 and is given in he appendix. To conclude his secion, we descibe and biefly analyze a simplified vesion of PRank. This vesion shaes he same algoihmic skeleon as PRank and diffes only in he way i modifies is anking ule. Rahe han modifying all of he hesholds in he eo se, his vesion chooses a single heshold o modify and updae w accodingly. Since a single heshold is modified on each ound, we use he abbeviaion Si-PRank o efe o his vesion. Moe fomally, we eplace sep 3 in Figue 2 wih he following seps, 3. Define updae index: If ŷ > y Then = ŷ 1 Else = ŷ 4. Updae: w +1 w + τ x b τ b +1 s b s (s ). b +1

11 Online Ranking by Pojecing 155 I is o veify ha his choice of a single heshold o updae peseves he oveall ode of he heshold. The poof is immediae consequence of he lemma 1. In addiion, following exacly he same poof echnique of heoem 1, we ge ha he misake bound of PRank also holds fo Si-PRank. 2 Supisingly, as we see lae in secion 6, in pacice Si-PRank pefoms slighly bee han PRank. 4 A Nom-Opimized Vesion of PRank The PRank algoihm pesened in he pevious secion pefoms he same fom of updae wheneve a anking eo occus. Fuhemoe, even when he pojecion of x ono w yields he coec ank value, he value of w x migh lie vey close o one of he hesholds b. The diffeence beween he pojecion of x and he closes heshold plays a simila ole o he noion of magin in classificaion poblems and was used in heoem 1 o deive a misake bound fo PRank. In his secion, we pesen a vesion of PRank ha solves on each ound a mini-opimizaion poblem ha balances beween wo opposing equiemens. On one hand, we equie ha he new anking ule v +1 be as simila as possible o he pevious anking ule v, which encompasses all ou knowledge on pas examples. On he ohe hand, we foce he new anking ule o ank he mos ecen example x coecly and wih a lage enough magin. Fomally, assuming ha hee was a ank pedicion eo on ound, hen we equie ha he new ule ( w +1, b +1 ) would saisfy, min {( w +1 x b +1 )y }} β, whee β is a posiive consan. These wo equiemens yield he following opimizaion poblem: min w, b 1 2 ( w, b) ( w, b ) 2 subjec o: ( w x b )y β fo = 1,...,k 1. (4.1) We call his vesion he Nom-Opimized PRank algoihm, o No-PRank in sho. The pseudocode of he algoihm appeas in Figue 3. Befoe analyzing he algoihm, le us fis fuhe develop he opimizaion poblem given in equaion 4.1 by expanding is Lagangian funcion, L = 1 2 ( w, b) ( w, b k 1 ) 2 τ [( w x b )y β]. (4.2) =1 2 In fac, an alenaive bound can be given fo Si-PRank, which saes ha he numbe of ounds on which an eo occus (indicaed by a sicly posiive value of he loss) is uppe bounded by R γ 2.

12 156 K. Camme and Y. Singe Inpu: Minimal magin paamee β Iniialize: Se w 1 = 0, b 1 1,...,b1 k 1 = 0, b1 k = Loop: Fo = 1, 2,...,T Receive a new insance x R n. Pedic: ŷ = min { : w x b < 0} {1,...,k} Receive a new ank-value y If y ŷ updae w and b : (ohewise se w +1 = w, b +1 = b ): 1. Fo = 1,...,k 1: Ify Then y = 1 Else y = Updae: se ( w +1, b +1 ) R n+k 1 o be he minimize of: min w, b 1 2 ( w, b) ( w, b ) 2 subjec o: ( w x b )y β fo = 1,...,k 1. Oupu: H( x) = min {1,...,k} { : w T+1 x b T+1 < 0}. Figue 3: The Nom-Opimized PRank algoihm. Hee, τ ae nonnegaive Lagange muliplies. Taking he deivaive of equaion 4.2 wih espec o w and compaing i o zeo, we ge w L = w w x Repeaing he pocess fo b,wege ( ) τ y = 0 w = w + τ y x. (4.3) b L = b b + τ y = 0 b = b τ y. (4.4) Plugging he value of w fom equaion 4.3 and b fom equaion 4.4 ino equaion 4.2, we ge he following dual poblem: ( ) 2 minq ( τ) = 1 τ 2 x 2 τ y + 1 τ [ ( τ y w x b ) ] β (4.5) s. 0 τ = 1,...,k 1.

13 Online Ranking by Pojecing 157 Befoe poceeding o discuss he fomal popeies of he No-PRank algoihm, i is woh noing ha he new veco obained a he end of ound, w +1 is a linea combinaion of w and he cuen insance x. Theefoe, i is ahe saighfowad o use No-PRank in conjuncion wih kenel mehods by mainaining w in is dual fom as a weighed combinaion of he insances x 1,..., x. Noe also ha he opimizaion poblem of No-PRank educes o a simple opimizaion poblem wih a quadaic objecive funcion and nonnegaiviy consains and can be solved using sandad convex opimizaion ools (Fleche, 1987). Le us now discuss he fomal popeies of No-PRank. The following simple lemma saes ha No-PRank is a consevaive online algoihm as i modifies is anking ule if and only if he minimal magin equiemens ae no aained fo he cuen insance. Lemma 2 (consevaiveness). Le ( x, y ) denoe he inpu o No-PRank on ound, and le y be 1 if y and +1 ohewise. Then if y ( w x b ) β fo all = 1,...,k 1he opimum of No-PRank s opimizaion poblem is aained a τ = 0 fo all. Poof. Conside he dual fom of No-PRank s opimizaion poblem given by equaion 4.5. The objecive funcion Q ( τ) is composed of hee summands. The fis wo ae clealy nonnegaive and aain a value of zeo iff all he muliplies τ ae zeo. If y ( w x b ) β, hen he hid summand is linea in τ wih nonnegaive coefficiens. Thus, is minimum is also aained when τ is zeo fo all. Nex we show ha No-PRank peseves he ode of he hesholds along is un and hus can always seve as a valid anking ule. Lemma 3 (ode pesevaion). Le w and b be he cuen anking ule, whee b 1 b k 1, and le ( x, y ) be an insance-ank pai fed o No-PRank on ound. Denoe by w +1 and b +1 he esuling anking ule afe he updae of No-PRank. Then b +1 1 b +1 k 1. Poof. In ode o show ha No-PRank mainains a coec (monoonically inceasing) ode of he hesholds, we use he vaiables employed by he algoihm along is un. Namely, we se y =+1fo < y and y = 1fo y. To pove ha b b+1 fo all, we expand b +1 and show ha b +1 b y +1 τ +1 y τ. (4.6) We need o analyze wo diffeen seings. The fis seing we analyze is when y +1 y, which implies ha y +1 = 1and y =+1. In his case, he igh-hand side of equaion 4.6 is a mos zeo, while he lef-hand side of

14 158 K. Camme and Y. Singe he equaion is a leas zeo. Thus, he claim holds. The ohe case is when y +1 = y = y. In his case, equaion 4.6 becomes b +1 b y(τ +1 τ ). (4.7) Assume by conadicion ha he opimal value of equaion 4.5 does no saisfy equaion 4.7 fo some. We now consuc anohe feasible se fo τ ha yields a lowe value of he objecive funcion Q. Le us define τ s yɛ s = + 1 τ s = τ s + yɛ s = ohewise, τ s fo some value of ɛ>0, which is deemined below. Infomally, he value of ɛ is se o be small enough so ha he consain τ s 0 sill holds. (This is possible since τ +1 > 0.) Since τ and τ diffe only a hei and + 1 componens, we ge Q( τ ) Q( τ) = 1 2 (τ 2 + τ 2 +1 ) + τ [y ( w x b ) β] + τ +1 [y +1( w x b +1 ) β] 1 2 (τ 2 + τ 2 +1 ) τ [y ( w x b ) β] τ +1 [y +1 ( w x b +1 ) β]. Expanding τ, we now ge Q( τ ) Q( τ) = 1 2 ((τ + yɛ) 2 + (τ +1 yɛ) 2 ) + (τ + yɛ)[y ( w x b ) β] + (τ +1 yɛ)[y +1 ( w x b +1 ) β] 1 2 (τ 2 + τ +1 2 ) τ [y ( w x b ) β] τ +1 [y +1 ( w x b +1 ) β] = (yɛ) 2 + yɛ(τ τ +1 ) + yɛ[y ( w x b ) β] yɛ[y +1 ( w x b +1 ) β]. Denoing boh y and y +1 simply as y and using he fac y { 1, +1}, we ge Q( τ ) Q( τ) = (yɛ) 2 + yɛ(τ τ +1 ) ɛb + ɛb +1 = ɛ[ɛ (y(τ +1 τ ) (b +1 b ))].

15 Online Ranking by Pojecing 159 Since we assumed by conadicion ha y(τ +1 τ ) (b +1 b ) = A > 0, we can choose 0 <ɛ A/2 and ge Q( τ ) Q( τ) = ɛ(a/2 A) = ɛa/2 < 0, which conadics he assumpion ha τ equaion 4.5. is he opimal soluion of We now un o he analysis of he pefomance of No-PRank. We fis bound he sum of he weigh employed by he dual pogam as hey accumulae along he un of No-PRank τ. Based on he bound on he weighs, we hen pove a misake bounds on he numbe of ounds on which an eo occued, ha is, y ŷ. As we see sholy, he bound depends on boh he value of he aainable magin γ and he magin paamee, β, employed by he algoihm. Theoem 3 (bound on weighs). Le ( x 1, y 1 ),...,( x T, y T ) be an inpu sequence o No-PRank whee x R n and y {1,...,k}. Le v = ( w, b ) be a anking ule wih b 1 b k 1 and w 2 + (b )2 = 1. Assume ha v classifies he enie sequence coecly wih a magin value γ = min, {( w x b )y } > 0. Then he oal sum of he weighs geneaed by No-PRank is bounded by T τ 2 β γ 2, =1 whee β is a pedefined paamee of he algoihm (see Figue 3). Poof. Le us concenae on an example ( x, y ) ha he algoihm eceived on ound. By consucion, he algoihm anked he example using he anking ule v, which is composed of w and he hesholds b. Similaly, we denoe by v +1 he updaed ule ( w +1, b +1 ) afe ound, ha is, ( ) w +1 = w + y τ x and b +1 = b y τ fo = 1, 2,...,k 1. To bound τ fom above, we deive bounds on v 2 fom boh above and below. Fis, we deive a lowe bound on v 2 by bounding v v +1. Subsiuing he values of w +1 and b +1, we ge k 1 v v +1 = v v + τ y ( w x b ). =1

16 160 K. Camme and Y. Singe Using he assumpion ha v anks he daa coecly wih a magin of a leas γ, we ge ha y ( w x b ) γ : k 1 v v +1 v v + τ γ = v v + γ =1 k 1 τ. =1 Unfolding he sum, we ge ha afe T ounds, he algoihm saisfies v v T+1 γ, τ. Plugging his esul ino Cauchy-Schwaz inequaliy, v T+1 2 v 2 ( v T+1 v ) 2, and using he assumpion ha v is of a uni nom, we ge he lowe bound, ( ) 2 v T+1 2 τ γ 2. (4.8), We nex bound he nom of v T+1 fom above. As befoe, assume ha an example ( x, y ) was anked using he anking ule v, and denoe by v +1 he anking ule afe he ound. We now expand he values of w +1 and b +1 in he nom of v +1 and ge v +1 2 = w 2 + b τ y ( w x b ) + x 2 ( ) 2 + (y τ )2. τ y We add and subac he em 2β τ on he igh-hand side of he above equaion and ge v +1 2 = w 2 + b τ [y ( w x b ) β] + x 2 ( ) 2 + ( y τ ) 2 + 2β τ y = w 2 + b 2 + 2Q( τ ) + 2β τ, (4.9) τ whee we used he definiion of Q fom equaion 4.5 o obain he las equaliy.

17 Online Ranking by Pojecing 161 Noe ha τ = 0 is a feasible soluion fo he opimizaion poblem posed in equaion 4.5. The value aained by Q fo his paicula choice of τ is zeo. Since τ is he minimize Q( τ), we mus have ha Q( τ ) 0. (4.10) Subsiuing equaion 4.10 in equaion 4.9, we obain he following bound on he nom of v +1 in ems of he nom of v : v +1 2 v 2 + 2β τ. (4.11) Thus, he anking ule we obain afe T ounds of he algoihm saisfies he uppe bound: v T+1 2 2β, τ. (4.12) Combining he lowe bound of equaion 4.8 wih he uppe bound of equaion 4.12, we have ha ( ) 2 γ 2 v T+1 2 2β τ., τ Dividing boh sides by γ 2, τ, we finally ge,, τ 2β γ 2. (4.13) We now discuss some popeies of he algoihm and is coesponding loss bound. As menioned above, he algoihm updaes is anking ule only on ounds on which he magin is less han β, which eflecs a minimal magin equiemen. Infomally, β can be viewed as a minimal diffeence equiemen beween he pojecion of he example x ono w and any of he hesholds b (see Figue 4). Thus, he lage β is, he bee is he sepaaion beween he ank levels. Fomally, he algoihm aemps o enclose all he examples in subinevals of he eal numbes defined by he hesholds. As saed above, based on heoem 3, we can deive a misake bound fo No- PRank. Supisingly, he dependency on he magin paamee β cancels ou, and he end esul is a misake bound ha depends on only he geomeical popeies of he poblem: he nomalized magin. Coollay 1 (misake bound). Assume he condiions of heoem 3 hold, and denoe by R = max x. Then he numbe of ounds fo which No-PRank made

18 162 K. Camme and Y. Singe β W X β b 1 b 2 b 3 b 4 Figue 4: An illusaion of he magin equied by he No-PRank algoihm. a misake is uppe bounded by 2 R2 + 1 γ 2. Poof. To pove he coollay, we use heoem 3 and show ha wheneve an eo occued on ound, he sum of he dual weighs τ is a leas β/(r 2 + 1). Since w +1 mus saisfy he consains of equaion 4.1, we now show ha fo = 1,...,k 1, ( w +1 x b +1 )y β. Subsiuing w +1 = w + x τ y and b+1 fom equaion 4.3 and b +1 = b τ y fom equaion 4.4, we ge ha he dual vaiables τ 1,...,τ k 1 saisfy [( β w + x s τ s y s ) ] x b + τ y y. Reaanging ems in he above equaion, we ge [( β w + x s τ s y s ) ] x b + τ y y = ( w x b )y + y x 2 s τ s y s + τ (y )2. (4.14) Since a ank pedicion eo occued on ound, hee exiss such ha ( w x b )y 0. In addiion, since y { 1, +1}, we have ha y s τ s y s s τ s. Using hese wo inequaliies in equaion 4.14 in conjuncion wih

19 Online Ranking by Pojecing 163 he fac ha τ 0, we ge β 0 + x 2 (R 2 + 1) τ + τ. τ We have hus shown ha if hee was a ank pedicion eo on ound, hen β R τ. (4.15) To pove he coollay, we combine heoem 3 wih equaion Denoe by M he numbe of ounds wih pedicion eos. We now show ha on each such ound, β/(r 2 + 1) τ, and fo he es of he ounds, we simply bound he sum τ fom below by 0. We hus ge ha M β R 2 + 1, τ. Applying heoem 3, we ge M β R β γ 2 M 2 R2 + 1 γ 2. Noe ha he misake bounds of heoem 1 and coollay 1 ae idenical up o a muliplicaive (k 1)/2 faco. Fuhemoe, if we employ he ivial fac ha y ŷ k 1, we can immediaely obain a bound on y ŷ fom coollay 1 ha is a 2 faco of he bound given in heoem 1. Howeve, he bound fo No-PRank is moe efined, as in addiion o he misake bound, we can also bound he oal sum of he weighs, τ. Unfounaely, since τ may be abiaily close o zeo, he moe efined analysis does no yield a bee misake bound. One possible diecion fo impoving he misake bound iself may be obained by modifying he minimal magin equiemen fom β o β y ŷ. Anohe viable appoach is o eplace he fixed magin consain ( w +1 x b +1 )y β wih a magin equiemen ha is dependen on he diffeence beween he coec label y and he specific heshold. Specifically, we define a new se of consains ( w +1 x b +1 )y βa, whee a = y fo < y and a = + 1 y fo y. Theefoe, he fuhe he heshold fom he ineval conaining he pojecion of he insance on he hypeplane, he lage he magin we equie. We leave hese possible exensions o fuue eseach.

20 164 K. Camme and Y. Singe 5 A Muliplicaive Vesion of PRank In his secion, we give a muliplicaive vesion of PRank ha we em Mu- PRank. This vesion is analogous o he basic PRank updae, bu i modifies w and b in a muliplicaive manne. Tha is, on each ound wih ank pedicion eo, he cuen weigh veco w and hesholds b ae muliplied by facos ha depend on x. The moivaion fo his vesion is he muliplicaive updaes employed by he wok of Wamuh and colleagues on online pedicion (see, e.g., Kivinen & Wamuh, 1997). Mu-PRank mainains a nomalized anking ule, ha is, ( w, b ) 1 = 1 fo all. As in PRank, on ound, Mu-PRank compues he veco τ, which deemines he updae of w and b. I hen akes he exponen of he updaed anking ule and nomalizes i so ha he l 1 nom of ( w +1, b +1 ) will be 1. The pseudocode of he algoihm is given in Figue 5. Examining he logaihm of b on each ound, i is ahe simple o ve- Inpu: Leaning ae η Iniialize: Se w 1 i = 1 n+k 1 i = 1,...,n, b 1 1,...,b1 k 1 = 1 n+k 1, b1 k = Loop: Fo = 1, 2,...,T Receive a new insance x R n, x 1 Pedic: ŷ = min { : w x b < 0} {1,...,k} Receive a new ank-value y If y ŷ updae w and b (ohewise se w +1 = w, b +1 = b ): 1. Fo = 1,...,k 1: Ify Then y = 1 Else y = 1 2. Fo = 1,...,k 1: If( w x b )y 0 Then τ = y Else τ = 0 3. Define: n Z = w i eηx i τ k 1 + b e ητ i=1 =1 4. Updae: Fo i = 1,...,n : wi +1 w i eηx i /Z Fo = 1,...,k 1:b +1 b e ητ /Z Oupu: H( x) = min {1,...,k} { : w T+1 x b T+1 < 0} Figue 5: The muliplicaive algoihm.

21 Online Ranking by Pojecing 165 ify ha, like PRank, Mu-PRank peseves he ode of he hesholds. The addiional sep Mu-PRank employs in is updae sage is he nomalizaion of is anking ule. Howeve, since his nomalizaion is applied o all he hesholds, i clealy keeps he ode of he hesholds inac. Moe fomally, log(b ) can be wien as ηu + log(c ), whee u is an inege and C is independen of. Applying exacly he same poof echnique used in lemma 1 o u gives he following coollay: Lemma 4 (ode pesevaion). Le w and b denoe he cuen anking ule and assume ha b 1 b k 1. Then, afe ( x, y ) is fed o Mu-PRank, he new ule ( w +1, b +1 ) peseves he ode of he hesholds, b +1 1 b +1 k 1. Nex, we analyze he misake bound of Mu-PRank. Noe ha Mu-PRank employs a leaning ae paamee, η. The misake bound of Mu-PRank given in he following heoem depends on his value. If an pioi lowe bound on he magin γ is known, we can fix he value of η so as o minimize he loss bound of Mu-PRank. The esuling bound is descibed in coollay 2, which follows heoem 4. Theoem 4 (misake bound). Le ( x 1, y 1 ),...,( x T, y T ) be an inpu sequence o Mu-PRank, whee x R n, x 1, and y {1,...,k}. Assume ha hee exiss a anking ule v = ( w, b ) wih b 1 b k 1 and w, b 1 = 1, which classifies he enie sequence coecly wih magin γ = min, {( w x b )y } > 0. Then he anking loss of Mu-PRank, T =1 ŷ y is, a mos, log ( k + n 1 ) (. log + ηγ ) 2 e η(k 1) +e η(k 1) Poof. As befoe, le v = ( w, b ) and v +1 = ( w +1, b +1 ) denoe he anking ules a he beginning and end of ound, especively. To pove he heoem, we analyze he decease in he Kullback-Leible (KL) divegence (Cove & Thomas, 1991) beween v and v. The KL divegence of wo discee pobabiliy disibuions p, q is D KL ( p q) = i p i log(p i /q i ). To pove he heoem, we examine he change of he KL divegence in wo consecuive ounds: = D KL ( v v +1 ) D KL ( v v ). We bound fom above and below. We fis bound his sum fom above.

22 166 K. Camme and Y. Singe As a eminde, we use n o denoe ŷ y. Using his noaion, we ge = = n i=1 ( w w ) i log i w i n ( w i log i=1 e ηx i [ n k 1 = w i + i=1 Z b =1 k 1 + τ =1 ( b b ) log b ) k 1 ( Z + b ) log =1 =1 e ητ ] log ( Z ) k 1 η τ ( w x b ) k 1 log(z ) ηγ τ =log(z ) ηγ n, (5.1) =1 whee we used he definiion of n and τ in conjuncion wih he fac ha v achieves a magin of γ o obain he inequaliy above. We now bound log(z ). We need he following inequaliy, e ηx a + x 2a eηa + a x 2a e ηa, (5.2) which holds fo a,η > 0 and x [ a, a]. (The poof of his inequaliy is an immediae applicaion of he convexiy of he exponen funcion.) Now ecall ha Z = n i=1 w i eηx i τ k 1 + b e ητ. (5.3) =1 We bound he lef-hand side and he igh-hand side of he above sum sepaaely. Fo he lef-hand side, we bound each em e ηx i τ. Using equaion 5.2 in conjuncion wih he fac ha x i 1and τ k 1, we ge e ηx i τ k 1 + x i 2(k 1) τ Similaly, τ 1 k 1, and hus e η(k 1) + k 1 x i 2(k 1) τ e η(k 1). (5.4) e ητ k 1 + τ 2(k 1) eη(k 1) + k 1 τ 2(k 1) e η(k 1). (5.5) Using he bounds fom equaions 5.4 and 5.5 in 5.3, we ge Z i w i [ k 1 + x i 2(k 1) τ e η(k 1) + k 1 x i 2(k 1) τ e η(k 1) ]

23 Online Ranking by Pojecing b [ k 1 + τ 2(k 1) eη(k 1) + k 1 τ ] 2(k 1) e η(k 1) = i w 1 i 2 (eη(k 1) + e η(k 1) ) + b 1 2 (eη(k 1) + e η(k 1) ) + τ ( w x b ) eη(k 1) + e η(k 1). (5.6) 2(k 1) The definiion of τ implies ha τ ( w x b ) 0. (Eihe hee was a anking eo o τ = 0). In addiion, since Mu-PRank nomalizes is anking ule a he end of each ound, we know ha ( w, b ) 1 = i w i + b = 1. Using hese facs in equaion 5.6, we ge he following bound on Z : Z 1 2 (eη(k 1) + e η(k 1) ). (5.7) Using equaion 5.7 in equaion 5.1, we obain an uppe bound on : [ ] 1 log 2 (eη(k 1) + e η(k 1) ) ηγ n. Since a anking eo occued, we know ha n 1. In addiion, he agumen of he logaihm is a leas 1; we can eaange ems and wie [ ] } 1 n {log 2 (eη(k 1) + e η(k 1) ) ηγ. (5.8) Summing ove, wege { [ 1 ( log e η(k 1) + e η(k 1))] } ηγ n. (5.9) 2 To bound fom below, we unavel he sum and ge = (D KL ( v v +1 ) D KL ( v v )) = D KL ( v v T+1 ) D KL ( v v 1 ) D KL ( v v 1 ), whee he inequaliy is due o he fac ha he KL divegence is always nonnegaive. Using he value of w 1, we ge ha D KL ( v v 1 ) = log(k 1 + n). Combining he lowe bound on wih equaion 5.9, we ge log(k 1 + n) { ( log 2 e η(k 1) + e η(k 1) ) } + ηγ n.

24 168 K. Camme and Y. Singe Reaanging ems, we finally ge he desied bound: n log ( k 1 + n ) (. log + ηγ ) 2 e η(k 1) +e η(k 1) As discussed above, he bound of heoem 4 depends on he leaning ae η.ifγ o a lowe bound on γ is known, we can se η o be ( ) 1 k 1 + γ η = 2(k 1) log. k 1 γ Fo his choice of η, we ge he following coollay: Coollay 2. η = If we un Mu-PRank wih ( ) 1 k 1 + γ 2(k 1) log, k 1 γ hen unde he assumpions of heoem 4, he cumulaive anking loss obained by Mu-PRank is bounded above by n (k 1) 2 log ( n + k 1 ) γ 2. This coollay implies ha he cumulaive anking loss of Mu-PRank is invesely popoional o squae of he magin γ. Noe ha a diec compaison of his bound o he misake bound of PRank (see heoem 1) is no possible as he noion of magin hee is diffeen. (Fo PRank, we used he l 2 nom of he insances and he anking ule o define he magin, while fo Mu- PRank, we acily use he l 1 nom of he anking ule in conjuncion wih he l of he insances.) This elaion beween he bounds also holds beween addiive and muliplicaive online algoihms fo classificaion (Rosenbla, 1958; Lilesone, 1987, 1988, 1989). Puing hese diffeences in he noion of magin aside, he wo bounds exhibi he same quadaic dependency on he magin. 6 Expeimens In his secion, we descibe expeimens we pefomed ha compaed he vaians of PRank discussed in he pevious secion wih wo ohe online leaning algoihms applied o anking: a muliclass genealizaion of he Pecepon algoihm (Camme & Singe, 2001b), denoed MCP, and he Widow-Hoff algoihm (Widow & Hoff, 1960) fo online egession leaning, which we denoe by WH. Fo WH we fixed is leaning ae o a

25 Online Ranking by Pojecing 169 consan value. The hypoheses ha he vaians of PRank and he wo ohe algoihms mainain shae similaiies bu ae diffeen in hei complexiy: PRank mainains a veco w of dimension n and a veco of k 1 modifiable hesholds b, oaling n + k 1 paamees; MCP mainains k pooypes, which ae vecos of dimension n, yielding kn paamees; WH mainains a single veco w of size n. Theefoe, MCP builds he mos complex hypohesis, while WH builds he simples. We descibe wo ses of expeimens wih wo diffeen daa ses. The daa se used in he fis expeimen is synheic and was geneaed in a simila way o he daa se used by Hebich e al. (2000). We fis geneaed poins x = (x 1, x 2 ) unifomly a andom fom he uni squae [0, 1] 2. Each poin was assigned a ank y fom he se {1,...,5} accoding o he following anking ule, y = max { :10((x 1 0.5)(x 2 0.5)) + ξ>b } whee b = (, 1, 0.1, 0.25, 1) and ξ is a nomally disibued noise wih a zeo mean and a sandad deviaion of We geneaed sequences of insance ank pais, each of lengh We fed he sequences o PRank, Si- PRank, MCP, and WH. We hen obained pedicions fo each insance. We conveed he eal-valued pedicions of WH ino anks by ounding each pedicion o is closes ank value. As in Hebich e al. (2000), we used a nonhomogeneous polynomial of degee 2, K( x 1, x 2 ) = (( x 1 x 2 ) + 1) 2 as he inne poduc opeaion beween each inpu insance and he hypeplanes ha he fou addiive algoihms mainain. A each ime sep, we compued fo each algoihm he accumulaive anking loss nomalized by he insananeous sequence lengh. Fomally, he ime-aveaged loss afe T ounds is (1/T) T ŷ y. We compued he loss fo each algoihm we esed fo T = 1,...,8000. To incease he saisical significance of he esuls, we epeaed he pocess 100 imes, picking a new andom insance ank sequence of lengh 8000 each ime, and hen aveaged he insananeous losses acoss he 100 uns. The esuls ae depiced on he lef-hand side of Figue 6. The size of he symbols in he plo is lage han 95% confidence inevals fo each esul depiced in he figue. In his expeimen, he pefomance of MCP is consanly wose han he pefomance of WH and PRank. WH iniially suffes he smalles insananeous loss, bu afe abou 500 ounds, boh PRank and Si-PRank sa o oupefom MCP and WH. Evenually he anking loss ha PRank and Si-PRank suffe is significanly lowe han boh WH and MCP. In he second se of expeimens, we used he EachMovie daa se (Mc- Jones, 1997). This daa se is used fo collaboaive fileing asks and conains aings of movies povided by 61,265 people. Each peson in he daa se viewed a subse of movies fom a collecion of 1623 iles. Each viewe aed each movie ha she saw using one of six possible aings: 0, 0.2, 0.4, 0.6, 0.8, 1. We chose subses of people who viewed a significan numbe of movies, exacing fo evaluaion people who have aed a leas 100 movies. Thee wee 7542 such viewes. We chose a andom one pe-

26 170 K. Camme and Y. Singe PRank Si PRank WH MC Pecepon 0.9 Rank Loss Round Figue 6: Compaison of he ime-aveaged anking loss of PRank, Si-PRank, WH, and MCP on synheic daa. son among hese viewes and se he peson s aings o be he age ank. We used he aings of he ohe viewes as feaues. Thus, he goal is o lean o pedic he ase of a andom use wih he use s pas aings seving as a feedback and he aings of fellow viewes as feaues. The pedicion ule associaes a weigh wih each fellow viewe and heefoe can be seen as leaning coelaions beween he ases of diffeen viewes. Nex, we subaced 0.5 fom each aing; heefoe he possible aing values ae 0.5, 0.3, 0.1, 0.1, 0.3, 0.5. This linea ansfomaion enabled us o assign a value of zeo o movies ha have no been aed. We fed each pai of feaue and ank level in an online fashion. Since we chose viewes who aed a leas 100 movies, we wee able o pefom a leas 100 ounds of online pedicions and updaes. We epeaed his expeimen 500 imes, each ime choosing a andom viewe as he age ank. The esuls ae given in he lef-hand plos of Figue 7. The eo bas in he plo indicae 95% confidence levels. We epeaed he expeimen using viewes who have seen a leas 200 movies. (Thee wee 1802 such viewes.) The esuls of his expeimen ae given on he igh-hand plos of Figue 7. The plos a he op of he figue compae he addiive algoihms: PRank, Si-PRank, MCP, and WH. Along he enie un of he algoihms, PRank and Si-PRank ae significanly

27 Online Ranking by Pojecing PRank Si PRank WH MC Pecepon PRank Si PRank WH MC Pecepon Rank Loss Rank Loss Round Round Rank Loss Si PRank η 0.50 η 1.00 η 4.00 η Rank Loss Si PRank η 0.50 η 1.00 η 4.00 η Round Round Figue 7: Compaison of he ime-aveaged anking loss of he vaians of PRank, WH, and MCP on he EachMovie daa se using viewes who aed a leas 200 movies (op) and a leas 100 movies (boom). bee han WH and consisenly bee han he muliclass pecepon algoihm, alhough he las employs a hypohesis ha is subsanially bigge. Compaing he wo addiive vaians of PRank, we see ha he anking loss of Si-PRank is slighly lowe han ha of PRank. The plos a he boom of he figue compae he bes of he addiive algoihms, Si-PRank, wih he Mu-PRank un wih diffeen leaning aes (η = 0.5, 1, 4, 16). One of he goals of his expeimen is o check he dependency of he pefomance of Mu-PRank on is leaning ae. I is clea fom he wo boom plos of Figue 7 ha he pefomance of Mu-PRank is sensiive o he choice of he leaning ae. Noe, hough, ha he bes leaning ae is poblem dependen: he bes leaning ae is η = 4 fo he fis paiion of EachMovie, while η = 0.5 esuls in he smalles anking loss in he case of he second paiion. This ype of behavio is also exhibied by muliplicaive algoihms in ohe poblems such as binay classificaion (Kivinen & Wamuh, 1997). Noneheless, Mu-PRank oupefoms mos of he ohe algoihms we compaed fo a boad ange

28 172 K. Camme and Y. Singe of leaning aes. Focusing fis on he lef plo, we obseve ha afe abou 60 ounds, Si-PRank achieves he bes pefomance, while a he sa of he aining pocess, is anking loss is almos infeio o mos of he copies of Mu-PRank. Summing up, boh he addiive and muliplicaive vesions of PRank oupefom egession and classificaion algoihms when evaluaed wih espec o he anking loss. While he supeio pefomance of PRank is no supising, i povides an empiical validaion of he fomal analysis pesened in he pevious secions. 7 Conclusion In his aicle, we descibed a family of algoihms fo insance anking. The oos of he algoihms go back he pecepon algoihm (Rosenbla, 1958). One of he majo esuls in he aicle is he descipion of a new appoach fo solving anking poblems. While mos of he pevious appoaches educe he poblem of anking o a classificaion of pais, we pesened an alenaive appoach ha builds a connecion beween ank levels and subinevals of he eals. An open poblem ha aises fom boh he heoeical analysis and he empiical esuls is deciding wha vesion of PRank o use. In paicula, PRank and Si-PRank shae he same misake bound, while in ou expeimens, Si-PRank pefomed bee han PRank. All of he vesions of PRank pesened in his aicle ae online algoihms, and fo hei analysis we used he misake-bound model. An ineesing eseach diecion is he design and analysis of algoihms fo bach seings in which all of he aining examples ae given a once. As menioned in secion 1, Shashua and Levin (2002) have descibed a bach algoihm fo anking. An ineesing quesion is how he diffeen vaians elae o he algoihm of Shashua and Levin. In addiion o coninuing ou eseach on online algoihms fo anking poblems, an ineesing eseach diecion is he design and genealizaion analysis of bach algoihms fo vaious anking losses. Appendix: Technical Poofs A.1 Poof of Theoem 2. To pove he heoem, we inoduce a sligh modificaion of heoem 1 by elaxing he assumpion ha v is of a uni nom and adding he nom of v o he misake bound of he basic PRank algoihm. We hus wie he misake bound of heoem 1 as T =1 ŷ y (k 1) (R2 + 1) v 2 γ 2. Since he case D = 0 educes o seing of heoem 1, we can assume ha D > 0. We pove he heoem by ansfoming he insepaable poblem ino a sepaable one. We do so by expanding each oiginal insance x R n ino

29 Online Ranking by Pojecing 173 a veco z R n+t as follows. The fis n coodinaes of z ae se o x. The n+ coodinae of z is se o, which is a posiive eal numbe whose value is se below. The es of he coodinaes of z ae se o zeo. We similaly exend he anking ule ( w, b ) o (ū, c ) R (n+t) R k 1 as follows. We ewie he value of d fom equaion 3.11 as { } d = max γ min {( w x b y )y }, 0,γ min {( w x b <y )y }, (A.1) and define s o be he following indicao funcion: 1 d = γ min y {( w x b s )y } = 0 d = 0. (A.2) +1 d = γ min <y {( w x b )y } Noe ha if s = 1, hen d = ( w x b )y fo some y, and hus y = 1. Similaly, if s =+1, hen d = ( w x b )y fo some < y and y =+1. We se he fis n columns ū o be w, and he n + coodinae of ū is se o be s d and he es of he coodinaes of ū ae se o zeo. We now show ha (ū, c ) achieves a magin value γ on he expanded sequence. Using he definiion of s and d, we ge (ū z c )y = ( w x + s d Noe ha by consucion, z 2 R b )y = ( w x b )y + s d y = ( w x b )y + d Since he nom of ( w, b ) is 1, we have ( w x b )y + γ ( w x b )y = γ. (A.3) ū 2 + c 2 = w 2 + b 2 + ( s d ) 2 = 1 + D2 2. We now apply he bound of heoem 1 in he fom discussed above and ge ha T =1 ŷ y (k 1) (R )(1 + D2 2 ) γ 2. (A.4) Seing 2 = D R in he equaion above yields he desied bound.

30 174 K. Camme and Y. Singe I emains o show ha he pedicions of he algoihm on each elemen of he oiginal sequence and on he expanded sequences ae idenical. This pa follows exacly he same line of poof used in heoem 1 fom Feund and Schapie (1998) and is hus omied. Acknowledgmens Thanks o Sanjoy Dasgupa and Rob Schapie fo numeous discussions on anking poblems and algoihms. Thanks also o Eleaza Eskin and Ui Maoz fo caefully eading he manuscip. This wok was suppoed in pa by he IST Pogamme of he Euopean Communiy, unde he PASCAL Newok of Excellence, IST This publicaion eflecs only he auhos views. Refeences Cohen, W., Schapie, R. E., & Singe, Y. (1999). Leaning o ode hings. Jounal of Aificial Inelligence Reseach, 10, Cove, T. M., & Thomas, J. A. (1991). Elemens of infomaion heoy. New Yok: Wiley. Camme, K., & Singe, Y. (2001a). Panking wih anking. In T. G. Dieeich, S. Becke, & Z. Ghahamani (Eds.), Advances in neual infomaion pocessing sysems, 14. Cambidge, MA: MIT Pess. Camme, K., & Singe, Y. (2001b). Ulaconsevaive online algoihms fo muliclass poblems. In Poceedings of he Foueenh Annual Confeence on Compuaional Leaning Theoy. Amsedam: Spinge. Cisianini, N., & Shawe-Taylo, J. (2000). An inoducion o suppo veco machines. Cambidge: Cambidge Univesiy Pess. Fleche, R. (1987). Pacical mehods of opimizaion (2nd ed.). New Yok: Wiley. Feund, Y., Iye, R., Schapie, R. E., & Singe, Y. (2003). An efficien boosing algoihm fo combining pefeences. Jounal of Machine Leaning Reseach, 4, Feund, Y., & Schapie, R. E. (1999). Lage magin classificaion using he pecepon algoihm. Machine Leaning, 37, Haingon, E. F. (2003). Online anking/collaboaive fileing using he pecepon algoihm. In Poceedings of he Twenieh Inenaional Confeence on Machine Leaning. Washingon, DC: AIII Pess. Hebich, R., Gaepel, T., & Obemaye, K. (2000). Lage maging ank boundaies fo odinal egession. In A. Smola, B. Schölkopf, & D. Schuumans (Eds.), Advances in lage magin classifies. Cambidge, MA: MIT Pess. Kemeny, J. G., & Snell, J. L. (1962). Mahemaical models in he social sciences. Cambidge, MA: MIT Pess. Kivinen, J., & Wamuh, M. K. (1997). Exponeniaed gadien vesus gadien descen fo linea pedicos. Infomaion and Compuaion, 132(1), Lilesone, N. (1987). Leaning when ielevan aibues abound. In 28h Annual Symposium on Foundaions of Compue Science (pp ). Los Angeles: IEEE.