Convex Games in Banach Spaces

Transcription

1 Convex Games n Banach Saces Karhk Srdharan TTI-Chcago karhk@c.edu Ambuj Tewar TTI-Chcago ewar@c.edu Absrac We sudy he regre of an onlne learner layng a mul-round game n a Banach sace B agans an adversary ha lays a convex funcon a each round. We characerze he mnmax regre when he adversary lays lnear funcons n erms of he Marngale ye of he dual of B. The cases when he adversary lays bounded and unformly convex funcons resecvely are also consdered. Our resuls connec onlne convex learnng o he sudy of he geomery of Banach saces. We also show ha arorae modfcaons of he Mrror Descen algorhm from convex omzaon can be used o acheve our regre uer bounds. Fnally, we rovde a verson of Mrror Descen ha adas o he changng exonen of unform convexy of he adversary s funcons. Ths adave mrror descen sraegy rovdes new algorhms even for he more famlar Hlber sace case where he loss funcons on each round have varyng exonens of unform convexy curvaure. 1 Inroducon Onlne convex omzaon [1, 2, 3 has emerged as an absracon ha allows a unfed reamen of a varey of onlne learnng roblems where he underlyng loss funcon s convex. In hs absracon, a T -round game s layed beween he learner or he layer and an adversary. A each round {1,..., T }, he layer makes a move w n some se W. In he learnng conex, he se W wll reresen some hyohess sace. Once he layer has made hs choce, he adversary hen cks a convex funcon l from some se F and he layer suffers loss l w. In he learnng conex, he adversary s move l encodes he daa seen a me and he loss funcon used o measure he erformance of w on ha daa. As wh any absracon, on one hand, we lose conac wh he concree deals of he roblem a hand, bu on he oher hand, we gan he ably o sudy relaed roblems from a unfed on of vew. An added benef of hs absracon s ha connecs onlne learnng wh geomery of convex ses, heory of omzaon and game heory. An moran noon n he onlne seng s ha of he cumulave regre ncurred by he layer whch s he dfference beween he cumulave loss of he layer and he cumulave loss for he bes fxed move n hndsgh. The goal of regre mnmzng algorhms s o conrol he growh rae of he regre as a funcon of T. There has been a huge amoun of work characerzng he bes regre raes ossble under a varey of assumons on he layer s and adversary s ses W and F. Wh a few exceons ha we menon laer, mos of he work has been n he seng where hese ses lve n some Eucldean sace R d. Whenever he resuls do no exlcly nvolve he dmensonaly d, hey are also usually alcable n any Hlber sace H. There also has been a lo of work dealng wh l saces for [1, 2. Bu here, he fac exloed s ha a srongly convex funcon, wh dmenson-ndeenden consan of srong convexy s avalable n hese Banach saces. There has been less work dealng wh arbrary Banach saces where srongly convex funcons mgh no exs. Our focus n hs aer s o exend he sudy of omal regre raes o he case when he se W lves n a general Banach sace B. Before we exlan wha our secfc conrbuons are, le us brefly menon wo examles o show why one mgh wan o move beyond Hlber saces and consder general Banach saces. A frs famly of examles s l saces wh 2. As grows, he l balls become larger and hus have beer aroxmaon roeres. On he oher hand, as we show below, he cumulave regre rae when comeng agans a fxed sze ball n l s OT 1/. So, here s a rade-off here beween

2 aroxmaon roeres and he regre rae or he esmaon error n a sochasc seng. In hgh-dmensons, one can easly consruc examles where he aroxmaon roery domnaes he rade-off and s advanageous o use > 2 even hough he esmaon error suffers see aendx for a worked ou examle. Anoher examle s redcon wh suared loss where he learner s ryng o redc a sgnal y gven nu x. A each se, he learner chooses a funcon f and suffers he loss y f x 2. Here, he vewon of consderng f as a on n a funcon sace s very fruful and very naural o assume ha he sace of funcons ha he learner can use s a Banach sace of funcons. For more deals, see [4. In he Hlber sace seng, s known ha he degree of convexy or curvaure of he funcons l layed by he adversary has a sgnfcan mac on he achevable regre raes. For examle, f he adversary can lay arbrary convex and Lschz funcons, he bes regre ossble s O T. However, f he adversary s consraned o lay srongly convex and Lschz funcons, he regre can be brough down o Olog T. Furher, s also known, va mnmax lower bounds [5, ha hese are he bes ossble raes n hese suaons. In a general Banach sace, srongly convex funcons mgh no even exs. We wll, herefore, need a generalzaon of srong convexy called -unform convexy srong convexy s 2-unform convexy. There wll, n general, be a number [2, such ha -unformly convex funcons are he mos curved funcons avalable on B. There are, agan, wo exremes: he adversary can lay eher arbrary convex-lschz funcons or -unformly convex funcons. We show ha he mnmax omal raes n hese wo suaons are of he order Θ T 1/ and Θ T 2 resecvely 1 where s he Marngale ye of he dual B of B. A Hlber sace has = = 2. We also gve uer and lower bounds for he nermedae case when he adversary layes -unformly convex funcons for >. Ths case, as far as we know, has no been analyzed even n he Hlber sace seng. Anoher naural game ha we have no seen analyzed before s he convex-bounded game: here he adversary lays convex and bounded funcons. Of course, beng Lschz on a bounded doman mles boundedness bu he reverse mlcaon s false: a bounded funcon can have arbrarly bad Lschz consan. For he convex-bounded game, we do no have a gh characerzaon bu we can gve non-rval uer bounds. However, hese uer bounds suffce o rove, for examle, ha he followng hree roeres of B are euvalen: 1 he convex-bounded game when he layer lays n he un ball of B has non-rval.e. ot mnmax regre; 2 he corresondng convex-lschz game has non-rval mnmax regre; and 3 he Banach sace B s suer-reflexve. We furher descrbe layer sraeges ha acheve he omal raes for hese convex games. These sraeges are all based on he Mrror Descen algorhm ha orgnaed n he convex omzaon leraure [6. Usually Mrror Descen s run wh a srongly convex funcon bu urns ou ha can also be analyzed n our Banach sace seng f s run wh a -unformly convex funcon Ψ. Moreover, wh he correc choce of Ψ, acheves all he uer bounds resened n hs aer. Thus, ar of our conrbuon s also o show he remarkable roeres of he Mrror Descen algorhm. Our fnal conrbuon s an adave algorhm, buldng on revous work, ha adas o he exonen of unform convexy n he adversary s funcons. Our resuls have novel mlcaons even n a Hlber sace. For examle, [7 showed how o ada o an adversary ha mxes lnear and srong convex funcons n s moves. We can now allow hs mx o also conss of funcons wh nermedae degrees of unform convexy. Relaed work The dea of exlong mnmax-maxmn dualy o analyze omal regre raes also aears n he recen work of Abernehy e al. [8. The earles aers we know of ha exlore he connecon of he ye of a Banach sace o learnng heory are hose of Donahue e al. [9 and Gurvs [10. Mendelson and Schechman [11 gave esmaes of he fa-shaerng dmenson of lnear funconals on a Banach sace n erms of s ye. In he conex of onlne regresson wh suared loss, Vovk [4 also gves raes worse han O T when he class of funcons one s comeng agans s no n a Hlber sace, bu n some Banach sace. He also menons Banach Learnng as an oen roblem n hs onlne redcon wk 2. For recen work exlorng Banach saces for learnng alcaons, see [12, 13, 14. These aers also gve more reasons for consderng general Banach saces n Learnng Theory. Oulne The res of he aer s organzed as follows. In Secon 2, we formally defne he mnmax and maxmn values of he game beween a layer and an adversary. We also nroduce he noons of Marngale ye and unform convexy from funconal analyss. Secon 3 consders convex- Lschz and lnear games. A key resul n ha secon s Theorem 4 whch gves a characerzaon 1 Our nformal Θ noaon hdes facors ha are ot ɛ for every ɛ > 0. 2 h://onlneredcon.ne/?n=oen.banachlearnng

3 of he mnmax regre of hese games n erms of he ye. Convex-bounded games are reaed n Secon 4 and he euvalence of suer-reflexvy of he sace o he exsence of layer sraeges achevng non-rval regre guaranees s esablshed Corollary 6. We nex consder he case when he adversary lays curved funcons n Secon 5. Here he regre deends on he exonen of unform convexy of he funcons layed by he adversary Theorem 8. In Secon 6, we descrbe layer sraeges based on he Mrror Descen algorhm ha acheve he uer bounds resened n Secon 3 and Secon 4. In Secon 7, usng he echnues of Barle e al. [7, we gve a layer sraegy ha adas o he exonen of unform convexy of he funcons beng layed by he adversary. We conclude n Secon 8 by exlorng drecons for fuure work ncludng a dscusson on how he deas n hs aer mgh lead o raccal algorhms. All roofs omed from he man body of he aer can be found n he aendx. 2 Prelmnares 2.1 Regre and Mnmax Value Our rmary objecs of sudy are ceran T -round games where he layer P makes moves n convex se W conaned n some real searable Banach sace B. The adverary A lays bounded connuous convex funcons on W chosen from a fxed funcon class F. oology nduced by he norm l := su lw. The game roceeds as follows. For = 1 o T P lays w W, A lays l F, P suffers l w. For gven seuences w 1:T, l 1:T, we defne he regre of P as, Regw 1:T, l 1:T := l w nf l w. Gven he ule T, B, W, F, we can defne he mnmax value of he above game as follows. Defnon 1. Gven T 1 and B, W, F sasfyng he condons above, defne he mnmax value, V T,B W, F := nf su nf su Regw 1:T, l 1:T. w 1 W l 1 F w T W l T F When T and B are clear from conex, we wll smly denoe he mnmax value by V W, F. A layer sraegy or P-sraegy W s a seuence W 1,..., W T of funcons such ha W : F 1 W. For a sraegy W, we defne he regre as, RegW, l 1:T := l W l 1: 1 nf In erms of layer sraeges, he mnmax value akes a smler form, l w. V T,B W, F = nf W su l 1:T RegW, l 1:T, where he suremum s over all seuences l 1:T F T. Le Q denoe dsrbuons over F T. We can defne he maxmn value, One has ha U T,B W, F := su nf E l 1:T Q [RegW, l 1:T. Q W V T,B W, F U T,B W, F. 1 Ths neualy wll be he sarng on for our subseuen analyss. For more abou he mnmax and maxmn values refer, [15, Marngale ye and Unform Convexy One of he goals of hs aer s o characerze he mnmax value n erms of he geomerc roeres of he sace B and degree of convexy nheren n he funcons n F. Among he geomerc characerscs of a Banach sace B, he mos useful for us s he noon of Marngale

4 ye or M-ye of B. A Banach sace B has M-ye f here s some consan C such ha for any T 1 and marngale dfference seuence d 1,..., d T wh values n B, [ T 1/ E d C E [ d. 2 We also defne he bes M-ye ossble for a Banach sace, B := su { : B has M-ye }. 3 A Banach sace B has M-coye f here s some consan C such ha for any T 1 and marngale dfference seuence d 1,..., d T wh values n B, T 1/ [ E [ d C E d. 4 A closely relaed noon n Banach sace heory s ha of suer-reflexvy. deals. Refer [16 for more Defnon 2. A Banach sace B s suer-reflexve f no non-reflexve sace s fnely reresenable n B. A resul of Pser [16 shows ha a Banach sace B has non-rval M-ye > 1 or euvalenly non-rval Marngale co-ye f and only f s suer-reflexve. To measure he degree of convexy of he funcons layed by he adversary, we need he noon of unform convexy. Le be he norm assocaed wh a Banach sace B. A funcon l : B R s sad o be C, -unformly convex on B f here s some consan C > 0 such ha, for any v 1, v 2 B and any θ [0, 1, lθv θv 2 θlv θlv 2 If C 1 we smly say ha he funcon l s -unformly convex. Cθ1 θ v 1 v 2. The followng remarkable heorem of Pser [17 shows ha he conce of M-yes and exsence of unformly convex funcons n he Banach sace are nmaely conneced. Theorem Pser. A Banach sace B has M-coye ff here exss a -unformly convex funcon on B. Now consder some [1, M B. Then, by defnon, B has M-ye. I s a fac ha B has M-ye ff B has M-coye 1 [16, Chaer 6. Thus, B has M-coye 1. Now, Pser s heorem guaranees he exsence of a 1-unformly convex funcon on B. For a convex funcon l : B R, s subdfferenal a a on v s defned as, lv = { B : v, lv lv + v v}, where B denoes he dual sace of B. Ths consss of all connuous lnear funcons on B wh norm defned as l := su w: w 1 lw. If lv s a sngleon hen we say l s dfferenable a v and denoe he unue member of lv by lv. If l s dfferenable a v 1, defne he Bregman dvergence assocaed wh l as, l v 1, v 2 = lv 1 lv 2 lv 2 v 1 v 2. Recall ha a funcon l : B R s L-Lschz on some se W B f for any v 1, v 2 W, we have lv 1 lv 2 L v 1 v 2. Gven a se W n a Banach sace B, we defne he followng naural ses of convex funcons on W, lnw := {l : l s lnear and 1-Lschz on W}, cvxw := {l : l s convex and 1-Lschz on W}, bddw := {l : l s convex and bounded by 1 on W}, cvx,l W = {l : l s -unformly convex and L-Lschz on W}. In he followng secons, we wll analyze he mnmax value V W, F when he adversary s se of moves s one hese 4 ses defned above. For readably, we wll dro he deendence of hese ses on W when s clear from conex. For examle, we wll refer o V W, cvxw smly as V W, cvx.

5 3 Convex-Lschz and Lnear Games Gven a Banach sace B wh a norm, denoe s un ball by U B := {v B : v 1}. Consder he case when he P s se W s he un ball U B for some B. Ths seng s no as resrcve as sounds snce any bounded symmerc convex se K n a vecor sace V gves a Banach sace B = V, K, where we eu V wh he norm, v K := nf {α > 0 : v αk}. 5 Moreover, he closure of K s he un ball of hs Banach sace. So, fx B and consder he case W = U B, F = cvxw. Theorem 14 gven n [5 gves us V W, cvx = V W, ln. We are herefore led o consder he case W = U B, F = lnw. Noe ha lnw s smly he un ball U B. The heorem below relaes he mnmax value V U B, ln o he behavour of marngale dfference seuences n B. Theorem 3. The mnmax value V U B, ln of he lnear game s bounded as, [ V U B, ln su E l M. where he suremum s over dsrbuons M of marngale dfference l T such ha each l U B. Proof. Recall ha we denoe a general dsrbuon over A s seuences by Q and P-sraeges by W. Euaon 1 gves us, V W, F su nf E l 1:T Q [RegW, l 1:T. Q W If we defne V Q := nf W E l1:t Q [RegW, l 1:T we can succncly wre, V W, F = su Q V Q. Now, le us fx a dsrbuon Q and denoe he condonal execaon w.r.. l 1: by E [ and he full execaon w.r.. l 1:T by E [. Subsung he defnon of regre and nong ha he nfmum n s defnon does no deend on he sraegy W, we ge [ T [ V Q nf E l W l 1: 1 E nf l w. 6 W Le us smlfy he nfmum over P-sraeges as follows, [ T nf E l W l 1: 1 = nf E [E 1 [l W l 1: 1 = W W = [ E nf E 1 [l w. w W Subsung hs no 6, we ge, [ [ V Q E nf E 1 [l w E w W = E [ su l w nf nf E [E 1 [l W l 1: 1 W l w [ E su w W E 1 [ l w Snce he losses l are lnear and W s he un ball, we can re-wre he above as [ V Q E l w E [ E 1 [ l w. 7 If we resrc ourselves o only dsrbuon Q such ha l 1,..., l T are marngale dfference seuences hen clearly E 1 [ l w = 0 and so [ su V Q su E l Q M w and so we ge he lower bound.

6 Gven he above resul, we can now characerze he mnmax value V U B, ln n erms of he B where B s he dual sace of B. Theorem 4. For all, such ha < B <, here exss a consan C such ha, Ω T 1/ = V U B, ln = V U B, cvx CT 1/. 8 If he suremum n 3 s acheved, he uer bound also holds for = B. Proof. To rove he lower bound, noe ha for any fne dmensonal Banach sace has B = 2 wh a ossbly dmenson deenden consan. In hs case, he lower bound of T for he lnear game game s easy: ck some non-zero vecor, say l U B, and use l or l a random n Theorem 3 and he lower bound follows. Therefore, assume B s nfne dmensonal. The lower bound n hs case s roved usng Lemma 12 whch n urn s roved usng deas from [16. Lemma 12 shows ha for any > B, [ T su M E d However we have from Theorem 3 ha T 1/ V U B, ln su M [ E d Hence we can conclude ha for any > asymocally T 1/ s domnaed by V U B, ln and hence he lower bound. As for he uer bounds, f B = 1 hen he uer bound s rval. On he oher hand, when M-ye s non-rval, hen M-ye mles M-coye = 1. Therefore, for each 1, B, B s of M-coye. By Theorem Pser, here exss a -unformly convex funcon on B. Usng hs funcon n he Mrror Descen algorhm, Prooson 9 yelds he reured uer bound. Alhough we have saed he above heorem for he secal case when W = U B and F = lnu B, acually gves us regre raes when P lays n r U B and A lays L-Lschz lnear funcons va he followng eualy whch s easy o rove from frs rncles, V r U B, L lnu B = r L V U B, lnu B. 9 For he secal case when B, B = l, l for 1/ + 1/ = 1, he raes gven by he heorem above become ΘT 1/ and ΘT 1/2 when 1, 2 and [2, resecvely. 4 Convex-Bounded Games Anoher naural game we consder s one n whch P lays from some convex se W and A lays some convex funcon bounded by 1. In he followng heorem, we bound he value of such a game. Theorem 5. For all, such ha < B <, here s a consan C such ha, Ω T 1/ V U B, bdd CT 1/+1/2. 10 where = 1. If he suremum n 3 s acheved, he uer bound also holds for = B. Proof of Theorem 5. Le us acually consder he case W = r U B and F = bddr U B. The bounds wll urn ou o be ndeenden of r. Noe ha we have he ncluson, bddr U B 1 r lnu B whch mles V r U B, bddr U B V r U B, 1r lnu B. The lower bound s now mmedae due o lower bound on lnear game on un ball Theorem 4 and roery 9. For he uer bound, noe ha any convex funcon bounded by 1 on he scaled ball r U B s 2 ɛr Lschz on he ball of radus 1 ɛr [18. Hence, by uer bound n Theorem 4 and roery

7 9, we see ha here exss a sraegy say W whose regre on he ball of radus r1 ɛ s bounded by C ɛ T 1 for any [1, B. Tha s l W argmn w r1 ɛub l w C ɛ T 1/, [1, B 11 Le w T = argmn l w. Now we consder wo cases, frs when w 1 ɛr U B In hs w r UB case he regre of he sraegy on he un ball s bounded by C ɛ T 1/ for all [1, B. On he oher hand f w / 1 ɛr U B, hen defne wɛ = r1 ɛw w. Noe ha wɛ r1 ɛu B. In hs case by convexy of T l w, we have ha Hence, we have ha l wɛ l wɛ l w r1 ɛ w l w + 1 T r1 ɛ w 1 l w + 1 r1 ɛ T w l 0 r1 ɛ w T 2 1 r1 ɛ w T However, snce w r we see ha T l wɛ T l w 2ɛT Combnng wh 11 we see ha for any [1, B, T l w T l w C ɛ T 1/ + 2ɛT Choosng ɛ = we ge he reured uer bound. C 2T 1/ Alhough we have saed he above resul for he un ball, he roof gven above shows ha he bounds are ndeenden of he radus of he ball n whch he layer s layng. Theorems 4 and 5 mly he followng neresng corollary. Corollary 6. The followng saemens are euvalen : 1. V U B, bdd = ot. 3. B has non-rval M-ye 2. V U B, cvx = ot. 4. Boh B and B are suer-reflexve. Proof of Corollary 6. The mlcaons 3 1 and 3 2 follow from he uer bounds n Theorems 4 and 5. The reverse mlcaons 1 3 and 2 3, n urn, follow from he lower bounds n hose heorems. The euvalence of 3 and 4 s due o dee resuls of Pser [19. The convex-lschz games and -unformly convex-lschz games consdered below deend, by defnon, no only on he layer s se W bu also on he norm of he underlyng Banach sace B. Ths s because A s funcons are reured o be Lschz w.r... However, noe ha he convex-bounded game can be defned only n erms of he layer se W. Hence, one would exec he value of he game o be characerzed solely by roeres of se W. Ths s wha he followng corollary confrms. Corollary 7. Le W be any symmerc bounded convex subse of a vecor sace V. The value of he bounded convex game on W s non-rval.e. ot ff he Banach sace V, W where W s defned as n 5 s suer-reflexve. 5 Unformly Convex-Lschz Games For any Hlber sace H, s known ha V U H, cvx 2,L s much smaller han V U H, ln,.e. he game s much easer for P f A lays 2-unformly convex also called srongly convex funcons. In fac, s known ha V U H, cvx 2,L = ΘL 2 log T whle V U H, ln = Θ T. Ths suggess ha we should ge a rae beween logt and T f A lays -unformly convex funcons n a Hlber sace H for some > 2. As far as we know, here s no characerzaon of he achevable raes for hese nermedae suaons even for Hlber saces. Our nex resul rovdes uer and lower bounds for V U B, cvx,l n a Banach sace, when he exonen of A s unform convexy les n an nermedae range beween s mnmum ossble value and s maxmum value. I s easy o see ha he mnmum ossble value s B / B 1.

8 Theorem 8. Le = B B 1 and,. Le = / 1 be he dual exonen of. Then, as long as cvx,l s non-emy, here exss K ha deends on L such ha for all > B, Ω L T 1 + V U B, cvx,l KT mn{2,1/ B }. 12 Proof. We sar by rovng he lower bound. To hs end noe ha f cvx,l s non-emy, hen he adversary lays L-Lschz, -unformly convex loss funcons. Noe ha gven such a funcon, here exss a norm such ha L e. an euvalen norm and 1 s a -unformly convex funcon [20. Gven hs we consder a game where adversary lays only funcons from lncvx,l W := {lw = w, x + 1 w : x L 1} Noe ha snce he above s L-Lschz w.r.., s auomacally L-Lchz w.r... Hence lncvx,l cvx,l, and so we have ha V U B, lncvx,l V U B, cvx,l However noe ha Also noe ha RegW, l 1:T = = = V U B, lncvx,l nf su E l1:t P RegW, l 1:T 13 W P x, w + w nf x, w + w w UB x, w + w + T su 1 x, w w w UB T x, w + w + T su 1 x, w w L w UB T x, w + w T 1 T T x + L Where he las se s by defnon of convex dual of. Now noe ha snce we have a suremum over dsrbuon n 13, and so we can lower bound he value by ckng dsrbuon such ha d 1,..., d T are marngale dfference seuences and d L 1U B. Thus we see ha V U B, lncvx,l su M = su M = su M nf W E nf W E L d, w + w L [ T w + L [ T 1 T E d + [ T 1 T E T 1 T 1 T d d [ T su M E d where he frs eualy s because w s only deenden on he hsory and so he condonal execaons over d are 0 and he las se s due o Jensen s neualy. Now noe ha usng Lemma 12 we see ha for any > we have ha V U B, lncvx,l = Ω 1 T L 15 he 1 1/L erm above comes from he fac ha he marngale dfferences come from ball of radus 1 1/L whle Lemma 12 s over un ball. Noe ha he above lower bound becomes 0 when L = 1 bu however n ha case means ha he adversary s forced o lay 1-Lschz, -unformly convex funcon. However snce from each -unformly convex L-Lschz convex funcon we can 14

9 buld an euvalen norm wh dsoron 1/L, hs means ha he funcon he adversary lays can be used o consruc he orgnal norm self. From he consrucon n [20 becomes clear ha he funcons he adversary can lay can be merely he norm lus some consan and so he lower bound of 0 s real. Now we urn o rovng he uer bound. Consder he regre of he mrror descen algorhm when we run usng a -unformly convex funcon Ψ ha s C-Lschz on he un ball. Here, for smlcy, we assume ha he suremum s acheved n 3 oherwse we can ck a Ψ ha s -unformly convex for = / 1 where = B 1/ log T and ay a consan facor. Noe ha n he case when > + 1, we have ha each σ = 0 and so by Theorem 10, Regw 1:T, l 1:T 2 mn 1,..., T L + C + 2 T 1 j j C 2 mn L + C T T C L+C Usng = T 1/ 2C we see ha Regw 1/ 1:T, l 1:T 82C 1/ L + CT 1/ On he oher hand when + 1, usng he uer bound n he heorem wh = 0 for all we see ha snce all = and all σ = 1 and L = L we fnd ha he regre of he adave algorhm s bounded as 2L + C 2L + C 4L + C T Regw 1:T, l 1:T + 4L + C 1 d Hence we see ha for < B 2, Regw 1:T, l 1:T 4L+C 2 T 2. Snce he regre of he adave algorhm bounds he value of he game, we see ha by ckng consan K = max{ 4L+C 2, 82C 1/ L + C} we ge he reured uer bound. The uer and lower bounds do no mach n general and s an neresng oen roblem o remove hs ga. Noe ha he uer and lower bounds do mach for he wo exreme cases and. When, hen boh lower and uer bound exonens end o 2 B. On he oher hand, when, boh exonens end o 1/ B. 6 Sraegy for he Player In hs secon, we rovde a sraegy known as Mrror Descen and s gven as Algorhm 1 below whch uses unformly convex funcon Ψ as nernal regularzer and s guaraneed o acheve low regre. Algorhm 1 Mrror Descen Parameers : η > 0, Ψ : B R whch s unformly convex for = 1 o T do Play w and receve l w +1 Ψ Ψw η where l w Udae w +1 argmn Ψ w, w +1 end for Examle : As a more concree examle for he above sraegy, f we consder he case where he layer lays from he un ball of a d dmensonal l sace. In hs case Ψw = 1 w where = whenever > 2 and = 2 oherwse. The corresondng dual hen s Ψ x = 1 x where = f > 2 and = 2 oherwse. Corresondngly we ge Ψw = w sgnw 1 w 1 1,..., sgnw d w d 1 Ψ x = x sgnx 1 x 1 1,..., sgnx d x d 1 We can use hs o udae w +1 and ge a concree algorhm from he above sraegy. The followng rooson gves a regre bound for Mrror Descen. Prooson 9. Suose W B s such ha w B. Le MD denoe he P-sraegy obaned by runnng Mrror Descen wh a funcon Ψ ha s -unformly convex on B and C-Lschz on W, and he learnng rae η = BC/T 1/ 1/L. Here, = / 1 s he dual exonen of. Then, for all seuences l 1:T such ha l s L-Lschz on W, we have, RegMD, l 1:T = O BC 1/ L T 1/ and 1

10 Proof. For B, w B we denoe he arng w by, w where, : B B R. Ths arng s blnear bu no symmerc. We wll frs show ha, for any w W, where = / 1. We have, η, w w Ψ w, w Ψ w, w +1 + η, 16 η, w w = η, w w +1 + w +1 w = η, w w +1 + η + Ψw +1 Ψw, w +1 w }{{}}{{} s 1 + Ψw Ψw +1, w +1 w }{{} s 3 17 s 2 Now, by defnon of he dual norm and he fac ha ab a + b for any a, b 0, we ge s 1 w w +1 η 1 w w η. By he defnon of he udae, w +1 mnmzes η Ψw, w +Ψw over w W. Therefore, s 2 0. Usng smle algebrac manulaons, we ge Pluggng hs no 17, we ge s 3 = Ψ w, w Ψ w, w +1 Ψ w +1, w. η, w w Ψ w, w Ψ w, w +1 + η w w +1 Ψ w +1, w }{{} s 4 Usng ha Ψ s -unformly convex on B mles ha s 4 0. So, we ge 16. We can now bound he regre as follows. For any w W, snce l w, we have, l w l w, w w. Combnng hs wh 16 and summng over gves, l w l w Ψ w, w 1 Ψ w, w T +1 η + η 1. Now, Ψ w, w T +1 0 and Ψ w, w 1 2BC. Furher L snce l s L-Lschz. Pluggng hese above and omzng over η gves he reured uer bound. Noe ha all he above algorhm needed o acheve low regre was a unformly convex funcon Ψ. Theorem Pser [16 gves us exacly hs, guaranees exsence of a 1-unformly convex funcon on a gven Banach sace for any [1, B hus makng sure ha he above mrror descen algorhm wh hs choce of Ψ gves us he omal rae for convex lschz games. 7 Adave Player Sraegy A naural exenson of he -unformly convex lschz game s a game where a round, A lays unformly convex funcons. In hs secon, we gve an adave layer sraegy for such games ha acheves he uer bound n Theorem 8 whenever he adversary lays only -unformly convex funcons on all rounds and n general ges nermedae raes when he modulus of convexy on each round s dfferen. Now for he sake of readably, we assume ha he suremum n 3 s acheved. The followng heorem saes ha he same adave algorhm acheves he uer bound suggesed n Theorem 8 for varous -unformly convex games. Furher he algorhm adjuss self o he scenaro when A lays a dfferen σ, -unformly convex funcon a each round. To see hs le, σ j = σ j 11 {j< +1}. In he above algorhm we se a each round ha sasfes, 2C = [ σ j j M j σ M + j M + [ σ j j M j 1 + j M 1, 18 where M = L +C. The followng heorem whch uer bounds he regre of he adave algorhm.

11 Algorhm 2 Adave Mrror Descen Parameers : Ψ : B R whch s -unformly convex C Lschz consan of Ψ on U B, w 1 0, Φ 1 0 for = 1 o T do Play w and receve l whch s L -Lschz and σ, -unformly convex Pck ha sasfes 18 Φ +1 Φ + l + Ψ w +1 Φ +1 Φ w Udae w +1 argmn Φ+1 w, w +1 end for Theorem 10. Le W = U B. Le AMD denoe he P-sraegy obaned by runnng Adave Mrror Descen wh a Ψ whch s = B B 1 unformly convex. Then, for all seuences l 1:T such ha l s L -Lschz and σ, -unformly convex, we have, 2σ M RegAMD, l 1:T mn [ 2 1:T σ + [ j σ C j j M j + j M j M j + j M Proof. Noe ha f = l + Ψ, Ψ s -unformly convex and l s σ, -unformly convex. Hence, f w σ +1, w w +1 w + w +1 w Where σ = σ 11 {< +1}. Now snce Φ +1 = f, we see ha Φ+1 w, w +1 = Φ+1w Φ +1w +1, w w +1 Φ+1 w +1, w = f w, w w +1 Φ+1 w +1, w f w, w w +1 σ w +1 w w +1 w =1 =1 Now consder any arbrary seuence β 1,..., β 2 of non-negave numbers such ha 2 =1 β = 1. In hs case noe ha by Fenchel-Young neualy, Φ+1 w, w +1 2 β f w, w w +1 σ w +1 w =1 =1 β f w + β + fw β L + C =1 σ / / =1 σ / =1 w +1 w + β + L + C / In he above we used he fac ha snce l s L -Lschz and Ψ s C-Lschz so ha, f w L + C we were able o ge rd of he because we use 1 and so L + C L + C. Now σ choosng, β L +C and < 2, β L +C we see ha σ Φ+1 w, w +1 L +C L σj =1 + +C j=1 L +C j L +C σj j=1 L +C j L +C σ L +C σj + + σ L +C L +C =1 j=1 L +C j σ L +C j j=1 L +C j L +C = =1 j=1 σ L +C σj L +C j L +C + j=1 1 σj 1 L +C j L +C where n he frs se we used he fac ha, 1 o remove hem from he denomnaor. Thus usng Lemma 13 we conclude ha σ L Regw 1:T, l 1:T +C 1 σ + j =1 j=1 L +C j L +C j=1 σ j L +C j + + 2C 1 j L +C Snce we choose s ha sasfy Euaon 18, usng Lemma 14 we ge he reured saemen.

12 Usng he above regre bound we ge he followng corollary showng ha he Adave Mrror Descen algorhm can be used o acheve all uer bounds on he regre resened n he aer. Corollary 11. There exss a Ψ whch s -unformly convex funcon, and usng hs funcon wh he Adave Mrror Descen AMD algorhm, we have he followng. 1. Regre of AMD for convex-lschz game maches uer bound n Regre of AMD for -unformly convex game maches uer bound n For he bounded convex game, here exss a C > 0 such ha usng AMD on 1 CT 1 2 ball acheves he uer bound n 10 for he game layed on he un ball. Proof. Clam 2 s shown n he consrucve roof of he uer bound of Theorem 8. As for clam 1, noe ha hs s he case of lnear funcons and so s he same as adversary ckng each σ = 0. Regre n hs case agan can be found n he roof of he uer bound of Theorem 8 and so clam 1 also holds. As for he las clam, gven clam 1, s evden from roof of Theorem 5. When 1,..., T = 2, AMD enjoys he same guaranee as Algorhm 4 n [7 see Theorem Dscusson In fuure work, we also lan o conver he layer sraeges gven here no mlemenable algorhms. Onlne learnng algorhms can be mlemened n nfne dmensonal reroducng kernel Hlber saces [21 by exlong he reresener heorem and dualy. We can, herefore, hoe o mlemen onlne learnng algorhms n nfne dmensonal Banach saces where some analogue of he reresener heorem s avalable. Der and Lee [12 have made rogress n hs drecon usng he noon of sem-nner roducs. For L Ω, µ saces wh even, hey showed how he roblem of fndng a maxmum margn lnear classfer can be reduced o a fne dmensonal convex rogram usng momen funcons. The yes and her assocaed consans of L saces are well known from classcal Banach sace heory. So, we can use her deas o ge onlne algorhms n hese saces wh rovable regre guaranees. Vovk [4 also defnes Banach kernels for ceran Banach saces of real valued funcons and gves an mlemenable algorhm assumng he Banach kernel s effcenly comuable. Hs neres s n redcon wh he suared loss. I wll be neresng o exlore he connecon of hs deas wh he seng of hs aer. Usng onlne-o-bach conversons, our resuls also mly error bounds for he esmaon error n he bach seng. If B < 2 hen we ge a rae worse han OT 1/2. However, we ge he ably o work wh rcher funcon classes. Ths can decrease he aroxmaon error. The sudy of hs rade-off can be helful. We would also lke o mrove our lower and/or uer bounds where hey do no mach. In hs regard, we should menon ha he uer bound for he convex-bounded game gven n Theorem 5 s no gh for a Hlber sace. Our uer bound s OT 3/4 bu can be shown ha usng he self-concordan barrer log1 w 2 for he un ball, we ge an uer bound of OT 2/3. Acknowledgmens We hank he Col 2010 revewers for her helful commens. We would also lke o hank Maxm Ragnsky for onng ou ceran suble ssues n our earler verson. References [1 M. Znkevch. Onlne convex rogrammng and generalzed nfnesmal graden ascen. In Proceedngs of he Tweneh Inernaonal Conference on Machne Learnng, [2 E. Hazan, A. Agarwal, and S. Kale. Logarhmc regre algorhms for onlne convex omzaon. Machne Learnng, 692 3: , [3 S.Shalev-Shwarz. Onlne Learnng: Theory, Algorhms, and Alcaons. PhD hess, Hebrew Unversy of Jerusalem, [4 V. Vovk. Comeng wh wld redcon rules. Machne Learnng, 692 3: , [5 J. Abernehy, P. L. Barle, A. Rakhln, and A. Tewar. Omal sraeges and mnmax lower bounds for onlne convex games. In Proceedngs of he Nneeenh Annual Conference on Comuaonal Learnng Theory, [6 A. Nemrovsk and D. Yudn. Problem comlexy and mehod effcency n omzaon. Nauka Publshers, Moscow, [7 P. L. Barle, E. Hazan, and A. Rakhln. Adave onlne graden descen. In Advances n Neural Informaon Processng Sysems 21, 2007.

13 [8 J. Abernehy, A. Agarwal, P. L. Barle, and A. Rakhln. A sochasc vew of omal regre hrough mnmax dualy. In Proceedngs of he Tweneh Annual Conference on Comuaonal Learnng Theory, [9 M. J. Donahue, L. Gurvs, C. Darken, and E. Sonag. Raes of convex aroxmaon n non-hlber saces. Consrucve Aroxmaon, 13: , [10 L. Gurvs. A noe on a scale-sensve dmenson of lnear bounded funconals n Banach saces. Theorecal Comuer Scence, 2611:81 90, [11 S. Mendelson and G. Schechman. The shaerng dmenson of ses of lnear funconals. The Annals of Probably, 323: , [12 R. Der and D. Lee. Large-margn classfcaon n Banach saces. In Processng of Elevenh Inernaonal Conference on Arfcal Inellgence and Sascs, [13 Hazhang Zhang, Yuesheng Xu, and Jun Zhang. Reroducng kernel Banach saces for machne learnng. Journal of Machne Learnng Research, 10: , [14 Fedor Zhdanov, Alexey Chernov, and Yur Kalnshkan. Aggregang algorhm comeng wh Banach laces, arxv rern arxv: avalable a h://arxv.org/abs/ [15 N. Cesa-Banch and G. Lugos. Predcon, learnng, and games. Cambrdge Unversy Press, [16 G. Pser. Probablsc mehods n he geomery of Banach saces. In G. Lea and M. Praell, edors, Probably and Analyss, volume 1206 of Lecure Noes n Mahemacs, ages Srnger, [17 G. Pser. Marngales wh values n unformly convex saces. Israel Journal of Mahemacs, 203 4: , [18 C. Zalnescu. Convex analyss n general vecor saces. World Scenfc Publshng Co. Inc., Rver Edge, NJ, [19 G. Pser. Sur les esaces de Banach u ne conennen as unformémen de l 1 n. C. R. Acad. Sc. Pars Sér. A-B, 277:A991 A994, [20 J. Borwen, A. J. Gurao, P. Hájek, and J. Vanderwerff. Unformly convex funcons on Banach saces. Proceedngs of he Amercan Mahemacal Socey, 1373: , [21 J. Kvnen, A. J. Smola, and R. C. Wllamson. Onlne learnng wh kernels. IEEE Transacons on Sgnal Processng, 528: , [22 K. Srdharan and A. Tewar. Convex games n banach saces. Techncal Reor no. TTIC-TR , Toyoa Technologcal Insue a Chcago, Aendx Lemma 12. For any Banach sace B and any > B we have ha [ T su M E d T 1/ as T, where M refers o dsrbuons over marngale dfference seuences d T such ha each d U B Lemma 13. For he Adave Mrror Descen Algorhm we have ha Regw 1:T, l 1:T Φ+1 w, w C Lemma 14. Defne for any seuence 1,..., S of any sze S, σ S L O S 1,..., S = +C + =1 j=1 σ j L +C j + j L +C Then as long as we ck ha sasfes Euaon 18, we have ha for any T O T 1,..., T 2 mn O{ 1,..., T } 1,..., T j=1 1 σ j L +C j + For roofs of he above wo lemma s refer o a longer verson of hs aer a [ C j L +C