Lecture 4: November 13

Transcription

1 Compuaional Learning Theory Fall Semeser, 2017/18 Lecure 4: November 13 Lecurer: Yishay Mansour Scribe: Guy Dolinsky, Yogev Bar-On, Yuval Lewi 4.1 Fenchel-Conjugae Moivaion Unil his lecure we saw our problem in he primal space x, fx)). In his lecure we will look a he dual space represenaion of our problem, meaning, looking a fx), fx)). For convex funcions, his represenaion conains all he daa we have in he regular problem while giving us a new geomeric view of our problem. Le us define he dual funcion:f y) = max w S y T u fw) Theorem 4.1 Assume ha x = arg max w S y T w fw). Then y fx). Hence, Proof: The definiion of f derives ha: u : f y) y T u fu) u : fu) y T u f y) From our assumpion ha: f y) = y T x fx), i implies ha, u : fu) y T u y T x + fx) = fx) + y T u x), which is he definiion of y fx) Examples Example from economics Assume ha a manufacurer produces d producs wih quaniies of q IR d +. Le us also assume he he cos funcion per quaniy is defined by a convex funcion Cq). Then he revenue is defined by: Revp, q) = p T q Cq), 1

2 2 Lecure 4: November 13 where p IR d is he price per uni of produc. The dual problem in ha case is: C p) = max q p T q Cq) = max q Revp, q). Namely, he dual problem, given prices oupus quaniies ha maximize revenue. In addiion, he marginal cos per per produc is Cq) meaning a opimum we have p = Cq). A single dimension example Define fw) = w log w where f : IR IR. Then: f y) = max w y T w w log w Hence, by aking he derivaive and comparing o 0 we ge: Therefore L-2 disance example y 1 logw = 0 w = e y 1. f y) = ye y 1 y 1)e y 1 = e y 1. Define fw) = 1 2 w 2 2 where f : IRd IR. As before: f y) = max w S y T w 1 2 w 2 2 Taking he derivaive and zeroing we ge w = y. This implies, f y) = y T y 1 2 y 2 2 = 1 2 y Fenchel-young inequaliy Theorem 4.2 Fenchel-young inequaliy: f y) + fu) y T u Proof: By definiion: f y) = max w S y T w fw) Hence: u : f y) y T u fu) Rearranging,

3 Fenchel-Conjugae 3 Theorem 4.3 fw) f w) Proof: Since f w) = f w)), We can se z = w and ge, u : f y) + fu) y T u. f w) = max y y T w f y) = max y y T w max z z T y fz))). f w) max y y T w w T y + fw)) = fw) I is possible o prove ha f = f if he epigraph of fis a close and convex group. As he graph of f is {x, fx)) x S} and he epigraph is {x, ) x S, fx) }. Lemma 4.4 If he epigraph of f is close and convex hen: Proof: y fx) x = arg max z y T z fz)) x f y) 1. Le us assume x = arg max z y T z fz)). Hence: From ha we conclude: Thus x f y). 2. Le x f y), Then As a resul, f y) = y T x fx) f w) f y) w T x fx)) y T x fx))) = x T w y) x f y) w : f w) f y) x T w y) w : x T y f y) f w) x T w y = arg max w x T w f w))

4 4 Lecure 4: November 13 Combining his wih he definiion of f, his leads us o f x) = x T y f y) Recall f = f since he epigraph of f is closed and convex) Hence f y) = x T y fx) = max z z T y fz)) x = arg max z z T y fz)). 3. We would like o show y fx) x = arg max z y T z fz)) y fx) z S : fz) fx) y T z x) z S : y T x fx) y T z fz) x = arg max z y T z fz)). Lemma 4.5 If f is differeniable hen y = f fy)). Proof: Se z such ha fy) + f z) = z T y. Then y fy) = y z y f z)) = z z f z) = z z y fy)) = y Therefore f fy)) = f z) = y. Theorem 4.6 If f = f Then x = f fx)). Proof: Recall he definiion of f ha f y) = max z z T y fz). If x maximizes he funcion hen y = fx). Hence, Since f = f Therefore f y) = x T y fx) f y) + f x) = x T y f x) = max z z T y f z) Meaning f y) = x. Combining y = fx) and x = f y) lead us o x = f y) = f fx)).

5 4.2. BERGMAN-DIVERGENCE Bergman-Divergence Bergman-Divergence of a Convex Funcion Le R be some convex funcion on a se S. We would like o use is dual space for an algorihm we will presen laer his lecure he Mirror-Decen Algorihm). We can go from a poin in S o a poin in he dual space by using R, bu we canno always use R o go back - he resuling poin is no necessarily in S. To fix his we will use he Bergman-Divergence: Definiion Bergman-Divergence of a convex funcion R is defined as: B R x y) = Rx) Ry) [ Ry)] T x y) We can now use argmin x S B R x y) as he projecion of poin y on S Examples L 2 -Norm Le Rw) = 1 2 w 2 2. So Rw) = w and we obain: B R x y) = 1 2 x y 2 2 y T x y) = 1 2 x y 2 2 y T x + y 2 2 = 1 2 x y 2 2 y T x = 1 2 x y 2 2 Negaive Enropy Le Rw) = i w i log w i. We obain Rw) =..., log w i ) + 1,...) T, so: B R x y) = i x i log x i i y i log y i i log y i ) + 1)x i y i )) = i x i log x i ) log y i ))) i x i + i y i = i x i log x i y i i x i + i y i If S = {w w i 0, w 1 = 1} is he simplex, i.e all disribuions, we obain ha B R x y) is he KL-divergence of x, y S. Also, in his case we obain ha he projecion on S is: argmin x S B R x y) = argmin x S i x i log x i y i + i y i 1 ) We will solve using Lagrange-mulipliers: F x, λ) = i x i log x i y i + i ) y i 1 λ x i 1 i

6 6 Lecure 4: November 13 i F = 1 + log x ) i λ = 0 x i y i i x i = y i e λ 1)) And since i x i = e λ 1) i y i = 1, we obain ha x i = S is he normalizaion of y. y i y i 1. Thus, he projecion of y on 4.3 Online Mirror Decen The Online Mirror Decen Algorihm The Online Mirror Decen algorihm is an online learning algorihm, similar o he ones we have already seen. The big difference is ha OMD uses he dual space o updae he curren poin, insead of he primal space, and projecs he updae on he primal space wih he Bergman-Divergence funcion. We will presen he algorihm wih linear loss funcions: Online Mirror Decen begin Se y 1 s.. Ry 1 ) = 0 Se w 1 = argmin w S B R w y 1 ) for [1, T ] do Play w and ge f x) = z T x Se y +1 s. Ry +1 ) = Ry ) η f w ) = Ry ) ηz Namely, y +1 = R 1 Ry ) ηz ) = R Ry ) ηz ) Se w +1 = argmin w S B R w y +1 ) end for end Online Mirror Decen Regre Analysis Theorem 4.7 Le R be some σ-srong-convex funcion. The OMD wih linear loss funcions oupus he same predicions as FoReL. Proof: We will denoe by w F and w O he predicions in ime of FoReL and OMD, respecively. Firs, we noice ha in OMD: Ry +1 ) = Ry ) ηz =... = η z i

7 4.4. EXPONENTIATED GRADIENT ALGORITHM 7 In FoReL, as we seen in Lecure 2, he updae rule is: Hence, and we obain: Therefore, ) w+1 F = argmin w S η zi T w + Rw) ) η zi T w + Rw) w+1) F = 0 η zi T + Rw+1) F = 0 Rw+1) F = η zi T = Ry +1 ) Since R is a σ-srong-convex funcion, we obain ha w F +1 = y +1. If y +1 S, we also have y +1 = w O +1 and we are done. Oherwise, we obain ha: w O +1 = argmin w S B R w y +1 ) = argmin w S Rw) Ry+1 ) [ Ry +1 )] T w y +1 ) ) = argmin w S Rw) [ Ry+1 )] T w ) = argmin w S Rw) + η Which is again he same as w+1. F ) z i w 4.4 Exponeniaed Gradien Algorihm The Exponeniaed Gradien Algorihm We can now proceed o rerieve he Randomized Weighed Majoriy algorihm he Exponeniaed Gradien Algorihm in his conex) from he Online Mirror Descen Algorihm. Regularizaion Analysis Seing he regularizaion funcion o Rw) = d w i logw i ), we have ha Rw) =..., logw i ) + 1,...) T. Now solving for max w Rw) s.. i w i = 1 using he Lagrangian F w, λ) = Rw) λ i w i 1) yields ha: w i F = 1+log w i λ log w i = log w j = λ 1 w i = 1. We herefore conclude ha Rw) log d. d

8 8 Lecure 4: November 13 An Online Mirror Descen Sep The OMD sep is defined as Ry +1 ) = Ry ) ηl, meaning ha: 1 + log y i) +1 = 1 + log y i) ηl i) y i) +1 = y i) e ηli), which is boh he definiion of RWM and coincides wih FoReL. Exponeniaed Gradien Algorihm begin Se y 1 = 1 Se w 1 = y 1 y 1 = 1 1 d for [1, T ] do Play w and ge l Se y i) +1 = y i) e ηli) Se w i) +1 = yi) d j=1 yj) end for end Exponeniaed Gradien Algorihm Regre Analysis Lemma 4.8 The regre of his algorihm is bounded a follows: u T=1 w u) T z i Ru) Rw 1 ) + T =1 B R Z 1; Z 1; 1 ), Where Z 1; = η z i and equaliy holds for u = arg min u Ru) i u T z i ) Proof: By he Fenchel-Young inequaliy, Ru) + R Z 1;T ) u T Z 1;T ) Ru) + u T Z 1;T ) R Z 1;T ), and from he definiion of OGD: Now using a elescopic series, we ge ha: w = R Z 1; 1 ). R Z 1;T ) = R 0) T=1 R Z 1; ) R Z 1; 1 )) ) Which by adding and subracing R Z 1; 1 ) T z o he sum and hen spliing i, equals: R 0) + T =1 R Z 1; 1 ) T z T =1 R Z 1; ) R Z 1; 1 ) R Z 1; 1 ) T z ) ) = R 0) + T =1 w T z T =1 B R Z 1; Z 1; 1 )

9 Exponeniaed Gradien Algorihm 9 Combining everyhing, yields he following inequaliies: Ru) + u T Z 1;T ) R 0) + T =1 w T z T =1 B R Z 1; Z 1; 1 ) Ru) + R 0) + T =1 B R Z 1; Z 1; 1 ) T =1 w u) T z Finally, we evaluae R 0): R 0) = max w 0 T w Rw) = max w Rw) = min w Rw) = Rw 1 ) Theorem 4.9 For he Normalized Exponeniaed Gradien Algorihm, he regre is: regre = T =1 w T u) T z log d η + η T =1 i w i) z i) 2 Proof: From he previous lemma, and since R θ) = 1 η log d e ηθ i ), i suffices o show ha B R Z 1; Z 1; 1 ) η i w i) z 2i). We evaluae B R Z 1; Z 1; 1 ) o ge: B R Z 1; Z 1; 1 ) = R Z 1; ) R Z 1; 1 ) + w T z = 1 η log However, e ηz 1; = e ηz 1; 1 e ηz, and herefore: B R Z 1; Z 1; 1 ) = 1 η log i w i) e ηzi) ) + w T z. i) i e ηz 1; j e ηzj) 1; 1 + w T z Since e a 1 a + a 2 for 1 a, ha value is bounded from above by: 1 η log i w i) 1 ηz i) + η 2 z i) 2 ) ) + w T z = 1 1 log ηw T η z + η 2 i w i) z i) 2 ) + w T z, Since e a 1 a we have log1 a) a, and bound from above by: B R Z 1; Z 1; 1 ) 1 η ηwt z + η 2 i w i) z i) 2 ) + w T z = η i w i) z i) 2.