Machine Learning Theory (CS 6783)

Transcription

1 Machie Learig Theory (CS 6783) Lecture 3 : Olie Learig, miimax value, sequetial Rademacher complexity Recap: Miimax Theorem We shall use the celebrated miimax theorem as a key tool to boud the miimax rate for olie learig problems. Below we state a geeralizatio of Vo Neuma s miimax theorem. Theorem (Browei 4). Let A ad B be Baach spaces. Let A A be oempty, weakly compact, ad covex, ad let B B be oempty ad covex. Let g : A B R be cocave with respect to b B ad covex ad lower-semicotiuous with respect to a A ad weakly cotiuous i a whe restricted to A. The if g(a, b) = if b B a A a A b B g(a, b) The above theorem states that uder the right coditios, oe ca swap ifimum ad remum. We shall use this i a sequetial maer to swap the order of the learer ad adversary ad use this to get a hadle o miimax rate for olie learig. For istace usig the above theorem, we ca show that for ay loss l, lower semicotiuous i its first argumet, as log as Y is well behaved (compact for istace), if ŷt q t l(ŷ t, y t ) + Φ(y t ) = q t (Y) where Φ is some arbitrary fuctio. p t (Y) 2 Miimax Rate for Olie Learig if l(ŷ t, y t ) + Φ(y t ) Recall that the miimax rate for a olie learig problem ca be writte as : V sq = if ŷ q... if ŷ q l(ŷ t, y t ) if q (Y) x X q (F) y Y That is i a sequetial fashio, o each roud, adversary picks the worst iput istace x t X, The learer the picks the optimal q t (Y) the adversary the picks the worst outcome y t Y, the learer draws predictio ŷ t q t with the aim of learer to miimize regret ad goal of adversary to maximize regret. We ow itroduce a shorthad otatio. We shall use the otatio Operator t... to refer to Operator Operator 2... Operator.... Hece for istace, V sq = if l(ŷ t, y t ) if q t (Y)

2 We ca also write the coditioal value as V (x, y,..., x t, y t ) = if x j X q j (Y) y j Y Claim 2. = ŷ j q j j=t+ if j=t+ l(ŷ t, y t ) if l(ŷ t, y t ) if Proof. = if q t (Y) = if q t (Y) = if q t (Y) = = =... = if q t (Y) if q t (Y) Thus we have the claim. if l(ŷ t, y t ) if l(ŷ t, y t ) + x X l(ŷ t, y t ) + x X l(ŷ t, y t ) + x X l(ŷ t, y t ) + x X if q (Y) y Y if y p p (Y) ŷ Y if p (Y) ŷ Y l(ŷ t, y t ) if p y (Y) p { } l(ŷ, y ) if ŷ q }{{} g(q,y ) l(ŷ, y ) if l(ŷ, y ) y p if ŷ Y y p if l(ŷ, y ) if y p = Notice that i the above claim, we have a distributios (possibly depedet) over istaces but have essetially elimiated the role of the learer ad moved to a completely stochastic object. From the above claim it is easy to show that the the miimax rate if govered by a quatity measurig rate of uiform covergece of class F over martigale differece sequeces. Claim 3. P (X Y) t where P is a joit distributio over the sequece of istaces ad t refers to the coditioal expectatio over istace (x t, y t ) give past istaces (x, y ),..., (x t, y t ) 2

3 Proof. = = = P (X Y) if l(ŷ t, y t ) if { } if l(ŷ t, y t ) t 3 Sequetial Rademacher Complexity I the statistical learig framework a key tool was symmetrizatio ad the use of Rademacher complexity. With the use of Rademacher complexity we were able to move our focus o how the fuctio class behaves o the etire space of istaces to oly how rich the class is effectively o samples of size. The questio ow, is whether there is a aalogue of this for uiform covergece over martigales. Surprisigly it turs out that there is ad this complexity we shall refer to as sequetial Rademacher complexity. Claim 4. Proof. = =... 2 x X p (Y ) x X p (Y ) x X p (Y ) ɛ... ɛ x X y Y ɛ t l(f(x t ), y t) y t pt y p ɛ x X y,y Y y,y p y,y p ɛ l(f(x t ), y t) + l(f(x ), y ) l(f(x ), y ) y t pt y p y t pt l(f(x t ), y t) + l(f(x ), y ) l(f(x ), y ) y t pt l(f(x t ), y t) + ɛ (l(f(x ), y ) l(f(x ), y )) y t pt l(f(x t ), y t) + ɛ (l(f(x ), y ) l(f(x ), y )) 3

4 proceedig i similar fashio ɛt y t,y t Y ɛt y t,y t Y 2 ɛt ɛ t (l(f(x t ), y t) ) ɛ t l(f(x t ), y t) + ɛ t = 2 ɛ... ɛ x X y Y x X y Y ɛ t ) ɛ t The above complexity ca be equivaletly writte as follows. ɛ t l(f(x t (ɛ :t )), y t (ɛ :t )) 2 x ɛ y =: 2R sq (l F) Where x ad y are X ad Y valued complete biary tree of depth. That is, for istace x = (x,..., x ) where each x t : {± } t X. To see that the two forms are equivalet, ote that, give ay trees x ad y, ote that ɛ... ɛ ɛ t x X y Y ɛ ɛ... x X y Y ɛ... ɛt+: ɛ ɛ l(f(x t (ɛ), y t (ɛ))) ɛ t + l(f(x (ɛ), y (ɛ))) t ɛ i l(f(x i ), y i ) + i= j=t+ l(f(x j (ɛ), y j (ɛ))) Sice the above statemet holds for ay trees x ad y we ca take the remum over the trees. O the other had, defie a pair of tree x ad y as follows : ) x = argmax ɛ ɛ t l(f(x t ), y t x X ɛt t=2 4

5 (ad similarly defie y ) ad subsequetly, give each ɛ :t defie x t (ɛ :t ) = argmax ɛt t ɛ i l(f(x i (ɛ)), y i (ɛ)) + x X x j X ɛ j i= y j Y j=t+ Clearly by defiitio of these trees, ɛ... ɛ x X y Y ɛ t ɛ ɛ j l(f(x j ), y j ) j=t l(f(x t (ɛ), yt (ɛ))) Sice we have both iequalities we coclude that the two forms are equivalet. I geeral for a give fuctio class G o space Z to reals we defie below the sequetial Rademacher complexity. Defiitio. Give a class G R Z, we defie the sequetial Rademacher complexity of the class G as, R sq (G) = ɛ ɛ t g(z t (ɛ)) z g G Pictorially, we ca view the Rademacher complexity as : 5