arxiv: v1 [math.pr] 4 Dec 2013

Transcription

1 Squared-Norm Empirical Process i Baach Space arxiv:32005v [mathpr] 4 Dec 203 Vicet Q Vu Departmet of Statistics The Ohio State Uiversity Columbus, OH vqv@statosuedu Abstract Jig Lei Departmet of Statistics Caregie Mello Uiversity Pittsburgh, PA jiglei@adrewcmuedu This ote exteds a recet result of Medelso o the remum of a quadratic process to squared orms of fuctios takig values i a Baach space Our method of proof is a reductio by a symmetrizatio argumet ad observatio about the subadditivity of the geeric chaiig fuctioal We provide a applicatio to the remum of a liear process i the sample covariace matrix idexed by fiite rak, positive defiite matrices Itroductio Let F be a class of R-valued fuctios o the probability space (Ω,P) ad X,,X be idepedet, idetically distributed radom variables Let ψα be the Orlicz ψ α orm for α ad γ α be Talagrad s geeric chaiig complexity of F uder the ψ α metric Medelso has proved the followig theorem: Theorem (Medelso (200)) If F is a symmetric class of mea-zero fuctios o (Ω,P) the there exists absolute costats c, c 2, ad c 3 such that for ay t c, with probability at least 2exp( c 2 t 2/5 ) f 2 (X i ) Ef 2 (X i ) c d ψ (F)γ 2 (F,ψ 2 ) 3t + γ2 2(F,ψ 2 ), f F where d ψ (F) = f F f(x ) ψ The goal of this ote is to exted the precedig theorem to a class G of fuctios that take values i a Baach space (B, ) We wish to boud g(x i ) 2 E g(x i ) 2 () This ote was first writte i 202 ad a versio has bee available o the first author s website It is ow posted o arxiv to provide a uiversal idetifier for citatios elsewhere It is ot iteded for publicatio i its curret form, but a revisio icorporatig later work will be posted separately This research was ported by NSF grats DMS ad BCS-09458

2 with a similar expoetial tail probability boud A special case is g(x ) = A T X, where X is a radom vector i R p ad A A, a class of p k matrices I this ote we will assume that G is coutable to avoid uecessary measurability issues Whe (B, ) = (R, ), this expectatio reduces to the oe addressed by Theorem Our goal is to boud the expectatio i terms of complexity parameters of the class G We will use a symmetrizatio argumet to put () ito a form suitable for Theorem The ext sectio cotais a abstract techical result cocerig subadditivity of the geeric chaiig γ α fuctioal (Talagrad 2005) that eables our reductio 2 The Geeric Chaiig Defiitio 2 Give a metric space (T,d) ad A a subset of T, defie the diameter of A to be (A) := d(t,u) (2) t,u A Defiitio 22 Give a metric space (T,d) ad A a partitio of T, for t T, defie A(t) to be the uique elemet of A that cotais t Defiitio 23 (Talagrad (2005)) Give a metric space (T, d), a admissible sequece is a icreasig sequece (A s ) of partitios of T such that A 0 = ad, for s, A s 2 2s For α, defie the γ α fuctioal by γ α (T,d) := if t T 2 s/α (A s (t)), where the ifimum is take over all admissible sequeces Lemma 2 If (T,d) ad (T 2,d) are metric spaces, the Moreover, if T T 2, the γ α (T T 2,d) 3 [ (T T 2 )+γ α (T,d)+γ α (T 2,d) ] γ α (T T 2,d) 9 [ γ α (T,d)+γ α (T 2,d) ] Proof Let (A s ) be a admissible sequece for T such that 2 s/α (A s (t)) 2γ α (T,d) for all t T, (3) ad (B s ) a admissible sequece for T 2 such that 2 s/α (B s (t)) 2γ α (T 2,d) for all t T 2 (4) We defie partitios C s of T T 2 as follows Let C 0 = T T 2, C = C 0, ad, for s 2, let C s be the partitio cosistig of the sets A (T \T 2 ) for A A s 2, B (T 2 \T ) for B B s 2, ad A B for A A s 2 ad B B s 2 2

3 It is straightforward to check that (C s ) is a icreasig sequece of partitios of T T 2, ad that C 0 = C = ad, for s 2, C s A s 2 + B s 2 + A s 2 B s 2 2 2s s s 2 2s Thus (C s ) is a admissible sequece for T T 2 Let t T T 2 Sice, for s 2, every elemet of C s is a subset of a elemet of A s 2 ad/or B s 2, 2 s/α (C s (t)) 3 (T T 2 )+ 2 s/α (C s (t)) s 2 by (3) ad (4) The 3 (T T 2 )+ 2 s/α (A s (t))+ 2 s/α (B s (t)) 3 [ (T T 2 )+γ α (T,d)+γ α (T 2,d) ], γ α (T T 2,d) t T T 2 2 s/α (C s (t)) 3 [ (T T 2 )+γ α (T,d)+γ α (T 2,d) ] For the moreover part, if u T T 2, the by the triagle iequality 3 Symmetrizatio (T T 2 ) = d(t,t 2 ) t,t 2 T T 2 d(t,u)+d(t 2,u) t,t 2 T T 2 2 [ (T )+ (T 2 ) ] 2 [ γ 2 (T,d)+γ 2 (T 2,d) ] The followig theorem is our mai result Its proof is a symmetrizatio argumet followed by a applicatio of Theorem to ad Lemma 2 Theorem 3 If X,X 2,,X are idepedet, idetically distributed radom variables, ad G is a symmetric class of fuctios takig values i a Baach space (B, ), the there exists absolute costats c, c 2, ad c 3 such that for all t c, with probability at least 2exp( c 2 t 2/5 ), g(x i ) 2 E g(x i ) 2 g(x ) ψ γ 2 (G,d) c 3 t where d is the metric o G defied by + γ2 2(G,d), (5) d(g,g 2 ) = g (X ) g 2 (X ) ψ2 (6) 3

4 Proof Let ǫ,,ǫ be idepedet, idetically distributed Rademacher radom variables that are idepedet of X,,X Defie the fuctio class ad let F = f : f(x,ǫ) = ǫ g(x) for some g G, F 0 = F F F 0 is a symmetric class of mea-zero fuctios by costructio So Theorem implies that, for t c, with probability at least 2exp( c 2 t 2/5 ), g(x i ) 2 E g(x i ) 2 = f 2 (X i,ǫ i ) Ef 2 (X i,ǫ i ) f F 0 g(x ) ψ γ 2 (F 0,ψ 2 ) c 3 t + γ2 2 (F 0,ψ 2 ) The rest of the proof deals with showig that Apply Lemma 2 to γ 2 (F 0,ψ 2 ) Cγ 2 (G,d) (7) γ 2 (F 0,ψ 2 ) 3 [ (F F)+γ 2 (F,ψ 2 )+γ 2 ( F,ψ 2 ) ] = 3 [ (F F)+2γ 2 (F,ψ 2 ) ] (8) For ay f,f 2 F with correspodig g,g 2 G, we have by the triagle iequality that f (X,ǫ ) f 2 (X,ǫ ) ψ2 = g (X ) g 2 (X ) ψ2 The by Theorem 36 of Talagrad (2005), g (X ) g 2 (X ) ψ2 =: d(g,g 2 ) γ 2 (F,ψ 2 ) Cγ 2 (G,d) (9) for a absolute costat C > 0 Usig the triagle iequality ad the symmetry of G, (F F) = max g (X ) g 2 (X ) ψ2, g (X ) + g 2 (X ) ψ2 max = max = d(g,g 2 ) γ 2 (G,d) d(g,g 2 ), 2 g(x ) ψ2 d(g,g 2 ), g(x ) ( g(x )) ψ2 Substitutig the precedig iequality ad (9) ito (8) proves (7) 4

5 4 Liear Trasformatios Let A R p k Observe that G = g : g(x) = A T x ad A A g(x i ) 2 2 = X i Xi T,AAT To apply Theorem 3 to this empirical process, we eed to boud g(x ) ψ 2 = A T X ψ 2 A T X ψ2 2 (0) ad d(g,g 2 ) = g(x ) g 2 (X ) 2 ψ2 = (A A 2 ) T X 2 ψ2 () The ext lemma allows us to relate the ψ 2 -orm o G to the Frobeius orm o A Lemma 4 Let Z be a p-variate radom vector ad A be a p k matrix The A T Z 2 ψ2 A F Z ψ2, (2) where Z ψ2 := u 2 Z,u ψ2 Proof Let a,,a k deote the colums of A The Thus, ad A T Z 2 2 ψ k ai,z 2 ψ = k ai,z 2 ψ 2 k a i 2 2 Z 2 ψ 2 A T Z 2 2 ψ 2 = A T Z 2 2 ψ A 2 F Z 2 ψ 2 Applyig Lemma 4 to eqs (0) ad () yields g(x ) ψ 2 X ψ2 A F d(g,g 2 ) X ψ2 A A 2 F Sice A F = vec(a) 2, the majorizig measure theorem (Talagrad 2005, Theorem 2) implies that γ 2 (G,d) c X ψ2 E A,Z, where Z is a p k matrix with iid stadard Gaussia etries ad c > 0 is a absolute costat Thus, we have the followig corollary of Theorem 3 5

6 Corollary 4 Let X,,X R p be iid mea 0 radom vectors, Σ = EX X T, ad σ = u 2 = X,u ψ2, ad S = X i Xi T If A R p k is symmetric, the there exist positive costats c, c 2, ad c 3 such that for all t c, with probability at least 2exp( c 2 t 2/5 ), σ S Σ,AA T c 3 t 2 A F ( E Z,A ) + σ2 ( E Z,A ) 2, where Z is a p k matrix with iid stadard Gaussia etries Furthermore, we have uder the same coditio, that σ E S Σ,AA T c 2 A F ( E Z,A ) + σ2 ( E Z,A ) 2, with a uiversal costat c Refereces Medelso, S (200) Empirical Processes with a Bouded ψ Diameter I: Geometric ad Fuctioal Aalysis 204, pp Talagrad, M (2005) The Geeric Chaiig Spriger-Verlag 6