Non-Asymptotic Theory of Random Matrices Lecture 8: DUDLEY S INTEGRAL INEQUALITY

Transcription

1 Non-Asympoic Theory of Random Marices Lecure 8: DUDLEY S INTEGRAL INEQUALITY Lecurer: Roman Vershynin Scribe: Igor Rumanov Tuesday, January 30, 2007 Le A : m n marix wih i.i.d. enries, m > n. We wan o esimae E sup x S n 1 Ax 2 m?, (we wan C n in place of? here, i would be beer esimae han in asympoic heory: C( m + n) ). Under he absolue value sign here sands a random variable, even family of random variables indexed by poins of sphere x S n 1, i.e. a random process. Random process: (X ) T is a collecion of random variables indexed by T. Classical: T = [a, b] - ime inerval. Examples of such processes are called Levy Processes. (Ex.: Brownian moion) General: T is arbirary, such as T = S n 1. Size of he random process : E supx T (index se has o be compac) How far a paricle can ge in ime T? (Ex.: The highes level of waer in a river in 10 years) Previous approach: Discreizaion of T. Consider an ε-ne N of T: cover by ε-balls : 1

2 Compue E sup N approximae X, Definiion 1 (Covering Numbers). Le (T, d) be a compac meric space, ε > 0. Then covering number N(T, ε) = minimal cardinaliy of an ε-ne of T = minimum possible number of ε-balls o cover T. Measure of compacness of T: log N(T,ε) is called meric enropy of T. Sharper approach: Muliscale discreizaion. Cover T progressively wih radius ε k -balls, k = 1,2,3,... The resul will be Dudley s Inegral Inequaliy Assumpions: 1) EX = 0 for all 2) Incremens X X s are proporional o he disance d(,s). X X s d(,s) is subgaussian for all,s: P( X X s > u d(,s)) Ce cu2 for u > 0, subgaussian incremens. (Here C and c are some consans). 2

3 Theorem 2 (Dudley [1, 2]). : For a process wih subgaussian incremens E supx C T 0 log N(T,ε)dε probabilisic geomeric (in T) (one can replace he upper limi of in he inegral wih diam(t)). Singulariy here is a 0. (For sphere N(T,ε) ( 1 ε )n ). ( logx = inverse of e x2 ). Proof: Le diam(t) = 1. (Exercise: general case) 1) Le 0 T be arbirary (reference poin), E sup T X = E sup T (X EX 0 ) E sup(x X 0 ), T by Jensen s inequaliy, because sup is convex funcion. 2) Muliscale discreizaion of T: CHAINING : π 1 () 0 T 1 Le N 1 be a 1/2-ne of T of size N 1 = N(T,1/2) Find π 1 () N 1 neares o 3

4 X X 0 = (X X π1 ()) + (X π1 () X 0 ) smaller han before (1/2) here are a mos N 1 such r.v. s (no oo many) π 1 () π 2 () 0 2 Le N 2 be a 1/4-ne of T of size N 2 = N(T,1/4) Find π 2 () N 2 neares o X X 0 = (X X π2 ()) + (X π2 ()) X π1 ()) + (X π1 () X 0 ) even smaller(1/4) here are (a mos?) N 1 N 2 N2 2 such r.v. s... k Le N k be a 2 k -ne of T of size N k = N(T,2 k ) Find π k () N k neares o... X X 0 = X πk () X πk 1 () chaining ideniy (π 0 () = 0 ), because X X πk () 0 a.s. (Exercise: use π k () ). 4

5 Nice properies of muliscale discreizaion: 1)Incremens are small: 2 k 2 (k 1) π k () π k 1 () d(π k (),π k 1 ()) d(π k (),) + d(π k 1 (),) 2 k + 2 (k 1) = 3 2 k. 2) There are a mos N k N k 1 N 2 k pairs of (π k(),π k 1 ()), whaever is. Incremens: P ( ) cu X πk () X πk 1 () > u a k C exp ( 2 a 2 ) k d(π k (),π k 1 ()) 2 (holds for a k > 0). Thus we can bound every incremen in he Chaining Ideniy: he failure (o bound) probabiliy is p = P( k, T : X πk () X πk 1 () > u a k ) In case of success: if k, T: ( ) = C exp c 2 2k u 2 a 2 k Nk 2 C exp( c 22k u 2 a 2 k ). X πk () X πk 1 () ua k, hen X X 0 u a k. Hence ( P sup X X 0 > u k a k ) p. ( ) 5

6 I remains o choose weighs a k. We have radeoff here: we wan a k o be small, bu for decreasing failure probabiliy a k have o be large. How large? Say, for u 1 we wan he summands in p be 2 k. Therefore a k = c 2 k log 2 k N 2 k (for u 1). Then p CNk 2 (2k Nk 2 ) u2 C 2 ku2. So subgaussian failure probabiliy obeys he bound p C 2 u2. This way we ge an esimae for he sum of weighs which appears in (*): ak = c 2 k log 2 k N 2 k (use a + b 2( a + b)) C k c ( 2 k log 2 k }{{} + 2 k log N k }{{} ) 2 k log N k = C k cons cons because diamt = 1, N k log N(T,2 k ) C IV log N(T,ε)dε := S 0 (compare series wih inegrals in he las inequaliy) We have ( ) P sup X X 0 > us Ce u2 for u 1. Thus, he random variable 1 S sup X X s is subgaussian and Dudley s inequaliy follows immediaely. Problem: o find sharp esimae (a funcion beer han S - will be done nex ime - see Lecure 9). 6

7 References [1] Michel Ledoux and Michel Talagrand. Probabiliy in Banach spaces, volume 23 of Ergebnisse der Mahemaik und ihrer Grenzgebiee (3) [Resuls in Mahemaics and Relaed Areas (3)]. Springer-Verlag, Berlin, [2] Michel Talagrand. The generic chaining. Springer Monographs in Mahemaics. Springer-Verlag, Berlin,