Sharp lower bounds on the least singular value of a random matrix without the fourth moment condition *

Transcription

1 Electron. Commun. Probb. 2 (215), no. 44, 1 9. DOI: /ECP.v2-489 ISSN: X ELECTRONIC COMMUNICATIONS in PROBABILITY Shrp lower bounds on the lest singulr vlue of rndom mtrix without the fourth moment condition * Pvel Yskov Abstrct We obtin non-symptotic lower bounds on the lest singulr vlue of X pn/ n, where X pn is p n rndom mtrix whose columns re independent copies of n isotropic rndom vector X p in R p. We ssume tht there exist M > nd α (, 2] such tht P( (X p, v) > t) M/t 2+α for ll t > nd ny unit vector v R p. These bounds depend on y = p/n, α, M nd re symptoticlly optiml up to constnt fctor. Keywords: Rndom mtrices; Singulr vlues; Hevy-tiled distributions. AMS MSC 21: 6B2. Submitted to ECP on Februry 3, 215, finl version ccepted on June 2, Introduction In this pper we obtin shrp lower bounds on the lest singulr vlue of rndom mtrix with independent hevy-tiled rows. For precise sttements, we need to introduce some nottion. Let X p be n isotropic rndom vector in R p, i.e. EX p Xp = I p for p p identity mtrix I p. Let lso X pn be p n rndom mtrix whose columns {X pk } n k=1 re independent copies of X p. Denote by s p (n 1/2 X pn) the lest singulr vlue of the mtrix n 1/2 X pn. The celebrted Bi-Yin theorem sttes tht, with probbility one, s p (n 1/2 X pn) = 1 y + o(1) when n, p = p(n) stisfies p/n y (, 1), nd the entries of X p re independent copies of rndom vrible ξ with Eξ =, Eξ 2 = 1, nd Eξ 4 <. In [5], Tikhomirov extended this result to the cse Eξ 4 =. Severl uthors hve studied non-symptotic versions of this theorem, relxing the independence ssumption, nd obtined bounds of the form s p (n 1/2 X pn) 1 Cy log y b tht hold with lrge probbility for some C,, b > nd ll smll enough y = p/n. See ppers [2], [3], [4], nd [6]. For generl isotropic rndom vectors X p with dependent entries not hving finite fourth moments, the optiml vlues of nd b re unknown. Assuming tht there exist M > nd α (, 2] such tht P( (X p, v) > t) M t 2+α for ll t > nd ny unit (in the l 2 -norm) vector v R p, (1.1) *Supported by RNF grnt from the Russin Scientific Fund. Steklov Mthemticl Institute, Russi. E-mil: yskov@mi.rs.ru

2 Shrp lower bounds on the lest singulr vlue we derive the optiml vlues of nd b in this pper. The pper is orgnized s follows. Section 2 contins the min results of the pper. Section 3 dels with the proofs. An Appendix with proofs of uxiliry results is given in Section 4. 2 Min results Our min lower bound is corollry of Theorem 2.1 in [6]. It is given below. Theorem 2.1. Let C 1 nd n > p 1. If (1.1) holds for M = C α/2 nd some α (, 2], then, with probbility t lest 1 e p, K α (Cy) α/(2+α), α (, 2) s p (n 1/2 X pn) 1 14 Cy log(c/y) α = 2 nd C/y > e Cy, α = 2 nd C/y e where y = p/n nd K α = 1/(α(1 α/2)) 2/(2+α). The next theorem contins our min upper bound for clss of rndom vectors X p = ηz p for Z p = (z 1,..., z p ) with i.i.d. entries {z i } p i=1 independent of η. (2.1) Theorem 2.2. Let (2.1) hold for ech p 1, where {z i } i=1 re independent copies of rndom vrible z with Ez =, Ez 2 = 1, nd η is rndom vrible with Eη 2 = 1. If there exist α (, 2] nd C > such tht then, for ech smll enough y >, P( η > t) Cα/2 t 2+α for ll lrge enough t >, (2.2) s p (n 1/2 X pn) 1 + o(1) 1 2 { K α (Cy) α/(2+α), α (, 2) Cy log(c/y), α = 2 lmost surely s n, where p = p(n) = yn + o(n) nd K α is given in Theorem 2.1. Theorem 2.2 nd the next proposition show tht, when y is smll enough, the lower bounds in Theorem 2.1 re symptoticlly optiml up to constnt fctor (equl to 14). Proposition 2.3. For ny given C > 1/4 nd α (, 2], there exists rndom vrible η such tht Eη 2 = 1, (2.2) holds, nd P( (X p, v) > t) (κc)α/2 t 2+α for ll t > nd ny unit vector v R p, where X p = ηz p, Z p is stndrd norml vector in R p tht is independent of η, nd κ > is universl constnt. The proof of Proposition 2.3 is given t the end of the pper, before the Appendix. 3 Proofs We will use below the following fct. By definition, s p (n 1/2 X pn) is the squre root of λ p (n 1 X pn X pn), where λ p (A) is the lest eigenvlue of p p mtrix A. In ddition, if 1 b for some, b, then 1 b. Moreover, if 1 b for some, b, then 1 b/2. Thus, to prove Theorems 2.1 nd 2.2 we need to derive pproprite lower nd upper bounds only for λ p (n 1 X pn X pn). ECP 2 (215), pper 44. Pge 2/9

3 Shrp lower bounds on the lest singulr vlue Proof of Theorem 2.1. By Theorem 2.1 in [6], for ll > nd y = p/n (, 1), λ p (n 1 X pn X pn) c p () C p() Cp (2)Z 5y +, n where Z = Z(p, n, ) is rndom vrible with EZ = nd P(Z < t) e t2 /2, t >, c p () = inf E min{(x p, v) 2, } nd C p () = sup E(X p, v) 2 min{(x p, v) 2, } with inf nd sup tken over ll unit vectors v R p. Since P ( Z < 2p ) e p nd y = p/n, we hve, with probbility t lest 1 e p, λ p (n 1 X pn X pn) c p () C p() 5y 2yC p (2). (3.1) To estimte c p () nd C p (), we will use the following lemm tht is proved in the Appendix. Lemm 3.1. Let >, X p be n isotropic rndom vector in R p, nd (1.1) hold for some M > nd α (, 2]. If α (, 2), then c p () 1 2M α α/2 nd C p () (2/α + 4/(2 α))m 1 α/2. In ddition, if α = 2, then c p () 1 M nd C p () 2M + M log( 2 /M) I( 2 > M). Tking First, ssume tht α (, 2). Using (1.1) nd Lemm 3.1, we get we hve In ddition, C p (2) 2 c p () C p() [ 4 1 α + 4 ] M 8M α/2 = 1 2 α α/2 α(2 α). [ ] 2My 1 2/(2+α) = = K α (M/y) 2/(2+α), α(2 α) y = 2M α/2 α(2 α) nd c p () C p() 1 4y. [ 2 α + 4 ] [ 4 M(2) α/2 2 α α + 4 ]M α/2 = 8M α/2 2 α α(2 α) = 4y nd 2yC p (2) 2y(8 2 y) = 4y = 4K α (M 2/α y) α/(2+α). Since C = M 2/α, we infer from (3.1) tht, with probbility t lest 1 e p, λ p (n 1 X pn X pn) 1 13y = 1 13K α (Cy) α/(2+α). Thus we get the desired lower bounds for α (, 2). Suppose now α = 2. Then M = C α/2 = C 1 nd log( 2 /C) log( 2 ) for ny >. Lemm 3.1 implies tht c p () C p() 1 3C + C log(2 )I( 2 > C). ECP 2 (215), pper 44. Pge 3/9

4 Shrp lower bounds on the lest singulr vlue Consider two possibilities log(c/y) > 1 nd log(c/y) 1. Assuming tht log(c/y) 1 nd tking = C/y, we hve 2 > C, log( 2 ) 1, nd 3C + C log( 2 ) 4C = 4 Cy. Additionlly, we get 5y = 5 Cy, C p (2) 2C + C log(4 2 ) (3 + log 4)C 9C/2 nd 2yC p (2) 3 Cy. As result, we conclude from (3.1) tht, with probbility t lest 1 e p, λ p (n 1 X pn X pn) 1 12 Cy. Suppose log(c/y) > 1. Set = (C/y) log(c/y). Then 2 > C, C/y C/y, nd 3C + C log( 2 ) 3C + C log(c/y)2 C/y 5 Cy log(c/y). Similrly, C p (2) 2C + C log(4 2 ) 7C/2 + C log( 2 ) (7/2 + 2)C log(c/y) nd 2yC p (2) 4 Cy log(c/y). Noting tht 5y = 5 Cy log(c/y), we infer tht, with probbility t lest 1 e p, Thus we hve proved the theorem. λ p (n 1 X pn X pn) 1 14 Cy. Proof of Theorem 2.2. We will use the following lemm (for the proof, see the Appendix). Lemm 3.2. Under the conditions of Theorem 2.2, λ p (n 1 X pn X pn) mx{, sup λ(s)} + o(1).s., n, (3.2) s> where p = p(n), p/n y (, 1), nd λ(s) = y/s + Eη 2 /(1 + sη 2 ). We estimte λ = λ(s) given in Lemm 3.2 s follows. Set ζ = η 2. Since Eζ = 1, λ(s) + y s = E ζ ( 1 + sζ = 1 + E ζ ) 1 + sζ ζ = 1 E sζ2 1 + sζ. It follows from the inequlity x/(1 + x) min{x, 1}/2, x, nd (4.1) tht E sζ2 1 + sζ 1 2 Eζ min{sζ, 1} = 1 2s [E(sζ 1)I(sζ > 1) + E min{(sζ)2, 1}]. As result, for ll s >, we get the following upper bound λ(s) 1 y s 1 2s [E(sζ 1)I(sζ > 1) + E min{(sζ)2, 1}]. (3.3) Recll lso tht, by (2.2) nd the definition of ζ (= η 2 ), there exists t 1 such tht P(ζ > t) Cα/2 t 1+α/2 for ll t t. (3.4) ECP 2 (215), pper 44. Pge 4/9

5 Shrp lower bounds on the lest singulr vlue As in the proof of Lemm 3.2 (see the Appendix), we get tht λ (s) = (y h(s))/s 2, s >, where h(s) = E(sζ) 2 /(1 + sζ) 2 is continuous strictly incresing function on R + with h() = nd h( ) = P(ζ > ) >. Hence, if y < P(ζ > ), λ(s) chieves its mximum in s = b with b = h 1 (y). Let α (, 2) nd tke y smll enough to mke b = h 1 (y) 1/(2 1/(1 α/2) t ). Then 1/b > t nd, by (3.4), E(bζ 1)I(bζ > 1) = 1 P(bζ > t) dt Moreover, (1/b) 1 α/2 /2 > t 1 α/2 nd, by (3.4), 1 1/b E min{(bζ) 2, 1} = P((bζ) 2 > t) dt = 2b 2 2b 2 1/b t C α/2 1 C α/2 (t/b) 1+α/2 dt = 2 α (Cb)α/2 b. zp(ζ > z) dz z dz = α/2 2Cα/2 b 2 (1/b)1 α/2 t 1 α/2 1 α/2 2C α/2 b 2 (1/b)1 α/2 /2 1 α/2 = (Cb)α/2 b 1 α/2. By (3.3), λ(b) g(b), where g(b) = 1 y/b Kb α/2 nd ( ) K = Cα/2 1 2 α/2 + 1 C α/2 = 1 α/2 α(1 α/2). By Young s inequlity, nd (K 2/α y) α 2+α = ( ) α y 2+α (Kb α/2 ) 2 2+α b λ(b) g(b) 1 (K 2/α y) α/(2+α). y/b (2 + α)/α + Kbα/2 (2 + α)/2 y b + Kbα/2 The right-hnd side of the lst inequlity cn be mde positive for smll enough y. Hence, combining the bove bounds with Lemm 3.2, we get the desired upper bound for λ p (n 1 X pn X pn) when α (, 2) (see lso the beginning of Section 3). Let now α = 2 nd tke y smll enough to mke b = h 1 (y) 1/t 2. Since t 1, we hve 1/b t 2 t nd, hence, the sme rguments s bove yield C E(bζ 1)I(bζ > 1) = P(bζ > t) dt 1 1 (t/b) 2 dt = Cb2, E min{(bζ) 2, 1} 2b 2 1/b Therefore, it follows from (3.3) tht λ(b) g(b), where Differentiting g yields t g (s) = y s 2 C 2 C z dz = 2Cb2 log 1 2Cb 2 log 1 = Cb 2 log(1/b). bt b g(s) = 1 y s Cs (log(1/s) + 1), s >. 2 Cs 1 (log(1/s) + 1) + 2 s = 2y Cs2 log(1/s) 2s 2. ECP 2 (215), pper 44. Pge 5/9

6 Shrp lower bounds on the lest singulr vlue If 2y/C is smll enough, then g = g(s) hs unique locl mximum in s 1 nd unique locl minimum in s 2, where s 1 < s 2, nd s 1, s 2 re solutions to the eqution f(s) = 2y/C with f(s) = s 2 log(1/s). The function f = f(s) is incresing on [, 1/ e], decresing on [1/ e, ] nd hs f() = f(1) =. Hence, s 2 > 1/2 nd b = h 1 (y) < 1/2 when y is smll enough. Thus, λ(b) g(b) 1 y Cs 1 s 1 2 (log(1/s 1) + 1) 1 y Cs2 1 log(1/s 1 ) = 1 2y. s 1 2s 1 s 1 Let us bound s 1 from bove. Tke s = (4y/C)/ log(c/y). If y is smll enough, then s < 1/ e s well s s 2 log(1/s ) = 4y/C [ 1 log(c/y) 2 log(c/y) + 1 ( 1 )] 2 log 4 log(c/y) = 2y 4 2y log log C/y + > 2y C C log(c/y) C. Therefore, s 1 < s nd λ(b) 1 2y s 1 1 2y s = 1 Cy log(c/y). The right-hnd side of the lst inequlity cn be mde positive for smll enough y. Hence, combining the bove bounds with Lemm 3.2, we get the desired upper bound for λ p (n 1 X pn X pn) in the cse with α = 2 (see lso the beginning of Section 3). Proof of Proposition 2.3. Let t = (1 + 2/α) 1 nd q = C/t 1+2/α. If α (, 2], then q C inf (1 + α (,2] 2/α)1+2/α = 4C > 1. Let η = ξζ, where ξ nd ζ re independent rndom vribles, ζ hs the Preto distribution P(ξ = q) = q 1 nd P(ξ = ) = 1 q 1, P(ζ > t) = { (t /t) 1+α/2, t t, 1, t < t. It is esy to see tht Eξ = 1. Moreover, P(ζ > t) (t /t) 1+α/2 for ll t > nd Eζ = P(ζ > t) dt = t + (t /t) 1+α/2 dt = t + 2t t α = 1. Hence, Eη 2 = Eξ Eζ = 1. In ddition, (2.2) holds since, for ll lrge enough t >, P( η > t) = q 1 P(ζ > t 2 /q) = q 1 (qt /t 2 ) 1+α/2 = qα/2 t 1+α/2 t 2+α We lso hve = Cα/2 t 2+α. (X p, v) = ξζ (Z p, v) d = ξζ Z for ll unit vectors v R p, where Z N (, 1) is independent of (ξ, ζ), «d =» mens equlity in lw. Hence, if t >, P( ξζ Z > t) = EP(sζ > t 2 ) s=ξz 2 E ( st /t 2) 1+α/2 I(s > ) s=ξz 2 E(t ξz 2 ) 1+α/2 t 2+α = t1+α/2 q α/2 E Z 2+α t 2+α = Cα/2 E Z 2+α t 2+α (κc)α/2 t 2+α, ECP 2 (215), pper 44. Pge 6/9

7 Shrp lower bounds on the lest singulr vlue where κ = sup (E Z 2+α ) 2/α. α (,2] Let us show tht κ <. If Z N (, 1), then f(α) = E Z 2+α = 2 2+α 2 Γ ( 3 + α ) π 2 is smooth function on [, 2] with f() = 1 nd, in prticulr, f () exists nd is finite. The function g(α) = f(α) 2/α is continuous on (, 2] nd g(α) = (1 + f ()α + o(α)) 2/α exp{2f ()}, α +. As result, κ = sup{g(α) : α (, 2]} is finite. This finishes the proof of the proposition. 4 Appendix Proof of Lemm 3.1. If U is non-negtive rndom vrible with EU = 1, then E min{u, } = P(U > t) dt = EU P(U > t) dt 1 M dt = 1 2M t1+α/2 α, α/2 where M = sup{t 1+α/2 P(U > t) : t > }. Putting U = (X p, v) 2 for given unit vector v R p nd tking the infimum over such v, we obtin the desired lower bound for c p (). Similrly, we hve where I 1 = EU min{u, } =E(U )I(U > ) + 2 P(U > ) + EU 2 I(U ) =E(U )I(U > ) + E min{u 2, 2 } =I 1 + I 2, (4.1) M P(U > t) dt t dt = 2M 2 1+α/2 α 1 α/2, I 2 = P(U 2 > t) dt. If α (, 2), then I 2 cn be bounded s follows Similrly, if α = 2, then I 2 2 Mdt M1 α/2 = t1/2+α/4 1/2 α/4. 2 I 2 M + I( 2 Mdt > M) = M + M log( 2 /M)I( 2 > M). M t Thus, we hve proved tht EU min{u, } M { (2/α + 4/(2 α)) 1 α/2, α (, 2), 2 + log( 2 /M)I( 2 > M), α = 2. Putting U = (X p, v) 2 for given unit vector v R p nd tking the supremum over such v, we get the desired upper bound for C p (). ECP 2 (215), pper 44. Pge 7/9

8 Shrp lower bounds on the lest singulr vlue Proof of Lemm 3.2. We hve n 1 X pn X pn = n 1 Z pn T n Z pn, where Z pn is p n mtrix with i.i.d. entries, T n is n n digonl mtrix whose digonl entries re independent copies of ζ = η 2, nd Z pn is independent of T n. By the Glivenko-Cntelli theorem, the empiricl spectrl distribution of T n converges.s. to the distribution of ζ. By Theorem 4.3 in [1], there is non-decresing cádlág function F = F (λ), λ R, such tht F (λ) = for λ <, F ( ) 1, nd ( P lim n 1 p p k=1 ) I(λ kn λ) = F (λ) = 1 for ll continuity points λ of F, (4.2) where p = p(n) = yn + o(n) nd {λ kn } p k=1 is the set of eigenvlues of p 1 X pn X pn. The Stieltjes trnsform F (dλ) f(z) = R λ z, z C+ = {w C : Iz > }, (4.3) of F cn be defined explicitly s unique solution in C + to the eqution ( f(z) = z 1 ) 1 y E ζ or, equivlently, z = f(z)ζ f(z) + 1 y E ζ 1 + f(z)ζ. (4.4) Define S G = {λ : G(λ + ε) > G(λ ε) for ny smll enough ε > } for non-decresing cádlág function G = G(λ), λ R. In other words, S G is the set of points of increse of G. Obviously, S G is closed set. Using (4.2) nd setting G = F s well s = inf{λ : λ S F }, we conclude tht S F nd when n. Consider the function λ p (n 1 X pn X pn) = p n λ p(p 1 X pn X pn) y + o(1).s. (4.5) z(s) = 1 s + 1 y E ζ 1 + sζ defined for s D, where D consists of ll s R\{} with s 1 / S G for G(λ) = P(ζ λ), λ R. This function differs from λ = λ(s) given in Lemm 3.2 by the fctor y, i.e. λ(s) = yz(s) for ll s >. Therefore, to finish the proof, we only need to show tht = mx{, sup z(s)}. s> Let us show tht = when z(s) for ll s >. The ltter cn be reformulted s follows: if >, then there is s > stisfying z(s) >. Suppose >. Then /2 R \ S F nd F (/2) =. Hence, F (dλ) f(/2) = > nd lim f(/2 + iε) = f(/2) >. R λ /2 ε + Tking z = /2 + iε in (4.4) nd tending ε to zero, we get /2 = z(s) > for s = f(/2). Assume further tht there is s > stisfying z(s) > or, equivlently, g(s) = E sζ 1 + sζ > y. ECP 2 (215), pper 44. Pge 8/9

9 Shrp lower bounds on the lest singulr vlue The function g = g(s) is continuous nd strictly incresing on R +. It chnges from zero to P(ζ > ) when s chnges from zero to infinity. The sme cn be sid bout h(s) = E (sζ)2 (1 + sζ) 2. Hence, y < P(ζ > ) nd there is b = b(y) > tht solves h(b) = y. By the Lebesgue dominted convergence theorem, z (s) = 1 s 2 1 y E ζ 2 (1 + sζ) 2 = y h(s) ys 2 for ny s >. Therefore, b is strict globl mximum point of z = z(s) on {s : s > }. The rest of the proof is bsed on Lemm 6.1 in [1] which sttes tht z (s) > nd s D if s = f(λ) for some λ R \ S F. Moreover, {z(s) : s D, z (s) > } R \ S F. We will now prove tht z(b). Suppose the contrry, i.e. > z(b). By definition, F (λ) = for ll λ <. Set z = z(b). Then z R \ S F, s = f(z ) = R F (dλ) λ z >, nd, by the bove lemm, z (s ) >. Tking z = z + iε in (4.4) nd tending ε to zero, we rrive t z(b) = z = z(f(z )) = z(s ). Since z (s ) > nd s >, we get the contrdiction to the fct tht b is strict globl mximum point of z = z(s) on {s : s > }. Let us finlly prove tht z(b). The function z = z(s) is continuous nd strictly incresing on the set (, b) with z(+) = nd z(b ) = z(b). By the bove lemm, Thus, z(b). This finishes the proof. References z((, b)) = (, z(b)) R \ S F. [1] Bi, Zh. nd Silverstein, J.: Spectrl nlysis of lrge dimensionl rndom mtrices. Second Edition. New York: Springer, (21). MR [2] Koltchinskii, V. nd Mendelson, S.: Bounding the smllest singulr vlue of rndom mtrix without concentrtion. rxiv: [3] Oliveir, R.I.: The lower til of rndom qudrtic forms, with pplictions to ordinry lest squres nd restricted eigenvlue properties. rxiv: [4] Srivstv, N. nd Vershynin, R.: Covrince estimtion for distributions with 2 + ε moments. Ann. Probb., 41, (213), MR [5] Tikhomirov, K.: The limit of the smllest singulr vlue of rndom mtrices with i.i.d. entries. rxiv: [6] Yskov, P.: Lower bounds on the smllest eigenvlue of smple covrince mtrix. Electron. Commun. Prob., 19 (214), pper 83, 1-1. MR ECP 2 (215), pper 44. Pge 9/9

10 Electronic Journl of Probbility Electronic Communictions in Probbility Advntges of publishing in EJP-ECP Very high stndrds Free for uthors, free for reders Quick publiction (no bcklog) Economicl model of EJP-ECP Low cost, bsed on free softwre (OJS 1 ) Non profit, sponsored by IMS 2, BS 3, PKP 4 Purely electronic nd secure (LOCKSS 5 ) Help keep the journl free nd vigorous Donte to the IMS open ccess fund 6 (click here to donte!) Submit your best rticles to EJP-ECP Choose EJP-ECP over for-profit journls 1 OJS: Open Journl Systems 2 IMS: Institute of Mthemticl Sttistics 3 BS: Bernoulli Society 4 PK: Public Knowledge Project 5 LOCKSS: Lots of Copies Keep Stuff Sfe 6 IMS Open Access Fund: