Convergence rates of spectral distributions of large sample covariance matrices

Transcription

1 Title Convergence rates of spectral distributions of large sample covariance matrices Authors) Bai, ZD; Miao, B; Yao, JF Citation SIAM Journal On Matrix Analysis And Applications, 2004, v. 25 n., p Issued Date 2004 URL Rights This work is licensed under a Creative Commons Attribution- NonCommercial-NoDerivatives 4.0 International License.

2 SIAM J. MATRIX ANAL. APPL. Vol. 25, No., pp c 2003 Society for Industrial and Applied Mathematics CONVERGENCE RATES OF SPECTRAL DISTRIBUTIONS OF LARGE SAMPLE COVARIANCE MATRICES Z. D. BAI, BAIQI MIAO, AND JIAN-FENG YAO Abstract. In this paper, we improve known results on the convergence rates of spectral distributions of large-dimensional sample covariance matrices of size p n. Using the Stieltjes transform, we first prove that the expected spectral distribution converges to the limiting Marčenko Pastur distribution with the dimension sample size ratio y = y n = p/n at a rate of On /2 )ify keeps away from 0 and, under the assumption that the entries have a finite eighth moment. Furthermore, the rates for both the convergence in probability and the almost sure convergence are shown to be O pn 2/5 ) and o a.s.n 2/5+η ), respectively, when y is away from. It is interesting that the rate in all senses is On /8 ) when y is close to. Key words. convergence rate, random matrix, spectral distribution, Marčenko Pastur distribution AMS subject classifications. Primary 60F5; Secondary 62H99 PII. S Introduction. The spectral analysis of large-dimensional random matrices has been actively developed in the last decades since the initial contributions of Wigner 955, 958); also see the recent reviewby Bai 999) and the book by Mehta 99). Various limiting distributions were discovered including the Wigner semicircular law Wigner, 955), the Marčenko Pastur lawmarčenko and Pastur, 967), the limiting lawfor multivariate F matrices Bai, Yin, and Krishnaiah 987) and Silverstein 985)) and the circular lawbai and Yin 986), Bai 997)). The spectrum separation problem for large-dimensional sample covariance matrices was investigated in Bai and Silverstein 998, 999). Let A be an n n symmetric matrix, and λ λ n be the eigenvalues of A. The spectral distribution F A of A is defined as F A x) = n number of elements in {k : λ k x}. Let X p =x ) p n be a p n observation matrix whose entries are mutually independent and have a common mean zero and variance. The entries of X p may depend on n but we suppress the index n for simplicity. In this paper, we consider the sample covariance matrix S = n X p X T p, where X T denotes the transpose of the matrix X. Assume that the ratio p/n of sizes tends to a positive limit y as n. Under suitable moment conditions on the x entries, it is known that the empirical spectral distribution ESD) F p := F S converges to the Marčenko Pastur distribution F y with Received by the editors February 5, 200; accepted for publication in revised form) by A. Edelman November 6, 2002; published electronically May 5, Department of Mathematics, Northeast Normal University, Changchun, China, and Department of Statistics and Applied Probability, National University of Singapore, Singapore, 7543 stabaizd@leonis.nus.edu.sg). The research of this author was partially supported by NSFC grant as well as by the NUS Research grant R Department of Statistics and Finance, University Science and Technology of China, Hefei, Anhui, China bqmiao@ustc.edu.cn). The research of this author was supported in part by the National Foundation of Natural Science of China. IRMAR, Université de Rennes, Campus de Beaulieu, F Rennes Cedex, France jian-feng. yao@univ-rennes.fr). 05

3 06 Z. D. BAI, BAIQI MIAO, AND JIAN-FENG YAO index y with density F yx) = 2πxy x a)b x) if a<x<b, 0 otherwise, where a = y) 2, b =+ y) 2. An important question here concerns the problem of the convergence rates. However, no significant progress was made before the introduction of a novel and powerful tool, namely, the Berry Esseen inequalities in terms of Stieltjes transforms, by Bai 993a, 993b). Using this methodology, Bai 993b) proved that the expected ESD EF p converges to F yn at a rate of On /4 )oron 5/48 ) depending on whether y n is far away or close to, respectively, where y n = p/n. In another work by Bai, Miao, and Tsay 997), these rates are also established for the convergence in probability of the ESD F p itself. In later works of Bai, Miao, and Tsay 999, 2002), the convergence rates for large Wigner matrices are significantly improved. In this work, we further investigate the convergence rates for empirical spectral distributions for large sample covariance matrices and improve those results in the theorems to follow. The following conditions will be used: C.) Ex =0, Ex 2 =, i p, j n. C.2) sup i,j,n E x 8 <. C.3) For any positive constant δ, Ex 8 I x δ n) = on 2 ). It is easy to see that condition C.3) guarantees that there is a sequence {δ = δ n 0} such that.) Ex 8 I x δ n) = on 2 δ 8 ). C.2 ) sup i,j,n E x k < for any integer k. Throughout the paper, we use the notation Z n = O p a n ) if the sequence a n Z n ) is tight and use Z n = o p a n ) when a n Z n tends to 0 in probability. We shall also set f = sup x fx). For simplicity, from nowon we drop the index n from y and use the notation y = y n = p/n. Finally, let us define.2) θ = θn, y) = 2log n y) +4 log n y) if y n /8 ) 2, 2 otherwise. We nowintroduce the main results of the paper. Theorem.. Assume that the conditions C.) C.3) are satisfied. Then, n /[4θ+2] ) EF p F y = O [, y + n /[8θ+4] ] Theorem.2. Assume that the conditions C.) C.3) are satisfied. Then, { n F p F y = O p max 2/5+θ)) [ y + n /5+θ)) ], n /[4θ+2] }) [. y + n /[8θ+4] ]

4 CONVERGENCE RATES OF SPECTRAL DISTRIBUTIONS 07 Theorem.3. Assume that the conditions C.) C.3) are satisfied. Then, with probability, { n 2/5+θ))+η F p F y = o max [ y + n /5+θ)) ], n /[4θ+2] }) [. y + n /[8θ+4] ] Remark on the convergence rates. If y is not close to, then θ c/ log n, and hence the convergence rates in the above three theorems are On /2 ), O p n 2/5 ), and o a.s. n 2/5+η ), respectively. When y> On /8 ), θ =/2, and hence the rates of the three theorems are On /8 ), O p n /8 ), and O a.s. n /8 ), respectively. When y goes to with intermediate rates, we may have intermediate convergence rates. It is worth noticing that the convergence rates given above for the case 0 <y also apply to the case y>, since the last case can be reduced to the first case by interchanging the roles of rowand column sizes p and n. The proofs of these main results will be given in section 3. For convenience, we first introduce some necessary notation and preliminary consequences in section 2. Some necessary lemmas are postponed to section Definitions and easy consequences. Throughout the paper, the transpose of a possibly complex matrix A is denoted by A T and its conjugate by A. For each fixed p, n, and k =,...,p, let us denote by x k =x k,...,x kn ) T the kth rowof X p arranged as a column vector, and let X p k) be the p ) n submatrix obtained from X p by deleting its kth row. Let us define 2.) α k := n X pk)x k, S k := n X pk)x T p k), B k := n XT p k)d k X p k), B := n XT p DX p, D k := S k zi p ), D := S zi p ), Γ k := D k D k, Λ k := D k S k D k. Here I m is the m-dimensional identity matrix and z a complex number with a positive imaginary part. Following Bai 993b), the Stieltjes transform of the spectral distribution F p of the sample covariance matrix S is defined for z = u + iv with v>0by m p z) = x z df px), and it is well known that m p z) = p trs zi p). Similarly, the Stieltjes transform of the spectral distribution F p k) satisfies m k) p z) = k) df p x) = x z p trs k zi p ). of the submatrix S k Finally, the Stieltjes transform of the limiting by noting that y = y n ) Marčenko Pastur distribution F y is mz) = x z df yx) = y + z y z)2 4yz 2.2) 2yz

5 08 Z. D. BAI, BAIQI MIAO, AND JIAN-FENG YAO for 0 <y. Here the square root z is the one with a positive imaginary part. Note that mz) is a root of the quadratic equation yzm 2 +y + z )m +=0, which implies that mz)m z) = yz, where m z) = y + z + + y z) 2 4y 2yz is the other root of the equation. We claim that 2.3) To see this, set We have 2.4) 2.5) mz) < m z) for all z = u + iv, v > 0. α + iβ = + y z) 2 4y with β 0. αβ = vu y ), β 2 α 2 =b u)u a)+v 2. First, note that β = 0 is impossible; otherwise we should have u =+y and 2.5) would be violated. Hence, β>0 and α 0 if and only if u +y. It is easy to see that mz) < m z) y + z α + iβ) < y + z +α + iβ) αy +u)+βv > 0. The last inequality clearly holds if u + y) oru y in this case the result was proved in Bai 993b, p. 65)). Now assume for a u y, +y) [a, b] that the inequality does not hold, i.e., αy +u) +βv 0. This implies that βv α [u y)] noting that α<0). Multiplying both sides by β and using 2.4), we get β 2 [u y)] [ + y u] b u)u a), which contradicts 2.5). The claim 2.3) is then proved. This claim implies that mz) / y z for any z. On the other hand, when u<a v, both real and imaginary parts of mz) are positive and increasing a consequence of the integral formula 2.2) of mz)). Thus, mz) can only reach its maximum when u>a v. When a<2v, we have mz) / 2 yv 2 y a+ v) = 2 2v y yv. When a 2v, by noticing that z 4 a 2 /4+v a + v), we obtain the same bound as in the first case. Therefore, we obtain 2.6) where mz) 2 2v y yv, 2.7) v y := v/[ a + v]=v/[ y + v].

6 CONVERGENCE RATES OF SPECTRAL DISTRIBUTIONS 09 Lemma 2.. Let x =x,...,x n ) T and y =y,...,y n ) T be independent real random vectors with independent elements. Suppose that for all j n, Ex j = Ey j =0, E x j 2 = E y j 2 =, E x j 4 L< and that A is an n n complex symmetric matrix. Let µ k = max j n E x j k, E y j k ). Then i) E x T Ay 2 = traā); ii) E x T Ax 2 LtrAĀ)+ tra 2 ; iii) E x T Ax tra 2 LtrAĀ); [ iv) E x T Ax tra 2k d k µ4k traā)k +LtrAĀ))k] for k 2 and some positive constant d k depending on k only. The proofs of i) iii) are elementary and therefore omitted. The statement iv) follows from Lemma 2.7 of Bai and Silverstein 998). Lemma 2.2. Let G and G 2 be probability distribution functions and z = u + iv, v>0. Then for each positive integer m, x z m dg x) G 2 x)) 2 v m G G 2. Proof. Let be G := G G 2. We have, by integration by parts, x z m dx)g [ ] = G x)d x z m Rez) [ ] = G [ ] x)d x z m + G x)d Rez) x z m { Rez) [ ] G [ ] } d x z m + d Rez) x z m { Rez) )} = G x z m + x z m = G 2 v m. Rez) We will need the following auxiliary variables: ε k = n x 2 kj ) + n n x k T B k x k EtrB), ε k = n j= n x 2 kj ) + n x k T B k x k trb k ), j= ε k = n trb k EtrB k )= z n trd k EtrD k ), π k = n EtrB k trb) = z n EtrD k trd), β k = n n x 2 kj ) + z + n x k T B k x k, j= β k = z + n trb k, β = z + n trb.

7 0 Z. D. BAI, BAIQI MIAO, AND JIAN-FENG YAO We summarize below some inequalities which will be used in the derivations. Let = EF p F y and M := sup i,j,n E x 4. For fixed n, p) and k p, we define the σ-algebra F k) = σx i : i p, i k), F k = σx i : k<i p).. from Lemma 3.3 of Bai 993a)): 2.8) 2. from Lemma 2.2 and 2.8)): 2.9) trd trd k = p )F k) p x) pf p x). 3. from equation 3.4) of Bai 993b)): 2.0) m p z) = 0 d[pf p x) p )F p k) x)] x z 2v. x z df px) = p trd = p 4. from Lemma 2.2 of Bai, Miao, and Tsay 997)): 2.) E m p z) Em p z)) 2 p v from β k Imβ k )=v + n trλ k )): ) 2.3) 2.4) β k + n trλ k ) v. k= β k Imβ k )=v + n ) αtk D k D k α k. + n αt k D 2 k α k + n αt k D k D k α k. Let λ kj, j =, 2,...,p, be the eigenvalues of S k which can be decomposed into a diagonal form on the basis of orthonormal and real eigenvectors. Let L be a complex matrix having the product form L = M l N l for some integers l, l and factors M, N equal to one of the matrices {D k, D k, S k }. An important feature that we will frequently use in what follows is that such a matrix L can be decomposed into a diagonal form on the same basis as the eigenvectors of S k. Moreover, the eigenvalues of L can be straightforwardly expressed in terms of the λ kj s. In particular, we have the following. Lemma 2.3. Assume that z T, where T. Then for all integers l, 2.5) 2.6) ) l trγ k ) l trγ k, trλ k ) l v 2 ) l T v 2 trλ k. β k.

8 CONVERGENCE RATES OF SPECTRAL DISTRIBUTIONS Proof. i) The inequality 2.5) follows from trγ k ) l = p j= λ kj z 2l v 2l ) ii) For the inequality 2.6), we have trλ k ) l = p j= p λ kj z 2 = v 2l ) trγ k. j= λ l kj λ kj z 2l. The conclusion follows from the fact that the function ϕλ) :=λ λ z 2 defined on 0, ) is convex and has a unique minimum value ϕ satisfying v 2 ϕ =2 u 2 + v 2 u) =2 z + u v2 T. Lemma 2.4. For the Marčenko Pastur distribution F y, we have b 2.7) a x z 2 df yx) 2 y v yv 2. 3/4 Proof. Since for x [a, b], x a)b x) xb a) =4x y, we have for any z, b b a x z 2 df yx) = x a)b x) dx a x z 2 2πxy b πy dx. 3/4 a x x z 2 The maximum of b a / x x z 2 ) dx can only be attained when u a. We then have b a x z 2 df yx) πy dx 3/4 0 x x z 2 = y 3/4 z 2 z 2u y 3/4 z /2 v. From this and by noticing that z /2 2 a + v) when u>a, the lemma is proved. 3. Proofs. We first truncate and centralize the random variables so that all random variables could be further considered as bounded up to some order of n). In the subsection 3.2, we introduce a Bai inequality for the proofs of the main theorems. These proofs are then given in subsequent sections. 3.. Truncation and centralization. Define ˆx = x I x δ n) and x =ˆx Eˆx ))/σ, where σ 2 = Eˆx Eˆx )) 2. Here δ = δ n is chosen such that δ n 0 with a slow rate and such that.) holds. We remind the reader that all the above variables depend on n, but the index is suppressed. Define p n matrices ˆX =ˆx ) and X = x ) and define p p matrices Ŝ = ˆX ˆX T n and S = X X T n. Denote the ESDs of Ŝ and S by ˆF p and F p, respectively.

9 2 Z. D. BAI, BAIQI MIAO, AND JIAN-FENG YAO We first estimate the truncation error F p ˆF p. By.), P ) x δ n n i,j = EI ) x δ n n δn n) 8 E [ x 8 I )] x δ n n i,j i,j 3.) cn 2. Let α 0, ). By the Markov inequality, P I x δ n n) n α cn 2+α, i,j which, together with the Borel Cantelli lemma, implies that rankx ˆX) 3.2) I x δ n n)=on α ) a.s. i,j By Lemma 2.6 of Bai 999), we have 3.3) F p ˆF p = O/n +α ) a.s. The estimation 3.3) reduces the proofs to showthat the three theorems remain true when F p is replaced with ˆF p. Furthermore, recalling the proof of Lemma 2.7 of Bai 999), we find that ˆF p x) F p x) dx = ˆλ k p λ k k= np tr ˆX X) ˆX X) T 2 ) /2 3.4) np tr ˆX ˆX T + X X T ), where ˆλ k and λ k, arranged in increasing order, are the eigenvalues of Ŝ and S, respectively. Under the uniform boundedness of the fourth moments of the entries, it is easy to showthat np tr ˆX ˆX T 3.5) x 2 a.s. np Also, 3.6) max Furthermore, by 3.5) and 3.6), ) /2 T tr X X = x 2 np np x 2 npmin σ ) σ 2 min σ 2. /2 /2 + /2 E x 2 I x δ n) 3.7) a.s.

10 CONVERGENCE RATES OF SPECTRAL DISTRIBUTIONS 3 3.8) Note that np tr ˆX X) ˆX X) T 2 np [ x 2 max /σ 2 + E 2 { x I x δ ] n)} = On 6 )a.s. Here, the convergence rates followfrom the facts that i) np x 2 a.s. ii) max /σ 2 max E 2 { x 2 I x δ n)} = Oδ 2 n 6 ). iii) E 2 { x I x δ n)} = Oδ 6 n 7 ). np It follows from 3.4) 3.8) that under conditions C.) C.3), 3.9) ˆF p x) F p x) dx = On 3 )a.s. Using Lemma 2.5 of Bai 993b), the proofs of the three theorems reduce to show that the main theorem remains true when ˆF p is replaced with F p. Note that the random variables x still satisfy the conditions C.) C.3). They also satisfy the additional condition x δ n here, the constant δ should be 3δ if δ is the one we previously selected. For brevity, we still use δ). Also, for simplicity, we shall drop the tilde sign from various variables The Bai inequality. Suppose that G is a function of bounded variation. The Stieltjes transform g of G is defined as gz) = x z dgx), where z = u + iv and v>0. Our main tool is the following inequality Bai 993a)). Proposition 3.. Let G be a distribution function and H be a function of bounded variation satisfying Gx) Hx) dx <. Denote their Stieltjes transforms by gz) and hz), respectively. Then [ A G H gz) hz) du + 2π Gx) Hx) dx π κ)2γ ) A v x >B + ] v sup Hx + y) Hx) dy, x y 2va

11 4 Z. D. BAI, BAIQI MIAO, AND JIAN-FENG YAO where the constants A>B, γ, and a are restricted by γ = π u a u 2 + du > 2, and κ = 4B 0, ). πa B)2γ ) Denote the Stieltjes transform of F p and F y recall our convention that y = y n = p/n) bym p z) and mz), respectively. Application of the Bai inequality with G, H) =F p,f y ), A = 25, and B = 5 gives, for some constant c>0, 3.0) [ A F p F y c m p z) mz) du + F p x) ex) dx A v x >5 + ] v sup F y x + u) F y x) du, x u 2va where ex) = for x>0 or ex) = 0 otherwise. We shall estimate these three terms in the above bound successively and start with the last one. a) Estimate for sup x u 2va F y x + u) F y x) du. Lemma 3.. We have, for any 0 <v<4 y, sup x u v F y x + u) F y x) du 2 + y) vv y, 3πy where v y = v/ a + v) is defined as in 2.7). Proof. It is enough to consider the part 0 u v in the integral only since the remaining part for v u 0 can be handled in a similar way. Set Φλ) := v 0 [F yx+ u) F y x)]du with x = a+λ; we are estimating the maximum of Φλ). Without loss of generality, we need only consider the case that λ 0 because v 0 [F yx+u) F y x)]du increases when x a. Then 3.) Φλ) = = = v du 0 x a+λ+v a+λ λ+v λ x+u F y t) dt a + λ + v t t a)b t)i[a,b] t)dt 2πyt u4 y u)i [0,b a] u) du. λ + v u 2πyu + a) Let φu) :=u + a) u4 y u). The derivative of logφu)) 2 is u 4 y u 2 u + a = 22 ya + y)u) u4 y u)u + a). Note that the above equality holds also for y = for which a = 0. Let ρ := + y) 2a y). Thus φu) is decreasing when u>ρand increasing when u<ρ. Since ) dφλ) dλ = λ+v [φu) φλ)] du, 2πy λ

12 CONVERGENCE RATES OF SPECTRAL DISTRIBUTIONS 5 it follows that for λ>ρ,φλ) is decreasing and then Φλ) Φρ); for λ<ρ v, Φλ) is increasing and then Φλ) Φρ v). Hence, Φλ) reaches its maximum only for some λ maxρ v, 0),ρ). Considering such a λ yields by 3.) Since Φλ) 2y/4 2πy λ+v λ =2πy 3/4 ) {λ + v + a) λ + v u udu u + a [ λ + v λ) )] λ + v λ a arctan arctan [ λ + v) 3/2 λ 3/2]}. a a 3 λ + v a arctan arctan a we get, by setting λ = λ + v λ, { a + λ + v) λ 3.2) Φλ) 2 πy 3/4 = 2 πy 3/4 [ λλ ] 3 λ 3. Let c 2 = +y 2 y. Since λ + v c 2 a and ) λ a a λ ) + v λ, λ + v + a ) a a + λ + v λ λ λ + λλ + 3 )} λ 2 λ + v + λ) 2 λ + v +2 λλ + v) 2 λv +2 λc 2 a, we have λ λ + v + λ) c 2 2 a +2c v c 2 a +2 v, λ + v + λ) 2c 3 a + v)v, where the last inequality follows from Hence Φλ) 2 πy 3/4 λ + v + λ) 3 λ + vv [ c 2 a + vv] 2 2c [ v + a ] v. c 6 a + v) v2 = 2 + y) 6πy v + y) v 2. This completes the proof of the lemma.

13 6 Z. D. BAI, BAIQI MIAO, AND JIAN-FENG YAO b) Estimate for v x >5 F px) ex) dx. Let λ p denote the largest eigenvalue of S. By Yin, Bai, and Krishnaiah 988), for any positive constant ε) and integer l n ) such that l n / log n and l n δn /4 / log n 0, we have 3.3) Therefore, for x 5 and any fixed t>0, 3.4) Eλ p ) ln cb + ε) ln. ) ln ) 2 b + ε b + ε b + ε P λ p >x) c c x x 5 Since F p x) =ex) =0forx 0, we have F p x) ex) dx = 3.5) = x >5 5 By 3.5) we finally get 5 p P λ k >x) dx k= ox 2 n t ) dx = on t ). x >5 Thus, for v>cn,wehave v c) 3.6) x >5 5 5 ) ln 2 [ F p x)] dx P λ p >x) dx F p x) ex) dx = On 2 )a.s. F p x) ex) dx = O a.s. n )a.s. = ox 2 n t ). Conclusion. Summarizing previous steps gives [ ] A F p F y c m p z) mz) du + O a.s. n )+v y. A To prove the main theorems, we need only estimate m p z) mz). Bai 993b) has proved that = n,y = EF p F y = On 5/48 ). In what follows, we shall treat as at least of the order On 5/48 ) Proof of Theorem.. We begin by estimating Em p z) mz), with various choices of v, subject to cn /2 v for some c>0. With the formula of m p given in 2.0), let us define δ p such that 3.7) Since m p z) = z + y +yzem p z) + δ p = Eβ + δ p. β k = Eβ ε ) k, β k

14 CONVERGENCE RATES OF SPECTRAL DISTRIBUTIONS 7 it is easy to see that Now Eδ p p Eβ 2 = p Eβ 2 p Eβ 2 + δ p = p k= ) Eε k + E ε2 k k= k= ε k = Eβ β k Eβ) 2 p β k ε k p k= ε 2 k β k k= ). [ Eε k + ε k )+π k + Eβ Eε2 k Eβ) 2 Eε3 k + ε 4 Eβ) 2 E k [ Eε k + ε k )+π k + k= )] ε 4 Eβ) 2 E k k= β k = Eβ 2 [I 0 + I + I 2 + I 3 ]. Eβ Eε2 k + Eβ) 2 Eε3 k k= We will estimate each I i to obtain a bound on Eδ p cf. 3.9) below). Since Eε k + ε k ) = 0, by 2.9), we have I 0 = z p k= k= π k = z EtrD k EtrD z /nv). pn k= From Lemma 4.2, Remark 4., and noticing that v v y,wehave Now Since p I E ε k 2 = E ε p Eβ p Eβ k 2 + E ε k 2 + π k 2 ) I 2 = p Eβ 2 k= k= k= k= k= c [ Eβ n + +v ] y nv 2 + +v y n 2 v 4 + ) n 2 v 2 Eε 3 k p Eβ E ε k 2 + E ε k 4 27 p c + v y) Eβ nv 2, p Eβ 3 E ε k 4 ). E ε k 4 + E ε k 4 + π k 4 ) = ci 2 + I 22 + I 23 ). k= trb k B k = tri p + zd k )I p + zd k ) 2p + z 2 trd k D k ), We have from the proof of Lemma 4., E ε k 4 cn 2 { +n 2 EtrB k B k ) 2} cn 2 { +n 2 EtrD k D k ) 2}. β k ) ]

15 8 Z. D. BAI, BAIQI MIAO, AND JIAN-FENG YAO Now EtrD k D k )) 2 = v 2 EImtrD k ))) 2 2v 2 [v 2 + EImtrD))) 2 ] =2v 4 +2p 2 v 2 EImm p z))) 2 2v 4 +4p 2 v 2 Em p z) 2 +4p 2 v 2 E m p z) Em p z) 2 cp 2 v 4 + v y ) 2 + cv 6 + v y ) cp 2 v 4 + v y ) 2, where the second inequality follows from 2.9) and the last steps follow from Proposition 4. and 3.8) Em p z) Em p z) mz) + mz) v 2 + α y v y ), with α y := 2 2/ y see 2.6)). Thus I 2 c { n 2 + n 2 v 4 + v y ) 2} cn 2 v 4 + v y ) 2. Also, considering D k instead of D as in Proposition 4. and applying 2.8), one can showthat for some L 0 such that for all L 0 n /2 v<, I 22 c + v y ) 2 n 4 v 8. Since π k z nv),wehavei 23 z 4 nv) 4, and hence, p E ε k 4 ci 2 + I 22 + I 23 ) k= Consequently, for some constant c>0, and I 3 c[n 2 v 4 + v y ) 2 + +v y ) 2 n 4 v 8 +nv) 4 ] cn 2 v 4 + v y ) 2. pv Eβ 2 I 2 c + v y) Eβ nv 2 + c + v y) 2 Eβ 3 n 2 v 4 E ε k 4 c n 2 v 5 Eβ 2 + v y) 2. k= Summing up the above results, we obtain Eδ p Eβ 2 [I 0 + I + I 2 + I 3 ] c [ Eβ 2 nv + +v y nv 2 Eβ + + v y) 2 ] n 2 v 5 Eβ 2 c [ +vy Eβ 2 nv 2 Eβ + + v y) 2 ] 3.9) n 2 v 5 Eβ 2.

16 CONVERGENCE RATES OF SPECTRAL DISTRIBUTIONS 9 For the positive constant L 0 required by Proposition 4.) and all v [L 0 n /2, ), define ϕ n v) = sup Eδ p γv with γ =/[0A +) 2 ]. u A Checking the proofs of 3.39) 3.40) of Bai 993b), we find that there is a constant c such that ϕ n v) 0= A A Em p z) mz) du<cv. In viewof 3.6), we can then find a positive constant c such that 3.20) ϕ n v) 0= <c v y. The proof of the theorem will be complete once we have shown that for all large n and all v [Ln /[4θ+2], ), we have ϕ n v) 0, where L is a constant such that L L 0 and [ cm0 2 c +)M 0 L c ) 2 M0 2 ] L 4 <γ and M 0 = γ +2c + α y. Assume the contrary; i.e., there exists a v [Ln /[4θ+2], ) for which ϕ n v ) > 0. By continuity of ϕ n, there exists a v 0 [Ln /[4θ+2], ) for which ϕ n v 0 )=0. As [ A, A] is compact, there exists a u 0 [ A, A] such that 0 = ϕ n v 0 )= Eδ p u 0,v 0 ) γv 0. Let z 0 = u 0 + iv 0. By 3.7), 3.8), and Lemma 2.2, with z = z 0, 3.2) Eβ = Eδ pz 0 )+ E[m p z 0 ) mz 0 )] + mz 0 ) Eδ p z 0 ) α yv y,0 v 0 γ +2c + α y ) v y,0 v y,0 = M 0. v 0 v 0 On the other hand, by definition.2) of θ, wehave v = a + v a v n v y Therefore, for any v [Ln /[4θ+2], ) we have vy ) 4 /L 2 nv 2, v θ 4θ+2. vy ) 6 /L 4 n 2 v 4. v Thus, from 3.9) we have for z = z 0 so v = v 0 and v y = v y,0 ) Eδ p z 0 ) cm [ 0 2 vy 2 c +)M 0 vy 2 v 2 nv c ] ) 2 M0 2 vy 4 n 2 v 7 [ vcm0 2 c +)M 0 L c ) 2 M0 2 ] L 4 <γv by noticing the selection of L. This leads to a contradiction of ϕ n z 0 ) = 0. The proof of Theorem. is complete.

17 20 Z. D. BAI, BAIQI MIAO, AND JIAN-FENG YAO 3.4. Proof of Theorem.2. As the proof of 3.6), one can showthat E F p F y [ ] A c E m p z) mz) du + v y c A [ A A ] A E m p z) Em p z) du + Em p z) mz) du + v y. A In the proof of Theorem., we have shown that A A Em pz) mz) du = Ov) if Ln /[2+4θ] <v<. Applying the Cauchy Schwarz inequality, Remark 4., and the result = On /[4θ+2] ) proved in Theorem., we conclude that A A E m p z) Em p z) du A A E m p z) Em p z) 2 ) /2 du cn /2 vy /2 v 2 v for some positive constant c and all cn 2/[5+θ] v <. Recall that we need a condition of v > Ln /[4θ+2] to guarantee A A Em pz) mz) du = Ov). The convergence rate we can guarantee is { O p max n 2/5+θ)) [ y + n /5+θ)) ], n /[4θ+2] [ y + n /[8θ+4] ] The proof of Theorem.2 in this case is complete Proof of Theorem.3. Similarly, we have [ ] A 3.22) F p F y c m p z) Em p z) du + v y. A }). Thus, to complete the proof of Theorem.3, setting v = n 2/[5+θ]+η it suffices to showthat A 3.23) v m p z) Em p z) du 0 a.s. A Now, applying Proposition 4., we obtain for each ξ>0, ) A P m p z) Em p z) du ξv A A vξ) 2k 2A) 2k E m p z) Em p z) 2k du [ A ξ 2k 2A) 2k c k n 2 v 6 ) k ] v y c kεξ) 2k n 5+θ)ηk.

18 CONVERGENCE RATES OF SPECTRAL DISTRIBUTIONS 2 The right-hand side of the above inequality is summable by choosing k such that 5ηk >. Recalling the condition used in the proof of Theorem., the convergence rate is { O a.s. max n 2/5+θ))+η [ y + n /5+θ)) ], n /[4θ+2] [ y + n /[8θ+4] ] Thus, 3.23) is proved and the proof of Theorem.3 is complete. 4. Intermediate lemmas. In this section, we establish a few more technical lemmas. Let ν l = sup i,j,n {E x l }. Lemma 4.. For each l with ν 4l <, there exist positive constants c l independent of n and v such that for all n, v satisfying nv T, we have E ε k 2l F k)) c l + λ p ) l/2 n l + ) l 4.) n trλ k and 4.2) E Proof. We have E ε k 2l F k)) = E n ε k )2l β k l ) F k) c l + λ p ) l/2 n l v l. n x 2 kj ) + 2l n x kb k x k trb k ) j= 2 2l n 2l E n x 2 kj ) := A + B. j= 2l + E F k) }) x kb k x k trb k 2l ) F k) For the first term A, by the Burkholder inequality Burkholder 973, p. 22)), we get 2l n E x 2 kj ) j= l n n c l E x 2 kj ) 2 c l n l E x 2 kj ) 2l c l ν 4l n l. j= For the second term B, denoting the eigenvalues of S k by λ kj and noticing that their maximum is less than the largest eigenvalue λ p of S, we then have = tr B k B k ) = trbk + ztrλ k, p j= p λ kj λ kj z + z j= λ kj λ kj z 2 λ/2 p j= p j= λ /2 p kj λ kj z + T j= λ kj λ kj z 2 λ /2 p + T )p +trλ k ).

19 22 Z. D. BAI, BAIQI MIAO, AND JIAN-FENG YAO Therefore by Lemma 2., E x kb k x k trb k 2l F k)) c l ν 4l + M l )trb k B k ) l c l λ /2 p + T ) l n l + n trλ k) l. Combining the bounds for A and B proves the first conclusion. The second conclusion immediately follows by taking into account inequality 2.2). Lemma 4.2. If n /2 v<, then there are positive constants C,C 2 such that for large n and each k p, i) EtrD k D k ) C p +v y v 2. ii) E ε k 2 C 2 + z 2 +v ) y n v 2. Proof. i) Recall that = EF p F y. By Lemma 2.2, x z 2 d EF px) F y x)) 2 v 2. Application of Lemma 2. and inequality 2.8) yields that EtrD k D k ) = p ) k) d[ EF x z 2 p x)] k) d[p ) EF x z 2 p x) p EF p x)] + p x z 2 d[ EF px) F y x)] + p x z 2 df yx) 2 v 2 + p2 v 2 + p x z 2 df yx). Here, the bound of the last term follows from Lemma 2.4. The proof of conclusion i) is complete. ii) This conclusion follows from i), 4.), and the fact trb k B k = tri p + zd k )I p + zd k ) 2p + z 2 trd k D k ). Lemma 4.3. Assume z T with T 2. Then there are constants C 0, C such that for all v C 0 n /2 and large n, we have 4.3) E βk ) C n + v y )v. k= Proof. From the definition of ε k, we notice that β k ) = β k + β k ε k ). By 2.9), βk β = n +ztrd k trd) + z ) 2T n v nv.

20 CONVERGENCE RATES OF SPECTRAL DISTRIBUTIONS 23 By 3.3), it is easy to see that for any fixed t>0 and all large n, 4.4) E [ + λ p ) l W ] 6 l E W + o W n t ), where W is a bounded random variable with a nonrandom bound W. By this and taking into account 2.0), 4.2), and 2.6), we obtain E βk ) k= E βk β + E β ) βk + E βk ) + E β k 2 k= k= c nv 2 E β k β β β k + k= E βk )+ k= k= E ε k β k 2 + k= c nv 2 + n /2 v + ) nv 2 3c nv 2 ) /2 k= k= E ε k 2 β k β k 2 + p E m pz) E E ε k 2 F k) )) /2 β k 2 + k= k= E E ε k 2 F k) ) v β k 2 E β k )+p E m p z) + on t ) E βk )+p E m p z) + on t ) k= for all v / n. Let C 0 = 6c ) 2. We have for all v C 0 /n E βk ) E β 2 k )+p E m p z) + on t ) k= k= 2p E m p z) + on t ) β k k= + p E m p z) 2p E m p z) Em p z)) +2p Em p z)) mz) +2p mz) + on t ) 2 pv +4p /v +2pv y /v. The proof is nowcomplete. Lemma 4.4. Let z k = EtrD F k ) EtrD F k ). Then trd EtrD = p k= z k and z k ) is a martingale difference with respect to F k ), k = p, p,...,0. Moreover, we have the following formula for z k : with 4.5) z k = { E a k F k ) E a k F k )} E b k F k ), a k = ε k + αt k D k 2 α k ) β k β k, b k = αt k D k 2 α k n tr[i + zd k)d k ] βk. Proof. Since EtrD k F k )= EtrD k F k ), we have z k = E[trD trd k ) F k ] E[trD trd k ) F k ].

21 24 Z. D. BAI, BAIQI MIAO, AND JIAN-FENG YAO On the other hand, trd trd k = + n αt k D k 2 α k β k = + n tr[i + zd k)d k ] β k = + n tr[i + zd k)d k ] βk The conclusion follows from and + E n tr[i + zd k)d k ] β k + ε k + αt k D k 2 α k ) β k β k + a k b k. ) F k + = E ) E αk T D 2 k α k F k) = n tr[i + zd k)d k ]. αt k D k 2 α k n tr[i + zd k)d k ] β k n tr[i + zd k)d k ] β k ) F k Proposition 4.. For each l>/2 with ν 4l <, there exist positive constants c l and L 0 independent of n and v such that for all n, v satisfying L 0 n /2 v<, E m p z) Em p z) 2l c l n 2l v 4l + v y ) l. Proof. In the proof of the proposition, c l and c l,0 will be used to denote universal positive constants which may depend on the moments up to order l of underlying variables and may represent different values at different appearances, even in one expression. Recall that we have m p z) Em p z) = [trd EtrD] = p where the {z k } are defined as in Lemma 4.4. We have E z k 2l { ) [ Fk = E E ak F k ) E a k F k )] E b k F k ) 2l } F k k= 2 2l E {[ E a k F k ) E a k F k )] 2l +[E b k F k )] 2l } Fk z k, 2 2l E {[ E a k F k )] 2l +[E b k F k )] 2l } Fk 2 2l { E a k ) 2l F k ) + E bk ) 2l F k )}. Note that by 2.3) and 2.4), a k v ε k /β k. Hence by Lemma 4., E a k 2l F k)) v 2l E ε ) 2l k F k) c l,0 + λ p ) l/2 n l v 3l βk l. β k On the other hand, by Lemma 2. and assuming l, E b k 2l ) F k) c l,0 nβk) 2l ν 4l + M l ) [ ] l tri + zd k )I + zd k )D k D k.

22 By 2.2) and 2.6), CONVERGENCE RATES OF SPECTRAL DISTRIBUTIONS 25 which implies β k tri + zd k )I + zd k )D k D k β k trλ 2 k nt v 3, E { b k 2l } Fk c l,0 n l v 3l E [ βk l ] F k. Therefore, for all l, E z k 2l ) F k cl,0 + λ p ) l/2 n l v 3l E [ βk l ] F k 4.6) c l,0 + λ p ) l/2 n l v 4l+ E [ β k F k ]. Applying Lemma 4.3 and 4.4), it follows that, for l, 4.7) E z k 2l c l,0 n l+ + v y )v 4l. k= Case l =. Since {z k } is a martingale difference sequence, the above inequality yields 4.8) E m p z) Em p z) 2 = n 2 E z k 2 c,0 n 2 + v y )v 4. k= The proposition is proved in this case. Case 2 <l<. By applying the Burkholder inequality for the martingale and using the concavity of the function x l,wefind E m p z) Em p z) 2l c l p 2l ) l [ )] l E z k 2 c l n 2l E z k 2 c l n 2l [ + v y )v 4] l, k= where the last step follows from the previous case l =. The lemma is then proved in this case. Case l>. We proceed by induction in this general case. First, by another Burkholder inequality for the martingale Burkholder 973), p. 39), we have ) l E m p z) Em p z) 2l c l p 2l E z k 2l + E E z k 2 F k ) 4.9) By 4.7) 4.0) ˆ= I + I 2. k= k= k= I c l,0 + v y )n 3l+ v 4l c l,0 + v y ) l n 2l v 4l. The proposition already has been proved for the case 2 <l. Suppose that the lemma is true for l 2 t. Now, we consider the case where 2 t <l 2 t+. Application of 4.6) with l = gives n E z k 2 ) Fk c,0 n v 3 + λ p ) /2 k= E βk ) Fk. k=

23 26 Z. D. BAI, BAIQI MIAO, AND JIAN-FENG YAO Hence, by taking into account 4.4) we get 4.) I 2 c l,0 nv) 3l E + λ p ) l/2 E βk F k ) k= c l,0 n 2l v 3l E βk l + on 4l v 4l ), k= since P λ p > 5) = on 4l ). Notice that if L 0 > 2, then nv 2 > 2 and that β βk β βk) = trd trd k p β βk pv 2 min β, βk ) this comes from 2.9) and ββ k v min β, β k )). This yields and β k β + p v 2 β k 2 β pβ β k) + βk) β β k) + v 2 β k= k= k= 2 β k) 2 β k) β k ) +2 β k 2 k= k= k= ε k 2 β k β k 2 +2p m pz). Therefore, by applying Lemma 4., and if we choose, L 0 > 2c l,0 ) /l so that c l,0 n l v 2l < /2, we have ) E βk l c l,0 v l E ε k 2l βk + p E m pz) l 2l k= k= c l,0 n l v 2l k= ) l ) E βk l + p E m p z) l k= 2c l,0 p E m p z) l. From the above inequality and 4.), we get by induction, I 2 c l n 2l v 3l E m p z) l + on 4l v 4l ) c l n 2l v 3l [ E m p z) Em p z) l + Em p z) mz) l + mz) l] + on 4l v 4l ) c l n 2l v 3l [ E m p z) Em p z) l + +v y ) l v l] + on 4l v 4l ) c l n 2l v 4l + v y ) l [ n 2 v 2 + v y ) ) l/2 + ] + on 4l v 4l ) 4.2) c l n 2l v 4l + v y ) l.

24 CONVERGENCE RATES OF SPECTRAL DISTRIBUTIONS 27 Therefore by 4.9) and 4.2), it follows that 4.3) E m p z) Em p z) 2l c l n 2l + v y ) l v 4l. The proof of Proposition 4. is complete. Remark 4.. Application of Proposition 4. to the case l = gives that there is some constant c > 0 such that 4.4) E trd k EtrD k 2 c + v y )v 4. It is also worth noticing that if we substitute D for any D k with k n, Proposition 4. as well as the above consequence 4.4) are still valid, with slightly different constants c l. Acknowledgments. The authors warmly thank the referees for their comments that actually lead to several important improvements. J. F. Yao thanks the National University of Singapore for the support during his visit to its Department of Statistics and Applied Probability. REFERENCES Z. D. Bai 993a), Convergence rate of expected spectral distributions of large random matrices, Part I: Wigner matrices, Ann. Probab., 2, pp Z. D. Bai 993b), Convergence rate of expected spectral distributions of large random matrices, Part II: Sample covariance matrices, Ann. Probab., 2, pp Z. D. Bai 997), Circular law, Ann. Probab., 25, pp Z. D. Bai 999), Methodologies in spectral analysis of large dimensional random matrices. A review, Statist. Sinica, 9, pp Z. D. Bai, B. Miao, and J. Tsay 997), A note on the convergence rate of the spectral distribution of large random matrices, Statist. Probab. Lett., 34, pp Z. D. Bai, B. Q. Miao, and J. Tsay 999), Remarks on the convergence rate of the spectral distributions of Wigner matrices, J. Theoret. Probab., 2, pp Z. D. Bai, B. Miao, and J. Tsay 2002), Convergence rates of the spectral distributions of large Wigner matrices, Int. Math. J.,, pp Z. D. Bai and J. W. Silverstein 998), No eigenvalues outside the support of the limiting spectral distribution of large-dimensional sample covariance matrices, Ann. Probab., 26, pp Z. D. Bai and J. W. Silverstein 999), Exact separation of eigenvalues of large dimensional sample covariance matrices, Ann. Probab., 27, pp Z. D. Bai and Y. Q. Yin 986), Limiting behavior of the norm of products of random matrices and two problems of Geman-Hwang, Probab. Theory Related Fields, 73, pp Z. D. Bai, Y. Q. Yin, and P. K. Krishnaiah 987), On the limiting empirical distribution function of the eigenvalues of a multivariate F-matrix, Probab. Theory Appl., 32, pp D. L. Burkholder 973), Distribution function inequalities for martingales, Ann. Probab.,, pp V. A. Marčenko and L. A. Pastur 967), Distribution of eigenvalues for some sets of random matrices, Mat. Sb., 72, pp M. L. Mehta 99), Random Matrices, Academic Press, New York. J. W. Silverstein 985), The limiting eigenvalue distribution of a multivariate F matrix, SIAM J. Math. Anal., 6, pp E. P. Wigner 955), Characteristic vectors of bordered matrices with infinite dimensions, Ann. of Math. 2), 62, pp E. P. Wigner 958), On the distribution of the roots of certain symmetric matrices, Ann. of Math. 2), 67, pp Y. Q. Yin, Z. D. Bai, and P. R. Krishnaiah 988), On the limit of the largest eigenvalue of the large dimensional sample covariance matrix, Probab. Theory Related Fields, 78, pp