arxiv: v1 [math.pr] 16 Jan 2009

Transcription

1 The Annals of Applied Probability 008, Vol. 18, No. 6, DOI: /08-AAP57 c Institute of Mathematical Statistics, 008 arxiv: v1 [math.pr] 16 Jan 009 THE ASYMPTOTIC DISTRIBUTION AND BERRY ESSEEN BOUND OF A NEW TEST FOR INDEPENDENCE IN HIGH DIMENSION WITH AN APPLICATION TO STOCHASTIC OPTIMIZATION By Wei-Dong Liu, 1 Zhengyan Lin 1 and Qi-Man Shao Zhejiang University, Zhejiang University and HKUST Let X 1,...,X n be a random sample from a p-dimensional population distribution. Assume that c 1n α p c n α for some positive constants c 1,c and α. In this paper we introduce a new statistic for testing independence of the p-variates of the population and prove that the limiting distribution is the extreme distribution of type I with a rate of convergence Olog n 5/ / n. This is much faster than O1/log n, a typical convergence rate for this type of extreme distribution. A simulation study and application to stochastic optimization are discussed. 1. Introduction and main results. Consider a p-variable population represented by a random vector X = X 1,...,X p with the covariance matrix Σ and the correlation coefficient matrix R and let {X 1,...,X n } be a random sample of size n from the population. In the applications of multivariate analysis in the century of data, both the dimension p and the sample size n can be very large, p may be comparable with n or even much larger than n; see, for example, Donoho [4], and Fan and Li [11]. Since classical limit theorems for fixed p may not be valid for large p, it is necessary to develop new limiting theorems for large p. Whether the p-variables are independent is usually a primary step because independence seems to be a granted assumption for many limiting theorems. When n/p γ > 0 and the population distribution is normal, several statistics have been developed to test Received September 007; revised February Supported by National Natural Science Foundations of China , and Specialized Research Fund for the Doctor Program of Higher Education Supported in part by Hong Kong RGC AMS 000 subject classifications. Primary 60F05; secondary 6F05. Key words and phrases. Independence test, extreme distribution, Berry Esseen bound, correlation matrices, stochastic optimization. This is an electronic reprint of the original article published by the Institute of Mathematical Statistics in The Annals of Applied Probability, 008, Vol. 18, No. 6, This reprint differs from the original in pagination and typographic detail. 1

2 W.-D. LIU, Z. LIN AND Q.-M. SHAO the complete independence of the p components of X. Johnstone [15] uses the largest eigenvalue of the sample covariance matrix and Ledoit and Wolf [17] use quadratic forms of the sample covariance matrix eigenvalues to test the null hypothesis H 0 :Σ = I p, where I p is the p p identity matrix, while Schott [0] uses the sums of sample correlation coefficient squares to test H 0 :R = I p. When the normality is not present, Jiang [14] constructs a test statistic based on the largest entries of the sample correlation matrix. Write X k = X k,1,x k,,...,x k,p, 1 k n and let where ρ n i,j = n X k,i L n = max 1 i<j p ρn i,j, n n X k,i X i X k,j n X i 1/ n X k,j n X j X n j 1/ and X n i = n X k,i /n. Jiang [14] proves the following limit theorem concerning the test statistic L n : If n/p γ 0, and E X 11 r < for some r > 30, then 1.1 lim Pn L n n 4log p + log p y = exp e y/ / 8π for y R, where and in the sequel logx = lnmaxx,e and log x = loglog x. Zhou [1] shows that the moment condition E X 1,1 r < for some r > 30 can be weakened to 1. x 6 P X 1,1 X 1, x 0 as x. The limit distribution appearing in 1.1 is called the extreme distribution of type I. It seems a common belief that the convergence rate of this type of extreme distribution is typically slow see Hall [13]. In fact, we shall prove [see Theorem 1. and 1.11] that even when X 1,1 has the standard normal distribution the rate of convergence is of order of Olog n/logn. The main purpose of this paper is to introduce a modified test statistic and show that the new one also has an extreme limit distribution of type I, but with a rate of convergence of Olog n 5/ / n. We shall also prove that the approximation rate of Pn L n 4logp + log p y to exp p p Pχ 1 4logp log p + y instead of the final limit exp e y/ / 8π is indeed of order of Olog n 5/ / n. This indicates that when a statistic has an extreme limiting distribution, one should use some intermediate approximation, not the final limiting distribution to approximate the distribution

3 A NEW TEST STATISTICS 3 of the statistic. Extreme limiting distributions are important in various applications, including assessing risk for highly unusual events, hydrologic assessment, analysis of network simulation and engineering see [1] and [16]. These results are partially motivated by the need for new approaches to applications of these types in practice. Throughout this paper let H 0 be the null hypothesis that the p components of the population X are independent and have the same distribution, and let X k = X k,1,x k,,...,x k,p,1 k n, be a random sample from the population X. Define 1.3 where L n = max 1 i<j p r i,j, 1.4 r i,j = A n,i,j + B n,i,j /D n,i,j, [n/] A n,i,j = X k,i B n,i,j = D n,i,j = k=[n/]+1 X k,i X k,i n X i X k,j X n i X n j, n X i X k,j n X k,j X n j, X n j, where [n/] denotes the integer part of n/. Our test statistic is chosen as 1.5 W n := nl n 4logp. It is easy to see that ri,j is a consistent estimator of the square of the correlation coefficient between X i and X j and that ri,j ρn i,j. Instead of assuming that n and p have the same order, we consider a more general case. Assume 1.6 c 1 n α p c n α, where c 1,c and α are positive constants. Our first result shows that the W n has an extreme limiting distribution of type I under a weaker moment assumption than 1. and the error of the approximation is of order of Olog n 5/ / n under E X 1,1 3+4α < and α > 3/4. Theorem 1.1. Assume that 1.6 is satisfied and that 1.7 x 1+α P X 1,1 X 1, xlogx 0 as x.

4 4 W.-D. LIU, Z. LIN AND Q.-M. SHAO Then under H 0, 1.8 PW n y exp 1 exp y as n for any y R. If E X 1,1 3+4α < and α > 3/4, then PW n y exp 1 exp y 1.9 Cn 1/ logn 5/ y R and C is a constant independent of n and p. The next result shows that 1.1 remains valid under the assumption 1.7. Moreover, the convergence rate to exp p p Pχ 1 4logp log p + y [χ 1 has a chi-square distribution with 1 degree of freedom] can also achieve the order of Ologn 5/ / n when the 7th moment of X 1,1 is finite. Theorem 1.. Under H 0, if 1.6 and 1.7 are satisfied, then 1.1 holds. If E X 1,1 3+4α < and α > 3/4, then Pn L n 4logp + log p y y R exp p p 1.10 Pχ 1 4logp log p + y Cn 1/ logn 5/. One can show that see a proof in Section 5 if α = 1 in 1.6, then exp p p Pχ 1 4logp log p + y exp e y/ / 8π 1.11 log n 1 exp y 8logn 8π 1 exp 8π y. So the rate of convergence in 1.1 is of order of Olog n/logn. The following remarks are noted. Remark 1.1. The logarithmic term logn 5/ in 1.10 may not be sharp. Since our aim is to get the main order n 1/, we will not seek the optimality of the logarithmic term in this paper. Remark 1.. It is not necessary to require p and n have the tight relation 1.6. For example, when 1.6 is replaced by c 1 n α 1 p c n α

5 A NEW TEST STATISTICS 5 for some positive constants c 1,c,α 1 and α, and assume that E X 1,1 3+4α r < for some r > 6, then, following the proofs of Theorems 1.1 and 1., we have PW n y exp 1 exp y Cn 1/ logp 5/ + p 1+ε y R and Pn L n 4logp + log p y y R exp p p Pχ 1 4logp log p + y Cn 1/ logp 5/ + p 1+ε for ε > 0, where the constant C may depend on ε. When logp is of order of n r 0 < r 1, we shall discuss the case in our future paper. Remark 1.3. It is not necessary to assume that the p components of X are identically distributed in Theorems 1.1 and 1.. For example, if inf n min 1 j p EX 1,j > 0 and x 1+α max P X 1,iX 1,j xlogx 0, 1 i<j p n 1 then 1.8 remains valid. If, in addition, n max 1 j p E X 1,j 3+4α <, α > 3/4, then 1.9 and 1.10 hold. The paper is organized as follows. In Section we conduct a simulation study and give a simple application of Theorem 1. to the sparsest solution of large underdetermined systems of linear equations. In Section 3 we present a general theorem from which Theorems 1.1 and 1. can be derived easily. An outline of the proof of the general result along with five propositions is given in Section 5, while the detailed proofs of the propositions are postponed to Section 6. Proofs of Theorems 1.1 and 1. are given in Section 4. Throughout the paper, C will denote a positive constant that doesn t depend on n and p but may be different at each appearance.. A simulation study and application to stochastic optimization. In this section we give a simulation study for performance of W n and L n and an application to the sparsest solution of large underdetermined systems of linear equations.

6 6 W.-D. LIU, Z. LIN AND Q.-M. SHAO Table 1 Estimated significance levels when α = 0.05 and X 1,1 N0,1 p Test statistics n = 16 n = 3 n = 64 n = 18 n = 56 4 W n L new L old W n L new L old W n L new L old W n L new L old W n L new L old W n L new L old A simulation study. The performance of the test statistics W n and L n is carried out via simulation. Estimates of the actual significance levels are obtained from 5000 independent simulations with the nominal significance level α = The simulation results for tests of H 0 based on W n in Theorem 1.1, L n denoted by L new in Theorem 1. and L n denoted by L old in 1.1 are given in Table 1 when X 1,1 has a standard normal distribution, and in Table when X 1,1 has a t-distribution with 7 degrees of freedom. The estimated significance levels are usually lower than the nominal level 0.05, which indicates that the tests are conservative. The performances of W n and L new are comparable and both are well when n is larger than p... An application to stochastic optimization. We apply Theorem 1. to the problem of finding sparse representations of single measurement vectors SMV in an over-complete dictionary. The SMV problem can be described as follows. Given a single measurement vector b and a dictionary A, one wants to solve the system of equations Ax = b, where A is a n p matrix, x is a p-variable vector and b is a n-variable vector. It is usually assumed that n p. A sparse representation means that vector x has a small number of nonzero components. Examples of such underdetermined

7 A NEW TEST STATISTICS 7 Table Estimated significance levels when α = 0.05 and X 1,1 t 7 p Test statistics n = 16 n = 3 n = 64 n = 18 n = 56 4 W n L new L old W n L new L old W n L new L old W n L new L old W n L new L old W n L new L old systems of equations include array signal processing, inverse problems and genomic data analysis. We refer to [, 6, 7, 8] and [9] and references therein for a comprehensive description of many important applications of a SMV problem. A sparse representation can be found by solving the following optimization problem: Q0 min x 0, s.t. Ax = b, where the quantity x 0 denotes the number of nonzero elements in the vector x. The problem Q0 is essentially a combinatorial optimization problem, which, in general, is extremely difficult to solve. The above problem can be convexified as a l 1 -norm minimization problem, and solved via linear programming. The l 1 -norm minimization problem is Q1 min x 1, s.t. Ax = b, where x 1 is the sum of the absolute values of the elements of vector x. It has been proved that the solutions between Q0 and Q1 are equivalent under various conditions. For example, letting G = A T A and M = max 1 i,j p,i j Gi,j, if x 0 < 1+M 1 /, then x is the unique solution of Q1 for b = Ax and this solution is identical to the unique solution

8 8 W.-D. LIU, Z. LIN AND Q.-M. SHAO of Q0 [3]. Here we assume that A is a random matrix. Let n,p satisfy the condition in Remark 1. and {X k,i,k,i 1} be independent centered random variables satisfying the conditions in Remark 1.3. Define the normalized k,i element of A by Y k,i := X k,i / n X k,i 1/. Following the proof of Theorem 1., we see that 1.10 remains valid for M = M n, where Mn = max G i,j = max 1 i,j p,i j 1 i<j p n X k,i X k,j n X k,i n X k,j. Hence, Q0 and Q1 are equivalent with probability 1 α 0 < α < 1 for every x with fewer than 1+m α / nonzeros, where m α = n/y α + 4log p log p and y α is the solution of exp p p Pχ 1 4logp log p + y α = 1 α. When X ij above are i.i.d. standard normal random variables, the result is similar to that given in [5]. 3. A general result. Instead of proving Theorems 1.1 and 1. separately, we give a general result in this section. Let d be a positive integer and ;k,i 1,1 m d} be an array of i.i.d. random variables. Put {X,X m k,i X k,i,j = Y 1 d k,i,j,...,y k,i,j, and W p,n = max Y m k,i,j = Xm k,i Xm k,j, X 1 i<j p k,i,j, where denotes the Euclidean norm in R d. i,j,k 1,1 m d Theorem 3.1. Suppose that EX = 0 and EX = 1. Let X be an independent copy of X, and 3.1 x 1+α P XX xlogx <. x Then for any 0 < ε 10 4 there exists a finite constant C such that W P p,n n α p y exp p p Pχ d α p + y 3. y R Cp 1+0 ε + C logn5/ n 1/ E XX 3 I{ XX n/log n 4 } + Cn 1+α P XX d 1/ ε nlogp, where α p = 4logp dlog p and χ d has a chi-square distribution with d degrees of freedom.

9 Now, set and Q n,i,j = A n,i A n,j, A n,i = A NEW TEST STATISTICS 9 d X m k,i, m=1 1 i p L n p,n = max X k,i,j. 1 i<j p Q n,i,j Theorem 3.. Under the conditions of Theorem 3.1 and EX 4 <, we have, for any 0 < ε 10 4, Pd nl p,n α p y exp p p Pχ d α p + y y R 3.3 where Cp 1+0 ε + C logn5/ n 1/ + Cn 1+α P XX d 1/ ε nlogp + τ n, τ n = Cnp 0 ε P X n1/4 log p 1/4. The proofs of Theorems 3.1 and 3. are postponed to Section Proofs of Theorems 1.1 and 1.. We are now ready to see that Theorems 1.1 and 1. are two special cases of Theorems 3.1 and 3.. Proof of Theorem 1.1. For the sake of simplicity, we assume that n is even. Otherwise use [n/] instead of n/ below. Without loss of generality, we assume EX 1,1 = 0 and EX1,1 = 1. Set n/ Ã n,i,j = X k,i X k,j, B n,i,j = D n,i,j = Xk,i Xk,j. k=n/+1 n/ X k,i X k,j = X k+n/,i X k+n/,j, Take d = in Theorem 3.1. Since Ãn,i,j + B n,i,j = Ãn,i,j, B n,i,j, by Theorem 3.1, Ãn,i,j + P B n,i,j 4logp y 4.1 max 1 i<j p n W p,n/ = P 4logp y e e y/ / n

10 10 W.-D. LIU, Z. LIN AND Q.-M. SHAO and hence, 4. max Ãn,i,j = O P nlogn 1 i<j p max B n,i,j = O P nlogn. 1 i<j p Noting that condition 1.7 implies E X 1,1 +4α ε < for any ε > 0 see Lemma 6.4, we can show, by Lemma 6.1, 4.3 and 4.4 for some δ > 0. Observe that n max X k,i 1 i p n and logn = O P n n Xk,i max n = O P n δ, 1 i p n ri,j = n/ X k,ix k,j X n/ 1,i,j X k,ix k,j + X 1,i,j [ n Xk,i n n X i ][ n Xk,j n n X j ] where + n k=n/+1 X k,i X k,j X nk=n/+1,i,j X k,i X k,j + X,i,j [ n Xk,i n n X i ][, n Xk,j n n X j ] X 1,i,j = 1 n X,i,j = 1 n n/ X i X j n n 1 = O P logn. + X n i n k=n/+1 X n/ j X n i 1 X k,i Xn j + n Xn X n j = O Plogn, i k=n/+1 X k,j X n i 1.8 now follows from Now we prove 1.9. Let E n,i,j = A n,i,j + B n,i,j. We first show that D n,i,j in the denominator of r i,j can be replaced by D n,i,j. Observe that n n [X k,i X i Xk,i ] = n X k,i n n. We have, by Lemma 6.1, n n [X P max k,i X i Xk,i ] 1 i p n logn/n 4 logn 1/4 4.5 pp X k,1 n n X n j

11 A NEW TEST STATISTICS Cn 1/, n X P max k,i n 1 i p n 1 Cn 1/ α Cn 1/ and 4.7 Therefore, P n max 1 i<j p P max X 1 i p k,i 4 nlogn Cn 1/. PW n y P E n,i,j 1 8 logn/ny + α p Cn D 1/ n,i,j n max 1 i<j p E n,i,j log n/ny + α p + Cn D 1/. n,i,j Now write F n,i,j = Ãn,i,j, B n,i,j. Note that F n,i,j A n,i,j + B n,i,j X 1,i,j, X,i,j. This together with 4.7 leads to n F n,i,j P max 1 i<j p D 1/ 1 8 logn/n 1/ y + α p 1/ n,i,j C logn/ n Cn 1/ PW n y n F n,i,j P max 1 i<j p D 1/ n,i,j logn/n 1/ y + α p 1/ + C logn/ n + Cn 1/. Take d = in Theorem 3.. It is easily seen that Pnmax 1 i<j p F n,i,j x = PnL p,n/ x for any x R. Write [ l n± y = 1 ± 8 logn n 1/ y + α p 1/ ± C logn ] n 1/. D n,i,j

12 1 W.-D. LIU, Z. LIN AND Q.-M. SHAO We have 4.8 PnL p,n/ l n y Cn 1/ PW n y PnL p,n/ l n+y + Cn 1/. By Theorem 3., we obtain PnL p,n/ l n+y exp 1 exp y where y R Note that 4.9 This implies exp p p Pχ l n+ y exp 1 y R exp y + Cn 1/ log n 5/ + Cp 1+0 ε + dn 1+α P XX 1 ε nlogp + τ n =: P n+ y + Cn 1/ log n 5/, y R P n+ y = exp p p e l n+y/ exp 1 exp y. Also, we can get 4.10 l n+ y α p log n 8 + C logn3/ y log n 8 n 1/. P n+ y Cn 3. y log n 8 l n+ y α p logn. y logn Therefore, following the inequality 1 exp 1 exp y C exp y for y 1, A direct elementary calculation gives P n+ y Cn 1/. y log n 4.11 log n3/ l n+ y α p y C log n 8 y logn n 1/,

13 A NEW TEST STATISTICS 13 hence, by the inequality e x 1 C x for x 1, P n+ y Cn 1/ logn 5/ + Cp 1 logn. log n 8 y logn The above arguments yield PnL p,n/ l n+y exp 1 exp y Cn 1/ logn 5/. y R Similarly, PnL p,n/ l n y exp 1 exp y Cn 1/ logn 5/. y R The proof is now complete by 4.8 and the above two inequalities. Proof of Theorem 1.. We assume that EX 1,1 = 0, EX1,1 = 1. The proof of 1.1 is similar to that of 1.8, hence, is omitted. Now we prove In view of Lemma 6.1 and the proofs of Theorem 1.1, we have exp p p Pχ 1 l n y Cn 1/ log n 5/ Pn L n α p y exp p p Pχ 1 l n+ y + Cn 1/ logn 5/. Moreover, it follows from 4.9, 4.10 and 4.11 that exp p p Pχ 1 l n± y y log n 8 y log n exp p p Pχ 1 α p + y Cn 1/, exp p p Pχ 1 l n± y exp p p Pχ 1 α p + y Cn 1/, and exp p p Pχ 1 l n± y log n 8 y logn exp p p Pχ 1 α p + y Cn 1/ log n 5/. This completes the proof of Theorem 1..

14 14 W.-D. LIU, Z. LIN AND Q.-M. SHAO 5. Proof of the general result. In this section we outline the proof of the general result, Theorems 3.1 and The truncation and notation. We first truncate X k,i,j. Let ε be a small positive number which will be specified later and put 5.1 and 5. Y m k,i,j = Xm k,i Xm k,j, k,i,j = Y m m k,i,ji{ Y k,i,j d 1/ ε nlogp}, Ỹ m k,i,j = Y m m k,i,ji{ Y k,i,j n/log n 4 }, Ŷ m Y m k,i,j = Y m k,i,j I{ n/log n 4 < Y m k,i,j d 1/ ε nlogp}, 1 k n,1 i,j p,1 m d k,i,j = Ỹ 1 d k,i,j,...,ỹ k,i,j, ˆXk,i,j = Ŷ 1 d k,i,j,...,ŷ k,i,j, X k,i,j = Y 1 k,i,j,..., Y d k,i,j, 1 k n, 1 i,j p, T p,n = max k,i,j, ˆTp,n = max ˆX k,i,j, 1 i<j p L n p,n = max k,i,j. 1 i<j p Q n,i,j 1 i<j p 5.. Auxiliary results. To prove the general result, we first collect some auxiliary results. As in many previous works on the Poisson approximation, we apply Lemma 5.1 below, which is a special case of Theorem 1 of Arratia, Goldstein and Gordon [1]. Lemma 5.1 Arratia, Goldstein and Gordon [1]. Let {η α,α I} be random variables on an index set I. For each α I, let B α be a subset of I with α B α. For a given t R, set λ = α I Pη α > t. Then 5.3 max P η α t e λ 1 λ 1 b 1 + b + b 3, α I where b 1 = Pη α > tpη β > t, α I β B α b = Pη α > t,η β > t, α I β B α,β α b 3 = α I E Pη α > t ση β,β / B α Pη α > t

15 A NEW TEST STATISTICS 15 and ση β,β / B α is the σ-algebra generated by {η β,β / B α }. In particular, if η α is independent of {η β,β / B α } for each α, then b 3 = 0. The following estimates are also essential for our proof. Proposition 5.1. Under the conditions of Theorem 3.1, we have log n θ y logn P k,1, ny n Pχ d α p + y 5.4 C logn5/ p n 1/ E XX 3 I{ XX n/log n 4 } + Cp 3, where y n = α p + y1 + O logn/n, θ = 8Γd/, and Γ is the gamma function. Proposition 5.. Under the conditions of Theorem 3.1, we have T P p,n n α p y exp p p Pχ d α p + y y log n θ Cn + Cp 1+0 ε. Proposition 5.3. Under the conditions of Theorem 3.1, we have T P p,n n α p y exp p p Pχ d α p + y y log n log n5/ C n 1/ E XX 3 I{ XX n/log n 4 } + Cp 1. Proposition 5.4. Under the conditions of Theorem 3.1, for any sequence v n satisfying v n nlogp, we have P k,1, v n, k,1,3 v n Cp 4+0 ε. Proposition 5.5. Under the conditions of Theorem 3., we have Pd n L p,n α p y exp p p Pχ d α p + y y log n θ 5.5 Cn + Cp 1+0 ε

16 16 W.-D. LIU, Z. LIN AND Q.-M. SHAO and 5.6 Pd n L p,n α p y exp p p Pχ d α p + y y logn C logn5/ n 1/ + τ n. Proofs of the propositions above will be given in Section Proof of Theorem 3.1. Clearly, PWp,n T p,n dp max max Y 1 k n 1 i<j p k,i,j d 1/ ε nlogp Cnp P XX d 1/ ε nlogp. To prove Theorem 3.1, it suffices to show that T P p,n y R n α p y exp p p Pχ d α p + y 5.8 Cp 1+0 ε + C logn5/ n 1/ E XX 3 I{ XX n/log n 4 }. Let I = {i,j;1 i < j p} in Lemma 5.1. For α = i,j I, set η α = k,i,j and B α = {k,l I; k = i or l = j}. Put t = nα p + ny. Noting that η α is independent of {η β,β / B α } for each α I, we have 5.9 where P T p,n t e λn b 1n + b n, λ n = 1/p pp k,1, > t, n b 1n p 3 P k,1, > t, b n p P 3 k,1, > t, k,1,3 > t. Now 5.8 follows from 5.9 and Propositions

17 A NEW TEST STATISTICS Proof of Theorem 3.. Assume that τ n 1, otherwise 3.3 is trivial. It suffices to show that Pd n L p,n α p y exp p p Pχ d α p + y y R Cp 1+0 ε + C logn5/ n 1/ + τ n. Following the proof of Theorem 3.1, by Lemma 5.1 again, we have 5.10 where and Let Pd n L p,n α p y e λn b 1n + b n, λ n = p p P k,1, > t 1,, b 1n p 3 P n k,1, > t 1,, b n pp pp k,1, > t 1,, k,1,3 > t 1,3 Â n,i = t i,j = d X m m=1 Q n,i,j d n α p + y. { k,i I X m k,i } n, logp ˆQ n,i,j = Ân,iÂn,j, ˆt i,j = ˆQn,i,j d n α p + y. The main idea of the proof is to replace Q n,i,j by ˆQ n,i,j and ˆQ n,i,j by some nonrandom constants. We use Lemma 6.1 and let δ = 1/4, M =, β = 1, a = 8D 1 1 Mδ 1 logp and x = 3 34d nlogp in 6.1. So Σ n,x,a = 0 and Let v n = { } Â n,1 n P dn EX I X 3 34 logp logp Cp 4. n { } n nα p log n θ 1 EX I X logp 1/ > logp n

18 18 W.-D. LIU, Z. LIN AND Q.-M. SHAO Then v n nlogp. Note that A n,i,j Ân.i.j and, hence, Q n,i,j ˆQ n,i,j. By Proposition 5.4, P y log n 8 k,1, > t 1,, k,1,3 > t 1,3 P y log n 8 k,1, > ˆt 1,, k,1,3 > ˆt 1, P k,1, > v n, k,1,3 > v n + Cp 4 Cp 4+0 ε. Moreover, by Proposition 5.1, for log n θ y logn, P k,1, > t 1, P k,1, > ˆt 1, P k,1, > nα p + y1 t n + Cp 4 Pχ d α p + y + C logn5/ p n 1/ + Cp 3, where t n = C logn/n. Note that P k,1, > t 1, P k,1, > ˆt 1, d P k,1, > ˆt 1,,X 1 n i,1. logp i=1 For y log n θ, letting β = 19ε, q = 4α, r = + α, δ = 1ε/5 and x = 1 ε nlogp in 6. yields P k,1, > ˆt 1,,X 1 n i,1 logp

19 A NEW TEST STATISTICS 19 P k,1, > nα p + y1 t n,x 1 n i,1 + Cp 4 logp P k,1, E k,1, > 1 ε nlogp,x 1 n i,1 logp, =i + Cp 4 C[p 1 10 ε + p 4 ] P X n1/4 logp 1/4 + Cp 4 Cp 1 10 ε P X n1/4 logp 1/4 + Cp 4. For log n θ y logn, it follows from Proposition 5.1 that P k,1, > ˆt 1, P k,1, > nα p + y1 + t n Cp 4 Pχ d α p + y C logn5/ p n 1/ Cp 3. Combing the above inequalities gives P k,1, > t 1, Pχ d α p + y 5.1 log n θ y logn log n5/ C p n 1/ + Cp 3 + Cnp 1 10 ε P X n1/4 logp 1/4. The proof of Theorem 3. is now complete by and Proposition Proofs of auxiliary results. To prove Propositions , we need some preliminary lemmas. The first is a Fuk Nagaev type inequality for vector-valued random variables. Lemma 6.1. Let ξ i, 1 i n, be independent random vectors in R d with Eξ i = 0 and E ξ i <, 1 i n. Put S n = n i=1 ξ i. Then for 0 < δ < 1, β > 0, a > δ 1 and any x > 0, P max S k x + 3E S n + 8 a 1 k n x Σ n,x,a

20 0 W.-D. LIU, Z. LIN AND Q.-M. SHAO 6.1 M Σn,x,a P ξ k > δx + C x 1 Mδx + exp 1 + βλ n 1 Mδa + exp, D β where Λ n = { n Eu,ξ k : u 1},, denotes the Euclidean inner product, Σ n,x,a = n E ξ k I{ ξ k x/a}, D β = 111+/β, M is a positive number satisfying Mδ < 1, and C is a constant which depends on M and δ. In particular, if max 1 k n E ξ i r K for some r > and K <, then for any q and 0 < β 1, there exist C 1, C depending only on β, q, K such that, for any x C n and 0 < δ βr 3q + 16r 3 1, P max S k x 1 k n 6. Proof. Put and write Then 6.3 P ξ k > δx + exp ξ i = ξ i I{ ξ i δx}, S n = ˆξ i = ξ i I{ ξ i x/a}, Ŝ n = x 1 + βλ n ξ i, i=1 i=1 ξ i = ξ i I{x/a < ξ i δx}, Sn = B n = 3E S n + 8 a x ˆξ i, + C 1 n q. ξ i i=1 E ξ k I{ ξ k x/a}. P max S k x + B n 1 k n P max S k x + B n 1 k n + P ξ k > δx

21 A NEW TEST STATISTICS 1 P max Ŝk 1 Mδx + B n / + 1 k n + P max S k Mδx + B n /. 1 k n Since Eξ k = 0 for 1 k n, we have P ξ k > δx max EŜk k n E Ŝn EŜn B n /. Hence, 6.4 P max Ŝk 1 Mδx + B n / 1 k n P max 1 k n Ŝk EŜk 1 Mδx + 3 E Ŝn EŜn + exp exp 1 Mδx 1 + βλ n 1 Mδa, D β where the last inequality follows from 3.4 in Einmahl and Li [10]. We now estimate Pmax 1 k n S k Mδx + B n /. It follows from the Hoeffding Bennett inequality that P max S k Mδx + B n / 1 k n n P ξ i Mδx + B n / P n ξ i E ξ i Mδx 3 n E ξ k I{ ξ k x/a} M Mδ x. So 6.1 is proved. In order to prove 6., we let a = maxd τ q1 Mδ 1 1 logn,δ , where δ 1 and τ are positive numbers which will be specified later. Then exp 1 Mδ 1a n q. D τ Note that E S n C nk 1/r. Since Σ n,y,a Knay 1 r, we have, for y n, ay 1 Σ n,y,a Cn 3 r/ logn r 1 C 3 n, and y Σ n,y,a Cn 1 r/ a r Cn 1 r/ logn r. Now take M = r 1 q + r and 0 < δ 1

22 W.-D. LIU, Z. LIN AND Q.-M. SHAO βr 16q + 8r 16 1 such that Mr/ 1 > q and Mδ 1 β/8. Let C 4 = 8β 1 3CK 1/r + 8C 3. By 6.1, it holds that, for y C 4 n, P max S k 1 + β/8y 1 k n P max S k y + 3E S n + 8 a 1 k n y Σ n,y,a P ξ k > δ 1 y + exp 1 β/8 y + C 1 n q, 1 + τλ n where we let τ satisfy 1 β/8 1 + τ β/8 > 1 + β 1. Setting C = 1 + β/8c 4, δ = 1 + β/8 1 δ 1 and x = y1 + β/8, we obtain 6.. The following moderate deviation for independent random variables will play an important role in our proof. Lemma 6.. Let ξ i, 1 i n, be independent random variables with Eξ i = 0. Put Assume that s n = Eξi, τ n = E ξ i 3, S n = ξ i. i=1 i=1 i=1 ξ i c n s n for 1 i n and some 0 < c n 1. Then 6.5 PS n xs n = e γx/sn 1 Φx1 + θ n,x 1 + xs 3 n τ n for 0 < x 1/18c n, where θ n,x 36 and γx is the Cramér Petrov series see Petrov [18] satisfying γx x 3 τ n s 3 n. In particular, we have 6.6 PS n xs n = 1 Φx1 + θ n,x 1 + x 3 s 3 n τ n for 0 < x 1/18c 1/3 n, where θ n,x 40. Proof. 6.5 is due to Sakhanenko [19] see Example 1 in Sakhanenko [19] and 6.6 follows from 6.5. The next lemma is simple but useful to provide a moderate deviation of the convolution of independent sums.

23 A NEW TEST STATISTICS 3 Lemma 6.3. Let U 1, U, V 1 and V be independent random variables. Assume that there exist 0 c 0 1 and x 0 such that, for any 0 x x 0, 6.7 P U 1 x = P V 1 x1 + θ 1,x and 6.8 P U x = P V x1 + θ,x, where θ 1,x c 0 and θ,x c 0. Then 6.9 PU 1 + U x = PV 1 + V x 1 + θ x for 0 x x 0, where θ x 3c 0. Proof. Observe that P U 1 x = 1 = P V 1 x for x 0, so 6.7 and 6.9 remain valid for x < 0 with θ 1,x = 0 = θ,x. Hence, for 0 x x 0, PU 1 + U x = E{PU 1 x U U } E{PV 1 x U U 1 + θ 1,x U } 6.10 Similarly, E{PV 1 x U U 1 + c 0 } 1 + c 0 PU x V 1 = E{PU x V 1 V 1 } 1 + c 0 E{PV x V c 0 } = 1 + c 0 PV 1 + V x 1 + 3c 0 PV 1 + V x PU 1 + U x 1 c 0 PU x V 1 This proves 6.9 by 6.10 and c 0 PV 1 + V x 1 c 0 PV 1 + V x. Remark 6.1. It is easy to see that Lemma 6.3 remains valid for m independent squared variables. Lemma 6.4. If condition 1.7 is satisfied, then 6.1 E X 1,1 +4α /1 + log X 1,1 4+4α <.

24 4 W.-D. LIU, Z. LIN AND Q.-M. SHAO Proof. It is easy to see that 1.7 implies E X 1,1 X 1, +4α /1 + log X 1,1 X 1, 4+4α <, which yields 6.1 by the independence of X 1,1 and X 1,. Proof of Proposition 5.1. Let k,1,, ˆXk,1,, Y 1 k,1, 5.1 and 5.. Let A = d n i=1 { Y i k,1, > } n logn 4. be defined in Then P k,1, ny n = P k,1, ny n,a + P k,1, 6.13 ny n,a c P k,1, ny n,a + P ˆX k,1, ny n. First, we prove that P n k,1, ny n,a is small, that is, P k,1, 6.14 ny n,a Cp 3. log n θ y logn Noting that for log n θ y logn P k,1, ny n,a d P k,1, ny n, Y 1 i,1, > i= d P i=1,k i = d P k,1, 1 ε nlogp i=1,k i P Y 1 n i,1, > log n 4, n logn 4 k,1, 1 ε nlogp, Y 1 i,1, > n log n 4

25 we have, by Lemma 6.1, P k,1, 1 ε nlogp,k i P X k,1, 1 ε nlogp,k i A NEW TEST STATISTICS 5 + dnp Y 1 1,1, d 1/ ε nlogp dnp Y 1 1,1, d 1/ ε nlogp + np X 1,1, δ nlogp + exp 41 ε nlogp + Cn q 1 + βλ n CnP Y 1 1,1, δ nlogp + exp 1 ε nlogp 1 + βλ n + Cn q, where δ and δ are some positive numbers, β is any positive number, q is a large number and Λ n = n 1{Eu,X 1,1, : u 1} = n 1. For log n θ y logn, by letting β small enough such that 1 ε 1 + β 1 > 1 ε, we have exp 1 ε nlogp 1 + βλ n By Markov s inequality and Lemma 6.4, np 1 ε P and Cp 1 ε /1+β Cp 1 ε. Y 1 1,1, > n log n 4 Cp 3 n P Y 1 1,1, δ nlogpp Y 1 n 1,1, > logn 4 Cp 3. This proves Now we come to estimate P n ˆXk,1, ny n. Observe that P ˆX k,1, ny n P b 1/ n ˆX k,1, E ˆX k,1, b 1/ n 6.16 ny n dna n P nb n 1/ ˆX k,1, E ˆX k,1, y n c n,

26 6 W.-D. LIU, Z. LIN AND Q.-M. SHAO where a n = E X 1,1, I{ X 1,1, n/log n 4 }, b n = VarŶ 1 1,1, and c n = C na n + C logne Y 1 1,1, I{ Y 1 1,1, n/log n 4 } C δ p +δ for any δ > 0. By Lemma 6., for 0 x logn 4/3 /100, we have 1 P EŶ k,1, nx b 1/ n Ŷ 1 k,1, = 1 Φx1 + θ n,x 1 + x 3 nb n 3/ τ n, where θ n,x 40 8 and τ n = ne Ŷ 1 1,1, 3. So by Lemma 6.3 see also Remark 6.1, we get, for 0 x logn 4/3 /100, P b 1/ n ˆX k,1, E ˆX k,1, nx 6.17 = Pχ d x 1 + θ n,x 1 + x3 nb n 3/ τ n, where θ n,x 3d Putting 6.16 with 6.17 together yields, for log n θ y logn and some 0 < C < not depending on y, P ˆX k,1, ny n Pχ d y n + c n y d 1 n exp y n c n / + CPχ d y n c n 1 + y n 3 nb n 3/ τ n Pχ d α p + y + Cp n 1/ logn 5/ + Cp c n logn 3/ + p log n 5/ n 1/ E Ŷ 1 1,1, 3 Pχ d α p + y + Cp 3 + p logn 5/ n 1/ E Ŷ 1 1,1, 3. Similarly, we have, for log n θ y logn, P ˆX k,1, ny n Pχ d α p + y Cp 3 + p logn 5/ n 1/ E Ŷ 1 1,1, 3. This completes the proof of Proposition 5.1 by combining the above inequalities.

27 Proof of Proposition 5.. Since A NEW TEST STATISTICS Pχ d x 1 d/ Γ 1 d/x d/ 1 exp x/ as x, we have 6.19 Noting that exp y log n θ p p Pχ d α p + y Cn. 6.0 P T p,n nα p ny P T p,n nα p nlog n θ, y log n θ Propositions 5.1 and 5.4 imply 6.1 P T p,n nα p nlog n θ e ϕn + Cp 1+0 ε, where ϕ n = p p P k,1, nα p nlog n. θ By Proposition 5.1 again, we have ϕ n log n for n large. Now Proposition 5. follows by 6.19, 6.0 and 6.1. Proof of Proposition 5.3. By 6.18, 6. 1 exp p p Pχ d α p + y Cn 1. y logn Moreover, by Proposition 5.1, P T p,n y log n 6.3 nα p + ny P T p,n nα p + nlogn p P k,1, nα p + nlogn log n5/ C n 1/ E XX 3 I{ XX n/log n 4 } + Cp 1 + Cn 1. Proposition 5.3 now follows from 6. and 6.3.

28 8 W.-D. LIU, Z. LIN AND Q.-M. SHAO Proof of Proposition 5.4. Observe that P i,1, v n, i,1,3 v n i=1 i=1 P i,1, E i,1, 1 εv n, i=1 i,1,3 E i,1,3 1 εv n i=1 6.4 P i,1, E i,1,, i,1,3 E i,1,3 1 εv n i=1 P i,1, E i,1,, i,1,3 E i,1,3 i=1 3ε nlogp. Take β = 19ε, q = 4α, r = + α, δ = βr 3q + 16r 3 1 = 1ε/5 and x = 3ε nlogp in 6.. Since i,1, ε nlogp, we have 6.5 P i,1, E i,1,, i,1,3 E i,1,3 δx = 0. It can be shown that the largest eigenvalue of the covariance matrix of 1,1, E 1,1,, 1,1,3 E 1,1,3 tends to 1 as n. Therefore, for 0 < ε < 10 4, 6.6 x exp εΛ n C exp 3ε logp ε Proposition 5.4 is proved by 6. and Cp 4+0 ε. Proof of Proposition 5.5. The proof of 5.5 follows from the proof of Proposition 5., while the proof of 5.6 is similar to that of Proposition 5.3. The details are omitted here. Proof of Let λ n = p p Pχ 1 yn, y n = 4logp log p + y. It is known that πyn yn e y n / Pχ 1 yn e y n /. πyn

29 Thus, By 6.7 again, Hence, λ n p p exp πyn A NEW TEST STATISTICS 9 y n λ n 1 p 1 logp πyn = 1 p 1 logp exp y. πyn exp y 1 O. logn λ n = 1 exp y 1 + A n exp y 1 + O, 8π 8π logn where A n log n/8log n as n. Finally, we can write e λn = 1 B n exp 1 exp y, B n log n 1 exp y. 8π 8logn 8π This proves Acknowledgments. We thank an anonymous referee and an Associate Editor for their insightful comments. REFERENCES [1] Arratia, R., Goldstein, L. and Gordon, L Two moments suffice for Poisson approximations: The Chen Stein method. Ann. Probab MR97770 [] Candès, E. J., Romberg, J. and Tao, T Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inform. Theory MR36170 [3] Chen, J. and X. Huo, X Sparse representations for multiple measurement vectors MMV in an over-complete dictionary. In International Conference on Acoustics, Speech and Signal Processing ICASSP-005. Philadelphia, PA. [4] Donoho, D. L High-dimensional date analysis: The curses and blessings of dimensionality. Available at donoho/. [5] Donoho, D. L For most large underdetermined systems of linear equations the minimal l 1-norm solution is also the sparsest solution. Comm. Pure Appl. Math MR17606 [6] Donoho, D. L. and Elad, M Optimally sparse representation in general nonorthogonal dictionaries via l 1 minimization. Proc. Natl. Acad. Sci. USA electronic. MR [7] Donoho, D. L., Elad, M. and Temlyakov, V. N Stable recovery of sparse overcomplete representations in the presence of noise. IEEE Trans. Inform. Theory MR3733 [8] Donoho, D. L. and Huo, X Uncertainty principles and ideal atomic decomposition. IEEE Trans. Inform. Theory MR [9] Donoho, D. L. and Stark, P. B Uncertainty principles and signal recovery. SIAM J. Appl. Math MR99798

30 30 W.-D. LIU, Z. LIN AND Q.-M. SHAO [10] Einmahl, U. and Li, D Characterization of LIL behavior in Banach space. Trans. Amer. Math. Soc [11] Fan, J. and Li, R Statistical challenges with high dimensionality: Feature selection in knowledge discovery. In International Congress of Mathematicians III Eur. Math. Soc., Zürich. MR75698 [1] Galambos, J., Lechner, J. and Simiu, E., eds Extreme Value Theory and Applications. I. Kluwer Academic, Dordrecht. [13] Hall, P On the rate of convergence of normal extremes. J. Appl. Probab MR [14] Jiang, T The asymptotic distributions of the largest entries of sample correlation matrices. Ann. Appl. Probab MR05906 [15] Johnstone, I. M On the distribution of the largest eigenvalue in principal components analysis. Ann. Statist MR [16] Leadbetter, M. R., Lindgren, G. and Rootzén, H Extremes and Related Properties of Random Sequences and Processes. Springer, New York. MR69149 [17] Ledoit, O. and Wolf, M. 00. Some hypothesis tests for the covariance matrix when the dimension is large compared to the sample size. Ann. Statist MR [18] Petrov, V. V Sums of Independent Random Variables. Springer, New York. MR [19] Sakhanenko, A. I Estimates of Berry Esseen type for the probabilities of large deviations. Sibirsk. Mat. Zh , 8. MR [0] Schott, J. R Testing for complete independence in high dimensions. Biometrika MR34197 [1] Zhou, W Asymptotic distribution of the largest off-diagonal entry of correlation matrices. Trans. Amer. Math. Soc MR37033 W.-D. Liu Department of Mathematics Zhejiang University Hangzhou China liuweidong99@gmail.com Q.-M. Shao Department of Mathematics Hong Kong University of Science and Technology Clear Water Bay Kowloon Hong Kong China maqmshao@ust.hk Z. Lin Department of Mathematics Zhejiang University Hangzhou China zlin@zju.edu.cn