On singular values distribution of a matrix large auto-covariance in the ultra-dimensional regime. Title

Transcription

1 itle On singular values distribution of a matrix large auto-covariance in the ultra-dimensional regime Authors Wang, Q; Yao, JJ Citation Random Matrices: heory and Applications, 205, v. 4, p. article no Issued Date 205 URL Rights Electronic version of an article published as [Random Matrices: heory and Applications, 205, v. 4, p. article no ] [0.42/S X] [copyright World Scientific Publishing Company] [ his wor is licensed under a Creative Commons Attribution- NonCommercial-NoDerivatives 4.0 International License.

2 On singular values distribution of a large auto-covariance matrix in the arxiv: v [math.pr] 27 Jan 205 ultra-dimensional regime Qinwen Wang and Jianfeng Yao Qinwen Wang Department of Mathematics Zhejiang University wqw883@gmail.com Jianfeng Yao Department of Statistics and Actuarial Science he University of Hong Kong Pofulam, Hong Kong jeffyao@hu.h Abstract: Let ε t t>0 be a sequence of independent real random vectors of p-dimension and let X = s+ t=s+ ε tε t s/ be the lag-s s is a fixed positive integer autocovariance matrix of ε t. his paper investigates the limiting behavior of the singular values of X under the so-called ultra-dimensional regime where p and in a related way such that p/ 0. First, we show that the singular value distribution of X after a suitable normalization converges to a nonrandom limit G quarter law under the forth-moment condition. Second, we establish the convergence of its largest singular value to the right edge of G. Both results are derived using the moment method. AMS 2000 subject classifications: 5A52, 60F5;. Keywords and phrases: Auto-covariance matrix, Singular values, Limiting spectral distribution, Ultra-dimensional data, Largest eigenvalue, Moment method. he research of J. Yao is partly supported by GRF Grant HKU 70543P. imsart-generic ver. 204/0/6 file: autocross.tex date: January 28, 205

3 Q. Wang and J. Yao/Singular values distribution of a ultra-large auto-covariance matrix 2. Introduction Let s be a fixed positive integer and ε t t +s a sequence of independent real random vectors, where ε t = ε it i p has independent coordinates satisfying Eε it = 0 and Eε 2 it =. Consider the so-called lag-s sample autocovariance matrix of ε t defined as X = s+ t=s+ ε t ε t s.. Motivated by their application in high-dimensional statistical analysis where the dimensions p and are assumed large tending to infinity, spectral analysis of such sample autocovariance matrices have attracted much attention in recent literature in random matrix theory. For example, perturbation theory on the matrix X has been carried out in Lam and Yao 202 and Li et al. 204 for estimating the number of factors in a large dimensional factor model of type y t = Λf t + ε t + µ,.2 where {y t } is a p-dimensional sequence observed at time t, {f t } a sequence of m-dimensional latent factor m p uncorrelated with the error process {ε t } and µ R p is the general mean. Since X is not symmetric, its spectral distribution is given by the set of its singular values which are by definition the square roots of positive eigenvalues of A := X X..3 o our best nowledge, all the existing results on X or A are found under what we will refer as the Marčeno-Pastur regime, or simply the MP regime, where p, and p/ c > 0..4 For example, Jin et al 204 derives the limit of the eigenvalue distributions ESD of the symmetrized auto-covariance matrix X 2 + X ; and Wang et al. 203 establishes the exact separation property of the ESD which also implies the convergence of its extreme imsart-generic ver. 204/0/6 file: autocross.tex date: January 28, 205

4 Q. Wang and J. Yao/Singular values distribution of a ultra-large auto-covariance matrix 3 eigenvalues. For the singular value distribution of X, the limit LSD has been established in Li et al. 203 using the method of Stieltjes transform and in Wang and Yao 204 using the moment method. he latter paper also establishes the almost sure convergence of the largest singular value of X to the right edge of the LSD, thans to the moment method. Related results are also proposed in Liu et al. 203 where the sequence ε t is replaced by a more general time series. In this paper, we investigate the same questions as in Wang and Yao 204 but under a different asymptotic regime, the so-called ultra-dimensional regime where p, and p/ 0..5 It is naturally expected that the limit under this regime will be much different than under the MP regime above. he findings of the paper confirm this difference by providing a new limit of the singular value distribution of X under the ultra-dimensional regime. In a related paper Wang et Paul 204, the authors also adopted the ultra-dimensional regime to derive the LSD for a large class of separable sample covariance matrices. However, the autocovariance matrix X considered in this paper is very different of these separable sample covariance matrices. Recalling the definition of A in.3, we have A i, j = 2 p l= m= n= ε i m+s ε lm ε j n+s ε ln. It follows by simple calculations that 0, i j, EA i, j = p/, i = j, and for i j, Var A i, j = EA 2 i, j = p 2. imsart-generic ver. 204/0/6 file: autocross.tex date: January 28, 205

5 Q. Wang and J. Yao/Singular values distribution of a ultra-large auto-covariance matrix 4 he row sum of the variances Var A i, j is thus of order p 2 / 2. herefore, in order to have the spectrum of A be of constant order when p/ 0, we should normalise it as A := A p2 / 2 = p X X..6 he main results of the paper are as follows. First in Section 2, we derive the almost sure limit of the singular value distribution of p X under the ultra-dimensional regime and assuming that the fourth moment of the entries {ε it } are uniformly bounded. his limit LSD simply equals to the image measure of the semi-circle law on [ 2, 2] by the absolute value transformation x x. Next in Section 3, we establish the almost sure convergence of the largest singular value of p X to 2 assuming that the entries {ε it } has a uniformly bounded moment of order 4 + ν for some ν > 0. Both results are derived using the moment method. Some technical details on the traditional truncation and renormalisation steps are postponed to the appendixes. 2. Limiting spectral distribution by the moment method In this section, we show that when p/ 0, the ESD of the singular values of tends to a nonrandom limit, which is lined to the well nown semi-circle law. p X heorem 2.. Suppose the following conditions hold: a. ε t t is a sequence of independent p-dimensional real valued random vectors with independent entries ε it, i p, satisfying Eε it = 0, Eε 2 it =, sup Eε 4 it <. 2. it b. Both p and tend to infinity in a related way such that p/ 0. hen, with probability one, the empirical distribution of the singular values of tends to the quarter law G with density function p X gx = π 4 x2, 0 < x imsart-generic ver. 204/0/6 file: autocross.tex date: January 28, 205

6 Q. Wang and J. Yao/Singular values distribution of a ultra-large auto-covariance matrix 5 Remar 2.. Recall that the quarter law G is the image measure of the semi-circle law by the absolute value transformation. It is also worth noticing that if there were no lag, i.e. s = 0, the matrix X would be a standard sample covariance matrix; and in this case the spectral distribution of p X I p would converge to the semi-circle law, see Bai and Yin 988. he case of a auto-covariance matrix X with a positive lag s > 0 is then very different. Since the singular values of p X are the square roots of the eigenvalues of p X X, in the remaining of this paper, we focus on the limiting behaviours of the eigenvalues of X p X. hese properties can then be transferred to the singular values of X p by the square-root transformation x x. heorem 2.2. Under the same conditions as in heorem 2., with probability one, the empirical spectral distribution F A of the matrix A in.6 tends to a limiting distribution F, which is the image measure of the semi-circle law on [ 2, 2] by the square transformation. In particular, its -th moment is: m = 2, 2.3 and its Stieltjes transform sz and density function fx are given by sz = z, z / 0, 4], 2.4 and fx = π x 4, 0 < x 4, 2.5 respectively. Remar 2.2. he -th moment in 2.3 is exactly the 2-th moment of the LSD of a standard Wigner matrix, which is also the number of Dyc paths of length 2 for the definition of Dyc paths, we refer to ao 202. Notice also that the density function f is unbounded at the origin. imsart-generic ver. 204/0/6 file: autocross.tex date: January 28, 205

7 Q. Wang and J. Yao/Singular values distribution of a ultra-large auto-covariance matrix 6 he remaining of the section is devoted to the proof of heorem 2.2 using the moment method. he -th moment of the ESD F A of A is m A = p tr A = i= p j= p + ε j i ε j i 2 ε j2 s+i 2 ε j2 s+i 3 ε j2 i 2 ε j2 i 2 ε j2 s+i 2 ε j2 s+i. 2.6 Here, the indexes in i = i,, i 2 run over, 2,, and the indexes in j = j,, j 2 run over, 2,, p. he core of the proof is to establish the following two assertions: I. Em A m = II. Varm A <. p= 2, 0; his is given in the Subsections 2., 2.2 and 2.3 below. It follows from these assertions that almost surely, m A m for all 0. Since the limiting moment sequence m clearly satisfies the Carleman s condition, i.e. >0 m /2 2 =, we deduce that almost surely, the sequence of ESDs F A wealy converges to a probability measure F whose moments are exactly m. Next, notice that m is exactly the number of Dyc paths of length 2 ao, 202, which is also the 2-th moment of the semi-circle law with support [ 2, 2], it follows that the LSD F equals to the image of the semi-circle law by the square transformation x x 2. he formula in 2.4 and 2.5 are thus easily derived and the proof of heorem 2.2 is complete. 2.. Preliminary steps and some graph concepts We now introduce the proofs for Assertions I and II. First we show that with a uniformly bounded fourth order moment, the variables {ε it } can be truncated at rate η /4 for some vanishing sequence η = η. his is justified in Appendix A. After these imsart-generic ver. 204/0/6 file: autocross.tex date: January 28, 205

8 Q. Wang and J. Yao/Singular values distribution of a ultra-large auto-covariance matrix 7 truncation, centralisation and rescaling steps, we may assume in all the following that Eε ij = 0, Eε 2 ij =, ε ij η /4, 2.7 where η is chosen such that η 0 but η /4. Now we introduce some basic concepts for graphs associated to the big sum in 2.6. Let ψe,, e m := number of distinct entities among e,, e m, i := i,, i 2, j := j,, j 2, i a, j b p, a, b =,, 2, At, s := {i, j : ψi = t, ψj = s}. Define Qi, j as the multigraph as follows: Let I-line, J-line be two parallel lines, plot i,, i 2 on the I-line, j,, j 2 on the J-line, called the I-vertexes and J-vertexes, respectively. Draw down edges from i 2u to j 2u, down edges from i 2u + s to j 2u, up edges from j 2u to i 2u, up edges from j 2u to i 2u+ + s all these up and down edges are called vertical edges and horizontal edges from i 2u to i 2u + s, horizontal edges from i 2u + s to i 2u with the convention that i 2+ = i, where all the u s are in the region: u. An example of the multi-graph Qi, j with = 3 is presented in the following Figure.! = " # = $ % = & I-line ' " '! = ' $ ' # = ' & ' % J-line p Figure : An example of the multigraph Qi, j with = 3. In the graph Qi, j, once a I-vertex i l is fixed, so is i l + s. For this reason, we glue all the I-vertexes which are connected through horizon edges and denote the resulting graph imsart-generic ver. 204/0/6 file: autocross.tex date: January 28, 205

9 Q. Wang and J. Yao/Singular values distribution of a ultra-large auto-covariance matrix 8 as MAt, s, where At, s is the index set that has t distinct I-vertexes and s distinct J- vertexes. An example of MA3, 4 that corresponds to the Qi, j in Figure is presented in the following Figure 2.! = " # = $ % = & I-line ' " '! = ' $ ' # = ' & ' % J-line p Figure 2: An example of MA3, 4 that corresponds to the Qi, j in Figure Proof of Assertion I Recall the expression of m A in 2.6, we have Em A = i= p j= p + E[ ε j i ε j i 2 ε j2 s+i 2 ε j2 s+i 3 ε j3 i 3 ε j3 i 4 ε j4 s+i 4 ε j4 s+i 5 ] ε j2 i 2 ε j2 i 2 ε j2 s+i 2 ε j2 s+i = pp p s + t + p + t,s MAt,s E [ ] ε j i ε j i 2 ε j2 s+i 2 ε j2 s+i 3 ε j2 i 2 ε j2 s+i 2 ε j2 s+i := t,s St, s, 2.8 where St, s = p + MAt,s pp p s + t + E [ ε j i ε j i 2 ε j2 s+i 2 ε j2 s+i ]. 2.9 hen we assert a lemma stating that St, s 0 except for one particular term. imsart-generic ver. 204/0/6 file: autocross.tex date: January 28, 205

10 Q. Wang and J. Yao/Singular values distribution of a ultra-large auto-covariance matrix 9 Lemma 2.. St, s 0 as p unless t = and s = +. Suppose Lemma 2. holds true for a moment, then according to 2.8 and 2.9, we have Em A = S, + + o = E[ ] #{MA, + } + o, 2.0 where E[ ] refers to the expectation part in 2.9 and #{MA, + } refers to the number of isomorphism class that have distinct I-vertexes and + distinct J-vertexes. First, we show the expectation part E[ ] equals when t = and s = +. Let v m denote the number of edges in MAt, s whose degree is m. hen we have the total number of edges having the following relationship: v + 2v v 4 = Since we have Eε ij = 0 in 2.7, all the multiplicities of the edges in the graph MAt, s should be at least two, that is v = 0. On the other hand, MAt, s is a connected graph with t + s vertexes and v + + v 4 = v v 4 edges, we have when t = and s = + : 2 + = t + s v + + v 4 + = v v v 2 + 3v v 4 + = 2 +, 2.2 where the last equality is due to 2. with v = 0. hen we have all the inequalities in 2.2 become equalities, that is, v v 4 + = 2 2v 2 + 3v v 4 + = 2 +, which leads to the fact that v 3 = v 4 = = v 4 = 0, v 2 = his means that all the edges in the graph MA, + is repeated exactly twice, so the part of expectation E [ ] ε j i ε j i 2 ε j2 s+i 2 ε j2 s+i = Eε 2 2 ji =. 2.4 imsart-generic ver. 204/0/6 file: autocross.tex date: January 28, 205

11 Q. Wang and J. Yao/Singular values distribution of a ultra-large auto-covariance matrix 0 Second, the number of isomorphism class in MAt, s with each edge repeated at least twice in the original graph Qi, j is given by the notation f t in Wang and Yao 204, where f t = 2. t t herefore, in this special case when t = and s = +, we have #{MA, + } = f = Finally, combine 2.0, 2.4 and 2.5, we have Em A = 2 + o. Assertion I is then proved. It remains to prove Lemma 2.. Proof. of Lemma 2. Denote b l as the degree that associated to the I-vertex i l l t in MAt, s, then we have b + + b t = 4, which is the total number of edges. On the other hand, since each edge in MAt, s is repeated at least twice otherwise, there exist at least one single edge, so the expectation will be zero, we have each degree b l at least four we glue the original I-vertexes i l and i l + s in MAt, s. herefore, we have 4 = b + + b t 4t, which is t. Now, consider the following two cases separately. Case : s > +. Recall the definition of v m in 2., which satisfies that v + 2v v 4 = 2v v 4 = 4 imsart-generic ver. 204/0/6 file: autocross.tex date: January 28, 205

12 Q. Wang and J. Yao/Singular values distribution of a ultra-large auto-covariance matrix and t + s v + + v 4 + = v v 4 +. We can bound the expectation part as follows: E[ε j i ε j i 2 ε j2 s+i 2 ε j2 s+i 3 ε j2 s+i 2 ε j2 s+i ] Eε 2 ji v2 Eε 4 ji v 4 η /4 v 3 +2v v 4 = η /4 3v 3 +4v v 4 2v 3 +v 4 + +v 4 = η /4 4 2v 2 +v 3 + +v 4 η /4 4 2t+s. 2.6 hen we have according to 2.9 that St, s p + t p s η /4 4 2t+s #{MAt, s} = ps 2 s t η4 2t+s #{MAt, s} p s = O, 2.7 η4 2t+s s t 2 where the last equality is due to the fact that #{MAt, s} is a function of is fixed, which could be bounded by a large enough constant. Since s > + and t + s 2, then s s 2 + t = s 2 + t 2 2 = s + t 2 0, 2 which is 0 < s s t. 2 So, 2.7 reduces to p s 2 St, O η 4 2t+s 0, 2.8 which is due to the fact that s > 0 and p/ 0. imsart-generic ver. 204/0/6 file: autocross.tex date: January 28, 205

13 Q. Wang and J. Yao/Singular values distribution of a ultra-large auto-covariance matrix 2 Case 2: s +, but not t = and s = +. For the same reason as before, we have t distinct I-vertexes, each degree is at least four, so we have another estimation for the expectation part: E[ε j i ε j i 2 ε j2 s+i 2 ε j2 s+i 3 ε j2 s+i 2 ε j2 s+i ] η /4 4 4t. 2.9 herefore, St, s p + t p s η /4 4 4t #{MAt, s} η 4 4t = O, 2.20 p + s which is also due to the fact that #{MAt, s} = O. Case 2 contains three situations:. t = and s < + : St, s O 0 ; p + s 2. t < and s = + : St, s O η 4 4t 0 ; η 4 4t 3. t < and s < + : St, s O p + s Combine 2.8 and 2.2, we have St, 0 as p unless t = s = Proof of Assertion II Recall Varm A = [ ] E p εqi,j ε Qi2,j 2 E εqi,j E εqi2,j 2. i,j,i 2,j imsart-generic ver. 204/0/6 file: autocross.tex date: January 28, 205

14 Q. Wang and J. Yao/Singular values distribution of a ultra-large auto-covariance matrix 3 If Qi, j has no edges coincident with edges of Qi 2, j 2, then E ε Qi,j ε Qi2,j 2 E εqi,j E εqi2,j 2 = 0 by independence between ε Qi,j and ε Qi2,j 2. If Q = Qi, j Qi 2, j 2 has an overall single edge, then E ε Qi,j ε Qi2,j 2 = E εqi,j E εqi2,j 2 = 0, so in the above two cases, we have Varm A = 0. Now, suppose Q = Qi, j Qi 2, j 2 has no single edge, Qi, j and Qi 2, j 2 have common edges. Let the number of vertexes of Qi, j, Qi 2, j 2, Q = Qi, j Qi 2, j 2 on the I-line be t, t 2, t, respectively; and the number of vertexes on the J-line be s, s 2, s, respectively. Since Qi, j and Qi 2, j 2 have common edges, we must have t t + t 2, s s + s 2. Similar to 2.6 and 2.9, we have two bounds for E εqi,j ε Qi2,j 2 : E εqi,j ε Qi2,j 2 η /4 8 2t+s, 2.23 or E εqi,j ε Qi2,j 2 η /4 8 4t For the same reason, we have also Eε Qi,j Eε Qi2,j 2 η /4 4 2t +s +4 2t 2 +s 2 < η /4 8 2t+s, 2.25 or E εqi,j ε Qi2,j 2 η /4 4 4t +4 4t 2 < η /4 8 4t, 2.26 where the last inequalities in 2.25 and 2.26 are due to the fact that t t + t 2, s s + s 2. imsart-generic ver. 204/0/6 file: autocross.tex date: January 28, 205

15 Q. Wang and J. Yao/Singular values distribution of a ultra-large auto-covariance matrix 4 Since Varm A = p t,s MAt,s [ E εqi,j ε Qi2,j 2 E εqi,j E εqi2,j 2 ] := t,s St, s Using 2.23, 2.24, 2.25 and 2.26, we can bound the value of St, s as follows: St, t p s s O η /4 8 2t+s p p s 2 2 = O, 2.28 s/2 t/2 /2 or St, t p s s O η /4 8 4t p = O p s Clearly, t + s 2 +, t 2 + s ; we have thus t + s t + t 2 + s + s 2 4. First, consider the case that s > t + where we use the bound in Since s 2 2 s/2 + t/2 + /2 = s/2 + t/2 2 3/2 3/2, which leads to s 2 2 3/2 + s/2 t/2 /2. imsart-generic ver. 204/0/6 file: autocross.tex date: January 28, 205

16 Q. Wang and J. Yao/Singular values distribution of a ultra-large auto-covariance matrix 5 Combine with 2.28, we have St, p s O p s t 3/2 2 O p 3/ Second, we use the bound in 2.29 for the case s t +. Recall that t + s 4, we have s + s t + s 4, which is 2s 4. hen, from 2.29, St, s O p s 2 2 O p = O p 3/ Combine 2.27, 2.30 and 2.3, we have Varm A Cp 3/2, which is summable with respect to p. Assertion II is then proved. 3. Convergence of the largest eigenvalue of A In this section, we aim to show that the largest eigenvalue of A tends to 4 almost surely, which is the right edge of its LSD. heorem 3.. Under the same conditions as in heorem 2., with sup it Eε 4 it < in 2. replaced by sup it E ε it 4+ν < for some ν > 0, the largest eigenvalue of A converges to 4 almost surely. Recall that in the proof of heorem 2.2, a main step is Lemma 2., which says that St, s 0 except for one term, which is when t = and s = +. One thing to mention here is that in order to prove this lemma, is assumed to be fixed. hen the number of imsart-generic ver. 204/0/6 file: autocross.tex date: January 28, 205

17 Q. Wang and J. Yao/Singular values distribution of a ultra-large auto-covariance matrix 6 isomorphism class in MAt, s is a function of, thus can be bounded by a large enough constant. So actually, we do not need to now the value of #{MAt, s} exactly. While in the case of deriving the convergence of the largest eigenvalue, should grow to infinity, so we can not trivially guarantee that the number of isomorphism class in MAt, s is still of constant order. herefore, the main tas in this section is to bound this value, maing St, s t or s + still a smaller order compared with the main term S, + when. Proposition 3.. Let the conditions in heorem 2. hold, with sup it Eε 4 it < in 2. replaced by sup it E ε it 4+ν < for some ν > 0, and = p, is an integer that tends to infinity and satisfies the following conditions: / log p, p/ 0, /p.0 3. hen we have Em A = 2 + o. Now suppose the above Proposition 3. holds true. We first show it will lead to heorem 3.. Proof. of heorem 3. Using Proposition 3., we have the estimation that then for any > 0, we have Em A = 2 + o, 3.2 P l > 4 + P tr A 4 + E tr A 4 + = p Em A 4 + imsart-generic ver. 204/0/6 file: autocross.tex date: January 28, 205

18 Q. Wang and J. Yao/Singular values distribution of a ultra-large auto-covariance matrix 7 p p / + o + o he right hand side tends to 4 4+ since / log p so p /. Once we fix this > 0, 3.3 is summable. he upper bound for l is trivial due to our heorem 2.2. Now it remains to prove our Proposition 3.. Proof. of Proposition 3. After truncation, centralisation and rescaling, we may assume that the ε it s satisfy the condition that Eε it = 0, Varε it =, ε it δ /2, 3.4 where δ is chosen such that δ 0 δ /2 ɛ 0 δ /2 δ 2 0 p δ More detailed justifications of 3.4 are provided in Appendix B. From the proof of heorem 2.2, we have Em A = St, s = S, + + o = 2 + o, t,s where S, + is the main term that contributes to Em A, while all other terms can be neglect. herefore, it remains to prove that when, we still have St, s = 2 o. t or s + We also consider two cases: imsart-generic ver. 204/0/6 file: autocross.tex date: January 28, 205

19 Q. Wang and J. Yao/Singular values distribution of a ultra-large auto-covariance matrix 8 Case : s > + Case 2 : s +, but not t = and s = +. Similar to 2.6 and 2.9, we have two bounds for the expectation part: E[ε j i ε j i 2 ε j2 s+i 2 ε j2 s+i 3 ε j2 s+i 2 ε j2 s+i ] δ /2 4 2t+s 3.6 or E[ε j i ε j i 2 ε j2 s+i 2 ε j2 s+i 3 ε j2 s+i 2 ε j2 s+i ] δ /2 4 4t. 3.7 Consider t = first. From Wang et al. 203, the number of isomorphism class #{MA, s} is bounded by 2, 2 s and combine this with 2.9 and 3.6, we have S, s p + ps δ /2 4 2s 2 2 s. 3.8 hen, s S, s 2 p + ps δ /2 4 2s 2 s= 2 s 2 = p + ps δ /2 4 2s s= s he right hand side of 3.9 can be bounded as 2 s= p + δ /2 4 which is dominated by the term when s = 2 since p δ 2 s 2p, δ 2. hen 3.9 reduces to p p + p2 = imsart-generic ver. 204/0/6 file: autocross.tex date: January 28, 205

20 Q. Wang and J. Yao/Singular values distribution of a ultra-large auto-covariance matrix 9 Next, we consider Case and Case 2 when t > separately. According to Wang et al. 203, the number of isomorphism class in MAs, t t > is bounded by f t 2 t, 3. s where f t = 2. t t Case s > + and t > : he part of expectation can be bounded by 3.6, and combining this with 2.9 and 3., we have St, s p + MAt,s p s t E[ε j i ε j i 2 ε j2 s+i 2 ε j2 s+i ] ps t δ /2 4 2t+s ft 2 t. 3.2 p + s Since s + 2, t 2, and a trivial relationship that t + s 2, we have 2+ t p s t St, s δ /2 4 2t+s ft 2 t. 3.3 p + t,s t=2 s s=+2 he summation over s in 3.3 can be bounded as follows: 2+ t δ 2s s p s 2 t 2+ t s 2p s δ 2, 3.4 and since p δ 2 s=+2 s=+2, the summation in 3.4 is dominated by the term of s = 2 + t. herefore, 3.3 reduces to p 2+ t t δ /2 4 2t+2+ t 2 t ft p + t=2 2 + t p t = ft = 2 p t. 3.5 t=2 t=2 t t imsart-generic ver. 204/0/6 file: autocross.tex date: January 28, 205

21 Q. Wang and J. Yao/Singular values distribution of a ultra-large auto-covariance matrix 20 For the same reason, the right hand side of 3.5 inside the summation can be bounded by p 2 2 t, p and since 2 /p = 2 / p, the dominating term in 3.5 is when t =, which reduces to 2 2 p = + 2 p Since p/ 0, we have 3.6 equals 2 o. 3.7 herefore, in this case, we have St, s = t,s 2 o. 3.8 Case 2 2 t and s + : For the same reason, combining the bound of the expectation part in 3.7 with 2.9 and 3., we have St, s = p + MAt,s p s t E[ε j i ε j i 2 ε j2 s+i 2 ε j2 s+i ] δ /2 4 4t p s t f p + t 2 t. 3.9 s herefore, we have t,s St, s + p s t δ 4 4t f t 2 t s= s t= We also consider the following three situations:. t = and s < +, imsart-generic ver. 204/0/6 file: autocross.tex date: January 28, 205

22 Q. Wang and J. Yao/Singular values distribution of a ultra-large auto-covariance matrix 2 2. < t < and s = +, 3. < t < and s < +, and show that for all the above three situations, we have 3.20 bounded by 2 o. For situation, 3.20 reduces to p s f s= = s p s s= 2 s, 3.2 which can be bounded as p 2 p s. s= herefore, the dominating term is when s =, thus 3.2 reduces to 2 p = which is due to the choice of that /p 0. 2 o, For situation 2, 3.20 reduces to δ 4 4t t f t 2 t t=2 = δ 4 4t t 2 2 t t=2 t t Since the right hand side of 3.22 can be bounded by δ 4 t= t, 3.23 δ 4 imsart-generic ver. 204/0/6 file: autocross.tex date: January 28, 205

23 Q. Wang and J. Yao/Singular values distribution of a ultra-large auto-covariance matrix 22 which is dominated by the term of t = since 22 δ 4 = 22 δ / bounded by δ = δ o, which is due to the fact that δ 4 2 = δ herefore, we have For situation 3, we have 3.20 reduce to = t=2 t=2 p s t δ 4 4t f t 2 t s p s t δ 4 4t 2 2 t t t s s= s= he part of summation over s is which could be bounded by s= p s 2 t s 2p s, s=, therefore, the dominating term is when s =. So 3.24 reduces to p δ 4 4t t t=2 2 2 t t t imsart-generic ver. 204/0/6 file: autocross.tex date: January 28, 205

24 Q. Wang and J. Yao/Singular values distribution of a ultra-large auto-covariance matrix 23 For the same reason, the right hand side of 3.25 can be bounded by p δ t 2, δ 4 t=2 which is dominated by the term of t = since 2 = 2. herefore, 3.25 δ 4 δ /4 4 reduces to p δ = O δ4 3 p 2, 3.26 and since δ4 3 p = δ 2 2 /p 0, we have 3.26 equals 2 o. Finally, in all the three situations, we have t,s St, s = he proof of Proposition 3. is complete. 2 o. Appendix A: Justification of truncation, centralisation and rescaling in 2.7 A.. runcation Define two p matrices E := ε ε 2 ε ε, E 2 := ε s+ ε s+2 ε s+ ε s+, A. then X = s+ t=s+ ε t ε t s = E 2E, A.2 imsart-generic ver. 204/0/6 file: autocross.tex date: January 28, 205

25 Q. Wang and J. Yao/Singular values distribution of a ultra-large auto-covariance matrix 24 and our target matrix Let A = p X X = p E 2E E E 2. A.3 ˆε ij = ε ij { εij η /4 }, ˆX and  are defined by replacing all the ε ij with ˆε ij in A.2 and A.3. Using heorem A.44 in Bai and Silverstein 200 and the inequality that ranab CD rana C + ranb D, we have F A x F Âx p ran p X = F p X X x F ˆX p ˆX x p ˆX = p ran E 2E Ê2Ê E p ran 2 Ê2 + p ran E Ê = p ran X ˆX = p ran E 2 E Ê2Ê = 2 E p ran Ê 2 p p { εij >η /4 }. A.4 i= j= Since sup it Eε 4 it <, we have always η 4 p i,j E ε ij 4 I εij >η /4 0 as p,. Consider the expectation and variance of p p i= j= { ε ij >η /4 } in A.4: 2 E p 2 Var p p i= p i= j= j= { εij >η /4 } { εij >η /4 } 2 p 4 p 2 p E ε ij 4 { εij >η /4 } = o, η 4 j= p E ε ij 4 { εij >η /4 } = o η 4 p. i= i= j= imsart-generic ver. 204/0/6 file: autocross.tex date: January 28, 205

26 Q. Wang and J. Yao/Singular values distribution of a ultra-large auto-covariance matrix 25 Applying Bernstein s inequality, for all small ε > 0 and large p, we have 2 p P p { εij >η /4 } ε 2e 2 ε2p. i= j= A.5 Finally, combine A.4, A.5 with Borel-Cantelli lemma, we have with probability, F A x F Âx 0. A.2. Centralisation Let ε ij = ˆε ij Eˆε ij, X and à are defined by involving the ε ij s in A.2 and A.3. Similar to A.4, we have F Âx F Ãx p ran p ˆX p X = Ê2 p ran Ê Ẽ2Ẽ 2 Ê p ran Ẽ = 2 p ran EÊ = 2 p 0, as p. herefore, we have F Âx F Ãx 0. A.3. Rescaling Let then for the same reason as A.4, we have σ 2 ij = E ε 2 ij, ˇε ij := ε ij /σ ij, F Ãx F Ǎx Ẽ2 p ran Ě2 + Ẽ p ran Ě imsart-generic ver. 204/0/6 file: autocross.tex date: January 28, 205

27 Q. Wang and J. Yao/Singular values distribution of a ultra-large auto-covariance matrix 26 = 2 Ẽ p ran Ě 2 p max ran Ẽ i j σ ij 2 p max min{p, } i j σ ij = O max. i j σ ij Since σ 2 ij = E ε 2 ij = Eˆε ij Eˆε ij 2 = Varˆε ij = Varε ij { εij η /4 } Varε ij =, as. herefore, we have F Ãx F Ǎx 0. Appendix B: Justification of truncation, centralisation and rescaling in 3.4 B.. runcation E, E 2, X and A are defined in A., A.2 and A.3. Let ˆε ij = ε ij { εij δ /2 }, ˆX and  are defined by replacing all the ε ij with ˆε ij in A.2 and A.3. With the assumption that sup it E ε it 4+ν <, we have always E ε it 4+ν { εit >δ /2 } sup 0 as p,. B. it δ 4+ν Since whose eigenvalues are the same as those of A = p E 2E E E 2, B := p E E E 2 E 2, imsart-generic ver. 204/0/6 file: autocross.tex date: January 28, 205

28 then we have Q. Wang and J. Yao/Singular values distribution of a ultra-large auto-covariance matrix 27 λ max A λ max  λmax = B λ max ˆB = p E E E2 E 2 op p Ê ÊÊ 2 Ê2 op p E E E2 E 2 p Ê ÊÊ 2 Ê2 op p E E E2 E 2 p Ê ÊE 2 E 2 + op p Ê ÊE2 E 2 = E E p Ê Ê E2 E 2 + Ê2 op p Ê Ê E2 E 2 Ê 2 p Ê ÊÊ 2 Ê2 op op := J + J 2. B.2 First, we have E E Ê Ê = max op x = xe E Ê Êx ] = max [xe E x = Ê E x + xê E Ê Êx max x = xe E Ê E x + max x = xê E Ê Êx := J + J 2, B.3 where J = max x = xe E Ê E x = max x i x j E E x = Ê E i, j p = max x i x j ε i ˆε i ε j x = i,j = p [ /2 max x 2 2 /2 i εi ˆε i x = = i i j p [ 2 /2 ] /2 = εi ˆε i = p = i /2 p 2 εi ˆε i i= j i,j ε 2 j = j= ε 2 j /2 x 2 j /2 j ε 2 j /2 ] imsart-generic ver. 204/0/6 file: autocross.tex date: January 28, 205

29 Q. Wang and J. Yao/Singular values distribution of a ultra-large auto-covariance matrix 28 p p /2 = O ε 2 i { εi >δ /2 } = i= /2 O p 2 sup E ε 2 i { εi >δ /2 } i p 2 /2 O sup E ε δ /2 2+ν i 4+ν { εi >δ /2 } i = o δp /2 ν/4, B.4 where the last inequality is due to B.. For the same reason, J 2 is also of the same order as B.4. herefore we have E E Ê Ê o δp /2 ν/4. B.5 op hen recall the definition of J in B.2, where J = E E p Ê Ê E2 E 2 op E E p Ê Ê E op 2 E op 2 o δ /2 ν/4 0, as p,, B.6 where the last inequality in B.6 is due to B.5 and the fact that E 2 E 2 op is the largest eigenvalue of the sample covariance matrix E 2E 2, which is of constant order. For the same reason, we also have J 2 the same order as J, which also tends to zero. Finally, according to B.2 we have λ max A λ max  0. B.2. Centralisation and Rescaling Let σ 2 it = Var ˆε it, ε it = ˆε it Eˆε it σ it, imsart-generic ver. 204/0/6 file: autocross.tex date: January 28, 205

30 Q. Wang and J. Yao/Singular values distribution of a ultra-large auto-covariance matrix 29 X and à are defined by replacing all the ε ij with ε ij in A.2 and A.3. In this subsection, we will show λ max  λ maxã 0, which is equivalent to showing λ max ˆB λ max B 0. First, since sup σ 2 i = sup Eε2 i E i i = sup ε E 2 i { εi >δ /2 } + i 2 ε i { εi δ /2 } E ε i { εi δ } /2 E ε i { εi >δ } 2 /2 2 sup E ε 2 2 sup i E ε i 4+ν { εi >δ /2 } i { εi >δ /2 } δ /2 2+ν i δ 2 = o 2+ν 2, B.7 where the last equality is due to B.. Finally, we have: sup i σ i = sup σ i i σ i = sup σi 2 i σ i σ i + sup = O σ 2 i i δ 2 o, B.8 2+ν 2 where the last inequality is due to B.7. Second, we have another estimation for the term sup i Eˆε i as follows: sup Eˆε i = sup E [ ε i { εi δ }] /2 = sup E [ ε i { εi >δ }] /2 i i i sup i E [ ] ε i 4+ν { εi >δ /2 } δ δ /2 3+ν = o. B.9 3+ν 2 hen similar to B.2, we have λ max ˆB λ max B imsart-generic ver. 204/0/6 file: autocross.tex date: January 28, 205

31 Q. Wang and J. Yao/Singular values distribution of a ultra-large auto-covariance matrix 30 Ê p Ê Ẽ Ẽ Ê 2 Ê2 + op p Ẽ Ê Ẽ2 Ẽ 2 Ê2 Ẽ 2 op := J 3 + J 4. Also, similar to B.3 and B.4, we have Ê Ê Ẽ Ẽ = max op x = xê Ê Ẽ Ẽx with Since = max x = xê Ê Ẽ Êx + max x = xẽ Ê Ẽ Ẽx := J 3 + J 32, B.0 J 3 = max x = xê Ê Ẽ Êx = max x i x j x = p = p = p = = O /2 p 2 ˆεi ε i i= p p = 2 ˆεi ε i = i= i= σ i i,j = j= /2 2 ˆεi ε i i= p = 2ˆε 2 i + i= p = ˆε 2 j ˆε i ˆε 2 i Eˆε i σ i i= σ 2 i Eˆε i 2 + p = = /2 p ˆε i ε i ˆε i i= 2 ˆε i Eˆε i σ i σ i B. { max O p sup 2 2 i σ i, O p sup Eˆε i, i O p sup } sup Eˆεi i σ i i { } δ 4 p δ 2 p δ 3 p max o, o, o, B.2 +ν 2+ν 3/2+ν where the last inequality is due to B.8 and B.9. hen according to B., we have the bound for the term J 3 : { δ 2 p J 3 max o ν/2 δp, o, o +ν 2 δ δp /4+ν/2 }. B.3 imsart-generic ver. 204/0/6 file: autocross.tex date: January 28, 205

32 Q. Wang and J. Yao/Singular values distribution of a ultra-large auto-covariance matrix 3 For the same reason, we have the term J 32 can be bounded by B.3 as well. herefore, we have J 3 = Ê p Ê Ẽ Ẽ Ê 2 Ê2 op p Ê Ê Ẽ Ẽ op Ê 2 Ê2 op = O p J 3 + J 32 { δ 2 δ δ } δ max o, o, o 0. ν/2 +ν 2 /4+ν/2 Similar, we also have J 4 0, which leads to the fact that λ max ˆB λ max B 0. B.4 References Bai, Z. D. and Yin, Y. Q. 988a. A convergence to the semicircle law. Ann. Probab. 62, Bai, Z.D. and Silverstein, J.W Spectral Analysis of Large Dimensional Random Matrices 2nd edition. Springer, 20. Jin, B. S., Wang, C., Bai, Z. D., Nair, K. K. and Harding, M. C Limiting spectral distribution of a symmetrized auto-cross covariance matrix. Ann. Appl. Probab. 243, Lam, C. and Yao, Q.W Factor modeling for high-dimensional time series: inference for the number of factors. Ann. Statist. 40, Li, Z., Pan, G.M. and Yao, J On singular value distribution of large-dimensional autocovariance matrices. Preprint, available at arxiv: Li, Z., Wang, Q. and Yao, J Identifying the number of factors from singular values of a large sample auto-covariance matrix Preprint, available at arxiv: Liu, H.Y., Aue, A. and Paul, D On the Marčeno-Pastur law for linear time series. Preprint, available at arxiv: imsart-generic ver. 204/0/6 file: autocross.tex date: January 28, 205

33 Q. Wang and J. Yao/Singular values distribution of a ultra-large auto-covariance matrix 32 ao, opics in Random Matrix heory. American Mathematical Society. Wang, L. and Paul, D Limiting spectral distribution of renormalized separable sample covariance matrices when p/n 0. J. Multivariate Anal. 26, Wang, C., Jin, B. S., Bai, Z. D., Nair, K. K. and Harding, M. C. 203 Strong Limit of the Extreme Eigenvalues of a Symmetrized Auto-Cross Covariance Matrix. Preprint, available at arxiv: Wang, Q. and Yao, J. 204 Moment approach for singular values distribution of a large auto-covariance matrix. Preprint, available at arxiv: imsart-generic ver. 204/0/6 file: autocross.tex date: January 28, 205