A multiple testing approach to the regularisation of large sample correlation matrices

Transcription

1 A multile testing aroach to the regularisation of large samle correlation matrices Natalia Bailey Queen Mary, University of London M. Hashem Pesaran University of Southern California, USA, and rinity College, Cambridge L. Vanessa Smith University of York November 5, 05 Abstract his aer rooses a regularisation method for the estimation of large covariance matrices that uses insights from the multile testing (M ) literature. he aroach tests the statistical signi cance of individual air-wise correlations and sets to zero those elements that are not statistically signi cant, taking account of the multile testing nature of the roblem. By using the inverse of the normal distribution at a redetermined signi cance level, it circumvents the challenge of estimating the theoretical constant arising in the rate of convergence of existing thresholding estimators, and hence it is easy to imlement and does not require cross-validation. he M estimator of the samle correlation matrix is shown to be consistent in the sectral and Frobenius norms, and in terms of suort recovery, so long as the true covariance matrix is sarse. he erformance of the roosed M estimator is comared to a number of other estimators in the literature using Monte Carlo exeriments. It is shown that the M estimator erforms well and tends to outerform the other estimators, articularly when the cross section dimension, N, is larger than the time series dimension, : JEL Classi cations: C3, C58 Keywords: Sarse correlation matrices, High-dimensional data, Multile testing, hresholding, Shrinkage he authors would like to thank Alex Chudik, Jianqing Fan, George Kaetanios, Yuan Liao, Ron Smith and Michael Wolf for valuable comments and suggestions, as well as Elizaveta Levina and Martina Mincheva for helful corresondence with regard to imlementation of their aroaches. he authors also wish to acknowledge nancial suort under ESRC Grant ES/I0366/.

2 Introduction Imroved estimation of covariance matrices is a roblem that features rominently in a number of areas of multivariate statistical analysis. In nance it arises in ortfolio selection and otimisation (Ledoit and Wolf (003)), risk management (Fan et al. (008)) and testing of caital asset ricing models (Sentana (009)). In global macro-econometric modelling with many domestic and foreign channels of interactions, error covariance matrices must be estimated for imulse resonse analysis and bootstraing (Pesaran et al. (004); Dees et al. (007)). In the area of bio-informatics, covariance matrices are required when inferring gene association networks (Carroll (003); Schäfer and Strimmer (005)). Such matrices are further encountered in elds including meteorology, climate research, sectroscoy, signal rocessing and attern recognition. Imortantly, the issue of consistently estimating the oulation covariance matrix, = ( ), becomes articularly challenging when the number of variables, N, is larger than the number of observations,. In this case, one way of obtaining a suitable estimator for is to aroriately restrict the o -diagonal elements of its samle estimate denoted by ^. Numerous methods have been develoed to address this challenge, redominantly in the statistics literature. See Pourahmadi (0) for an extensive review and references therein. Some aroaches are regression-based and make use of suitable decomositions of such as the Cholesky decomosition (see Pourahmadi (999, 000), Rothman et al. (00), Abadir et al. (04), among others). Others include banding or taering methods as roosed, for examle, by Bickel and Levina (004, 008a) and Wu and Pourahmadi (009), which assume that the variables under consideration follow a natural ordering. wo oular regularisation techniques in the literature that do not make use of any ordering assumtions are those of thresholding and shrinkage. hresholding involves setting o -diagonal elements of the samle covariance matrix that are in absolute terms below certain threshold values to zero. his aroach includes universal thresholding ut forward by El Karoui (008) and Bickel and Levina (008b), and adative thresholding roosed by Cai and Liu (0). Universal thresholding alies the same thresholding arameter to all o -diagonal elements of the unconstrained samle covariance matrix, while adative thresholding allows the threshold value to vary across the di erent o -diagonal elements of the matrix. Furthermore, the selected non-zero elements of ^ can either be set to their samle estimates or can be adjusted downward. his relates to the concets of hard and soft thresholding, resectively. he thresholding aroach traditionally assumes that the underlying (oulation) covariance matrix is sarse, where sarseness is loosely de ned as the resence of a su cient number of zeros on each row of such that it is absolute summable row (column)-wise, or more generally in the sense de ned by El Karoui (008). However, Fan et al. (0, 03) show that such regularisation techniques can be alied even if the underlying oulation covariance matrix is not sarse, so long as the non-sarseness is characterised by an aroximate factor structure. he main challenge in alying this aroach lies in the estimation of the thresholding arameter. he method of cross-validation is rimarily used for this urose which has its own limita-

3 tions and may not be aroriate in alications where the underlying model generating the observations is unstable over time. In contrast to thresholding, the shrinkage aroach reduces all samle estimates of the covariance matrix towards zero element-wise. More formally, the shrinkage estimator of is de ned as a weighted average of the samle covariance matrix and an invertible covariance matrix estimator known as the shrinkage target - see Friedman (989). A number of shrinkage targets have been considered in the literature that take advantage of a riori knowledge of the data characteristics under investigation. Examles of covariance matrix targets can be found in Ledoit and Wolf (003), Daniels and Kass (999, 00), Fan et al. (008), and Ho (009), among others. Ledoit and Wolf (004) suggest a modi ed shrinkage estimator that involves a linear combination of the unrestricted samle covariance matrix with the identity matrix. his is recommended by the authors for more general situations where no natural shrinking target exists. On the whole, shrinkage estimators tend to be stable, but yield inconsistent estimates if the urose of the analysis is the estimation of the true and false ositive rates of the underlying true sarse covariance matrix (the so called suort recovery roblem). his aer considers an alternative to cross-validation by making use of a multile testing (M ) aroach to set the thresholding arameter. he idea has been suggested by El Karoui (008,. 748) but has not been theoretically develoed in the literature. As noted by El Karoui, hard thresholding can also be imlemented by testing the N(N )= null hyotheses that = 0, for all i 6= j. However, such tests will not be standard and their critical value must be determined from the knowledge of the inferential roblem and the fact that N and both tend to in nity. he M aroach can readily accommodate both Gaussian and non-gaussian observations and does not require cross-validation which is often quite time consuming to aly. he M rocedure is shown to be equivalent to the alication of the multile testing rocedure due to Bonferroni (935) to the individual rows of, searately, when = 0 imlies indeendence, and to all distinct non-diagonal elements of, if = 0 does not imly indeendence. We show that the M estimator of R, the correlation matrix associated with, converges in sectral norm at the rate of m O N, where m N is the maximum number of non-zero elements in the o -diagonal rows q of R. his comares favourably with the corresonding O m N rate established m N N log(n) in the literature. Similarly, we show that the M estimator converges in Frobenius norm q at the rate of O, even if the underlying observations are non-gaussian. o the best of our knowledge, the only work that addresses the theoretical roerties of the thresholding estimator for the Frobenius norm is Bickel and Levina (008b), who establish the q rate of O, assuming the observations are Gaussian. he M estimator also m N N log(n) consistently recovers the suort of the oulation covariance matrix under non-gaussian observations. he erformance of the M estimator is investigated using a Monte Carlo simulation

4 study, and its roerties are comared to a number of extant regularised estimators in the literature. he simulation results show that the roosed multile testing estimator is robust to the tyical choices of used in the literature (0%, 5% and %), and erforms favourably comared to the other estimators, esecially when N is large relative to. he M rocedure also dominates other regularised estimators when the focus of the analysis is on suort recovery. he rest of the aer is organised as follows: Section outlines some reliminaries, introduces the M rocedure and derives its asymtotic roerties. he small samle roerties of the M estimator are investigated in Section 3. Concluding remarks are rovided in Section 4. Some of the technical roofs and additional simulation results are rovided in a Sulementary Aendix. Notation: We denote the largest and the smallest eigenvalues of the N N real symmetric matrix A = (a ) by max (A) and min (A) ; resectively, its trace by tr (A) = P N i= a ii, its PN maximum absolute column sum norm by kak = max jn i= ja j, its maximum PN absolute row sum norm by kak = max in j= ja j, its sectral radius by % (A) = j max (A)j, its sectral (or oerator) norm by kak sec = = max (A 0 A), its Frobenius norm by kak F = tr (A 0 A). Note that kak sec = % (A). a N = O(b N ) states the deterministic sequence fa N g is at most of order b N, x N = O (y N ) states the vector of random variables, x N ; is at most of order y N in robability, and x N = o (y N ) is of smaller order in robability than y N,! denotes convergence in robability, and! d convergence in distribution. All asymtotics are carried out under N! jointly with!. Regularising the samle correlation matrix: A multile testing (M) aroach Let fx it ; i N; t g, N N; Z, be a double index rocess where x it is de- ned on a suitable robability sace (; F; P ), and denote the covariance matrix of x t = (x t ; x t ; :::; x Nt ) 0 by V ar (x t ) = = E (x t ) (x t ) 0 ; () where E(x t ) = = ( ; ; :::; N ) 0, and is an N N symmetric, ositive de nite real matrix with (i; j) element,. We consider the regularisation of the samle covariance matrix estimator of, denoted by ^, with elements ^ ; = X (x it x i ) (x jt x j ) ; for i; j = ; ; :::; N; () t= where x i = P t= x it. o this end we assume that is (exactly) sarse de ned as follows. 3

5 Assumtion he oulation covariance matrix, = ( ); where min () " 0 > 0, is sarse in the sense that m N de ned by m N = max in NX I ( 6= 0) ; (3) j= is bounded in N, where I(A) is an indicator function that takes the value of if A holds and zero otherwise. he remaining N(N m N ) non-diagonal elements of are zero. A comrehensive discussion of the concet of sarsity alied to and alternative ways of de ning it are rovided in El Karoui (008) and Bickel and Levina (008b). De nition is a natural choice when considering concurrently the roblems of regularisation of ^ and suort recovery of. We also make the following assumtion about the bivariate moments of (x it ; x jt ). Assumtion he observations f(x i ; x j ); (x i ; x j ); ::::; (x i ; x j )g are drawn from a general bivariate distribution with mean i = E(x it ), j i j < K, variance ii = V ar(x it ), 0 < ii < K, and correlation coe cient = = ii jj satisfying 0 < min < < max <. Also, it is assumed that the following nite higher-order moments exist (; ) = E y ity jt ; (3; ) = E(y 3 ity jt ), and (; 3) = E(y it y 3 jt), (4; 0) = E(y 4 it) < K; and (0; 4) = E(y 4 jt) < K; where y it = (x it i )= ii, and E y r ity s jt = (r; s); for all r; s 0. We follow the hard thresholding literature but, as noted above, we emloy multile testing rather than cross-validation to decide on the threshold value. More seci cally, we set to zero those elements of R = ( ) that are statistically insigni cant and therefore determine the threshold value as a art of a multile testing strategy rather than by crossvalidation. We aly the thresholding rocedure exlicitly to the correlations rather than the covariances. his has the added advantage that one can use a so-called universal threshold rather than making entry-deendent adjustments, which in turn need to be estimated when thresholding is alied to covariances. his feature is in line with the method of Bickel and Levina (008b) or El Karoui (008) but shares the roerties of the adative thresholding estimator develoed by Cai and Lui (0). Seci cally, denote the samle correlation of x it and x jt, comuted over t = ; ; :::;, by ^ ; ^ ; = ^ ji; = ; (4) ^ii; ^ jj; where ^ ; is de ned by (). For a given i and j, it is well known that under H 0; : = 0, ^; is asymtotically distributed as N(0; ) for su ciently large. his suggests using = as the threshold for ^;, where () is the inverse of the cumulative distribution of a standard normal variate, and is the chosen nominal size of the test, 4

6 tyically taken to be % or 5%. However, since there are in fact N (N ) = such tests and N is large, then using the threshold = for all N(N )= airs of correlation coe cients will yield inconsistent estimates of and fails to recover its suort. A oular aroach to the multile testing roblem is to control the overall size of the n = N(N )= tests jointly (known as family-wise error rate) rather than the size of the individual tests. Let the family of null hyotheses of interest be H 0 ; H 0 ; :::; H 0n ; and suose we are rovided with the corresonding test statistics, Z ; Z ; ::::; Z n, with searate rejection rules given by (using a two-sided alternative) Pr (jz i j > CV i jh 0i ) i ; where CV i is some suitably chosen critical value of the test, and i is the observed -value for H 0i. Consider now the family-wise error rate (FWER) de ned by F W ER = Pr [[ n i= (jz i j > CV i jh 0i )] ; and suose that we wish to control F W ER to lie below a re-determined value,. One could also consider other generalized error rates (see for examle Romano et al. (008)). Bonferroni (935) rovides a general solution, which holds for all ossible degrees of deendence across the searate tests. Using the union bound, we have Pr [[ n i= (jz i j > CV i jh 0i )] nx Pr (jz i j > CV i jh 0i ) i= nx i : i= Hence to achieve F W ER, it is su cient to set i =n. Alternative multile testing rocedures advanced in the literature that are less conservative than the Bonferroni rocedure can also be emloyed. One rominent examle is the ste-down rocedure roosed by Holm (979) that, similar to the Bonferroni aroach, does not imose any further restrictions on the degree to which the underlying tests deend on each other. More recently, Romano and Wolf (005) roosed ste-down methods that reduce the multile testing rocedure to the roblem of sequentially constructing critical values for single tests. Such extensions can be readily considered but will not be ursued here. In our alication we scale by a general function of N, which we denote by f(n) and then derive conditions on f(n) which ensure consistent suort recovery and a suitable convergence rate of the error in estimation of R = ( ). In articular, we show that the sectral norm of R and its suort recovery can be consistently estimated so long as f(n) rises linearly in N, and does not deend on whether x it and x jt are deendently distributed when = 0. However, we show that under the Frobenius norm the form of f(n) deends on whether the airs (x it ; x jt ), for all i 6= j dislay non-linear deendence, in the sense that they are deendent even if = 0. As will be shown in Section., under the null hyothesis, H 0; : = 0 for all i and j, i 6= j, the degree of non-linear deendence is de ned by the 5

7 arameter max = su ( ) where = (; ) = 0. Under indeendence, max = and f(n) = N, while under non-linear deendence we have max > and f(n) = O (N max ). More recisely, the multile testing (M ) estimator of R, denoted by R e M = ~ ; is given by where ~ = ^ I ^ > = c (N) ; i = ; ; :::; N ; j = i + ; :::; N; (5) c (N) = : (6) f(n) It is evident that since c (N) is selected a riori and does not need to be estimated, the multile testing rocedure in (5) is also comutationally simle to imlement. his contrasts with traditional thresholding aroaches which face the challenge of evaluating the theoretical constant, C, arising in the rate of convergence of their estimators. A searate cross-validation rocedure is tyically emloyed for the estimation of C that has its own limitations. Finally, the M estimator of is now given by e M = ^D = e RM ^D = ; where ^D = diag(^ ; ; ^ ; ; :::; ^ NN; ). he M rocedure can also be alied to defactored observations following the de-factoring aroach of Fan et al. (0, 03).. heoretical roerties of the M estimator Next we investigate the asymtotic roerties of the M estimator de ned by (5). We begin with the following roosition. Proosition Let y it = (x it i )= ii, where i = E(x it ), j i j < K, and ii = V ar(x it ), 0 < ii < K, for all i and t, and suose that Assumtion holds. Consider the samle correlation coe cient de ned by (4) which can also be exressed in terms of y it as ^ ; = h P P t= (y it y i ) (y jt y j ) t= (y it y i ) i = h P t= (y jt y j ) i = : (7) hen where ; = E ^ ; = + K m( ) + O ; (8)! K v ( ) ; = V ar ^ ; = + O ; (9) K m ( ) = ( )+ 8 3 [ (4; 0) + (0; 4)] 4 [ (3; ) + (; 3)] + (; ) ; 6 (0)

8 K v ( ) = ( ) + 4 [ (4; 0) + (0; 4)] 4 [ (3; ) + (; 3)] + ( + ) (; ) ; (4; 0) = (4; 0) 3 (; 0) = E(y 4 it) 3; (0; 4) = (0; 4) 3 (0; ) = E(y 4 jt) 3; (3; ) = (3; ) 3 (; 0) (; ) = E(y 3 ity jt ) 3 ; (; 3) = (; 3) 3 (0; ) (; ) = E(y 3 jty it ) 3 ; (; ) = (; ) (; 0) (0; ) (; ) = (; ) ; and = ( ; (0; 4)+ (4; 0); (3; )+ (; 3); (; )) 0. Furthermore jk m ( )j < K, K v ( ) = lim! V ar ^;, and Kv ( ) < K. Proof of Proosition. he results for E ^ ; and V ar ^; are established in Gayen (95) using a bivariate Edgeworth exansion aroach. his con rms earlier ndings obtained by schurow (95, English ranslation, 939) who shows that results (8) and (9) hold for any law of deendence between x it and x jt. See, in articular,. 8 and equations (53) and (54) in Gayen (95). Using (9) and () we have lim! V ar ^; = Kv ( ). Finally, the boundedness of jk m ( )j and K v ( ) follows directly from the assumtion that the fourth-order moment of y it exists for all i and t. he existence of the other moments, E(yity 3 jt ) and E yityjt, follows by alication of Holder s and Cauchy Schwarz inequalities as given below: E(y it yjt) E(jyit j 4 ) = E jyjt j 4 = < K () and E(y it yjt) 3 E( y it yjt 3 ) E(jy it j 4 ) h =4 E yjt 3 4=3i 3=4 = E(jy it j 4 ) =4 E jyjt j 4 3=4 = E(jyit j 4 ) < K: Remark From Gayen (95).3 (eq (54)bis) it follows that K v ( ) > 0 for each correlation coe cient = = ii jj satisfying 0 < min < < max <. Further, under the null H 0; : = 0, () becomes K v ( ) = + (; ) = (; ) > 0. We introduce the following assumtion which is insired from the above roosition. Assumtion 3 he standardised correlation coe cients, z ; = ^ ; E ^ ; = qv ar ^ ;, for all i and j (i 6= j) admit the Edgeworth exansion Pr (z ; a ; jp ) = F ; (a ; jp ) () = (a ; ) + = (a ; ) G (a ; jp ) + (a ; ) G (a ; jp ) + ::::; where E ^ ; and V ar ^; are de ned by (8) and (9) of Proosition, (a; ) and (a ; ) are the cumulative distribution and density functions of the standard Normal (0; ), 7

9 resectively, and G s (a ; jp ), s = ; ; ::: are olynomials in a ;, whose coe cients deend on the underlying bivariate distribution of the observations (x it,x jt for t = ; ; :::; ) which is denoted by P. Remark While Assumtion C of Cai and Liu (0) characterising the tail-roerty of y it can be used, we ot to focus on the standardised correlation coe cient, z ;. his is a self-normalised rocess where E ^ ; and V ar ^; are given by (8) and (9) resectively. hen, for a nite, all moments of z ; exist and as!, z ;! d z s N(0; ). Hence, following the theorem of Sargan (976) on.43 the Edgeworth exansion is valid. Given Assumtions -3, rst we establish the rate of convergence of the M estimator under the sectral (or oerator) norm which imlies convergence in eigenvalues and eigenvectors (see El Karoui (008), and Bickel and Levina (008a)). heorem (Convergence under the sectral norm) Denote the samle correlation coe - cient of x it and x jt over t = ; ; :::; by ^ ; (as de ned in (7) of Proosition ) and the oulation correlation matrix by R = ( ). Suose that Assumtions -3 hold. Let f(n) be an increasing function of N, a nite constant (0 < < ), and suose there exist nite 0 and N 0 such that for all > 0 and N > N 0, f(n) > 0 and hen so long as N=! 0 we have E R e M ln f(n)=! 0; as N and! : R = O sec mn ; (3) where m N is de ned by (3), R e M = (~ ; ) = ^ ; I ^; > = c (N) ; and c (N) = > 0: f(n) Proof. See Aendix. Under the conditions of heorem, and since by Assumtions and, min (R) " 0 > 0, then the eigenvalues of RM e are bounded away from zero with robability aroaching, and we have erm R = erm R e R M R sec sec erm R min RM e sec min (R) mn = O : Also see Aendix A of Fan et al. (03) and roof of lemma A. in Fan et al. (0). Similarly, we establish the rate of convergence of the M estimator under the Frobenius norm. 8

10 heorem (Convergence under the Frobenius norm) Denote the samle correlation coe - cient of x it and x jt over t = ; ; :::; by ^ ; (as de ned in (7) of Proosition ) and the oulation correlation matrix by R = ( ). Suose that Assumtions -3 hold. Let f(n) be an increasing function of N, a nite constant (0 < < ), and suose there exist nite 0 and N 0 such that for all > 0 and N > N 0, and f(n) > 0; ln f(n)=! 0; as N and! ; max lim N! ln [f(n)] ln(n) ; (4) where max = su [ ], = (; ) = 0, with (; ) de ned in Assumtion. hen as long as N=! 0 we have E R e r! mn N M R = O ; (5) F where m N is de ned by (3), R e M = (~ ; ) = ^ ; I ^; > = c (N) ; and c (N) = > 0: f(n) Proof. See Aendix. Remark 3 For the convergence of the Frobenius norm, E R e M R, at the rate of F mn O N=, the rate at which f(n) rises with N is dictated by the magnitude of max. For examle if max = ; setting f(n) = N meets all the conditions of heorem. But for values of max >, we need f(n) to rise with N at a faster rate. For max (; ], it is su cient to set f(n) = N(N )=. It is easily seen that in this case lim N! ln f(n) ln(n) ln(n) + ln(n ) ln() max = lim N! ln(n) max = max ; and the conditions are met if max. Similarly, for max 3 we need to secify f(n) = O(N 3 ). Remark 4 While in ractice we nd it reasonable to set max no greater than two, further research is required in determining this data deendent measure, which is beyond the scoe of the resent aer. Convergence under the sectral norm is less demanding than under the Frobenius norm. Unlike heorem, the statement of heorem does not require a condition on max. his imlies that under the sectral norm, when controlling the errors in estimation of R; it is su cient for f(n) to rise linearly with N irresective of the value of max. 9

11 Remark 5 he orders of convergence in (3) and (5) are in line with the results in the thresholding literature. See, for examle, heorem of Cai and Liu (0, CL), and Bickel and Levina (008b, BL), with q = 0, that state the convergence rate using the sectral norm in terms of robability, e q = O m N sec log(n), where e is the thresholded estimator of ^ using either the CL or BL aroaches. Similarly, heorem of Bickel and Levina (008b), with q = 0, using the Frobenius norm under the Gaussianity assumtion, obtains a convergence rate of e q = O. In fact (3) and (5) are F m N N log(n) imrovements on the existing rates since the log (N) factor is absent in both cases. he rate of O mn is achieved in the shrinkage literature as well if the assumtion of sarseness is imosed. Here m N also can be assumed to rise with N in which case the rate of convergence becomes slower. his comares with a rate of O for the samle N= covariance (correlation) matrix - see heorem 3. in Ledoit and Wolf (004 - LW). Note that LW use an unconventional de nition for the Frobenius norm (see their De nition. 376). Remark 6 Results (3) and (5) also hold if a concet of aroximate sarseness is used in lace of Assumtion, such that m N is de ned more generally as m N = max in P N j= j j q, for some q [0; ). See Bickel and Levina (008b) or Fan et al. (03). Remark 7 It is interesting to note that alication of the Bonferroni rocedure to the roblem of testing = 0 for all i 6= j, is equivalent to setting f(n) = N(N )=: Our theoretical results suggest that this is too conservative if = 0 imlies x it and x jt are indeendent, but could be aroriate otherwise. In our Monte Carlo study we consider observations with linear and non-linear deendence, and exeriment with f(n) = N and f(n) = N(N )=: We nd that the simulation results conform closely to our theoretical ndings. Consider now the issue of consistent suort recovery of R (or ), which is de ned in terms of true ositive rate (PR) and false ositive rate (FPR) statistics. Consistent suort recovery requires P R! and F P R! 0 with robability as N and!, and does not follow from the results obtained above on the convergence rates of di erent estimators of R. he roblem is addressed in the following theorem. heorem 3 (Suort Recovery) Consider the true ositive rate (PR) and the false ositive rate (FPR) statistics comuted using the multile testing estimator ~ ; = ^ ; I ^ ; > = c (N) ; 0

12 given by P R = F P R = P P I(~ ; 6= 0; and 6= 0) i6=j P P I( 6= 0) i6=j (6) P P I(~ ; 6= 0; and = 0) i6=j P P ; (7) I( = 0) i6=j where ^ ; is the air-wise correlation coe cient de ned by (7), c (N) = > f(n) 0, 0 < <, f(n) is an increasing function such that c (N)!, as N!, ln f(n)=! 0 and c (N)=! 0, as N and!. Suose also that Assumtions -3 hold. hen with robability tending to, P R =, and with robability tending to, F P R = 0, if there exist N 0 and 0 such that for N > N 0 and > 0, min c (N) > 0, where min = min > 0. Proof. See Aendix. Remark 8 he roof of suort recovery does not deend on (; ). Also it only requires that f(n) rises with N linearly. For examle, setting f(n) = N it is easily seen that ln f(n)= = ln(n )=! 0; as N and!, and c (N) =!, as f(n) N!, and conditions of heorem 3 are met. Interestingly, this suggests that consistent suort recovery is ensured if Bonferroni s M rocedure is alied to R (or ) row-wise. One is likely to encounter loss of ower if Bonferroni s rocedure is alied to all the distinct o -diagonal elements of R. A similar argument can be made for Holm s M rocedure, although the alication of Holm s rocedure row-wise can result in contradictions due to the symmetry of the correlation matrix. 3 Monte Carlo simulations We investigate the numerical roerties of the roosed multile testing (M ) estimator using Monte Carlo simulations. We comare our estimator with a number of thresholding and shrinkage estimators roosed in the literature, namely the thresholding estimators of Bickel and Levina (008b, BL) and Cai and Liu (0, CL), and the shrinkage estimator of LW. As mentioned earlier the thresholding methods of BL and CL require the comutation of a theoretical constant, C, that arises in the rate of their convergence. For this urose, cross-validation is tyically emloyed which we use when imlementing these estimators. For the CL aroach we also consider the theoretical value of C = roosed by the authors. A review of these estimators along with details of the associated cross-validation rocedure can be found in the Sulementary Aendix B. We begin by generating the standardised variates, y it, as y t = P u t ; t = ; ; :::; ;

13 where y t = (y t ; y t ; :::; y Nt ) 0, u t = (u t ; u t ; :::; u Nt ) 0, and P is the Cholesky factor associated with the choice of the correlation matrix R = P P 0. We consider two alternatives for the errors, u it : (i) the benchmark Gaussian case where u it s IIDN(0; ) for all i and t, and (ii) the case where u it follows a multivariate t-distribution with v degrees of freedom generated as = v u it = " it, for i = ; ; :::; N; v;t where " it s IIDN(0; ), and v;t is a chi-squared random variate with v > 4 degrees of freedom, distributed indeendently of " it for all i and t. As fourth-order moments are required by Assumtion we set v = 8 to ensure that E (yit) 4 exists and max. Note that under = 0; = (; = 0) = (v )=(v 4), and with v = 8 we have = = :5. herefore, in the case where the standardised errors are multivariate t-distributed to ensure that conditions of heorem are met we must set f(n) = N (N ) =. (See also Remark 3 and Lemma 7 in the Sulementary Aendix A). One could further allow for fat-tailed " it shocks, though fat-tail shocks alone (e.g. generating u it as such) do not necessarily result in > as shown in Lemma 8 of the Sulementary Aendix A. he same is true for normal shocks under case (i), where (; ) = whether P = I N or not. In such cases setting f(n) = N is then su cient for conditions of heorem to be met. Next, the non-standardised variates x t = (x t ;x t ; :::;x Nt ) 0 are generated as x t = a + f t + D = y t, (8) where D = diag( ; ; ::::; NN ), a = (a ; a ; :::; a N ) 0 and = ( ; ; :::; N ) 0. We reort results for N = f30; 00; 00g and = 00; for the baseline case where = 0 and a = 0 in (8). he roerties of the M rocedure when factors are included in the data generating rocess are also investigated by drawing i and a i as IIDN (; ) for i = ; ; ::::; N, and generating f t, the common factor, as a stationary AR() rocess, but to save sace these results are made available uon request. Under both settings we focus on the residuals from an OLS regression of x t on an intercet and a factor (if needed). In accordance with our theoretical assumtions we consider two exactly sarse covariance (correlation) matrices: Monte Carlo design A: Following Cai and Liu (0) we consider the banded matrix = ( ) = diag(a ; A ); ji jj where A = A+I N=, A = (a ) i;jn= ; a = ( ) 0 + with = max( min (A); 0)+0:0 to ensure that A is ositive de nite, and A = 4I N= : is a two-block diagonal matrix, A is a banded and sarse covariance matrix, and A is a diagonal matrix with 4 along the diagonal. Matrix P is obtained numerically by alying the Cholesky decomosition to the correlation matrix, R = D = D = = PP 0, where the diagonal elements of D are given by ii = + ; for i = ; ; :::; N= and ii = 4; for i = N= + ; N= + ; :::; N: Monte Carlo design B: We consider a covariance structure that exlicitly controls for the number of non-zero elements of the oulation correlation matrix. First we draw the N

14 vector b = (b ; b ; :::; b N ) 0 with elements generated as Uniform (0:7; 0:9) for the rst and last N b (< N) elements of b, where N b = N, and set the remaining middle elements of b to zero. he resulting oulation correlation matrix R is de ned by R = I N + bb 0 diag (bb 0 ), (9) for which min c (N) > 0 and min = min > 0, in line with heorem 3. he degree of sarseness of R is determined by the value of the arameter. We are interested in weak cross-sectional deendence, so we focus on the case where < = following Pesaran (05), and set = 0:5. Matrix P is then obtained by alying the Cholesky decomosition to R de ned by (9). Further, we set = D = RD =, where the diagonal elements of D are given by ii s IID (= + ()=4), i = ; ; :::; N. An additional two covariance seci cations based on aroximately sarse matrices as de ned in Bickel and Levina (008b,. 580 for 0 < q < ), namely the correlation matrices corresonding to an AR() and satial AR(), SAR(), rocess resectively, along with their associated simulation results can be found in the Sulementary Aendix D. 3. Finite samle ositive de niteness As with other thresholding aroaches, multile testing reserves the symmetry of ^R and is invariant to the ordering of the variables but it does not ensure ositive de niteness of the estimated covariance matrix when N >. A number of methods have been develoed in the literature that roduce sarse inverse covariance matrix estimates which make use of a enalised likelihood (D Asremont et al. (008), Rothman et al. (008, 009), Yuan and Lin (007), and Peng et al. (009)) or convex otimisation techniques that aly suitable enalties such as a logarithmic barrier term (Rothman (0)), a ositive de niteness constraint (Xue et al. (0)), an eigenvalue condition (Liu et al. (03), Fryzlewicz (03), Fan et al. (03, FLM)). Most of these aroaches are rather comlex and comutationally extensive. A simler alternative, which concetually relates to soft thresholding (such as smoothly clied absolute deviation by Fan and Li (00) and adative lasso by Zou (006)), is to consider a convex linear combination of RM e and a well-de ned target matrix which is known to result in a ositive de nite matrix. In what follows, we ot to set as benchmark target the N N identity matrix, I N, in line with one of the methods suggested by El Karoui (008). he advantage of doing so lies in the fact that the same suort recovery achieved by R e M is maintained and the diagonal elements of the resulting correlation matrix do not deviate from unity. Given the similarity of this adjustment to the shrinking method, we dub this ste shrinkage on our multile testing estimator (S-M ), er S-M () = I N + ( ) e R M ; (0) with shrinkage arameter ( 0 ; ]; and 0 being the minimum value of that roduces a non-singular e R S-M ( 0 ) matrix. Alternative ways of comuting the otimal weights on the 3

15 two matrices can be entertained. We choose to calibrate,, since oting to use 0 in (0), as suggested in El Karoui (008), does not necessarily rovide a well-conditioned estimate of e R S-M. Accordingly, we set by solving the following otimisation roblem = arg min 0 + R 0 er S-M () F ; () where is a small ositive constant, and R 0 is a reference invertible correlation matrix. Finally, we construct the corresonding covariance matrix as e S-M ( ) = ^D = e RS-M ( ) ^D = : Further details on the S-M rocedure, the otimisation of () and choice of reference matrix R 0 are available in the Sulementary Aendix C. 3. Alternative estimators and evaluation metrics Using the earlier set u and the relevant adjustments to achieve ositive de niteness of the estimators of where required, we obtain the following estimates of : M N : thresholding based on the M aroach alied to the samle correlation matrix ( e M ) using f(n) = N ( e MN ) M N(N )= : thresholding based on the M aroach alied to the samle correlation matrix ( e M ) using f(n) = N(N )= ( e MN(N )= ) BL ^C: BL thresholding on the samle covariance matrix using cross-validated C ( e BL; ^C) CL : CL thresholding on the samle covariance matrix using the theoretical value of C = ( e CL; ) CL ^C: CL thresholding on the samle covariance matrix using cross-validated C ( e CL; ^C) S-M N : sulementary shrinkage alied to M N ( e S-MN ) S-M N(N )= : sulementary shrinkage alied to M N(N )= ( e S-MN(N )= ) BL ^C: BL thresholding using the Fan, Liao and Mincheva (03, FLM) cross-validation adjustment rocedure for estimating C to ensure ositive de niteness ( e BL; ^C) CL ^C: CL thresholding using the FLM cross-validation adjustment rocedure for estimating C to ensure ositive de niteness ( e CL; ^C) LW^: LW shrinkage on the samle covariance matrix (^ LW ^). In accordance with the theoretical results in heorem and in view of Remark 3, we consider two versions of the M estimator deending on the choice of f(n) = fn ; N(N )=g : he BL ^C, and CL and CL ^C estimators aly the thresholding rocedure without ensuring that the resultant covariance estimators are invertible. he next ve estimators yield invertible covariance estimators. he S-M estimators are obtained using the sulementary shrinkage aroach described in Section 3.. BL ^C and CL ^C estimators are obtained by alying the additional FLM adjustments. he shrinkage estimator, LW^, is invertible by construction. In the case of the M estimators where regularisation is erformed on the correlation matrix the associated covariance matrix is estimated as ^D = e RM ^D =. 4

16 For both Monte Carlo designs A and B, we comute the sectral and Frobenius norms of the deviations of each of the regularised covariance matrices from their resective oulation : sec and ; () F where is set to one of the following estimators f e MN ; e MN(N )= ; e BL; ^C; e CL; ; e CL; ^C; e S-MN ; e S-MN(N )= ; e BL; ^C; e CL; ^C; ^ LW ^g. he threshold values, ^C and ^C, are obtained by cross-validation (see Sulementary Aendix B.3 for details). Both norms are also comuted for the di erence between, the oulation inverse of, and the estimators f e S-M N ; e S-M N(N )= ; e BL; ^C ; e CL; ^C ; ^ LW ^g. Further, we investigate the ability of the thresholding estimators to recover the suort of the true covariance matrix via the true ositive rate (PR) and false ositive rate (FPR), as de ned by (6) and (7), resectively. he statistics PR and FPR are not relevant to the shrinkage estimator LW^ and will not be reorted for this estimator. 3.3 Robustness of M to the choice of the -value and f(n) We begin by investigating the sensitivity of the M estimator to the choice of the -value,, and the scaling factor f(n) used in the formulation of c (N) de ned by (6). For this urose we consider the tyical signi cance levels used in the literature, namely = f0:0; 0:05; 0:0g; and f(n) = fn ; N(N )=g. able summarises the sectral and Frobenius norm losses (averaged over 000 relications) for both Monte Carlo designs A and B, and for both distributional error assumtions (Gaussian and multivariate t). First, we note that neither of the norms is much a ected by the choice of the values when the scaling factor is N (N ) =, irresective of whether the observations are drawn from a Gaussian or a multivariate t distribution. Perhas this is to be exected since for N su ciently large the e ective -value which is given by =N(N ) is very small and the test outcomes are more likely to be robust to the changes in the values of as comared to the case when the scaling factor used is N. he results in able also con rm our theoretical nding of heorem that in the case of Gaussian observations, where max =, the scaling factor N is likely to erform better as comared to N(N )=, but the reverse is true if the observations are multivariate t distributed and the scaling factor N(N )= is to be referred (see also Remark 3). We also note that all the norm losses rise with N given that is ket at 00 in all the exeriments. We obtain similar results when we consider other Monte Carlo designs with aroximately sarse covariance matrices. o save sace the results for these designs are rovided in the Sulementary Aendix D. Overall, we nd that the results are more robust when the scaling factor N(N )= is used. 3.4 Norm comarisons of M, BL, CL, and LW estimators In comaring our roosed estimators with those in the literature we consider a fewer number of Monte Carlo relications and reort the results with norm losses averaged over 00 5

17 relications, given the use of the cross-validation rocedure in the imlementation of BL and CL thresholding. his Monte Carlo seci cation is in line with the simulation set u of BL and CL. Our reorted results are also in agreement with their ndings. ables and 3 summarise the results for the Monte Carlo designs A and B, resectively. Based on the results of Section 3.3, we rovide norm comarisons for the M estimator using the scaling factor N(N )=; and the conventional signi cance level of = 0:05. Initially, we consider the threshold estimators, M, BL and the two versions of the CL estimators (CL and CL ^C) without further adjustments to ensure invertibility. First, we note that the M and CL estimators (both versions) dominate the BL estimator in every case, without any excetions and for both designs. he same is also true if we comare M and CL estimators to the LW shrinkage estimator, although it could be argued that it is more relevant to comare the invertible versions of the M and CL estimators (namely e CL; ^C and e S-M ) with ^ LW ^. In such comarisons ^ LW ^ erforms relatively better, nevertheless, ^ LW ^ is still dominated by e S-M, with a few excetions in the case of design A and rimarily when N = 30. However, no clear ordering emerges when we comare ^ LW ^ with e CL; ^C. 3.5 Norm comarisons of inverse estimators Although the theoretical focus of this aer has been on estimation of rather than its inverse, it is still of interest to see how well e S-M, e BL; ^C, e CL; ^C, and ^ LW ^ estimate, assuming that is well de ned. able 4 rovides average norm losses for Monte Carlo design B whose is ositive de nite. for design A is ill-conditioned and will not be considered any further here. As can be seen from the results in able 4, e S-M erforms much better than e BL; ^C and e CL; ^C for Gaussian and multivariate t-distributed observations. In fact, the average sectral norms for e BL; ^C and e CL; ^C include some sizeable outliers, esecially for N 00. However, the ranking of the di erent estimators remains the same if we use the Frobenius norm which aears to be less sensitive to the outliers. It is also worth noting that e S-M erforms better than LW^, for all samle sizes and irresective of whether the observations are drawn as Gaussian or multivariate t. 3.6 Suort recovery statistics able 5 reorts the true ositive and false ositive rates (PR and FPR) for the suort recovery of using the multile testing and thresholding estimators. In the comarison set we include two versions of the M estimator ( e MN and e MN(N )= ); e BL; ^C; e CL; ; and e CL; ^C. Again we use 00 relications due to the use of cross-validation in the imlementation of BL and CL thresholding. We include the M estimators for both choices of the scaling factor, f(n) = N and f(n) = N(N )=, comuted at = 0:05, to see if our theoretical result, namely that for consistent suort recovery only the linear scaling factor, N, is needed, is borne out by the simulations. For consistent suort recovery we would like to see 6

18 F P R values near zero and P R values near unity. As can be seen from able 5, the F P R values of all estimators are very close to zero, so any comarisons of di erent estimators must be based on the P R values. Comaring the results for e MN and e MN(N )= we nd that as redicted by the theory (heorem 3 and Remark 8), P R values of e MN are closer to unity as comared to the P R values of e MN(N )=. Similar results are obtained for the M estimators for di erent choices of the values. able 6 rovides results for = f0:0; 0:05; 0:0g; and for f(n) = fn ; N(N )=g using,000 relications. In this table it is further evident that, in line with the conclusions of Section 3.3, both the P R and the F P R statistics are relatively robust to the choice of the values irresective of the scaling factor, f(n), or whether the observations are drawn from a Gaussian or a multivariate t distribution. his is esecially true under design B, since for this seci cation we exlicitly control for the number of non-zero elements in, that ensures the conditions of heorem 3 are met. urning to a comarison with other estimators in able 5, we nd that the M and CL estimators erform substantially better than the BL estimator. Further, allowing for non-linear deendence in the errors causes the suort recovery erformance of BL ^C; CL and CL ^C to deteriorate noticeably while M N and M N(N )= remain remarkably stable. Finally, again note that P R values are higher for design B. Overall, the estimator e MN does best in recovering the suort of as comared to other estimators, although the results of CL and M for suort recovery are very close, which is in line with the comarative analysis carried out in terms of the relative norm losses of these estimators. 3.7 Comutational demands of the di erent thresholding methods able 7 reorts the relative execution times of the di erent thresholding methods studied. All times are relative to the time it takes to carry out the comutations for the M N(N )= estimator. It took 0:00, 0:03, and 0:06 seconds to aly the M method in Matlab to a samle of N = f30; 00; 00g; resectively, and = 00 observations using a deskto c. he slight di erence in execution time between M N and M N(N )= amounts to the stricter condition imosed by the -value on the M N(N )= rocedure, which roduces a slightly sarser version of. e In contrast, the BL ^C and CL ^C thresholding aroaches are comutationally much more demanding. heir comutations took between about 8 and 493 times longer than the M aroach, for the same samle sizes and comuter hardware. he BL ^C method was less demanding than the CL ^C method - it took between about 8 and 500 times longer than the M aroach. Even CL, which does not require estimation of the threshold arameter, took u to 7 times longer than the M aroach. hus, comared with other thresholding methods M has a clear comutational advantage. 4 Concluding Remarks his aer considers regularisation of large covariance matrices articularly when the cross section dimension N of the data under consideration exceeds the time dimension : In this 7

19 case the samle covariance matrix, ^; becomes ill-conditioned and is not a satisfactory estimator of the oulation covariance. A regularisation estimator is roosed which uses multile testing rather than crossvalidation to calibrate the threshold value. It is shown that the resultant estimator has a convergence rate of m N = under the sectral norm and (m N N= ) = under the Frobenius norm, where is the number of observations, and m N is bounded in N (the dimension of ), which rovide slightly better rates than the convergence rates established in the literature for other regularised covariance matrix estimators. Our results derived under the Frobenius norm exlicitly relate the scaling function in the multile testing roblem to the ossible non-linear deendence of the underlying data, and together with the sectral norm results are valid under both Gaussian and non-gaussian assumtions. his comliments the existing theoretical results in the literature for the Frobenius norm of the thresholding estimator derived only under the assumtion of Gaussianity. As comared to the threshold estimators that use cross-validation, the M estimator is also comutationally simle and fast to imlement. he numerical roerties of the roosed estimator are investigated using Monte Carlo simulations. It is shown that the M estimator erforms well, and generally better than the other estimators roosed in the literature. he simulations also show that in terms of sectral and Frobenius norm losses, the M estimator is reasonably robust to the choice of in the threshold criterion, ^ > =, articularly when f(n) is set to f(n) N(N )=. For suort recovery, better results are obtained if f(n) is set to N. 8

20 able : Sectral and Frobenius norm losses for the M () estimator using signi cance levels = f0:0; 0:05; 0:0g and the scaling factors f(n) = fn ; N(N )=g, for = 00 Monte Carlo design A f (N) = N f (N) = N (N ) = N M N (:0) M N (:05) M N (:0) MN(N ) u it s Gaussian (:0) MN(N ) (:05) MN(N ) Sectral norm 30.70(0.49).68(0.49).7(0.49).84(0.50).75(0.50).7(0.50) 00.6(0.50).5(0.50).50(0.50) 3.0(0.50).84(0.50).76(0.50) (0.48).9(0.49).89(0.49) 3.58(0.47) 3.37(0.47) 3.9(0.47) Frobenius norm (0.45) 3.4(0.50) 3.0(0.54) 3.4(0.4) 3.5(0.44) 3.9(0.44) (0.45) 6.5(0.5) 6.60(0.55) 7.57(0.4) 7.7(0.4) 7.00(0.4) (0.46) 9.60(0.53) 9.73(0.58).49(0.4) 0.89(0.4) 0.63(0.4) u it s multivariate t distributed with 8 degrees of freedom Sectral norm 30.6(.08).43(.0).55(.7).6(0.95).4(.03).5(.06) (4.84) 4.(5.9) 4.47(5.48) 3.74(3.94) 3.7(4.8) 3.7(4.44) (3.46) 5.04(4.34) 5.45(4.77) 4.3(.97) 4.9(.37) 4.0(.58) Frobenius norm (.4) 4.36(.3) 4.6(.40) 4.08(0.95) 4.03(.06) 4.04(.) (5.7) 9.76(5.67) 0.5(5.88) 8.9(4.9) 8.74(4.57) 8.70(4.74) 00.96(4.3) 4.5(5.4) 5.8(5.95) 3.06(.6).7(.77).63(3.05) Monte Carlo design B u it s Gaussian Sectral norm (0.6) 0.50(0.6) 0.53(0.6) 0.49(0.8) 0.48(0.7) 0.48(0.7) (0.34) 0.76(0.3) 0.78(0.3) 0.85(0.4) 0.79(0.37) 0.77(0.37) (0.) 0.74(0.0) 0.77(0.0) 0.8(0.3) 0.75(0.6) 0.73(0.6) Frobenius norm (0.7) 0.9(0.8) 0.98(0.9) 0.87(0.8) 0.86(0.7) 0.86(0.7) 00.56(0.4).66(0.4).77(0.4).64(0.3).58(0.7).57(0.7) 00.6(0.8).3(0.0).50(0.).(0.).6(0.0).5(0.0) u it s multivariate t distributed with 8 degrees of freedom Sectral norm (0.39) 0.78(0.43) 0.84(0.45) 0.67(0.34) 0.67(0.37) 0.69(0.38) 00.6(0.97).3(.0).4(.8).(0.77).0(0.83).0(0.87) 00.36(.73).65(.05).83(.0).3(.).4(.8).6(.37) Frobenius norm 30.3(0.43).4(0.48).54(0.5).5(0.36).8(0.39).0(0.4) 00.40(.).90(.3) 3.6(.40).6(0.8).6(0.90).0(0.96) (.4) 4.5(.54) 5.8(.7).97(.30) 3.0(.53) 3.07(.65) Note: Norm losses are averages over,000 relications. Simulation standard deviations are given in arentheses. he M estimators are de ned in Section 3.. (:0) 9

21 able : Sectral and Frobenius norm losses for di erent regularised covariance matrix estimators ( = 00) - Monte Carlo design A N = 30 N = 00 N = 00 Norms Norms Norms Sectral Frobenius Sectral Frobenius Sectral Frobenius u it s Gaussian Error matrices ( ) M N(N )=.8(0.54) 3.3(0.4).75(0.50) 7.(0.4) 3.37(0.43) 0.9(0.39) BL ^C 5.30(.6) 7.6(.3) 8.74(0.06) 6.90(0.0) 8.94(0.04) 4.6(0.3) CL.87(0.55) 3.39(0.44).99(0.49) 7.57(0.44) 3.79(0.47).88(0.4) CL ^C.8(0.58) 3.33(0.56).54(0.50) 6.8(0.5) 3.0(0.46) 0.(0.59) S-M N(N )= 3.0(0.79) 4.9(0.64) 5.73(0.34) 0.77(0.46) 6.40(0.) 6.44(0.35) BL ^C 7.09(0.0) 8.6(0.09) 8.74(0.06) 6.90(0.0) 8.94(0.04) 4.5(0.0) CL ^C 7.05(0.6) 8.58(0.) 8.7(0.07) 6.85(0.) 8.94(0.04) 4.3(0.09) LW^.99(0.47) 6.49(0.9) 5.0(0.34) 6.70(0.9) 6.8(0.0) 6.84(0.4) u it s multivariate t distributed with 8 degrees of freedom Error matrices ( ) M N(N )=.6(0.76) 4.03(0.99) 3.43(.09) 8.43(.6) 3.97(0.9).66(.83) BL ^C 6.90(0.8) 8.75(0.55) 8.74(0.0) 7.6(0.30) 9.00(0.4) 4.93(.0) CL.55(0.93) 4.53(.00) 4.63(.) 0.35(.48) 5.9(0.8) 6.43(.74) CL ^C.7(0.76) 4.4(0.94) 3.85(.5) 9.44(.33) 5.04(.04) 5.65(4.7) S-M N(N )= 3.8(0.8) 4.68(0.8) 5.75(0.45).33(0.6) 6.4(0.3) 7.0(0.74) BL ^C 7.06(0.3) 8.84(0.30) 8.74(0.0) 7.5(0.3) 8.95(0.08) 4.84(0.55) CL ^C 7.0(0.6) 8.77(0.30) 8.73(0.) 7.3(0.9) 8.94(0.08) 4.77(0.53) LW^ 3.35(0.5) 7.35(0.50) 5.67(0.46) 8.04(0.45) 6.60(0.43) 8.8(0.53) Note: Norm losses are averages over 00 relications. Simulation standard deviations are given in arentheses. = f e MN(N )= ; e BL; ^C; e CL; ; e CL; ^C; e S-MN(N )= ; e BL; ^C; e CL; ^C; ^ LW g. MN(N e ^ )= and e S-MN(N )= are comuted using = 0:05. BL is Bickel and Levina universal thresholding, CL is Cai and Liu adative thresholding, e BL; ^C is based on ^C which is obtained by cross-validation, e BL; emloys ^C the further adjustment to the cross-validation coe cient, ^C, roosed by Fan, Liao and Mincheva (03), e CL; is CL s estimator with C = (the theoretical value of C), ^ LW ^ is Ledoit and Wolf s shrinkage estimator alied to the samle covariance matrix. 0