Covariance estimation using random matrix theory

Transcription

1 Covariance estimation using random matrix theory Randolf Altmeyer joint work with Mathias Trabs Mathematical Statistics, Humboldt-Universität zu Berlin Statistics Mathematics and Applications, Fréjus 03 September 2015

2 COVARIANCE ESTIMATION Estimation problem Given: random vector X R p, n iid samples X k (1) estimate covariance matrix Σ = E [(X E[X ])(X E[X ]) ] (2) estimate eigenvalues of Σ canonical estimator : empirical covariance ( X = empirical mean) ˆΣ = 1 n n ( Xk X )( X k X ) k=1

3 COVARIANCE ESTIMATION FOR LARGE p typical application: 500 stocks (= p) and 1000 daily returns (= n) estimate correlation in daily returns use of ˆΣ justified in p n -case justified? (1) No! ˆΣ is only consistent if p fixed and n (2) No! eigenvalues of ˆΣ do not consistently estimate eigenvalues of Σ Example. X k iid N(0,Id), limn p n = 1 2 extreme eigenvalues of ˆΣ converge a.s. to ( ± 2) instead of

4 CRASH COURSE IN RANDOM MATRIX THEORY spectral distribution: for symmetric p p matrix A define measure F A = 1 p p δ λj, (λ j ) j=1,...,p eigenvalues of A j=1 Example. A = Id F A = δ 1 interesting (spectral) statistics of A can be expressed via F A : ( tr A k) = p λj k j=1 = p x k df A (x)

5 CRASH COURSE IN RANDOM MATRIX THEORY random matrix theory analyse the eigenvalue structure of A via the measure F A find limiting spectral distribution H, i.e. F A w H as p important tool: Stieltjes transform, for z C\R s F A(z) := R 1 λ z df A (λ) = 1 p p j=1 1 λj A z = 1 p tr (A zid p) 1

6 THEOREM OF MARCHENKO-PASTUR X k iid N (0,Σ), k = 1,...,n, Σ R p p Theorem (Marchenko-Pastur; Silverstein (1995)) If p/n c (0, ) and if F Σ w H, then F ˆΣ w F a.s. where s F (z) is the unique solution in { m C + : 1 c } z + cm C + of z = 1 s G (z) R + c λ 1 + λs G (z) dh (λ), s F (z) = 1 c 1 cz c s G (z). Example: Σ = Id H = δ 1, F has density

7 MARCHENKO-PASTUR AND INFERENCE inference on H: How to estimate density, distribution function or functionals? inference on Σ: data ˆΣ F ˆΣ F Marchenko Pastur H F Σ

8 INVERTING THE MARCHENKO-PASTUR EQUATION El Karoui (2008), Ledoit and Wolf (2014): optimization procedure, H has to be discrete, only consistency Bai et al. (2010), Rao et al. (2008): method of moments, i.e. connect 1 p tr (ˆΣ k ) = x k df ˆΣ (x) with x k dh (x) Mestre (2008), Li and Yao (2014): for linear spectral statistics H (f ) := f (x)dh (x), f complex analytic

9 THE CONTOUR METHOD for f complex analytic: ( ) 1 f (z) H (f ) = 2πi C z x dz dh (x) = 1 f (z)s H (z) dz 2πi C Marchenko Pastur H (f ) = K (c,f ) + 1 ( zs G 2πic (z)f 1 ) dz C s G (z) estimator: Ĥ n (f ) = K ( p ) n,f + n ( zs G 2πip n (z)f 1 ) dz C n s Gn (z) CLT: n(ĥ n (f ) H p (f )) d N(µ,σ 2 ) for some very complicated µ,σ 2, where H p = 1 p p j=1 δ λ Σ j

10 BEYOND COMPLEX ANALYTIC FUNCTIONS x e iux is complex analytic for every u R estimate the characteristic function of H by ˆϕ n (u) := Ĥ n (e iu ) = 1 n p + n 2πip C n ( zs G n (z)exp ( iu s Gn (z) )) dz. Theorem With ϕ p (u) = H p (e iu ) we have with respect to the uniform topology on compacts n( ˆϕ n ϕ p )) d G, where G is an uncentered Gaussian process.

11 BEYOND COMPLEX ANALYTIC FUNCTIONS if H has density h and f L 2, then by Plancherel H (f ) = 1 1 F f,f h = 2π 2π F f,ϕ H. estimate H(f ) for f L 2 and spectral cut-off A n by H n (f ) := 1 F f, ˆϕn 1 2π [ An,A n ] by Paley-Wiener-Theorem: if supp(f f ) compact, then f is entire H n (f ) = Ĥn (f )

12 BEYOND COMPLEX ANALYTIC FUNCTIONS Theorem If R F f (u) (1 + u )s du < for some s > 0, then ˆϕ n (u) := Ĥn(e iu ) = 1 n p + 1 H n (f ) H p (f ) = O P ( (logn) s ). n ( 2πip C n (zs G n (z)exp iu s Gn (z) )) dz. main reason for log-rate: if u R, z C\R and s Gn (z) = x + iy, then ( exp iu ) ( = iux s Gn (z) exp x 2 + y 2 uy ) x 2 + y 2 = exp( uy ) x 2 + y 2

13 REFERENCES I Bai, Z., J. Chen, and J. Yao (2010, December). On estimation of the population spectral distribution from a high-dimensional sample covariance matrix. Australian & New Zealand Journal of Statistics 52(4), El Karoui, N. (2008). Spectrum estimation for large dimensional covariance matrices using random matrix theory. Ledoit, O. and M. Wolf (2014, June). Spectrum Estimation: A Unified Framework for Covariance Matrix Estimation and PCA in Large Dimensions. pp. 40. Li, W. and J. Yao (2014, April). On generalized expectation-based estimation of a population spectral distribution from high-dimensional data. Annals of the Institute of Statistical Mathematics. Mestre, X. (2008). Improved Estimation of Eigenvalues and Eigenvectors of Covariance Matrices Using Their Sample Estimates. IEEE Transactions on Information Theory 54.

14 REFERENCES II Rao, N. R., J. A. Mingo, R. Speicher, and A. Edelman (2008). Statistical Eigen-Inference from large wishart matrices. Annals of Statistics 36, Silverstein, J. (1995, November). Strong Convergence of the Empirical Distribution of Eigenvalues of Large Dimensional Random Matrices. Journal of Multivariate Analysis 55(2),