An introduction to G-estimation with sample covariance matrices

Transcription

1 An introduction to G-estimation with sample covariance matrices Xavier estre Centre Tecnològic de Telecomunicacions de Catalunya (CTTC) "atrices aleatoires: applications aux communications numeriques et a l estimation statistique" ENST (Paris), January 0, 2007

2 Outline Convergence of the eigenvalues and eigenvectors of the sample covariance matrix. Characterization of the clusters/support of the asymptotic sample eigenvalue distribution. The principle of G-estimation:, N-consistency versus N-consistency. Some examples. An application in array signal processing: subspace-based estimation of directions-of-arrival (DoA). Xavier estre: An introduction to G-estimation with sample covariance matrices. 2/4

3 Thesamplecovariancematrix We assume that we collect N independent samples (snapshots) from an array of antennas: ˆR = N P N n= y(n)yh (n). It is usual to model the observations {y(n)} as independent, identically distributed random vectors with zero mean and covariance R. Xavier estre: An introduction to G-estimation with sample covariance matrices. 3/4

4 Thesamplecovariancematrix We assume that we collect N independent samples (snapshots) from an array of antennas: ˆR = N P N n= y(n)yh (n). Example:R has 4 eigenvalues {, 2, 3, 7} with equal multiplicity. 25 Histogram of the eigenvalues of the sample covariance matrix, =80, N= Histogram of the eigenvalues of the sample covariance matrix, =400, N= Number of eigenvalues 5 0 Number of eigenvalues lambda lambda Xavier estre: An introduction to G-estimation with sample covariance matrices. 4/4

5 Thesamplecovariancematrix We assume that we collect N independent samples (snapshots) from an array of antennas: ˆR = N P N n= y(n)yh (n). Example:R has 4 eigenvalues {, 2, 3, 7} with equal multiplicity. 25 Histogram of the eigenvalues of the sample covariance matrix, =80, N= Histogram of the eigenvalues of the sample covariance matrix, =400, N= Number of eigenvalues 5 0 Number of eigenvalues lambda lambda Xavier estre: An introduction to G-estimation with sample covariance matrices. 5/4

6 The sample covariance matrix: asymptotic properties When both, N, /N c, 0 <c<, the e.d.f. of the eigenvalues of ˆR tends to a deterministic density function. Example: R has 4 eigenvalues {, 2, 3, 7} with equal multiplicity. Aysmptotic density of eigenvalues of the sample correlation matrix.6 c=0.0 c=0. c= Eigenvalues Xavier estre: An introduction to G-estimation with sample covariance matrices. 6/4

7 Convergence of the eigenvalues and eigenvectors of the sample covariance matrix Can we characterize the asymptotic sample eigenvalue behavior analytically? What about the eigenvectors? We will consider the following two quantities: ˆb (z) = ³ tr ˆR zi ³, ˆm (z) =a H ˆR zi b where z C and a, b C. Note that ˆb (z) depends only on the eigenvalues of ˆR,whereas ˆm (z) depends on both the eigenvalues and the eigenvectors of this matrix. uch of the asymptotic behavior of the eigenvalues and eigenvectors of ˆR can be inferred from this two quantities. Xavier estre: An introduction to G-estimation with sample covariance matrices. 7/4

8 Stieltjes transforms The function ˆb (z) is the classical Stieltjes transform of the empirical distribution of eigenvalues: ˆb (z) = X Z ˆλ m z = λ z df ˆR (λ) where F ˆR (λ) = nm # =... : ˆλ o m λ = X I ³ˆλm λ. In the same way, ˆm (z) is the Stieltjes transform of a different empirical distribution function X a H ê m ê H m ˆm (z) = b Z = ˆλ m z λ z dgˆr (λ) where GˆR (λ) = X a H ê m ê H mb I ³ˆλm λ. Xavier estre: An introduction to G-estimation with sample covariance matrices. 8/4

9 Graphical representation of the considered empirical distribution functions Xavier estre: An introduction to G-estimation with sample covariance matrices. 9/4

10 Convergence of the eigenvalues and eigenvectors of the sample covariance matrix Theorem. Under some statistical assumptions, ˆm (z) m (z) 0, ˆb (z) b (z) 0 almost surely for all z C + = {z C :Im[z] > 0} as,n at the same rate, where b (z) = b is the unique solution to the following equation in the set {b C : ( c)/z + cb C + }: b = X λ m ( c czb) z and m (z) = f (z) a H (R f (z)i ) z b, f (z) = z c cz b (z). Assuming that b(z) = lim b (z) exists, then the asymptotic sample eigenvalue density can be obtained as q(x) = π lim y 0 Im b(x + + j y). Xavier estre: An introduction to G-estimation with sample covariance matrices. 0/4

11 Determining the support of q(x) Consider the following equation in f, whichhas 2Q solutions (counting multiplicities) Ψ (f) = X µ 2 λm = λ m f c. Xavier estre: An introduction to G-estimation with sample covariance matrices. /4

12 Determining the support of q(x) n o The solutions are denoted f,f+,...,f Q,f+ Q. For each eigenvalue λ m, there exists a single q {...Q}, suchthatλ m fq,f q +. In other words, each eigenvalue is associated with a single cluster, but one cluster may be associated with multiple eigenvalues. Xavier estre: An introduction to G-estimation with sample covariance matrices. 2/4

13 Determining the support of q(x) It turns out that Q is the number of clusters in q(x), and the position of each cluster is given by where the function Φ(f) is defined as x + q = Φ(f + q ), x q = Φ(f q ) Φ(f) =f à c X Hence, the support of q(x) can be expressed as S = x,x+ λ m λ m f!.... h x Q,x+ Q i. Note: It may happen that λ m / x q,x + q.evenwhenλm f q,f + q. Xavier estre: An introduction to G-estimation with sample covariance matrices. 3/4

14 Determining the support of q(x) Xavier estre: An introduction to G-estimation with sample covariance matrices. 4/4

15 Determining the support of q(x) Xavier estre: An introduction to G-estimation with sample covariance matrices. 5/4

16 Splitting number for two clusters in q(x) Given two consecutive eigenvalues {λ m,λ m+ } there exists a minimum number of samples per antenna to guarantee that the corresponding eigenvalue clusters split. µ N > X µ 2 λk min λ k ξ k= where ξ is the mth real valued solution to Ψ 0 (f) =0. Xavier estre: An introduction to G-estimation with sample covariance matrices. 6/4

17 odeling the finite sample size effect using random matrix theory Random atrix Theory offers the possibility of analyzing the behavior of different quantities depending on ˆR when the sample size and the number of sensors/antennas have the same order of magnitude. Assume ³ that we want to analyze the asymptotic behavior of a certain scalar function of ˆR,namely φ ˆR. Traditional ³ Approach: Assuming that the number of samples is high, we might establish that φ ˆR φ (R ) in some stochastic sense as N while remains fixed. New Approach: In order to characterize the situation where, N havethesameorderof magnitude, one might consider the limit N,, /N c, 0 <c<. Note that, in general, ³ φ ˆR φ (R ) 9 0, N,,/N c For example: h i tr ˆR c tr R 0, c <. Xavier estre: An introduction to G-estimation with sample covariance matrices. 7/4

18 Using random matrix theory to analyze the finite sample size effect Assume that the quantity that we need to characterize can be expressed in terms of a Stieltjes transform: h i tr ˆR = X = X = ˆλ m ˆλ ˆb (0) m z à z=0 n o! ˆR = i,j uh i ˆR u X j = u H ê m ê H X m u H i ê m ê H i u j = mu j =ˆm (0) ˆλ m ˆλ m z z=0 h i T where u i = 0 T {z} 0T.Now, ith position ˆb (z) b (z) 0 = h i tr ˆR b (0) 0, b (0) = c tr R o ˆm (z) m (z) 0 = nˆr m (0) i,j 0, m (0) = n o ˆR c i,j Xavier estre: An introduction to G-estimation with sample covariance matrices. 8/4

19 Estimation under low sample support with random matrix theory When designing an estimator of a certain scalar function of R,namelyϕ (R ), one can distinguish between: Traditional N-consistency: Consistency when N while remains fixed., N-consistency: Consistency when,n at the same rate. We observe that,n-consistency guarantees a good behavior when the number of samples N has the same order of magnitude as the observation dimension. The objective of G-estimation (V.L. Girko) is to provide a systematic approach for the derivation of, N-consistent estimators of different scalar functions of the true covariance matrix. For example, the G-estimator of tr R will be ( c) h i tr ˆR. Xavier estre: An introduction to G-estimation with sample covariance matrices. 9/4

20 The principle of G-estimation We will assume that the quantity that we need to estimate, namely ϕ (R ),canbeexpressedin terms of the Stieltjes transforms associated with R,namely For example, b (ω) = tr h(r ωi ) i, m (ω) =a H (R ωi ) b. ª R i,j = uh i R u j = u H i à X! e m e H m u j = m (0). If we find, N-consistent estimators of these two functions b (ω), m (ω), we will implicitly find (under some regularity conditions), N-consistent estimators of ϕ (R ). The problem is, we don t have access to b (ω) and m (ω) because the true covariance matrix R is unknown. We only have access to the sample covariance matrix, and hence ˆb (z) = tr ³ ˆR zi, ˆm (z) =a H ³ ˆR zi b. λ m Xavier estre: An introduction to G-estimation with sample covariance matrices. 20/4

21 The principle of G-estimation (ii) Recall that, by definition, ˆb (z) and ˆm (z) are,n-consistent estimators of b (z) and m (z) respectively when z C +.Now b (z) is solution to the equation which can also be expressed as b (z) = f (z) z b (z) = X X λ m c cz b (z) z λ m f (z), f (z) = Observe the similarity with b (ω), the quantity that we need to estimate b (ω) = X λ m ω. z c cz b (z). Xavier estre: An introduction to G-estimation with sample covariance matrices. 2/4

22 The principle of G-estimation (iii) Compare the quantity that needs to be estimated b (ω) with z b (z)/f (z): b (ω) = X λ m ω X λ m f (z) = z b (z) f (z). Now, if the equation in z ω = f (z) has a single solution, namely z = f (ω), then X λ m ω = X b (z) = z = λ m f (z) f (z) b (z) c cz b (z) z=f z=f z=f (ω) (ω) As a conclusion, b (ω) is,n-consistently estimated by ³ G (ω) =ˆb (z) c czˆb (z) z= ˆf. (ω) (ω). Xavier estre: An introduction to G-estimation with sample covariance matrices. 22/4

23 An example of G-estimator Assume that we want to estimate ϕ (R )= tr R 2. We express ϕ (R ) as a function of b (ω), namely ϕ (R )= tr R 2 = X λ 2 m = d dω " X # λ m ω ω=0 = db (ω) dω ω=0. The G-estimator is constructing replacing b (ω) with the G-estimator G(ω), namely ˆϕ (R )= dg (ω) = d ³ ˆb (z) c czˆb (z) = dω ω=0 dω z= ˆf (ω) ω=0 =( c) 2 h i µ h i 2 tr ˆR 2 c ( c) tr ˆR. where we have used the chain rule, the inverse function theorem and the fact that 0=f (z) has a single solution z =0whenever c <. Xavier estre: An introduction to G-estimation with sample covariance matrices. 23/4

24 G-estimation of functions of m (ω) Assume that the function that we need to estimate ϕ (R ) can be expressed in terms of the Stieltjes transform m (ω). Inthiscase,theproceduretoderiveaG-estimatorofϕ (R ) boils down to finding a G-estimator of m (ω). We recall that m (z) ˆm (z) 0 as,n at the same rate, where m (z) = f (z) a H (R f (z)i ) b = f (z) z z X a H e m e H mb λ m f (z) Compare now the above equation with the quantity that needs to be estimated, namely X a H e m e H m (ω) = mb λ m ω X a H e m e H mb λ m f (z) = z m (z) f (z) = m (z) c cz b (z). Hence, if ω = f (z) has a unique solution, then m (ω) is, N-consistently estimated by ³ H(ω) = ˆm (z) c czˆb (z) z= ˆf. (ω) Xavier estre: An introduction to G-estimation with sample covariance matrices. 24/4

25 Another example of G-estimator Assume that we want to estimate ϕ (R )= ª R i,j. We express ϕ (R ) as a function of m (ω), namely ϕ (R )=u H i R u j = X u H i e m e H mu j λ m = " X u H i e m e H mu j λ m ω # ω=0 = m (ω) ω=0. The G-estimator is constructing replacing m (ω) with the G-estimator H(ω), namely ³ n o ˆϕ (R )=H(ω) ω=0 = ˆm (z) c czˆb (z) z= =( c) ˆm (0) = ( c) ˆR ˆf (0). i,j wherewehaveusedthefactthat0=f (z) has a single solution z =0whenever c<. Xavier estre: An introduction to G-estimation with sample covariance matrices. 25/4

26 Assume that we want to estimate ϕ (R )= Using the integral Yet another example of G-estimator n Z R /2 o i,j (whitening filter). = 2 x π 0 x + t2dt, x > 0, We express ϕ (R ) as a function of m (ω), namely n R /2 o i,j = 2 π Z u H i 0 R + t 2 I uj dt = 2 Z m ( t 2 )dt π If c<, the equation f (z) = t 2 has a unique solution for any t R. The G-estimator is constructing replacing m (ω) with the G-estimator H(ω), namely ˆϕ (R )= 2 Z ³ ˆm (z) c czˆb (z) π 0 z= dt = ˆf ( t2 ) = 2 Z Ã! X u H i ê m ê H mu j c cz X dt π 0 ˆλ m ˆf ( t2 ) ˆλ k ˆf ( t2 ) 0 k= Xavier estre: An introduction to G-estimation with sample covariance matrices. 26/4

27 Solutions to the equation ω = f (z) It turns out that f (z) can be obtained as a particular root of the polynomial equation in f Ã! Φ(f) =f c X λ m = z. λ m f If there exists a root f C +, it is unique and we choose f (z) =f. Otherwise, all the roots are real, but there is only one root such that Φ 0 (f) > 0 or, equivalently, X µ 2 λm λ m f c and we choose f (z) =f. Xavier estre: An introduction to G-estimation with sample covariance matrices. 27/4

28 Solutions to the equation ω = f (x) for ω R c< c> The equation has a unique solution when ω R\ f,f+... h f Q,f+ Q i. Xavier estre: An introduction to G-estimation with sample covariance matrices. 28/4

29 Application to subspace-based DoA estimation. Introduction and signal model We consider DoA detection based on subspace approaches (USIC), that exploit the orthogonality between signal and noise subspaces. Consider a set of K sources impinging on an array of sensors/antennas. We work with a fixed number of snapshots N, {y(),...,y(n)} assumed i.i.d., with zero mean and covariance R. The true spatial covariance matrix can be described as R = S (Θ) Φ S S (Θ) H + σ 2 I where S (Θ) is an K matrix that contains the steering vectors corresponding to the K different sources, S (Θ) = s (θ ) s (θ 2 ) s (θ K ) and σ 2 is the background noise power. Xavier estre: An introduction to G-estimation with sample covariance matrices. 29/4

30 Introduction and signal model (ii) The eigendecomposition of R allows us to differentiate between signal and noise subspaces: R = Λ E S E S 0 H N 0 σ 2 ES E I N. K It turns out that E H N s (θ k)=0, k =...K. Since R is unknown, one must work with the sample covariance matrix ˆR = NX y(n)y H (n). N n= The USIC algorithm uses the sample noise eigenvectors, and searches for the deepest local minima of the cost function η USIC (θ) =s H (θ) Ê N Ê H N s (θ). It is interesting to investigate the behavior of η USIC (θ) when, N have the same order of magnitude. Xavier estre: An introduction to G-estimation with sample covariance matrices. 30/4

31 Noise sample eigenvalue cluster separation assumption In our asymptotic approach, we assume that there exists separation between noise and signal eigenvalue clusters in the asymptotic sample eigenvalue distribution. This implies that there is a minimum number of samples per antenna: µ N > X µ 2 λm λ m ξ where ξ is the smallest real valued solution to the following equation: X λ 2 m (λ m ξ) 3 =0. Xavier estre: An introduction to G-estimation with sample covariance matrices. 3/4

32 Asymptotic behavior of USIC The USIC algorithm suffers from the breakdown effect. The performance breaks down when the number of samples or the SNR falls below a certain threshold. Cause:Ê N is not a very good estimator of E N when, N have the same order of magnitude. The performance breakdown effect can be easily analyzed using random matrix theory, especially under a noise eigenvalue separation assumption: η USIC (θ) η USIC (θ) 0 Ã! X η USIC (θ) =s H (θ) w(k)e k e H k s (θ) w(k) = k= P ³ σ 2 K r= K+ λ r σ µ 2 λ r µ σ 2 λ k σ µ 2 λ k µ where {µ r,r=,...,} are the solutions to P r= λ r λ r µ = c. k K k> K Xavier estre: An introduction to G-estimation with sample covariance matrices. 32/4

33 Asymptotic behavior of USIC: an example We consider a scenario with two sources impinging on a ULA (d/λ c =0.5, =20)fromDoAs: 35, USIC asymptotic pseudospectrum, =20, DoAs=[35,37]deg N=25 N=5 50 Position of the two deepest local minima of the asymptotic USIC cost function N=5 N= Azimuth (deg) SNR=7dB SNR=2dB Azimuth (deg) SNR (db) Xavier estre: An introduction to G-estimation with sample covariance matrices. 33/4

34 ,N-consistent subspace detection: G-USIC We propose to use an, N-consistent estimator of the cost function η (θ) =s H (θ) E N E H N s (θ): Ã! X η G-USIC (θ) =s H (θ) φ(k)ê k ê H k s (θ) φ(k) = + P k= ³ ˆλr r= K+ ˆλ k ˆλ r P K r= ³ ˆλr ˆλ k ˆλ r where now ˆµ,...,ˆµ are the solutions to the equation X k= ˆµ r ˆλ k ˆµ r ˆµ r ˆλ k ˆµ r ˆλ k ˆλ k ˆµ = c. k K k> K Xavier estre: An introduction to G-estimation with sample covariance matrices. 34/4

35 Sketch of the derivation The quantity that needs to be estimated can be written as η (θ) =s H (θ) E N E H Ns (θ) =ks (θ)k 2 s H (θ) E S E H S s (θ) s H (θ) E S E H S s (θ) = I m (ω)dω 2π j C where m (ω) =s H (θ)(r ωi ) s (θ) and C is a negatively oriented contour enclosing only the signal eigenvalues {λ K+,...,λ }. The idea of the derivation is based on obtaining a proper parametrization of the contour C. Xavier estre: An introduction to G-estimation with sample covariance matrices. 35/4

36 Sketch of the derivation (ii) The derivation is based on the following random matrix theory result. Let ³ ˆm (z) =s H (θ) ˆR zi s (θ), ˆb (z) = ³ tr ˆR zi It turns out that ˆm (z) m (z) 0, ˆb (z) b (z) 0 almost surely for all z C + as, N at the same rate, where b (z) =b is the unique solution to the following equation in the set {b C : ( c)/z + cb C + }: and b = X λ m ( c czb) z m (z) = f (z) s H (θ)(r f (z)i ) s (θ), f (z) = z z c cz b (z). Xavier estre: An introduction to G-estimation with sample covariance matrices. 36/4

37 Sketch of the derivation (iii) It turns out that, surprisingly enough, f (x) givesusavalidparametrizationofc : η (θ) =s H (θ) E N E H N s (θ) =ks (θ)k2 π Im Z σ2 σ m (f (x))f 0 (x)dx Xavier estre: An introduction to G-estimation with sample covariance matrices. 37/4

38 Sketch of the derivation (iv) From this point, we can express the integral in terms of m (z) and b (z), namely η (θ) =ks (θ)k 2 I m (ω)dω = ks (θ)k 2 Z σ2 2π j C π Im m (x) c + cx2 b0 (x) c cx b (x) dx and we only need to replace m(z) and b(z) with their,n-consistent estimates: " Z # η G-USIC (θ) =ks (θ)k 2 σ2 π Im ˆm (x) c + cx2ˆb0 (x) c cxˆb (x) dx. Integrating this expression, we get to the proposed estimator: Ã! X η G-USIC (θ) =s H (θ) φ(k)ê k ê H k s (θ) σ k= σ Xavier estre: An introduction to G-estimation with sample covariance matrices. 38/4

39 Performance evaluation USIC vs. G-USIC Comparative evaluation of USIC and G-USIC via simulations. Scenarios with four ( 20, 0, 35, 37 )andtwo(35, 37 ) sources respectively, ULA ( =20, d/λ c =0.5). 0 0 Example of USIC and GUSIC cost function, SNR=8dB, =20, N=5, DoAs=35, 37, 0, 20 deg ean Squared Error USIC GUSIC CRB =20, N= SE 0 0 =20, N= Angle of arrival (azimuth), degrees USIC GUSIC SNR (db) Xavier estre: An introduction to G-estimation with sample covariance matrices. 39/4

40 Some references J. Silverstein, Strong convergence of the empirical distribution of eigenvalues of large dimensional random matrices, Journal of ultivariate Analysis, vol. 5, pp , 995. V. Girko, An Introduction to Statistical Analysis of Random Arrays. The Netherlands: VSP, 998. J. Silverstein and P. Combettes, Signal detection via spectral theory of large dimensional random matrices, IEEE Trans. on Signal Processing, vol. 40, pp , Aug X. estre, Improved estimation of eigenvalues of covariance matrices and their associated subspaces using their sample estimates, Preprint. Available at Z. Bai and J. Silverstein, No eigenvalues outside the support of the limiting spectral distribution of large dimensional sample covariance matrices, Annals of probability, vol. 26, pp , 998. Z. Bai and J. Silverstein, Exact Separation of Eigenvalues of Large Dimensional Sample Covariance atrices, Annals of probability, vol. 27, pp , 999. Xavier estre: An introduction to G-estimation with sample covariance matrices. 40/4

41 Thank you for your attention!!! Xavier estre: An introduction to G-estimation with sample covariance matrices. 4/4