Can the Threshold Performance of Maximum Likelihood DOA Estimation be Improved by Tools from Random Matrix Theory?

Transcription

1 Advances in Signal Processing 2(): 8-28, 204 DOI: 0.389/asp Can the Threshold Perforance of axiu Likelihood DOA Estiation be Iproved by Tools fro Rando atrix Theory? Yuri I. Abraovich, Ben A. Johnson 2, Institute for Telecounications Research University of South Australia, awson Lakes, SA 5045 Australia 2 Colorado School of ines, Golden, CO 8040 Corresponding Author: ben.a.johnson@ieee.org Copyright c 204 Horizon Research Publishing All rights reserved. Abstract For direction of arrival (DOA) estiation in the threshold region, it has been shown that use of Rando atrix Theory (RT) eigensubspace estiates provides significant iproveent in USIC perforance. Here we investigate whether these RT ethods can also iprove the threshold perforance of unconditional (stochastic) axiu likelihood DOA estiation (LE). Keywords Signal Detection and Estiation, axiu-likelihood Estiation, Rando atrix Theory Introduction The pre-asyptotic perforance of traditional stochastic (unconditional) LE of DOAs for closely-spaced Gaussian signals iersed in white Gaussian noise using an -sensor array and a finite nuber T independent identically distributed (i.i.d.) training saples continues to be the subject of investigations [3 5, 5] despite being a relatively old proble [6]. The ain reason, as forulated in [4], is that in the pre-asyptotic doain, no general non-asyptotic results are [currently] available for the perforance evaluation of the L ethod, and each proble requires a special investigation. As an alternative, instead of focusing on the accurate non-asyptotic analysis, one ay consider the asyptotic odel where both quantities and T grow without bound, while their quotient converges to a fixed finite quantity:, T, T c, 0 < c < () This condition is known as the Kologorov asyptotic condition [6], and underpins a field of analysis referred to as Generalized Statistical Analysis (GSA) or Rando atrix Theory (RT). Of course, in any practical situation, one deals with finite and T values, and therefore generalized G-asyptotic (i.e. asyptotic per the conditions in ()) results ay still not be sufficiently accurate. However, in nuerous studies, it has been deonstrated that estiators that are consistent G-asyptotically are ore robust in the presence of finite saples T than other estiators which are only consistent for T, =constant [6, 0,, 8]. Specifically, let us suppose that i.i.d. observations x,..., x T of rando vector ξ with diension are given, and we wish to estiate soe value φ(r ), where φ is a continuous function of the entries of the (true) covariance atrix R of vector ξ. If T is large and is fixed and does not change as T grows (i.e. the standard asyptotic case), then as an estiator of φ(r ), we ay take φ( ˆR ), where pli φ( ˆR ) = φ(r ) (2) T and ˆR is the standard saple covariance atrix for zero-ean data (= T T xxh ). oreover, if x j, j =,..., T is a set of i.i.d. coplex (circular) Gaussian saples (i.e. x j CN (0, R )), then ˆR and φ( ˆR ) are not only consistent (in the standard asyptotic fraework), but also the axiu likelihood estiate of R and φ(r ) respectively [7]. However, this failiar assertion is not in general true under the G-asyptotic condition

2 Advances in Signal Processing 2(): 8-28, (). There, for a wide range of functions φ(r ), one can find a easurable function ψ of the entries of the atrix R for which pli,t [φ( ˆR ) ψ(r )] = 0. (3) But in general, the functions φ(r ) and ψ(r ) do not coincide, in which case φ( ˆR ) is not necessarily a G- consistent estiate of φ(r ). However, with the help of function ψ(r ), one ay try to find a easurable function g( ˆR ) such that pli,t [g( ˆR ) φ(r )] = 0. (4) and where the distribution of noralized difference g( ˆR ) φ(r ) is asyptotically noral. The function g( ˆR ) is then called a G-estiator [6], and is consistent under condition (). In [0,], this RT ethodology has been used to find G-estiates of eigenvalues for a covariance atrix with known ultiplicity of its eigenvalues. Specifically, for an -variate covariance atrix R, let γ < γ 2 <... < γ be the set of distinct eigenvalues ( ) after accounting for individual ultiplicity K of the true eigenvalues (i.e. = K = ). Associated with each eigenvalue γ, there is a coplex subspace of diension K. This subspace is deterined by an K atrix of corresponding eigenvectors, denoted by E, such that EE H = I K. Note that this specification is unique up to right ultiplication by the orthogonal atrix, and therefore the proble of eigendecoposition for the atrix R R = γ j E j Ej H (5) is ore convenient to forulate as a proble of estiation of orthogonal projection atrices, defined as given the saple covariance atrix ˆR that has the eigendecoposition P = E E H, =,...,, (6) ˆR = ˆλ j Û j Ûj H (7) To use this eigendecoposition to estiate the orthogonal projection atrices P, let K be a set of indices K = K j, K j +,..., K j (8) with the cardinality of K equal to the ultiplicity of the eigenvalue γ, naely K. The classical (and indeed axiu likelihood for ultivariate Gaussian observations) estiator of the -th eigenvalue and orthogonal projection atrices P in (6) is given by One can see that ˆγ L and estiates of γ and P. ˆP L ˆγ L = K L ˆλk ; ˆP k K = k K Û k Û H k (9) are specified as φ( ˆR ) in (2) and indeed are strongly T -consistent (and L) Yet in Theore 2 of [0], X. estre proved that under certain conditions (As-As4), the traditional estiators of eigenvalues and eigensubspaces (ˆγ L and ˆη j L = S H k K Û k Ûk HS 2, respectively, where S and S 2 are two deterinistic vectors) can be shown to converge under condition () (i.e., T with /T = c constant and finite) to: γ L 0 and ˆη j L η L 0 (0) where the (non-rando) values γ L ˆγ L and η L γ L are defined as = γ c r K r γ r () γ r γ and w (k) = η L = w (k)s H E k Ek H S 2 (2) { K r K r γ γ r γ µ γ r µ ( ) γ γ r γ µ γ r µ k = k (3)

3 20 Can the Threshold Perforance of axiu Likelihood DOA Estiation be Iproved by Tools fro Rando atrix Theory? where µ is the -th solution to the following equation in µ: K r γ r γ r µ = c (4) under the convention µ < µ 2 <... < µ. Thus, the traditional L estiators for ˆγ L and ˆη L are not G-consistent estiates, and therefore could potentially be iproved within the RT ethodology. Iproved G-consistent estiates for γ and P under the conditions (As-As4) have been derived by X. estre, Theore 3 in [0]: ˆγ G = T (ˆλ k ˆµ k ), K k K ˆη G = ρ (k)s H ÛkÛ k H S 2 ; (5) where { φ (k), k / K ρ j (k) = + ψ (k), k K φ (k) = ( ) ˆλr ˆµ r r K ˆλ k ˆλ r ˆλ k ˆµ r ψ (k) = ( ) ˆλr ˆµ r ˆλ k ˆλ r ˆλ k ˆµ r r / K (6) and ˆµ ˆµ 2 ˆµ are the real-valued solutions to the following equation in ˆµ: ˆλ k ˆλ k ˆµ = c ; c = T (7) One can prove (see [0]) that for any fixed, as T, we have ˆµ k ˆλ k, T (ˆλ k ˆµ k ) ˆλ k (8) and therefore ˆγ G K k K ˆλk = ˆγ L (9) On the other hand, φ (k) 0 and ϕ (k) 0, which iplies that ˆη G ˆη L, as T. Therefore, the RT ethodology provides a unique set of G-consistent estiates for ˆγ G and ˆη G that tend to the classical LE estiates under traditional asyptotic assuptions ( = constant, T ). Specifically, the G-consistent USIC (or G-USIC) pseudo-spectru estiate ˆη G (θ) = ρ (k)s H (θ)ûkû k H S(θ) (20) has been suggested in [9] and deonstrated significant perforance iproveent copared with the traditional USIC function ˆη L (θ). Recently in [8], we showed that for closely spaced sources, despite the substantial iproveents deonstrated by G-USIC (20) with respect to conventional USIC, the threshold perforance of G-USIC (where the estiator s ean squared error departs rapidly fro the Craér-Rao lower bound as SNR and/or saple support is reduced) reains significantly worse than perforance of the rigorously ipleented (via global search) L DOA estiation. Since the RT-ethodology, and specifically the G-consistent estiates in (5), are able to iprove conventional USIC perforance and differ fro the traditional L eigenvalue and subspace estiates, it is quite legitiate to investigate whether these estiates ay be used to iprove the threshold perforance of L DOA estiation itself. In this correspondence, we try to address this question. Note that for S = S 2 = S(θ) where S(θ) is an -eleent antenna steering (anifold) vector specified by the DOA θ, and for =, ˆη (θ) is the traditional USIC pseudo-spectru function.

4 Advances in Signal Processing 2(): 8-28, G-asyptotic L DOA Estiation (G-LE) Forulation For the stochastic (unconditional) L odel under consideration, the set of T i.i.d. data x j CN (0, R 0 ), j =,..., T is described by the true/exact covariance atrix R 0, coprised of independent planewave sources with direction of arrival θ, ebedded in white noise. Under this odel, the traditional L DOA estiates are found as the global axiu of the stochastic (unconditional) likelihood function (LF) coputed for the odel covariance atrix R(Ω ), uniquely specified by 2 + paraeters in Ω ( source DOAs and powers as well as the noise power) and denoted L[R(Ω )]: ˆΩ = arg ax L[R(Ω )], Ω [ exp[ tr R (Ω ) L[R(Ω )] = ˆR] ] T (2) π det R(Ω ) or, equivalently, as the iniu of the (scaled) 2 log-likelihood function (LLF) ll[r(ω )] = tr R (Ω ) ˆR + log det R(Ω ) Note that for any given R(Ω ), the LLF (22) ay be viewed as the L estiate of the function (22) φ[r(ω ), R 0 ] φ[r 2 (Ω )R 0 R 2 (Ω )] = = tr [R 2 (Ω )R 0 R 2 (Ω )] + log det R(Ω ) (23) with an unknown R 0, here represented by its generic (positive definite Heritian, for T ) L estiate saple covariance atrix ˆR fored fro the training data x j. Since the likelihood function (22) is just φ( ˆR (Ω )) (per the notation in (2)), under the RT ethodology, we need to construct the G-consistent estiate g( ˆR (Ω )) (per the notation in (4)) of the function φ 0 (the portion of φ fro (23) dependent on R 0, which can be siplified to tr R (Ω )R 0 ). Yet, according to Lea 3. in [6] (see eqn. 3.0, p. 82), for a very broad class of epirical covariance atrices that ebraces the coplex Wishart case ˆR (Ω ) = T R 2 (Ω )R 2 0 ĈR 2 0 R 2 (Ω ); Ĉ CW(, T, I ), (24) under the Kologorov asyptotic condition (), the following property is proven: [ pli T, T c tr ˆR ] E( tr ˆR ) = 0, (25) where E( ) is the expectation operator and CW(, T, I ) is the coplex Wishart distribution [7]. This property eans that for the function φ 0, we have a special case where its G-consistent estiate g( ˆR (Ω )) coincides with its L estiate φ 0 ( ˆR): g( ˆR (Ω )) = φ( ˆR (Ω )) = tr [R (Ω ) ˆR] (26) In essence, the property (25) eans that for the ultivariate coplex Gaussian data case, the (scaled) unconditional log-likelihood function (22) ay be treated as the L estiate and at the sae tie, the G-consistent estiate of the function φ[r(ω ), R 0 ] given in (23). Lea 3. fro [6], and ore iportantly, its interpretation as G-consistency of the standard log-likelihood function 3, leaves an apparent contradiction which needs to be resolved. Specifically, since according to Theore 2 fro [0], individual L estiates of eigenvalues are not G-consistent and can be iproved, per (5). At the sae tie, the su of all eigenvalues (i.e. the saple covariance atrix trace) is G-consistent, and therefore cannot be iproved within the RT ethodology 4. This contradiction could be resolved by the direct proof that the su 2 This scaling by the factor / is iaterial for a fixed, but is subsequently required for G-asyptotic analysis with (, T ), /T c, for ll[r(ω )] to reain finite as 3 It is iportant to ephasize that G-consistency of the likelihood function (or the G-USIC function, for that atter) does not iply that the resultant DOA estiates are always G-consistent as well. On the contrary, the fact that the globally optial L DOA estiates (as well as the G-USIC DOA estiates) exhibit perforance breakdown in their respective threshold regions deonstrates that G-consistency of the likelihood or G-USIC function is only a necessary, but not at all sufficient condition for G-consistent DOA estiation. 4 While Theore 3 in [0], provides a unique set of G-consistent estiates (5) within the RT ethodology, it does not resolve questions regarding the broader uniqueness of G-consistent estiates. In this paper, per our title, we confine ourselves to the application of these (strongly) consistent RT estiates for individual eigenvalues and eigenvectors. While the possibility of another ethodology which provides better G-consistent estiates cannot be precluded, to our knowledge, such a ethodology has not yet been developed.

5 22 Can the Threshold Perforance of axiu Likelihood DOA Estiation be Iproved by Tools fro Rando atrix Theory? of G-consistent individual eigenvalue estiates ˆγ G in (5) is strictly equal to the su of saple eigenvalues ˆλ k of ˆR (Ω ), ie. K ˆγ G = ˆλ k = tr [ ˆR (Ω )] (27) = According to (5), we therefore have to prove that Theore Under conditions As-As3 in [0], the estiate K ˆγ G = T ( ˆλ k ˆµ k ) = ˆλ k (28) = is the G-consistent estiate of the trace of R, regardless of the eigenvalue splitting condition As4. The proof of this equivalence is shown in Appendix A, and shows that the trace of a saple atrix ˆR is always the G-consistent estiate of the trace of the original (true) covariance atrix R, even when for soe individual eigenvalues of this atrix, their G-consistent estiates ay not exist due to As4 not being satisfied. As an aside for the interested reader, estre s assuptions As to As3 (the details of which are provided in [0]) ipose general conditions on the covariance atrix and eigenvalues which are always eet under our proble with the saple Wishart atrix ˆR (Ω ), but assuption As4 ay or ay not be et. The physical eaning of the As4 condition, also referred to as the eigenvalue splitting condition, is that the asyptotic eigenvalue distribution of ˆR can be considered as a collection of clusters, each one centered around a different eigenvalue of R = E ˆR = R 2 (Ω )R 0 R 2 (Ω ). The size of the support of these clusters depends on the ratio of the nuber of sensors/antennas to the nuber of saples (ie. the ratio c = /T ) fro which ˆR is constructed. If the ratio /T is high, the clusters with close eigenvalues γ j of R ay erge and be represented by a single estiate with a certain ultiplicity K. If, on the contrary, the ratio /T is low, each saple eigenvalue ˆλ k will be associated with a different distinct cluster, well separated fro the rest of the saple eigenvalue distribution. This analysis provides an answer, therefore, to the ain question posed in the title of this correspondence: Within the existing RT fraework, iproved G-consistent estiates of individual eigenvalues ˆγ G and bilinear fors ˆη G given in (5) cannot ake the traditional log-likelihood function ore G-consistent, and therefore provide any iproveent into the threshold perforance of L DOA estiation, in full correspondence with Lea 3. fro [6] shown in (25) above. To confir this further and support nuerical evaluation, we still ay consider an alternative reconstruction of the G-consistent log-likelihood function using the individual estiates (5) in a anner siilar to the G-USIC derivation in [9]. Assuing that the eigenvalue splitting condition As4 fro [0] is satisfied for all the ( + ) different eigenvalues 5, we ay construct the G-LLF (log likelihood function) as g 0 [R(Ω )] = tr R (Ω ) ˆR G, ˆRG = + ˆλ G j ˆP G j. (29) where, based on (5), G-consistent estiates of the eigenspace P j in R 0 (R 0 = λ P + + j=2 λ jp j ) are given by ˆP G j = ρ j (k)ûkû k H ; ˆR = ˆλ j Û j Ûj H (30) Recall that G-USIC was calculated as per (20), and therefore (29)-(30) reseble this construction, but with soe differences. First of all, in contrast to standard eigendecoposition, we note that individual eigensubspace estiates ˆP j G in (30) are no longer utually orthogonal, though they can be shown to su to the identity atrix since the weights ρ j (k) in (6) su to unity. Therefore, if we introduce the diagonal atrix D(j) diag[ρ j (),..., ρ j ()], we iediately discover that according to (30), we get + ˆR G = Û ˆλ G j D j Û H. (3) While individual (G-consistent) estiates ˆP j G in (30) are different fro the L estiates ˆP j = k K j Û k Ûk H, the eigenvectors in ˆR G L are exactly the sae as in the L estiation ˆR (Theore 9.3. in [3]). 5 Recall that R 0 (as opposed to the whitened covariance atrix R ) consists of an diensional signal subspace and a noise subspace of diension

6 Advances in Signal Processing 2(): 8-28, Let us deonstrate that reconstructed eigenvalues λ R k in + ˆλ G j D j = diag[ˆλ R,..., ˆλ R k ] (at least G-asyptotically) tend to the sae liit as the (not G-consistent) LE estiate ˆγ L li,t /T c ˆλ R k li,t /T c ˆγ L j k : = γ k + c + K r γ r (32) γ r γ k Using (5)-(6), for the noise subspace eigenvalues (ˆλ R,..., ˆλ R ), we get ˆλ R k = ˆλ G k + ˆλ r ˆλ k ˆλ r ˆµ r ˆλ k ˆµ r ˆλ G r [ ] ˆλ r ˆµ r ˆλ k ˆλ, (33) r ˆλ k ˆµ r for k =,..., ( ) and with ˆλ G = ˆλ G 2 =... = ˆλ G, which is identical to ˆλ R k = ˆλ G k + ˆλ k (ˆλ G k ˆλ G r )(ˆλ r ˆµ r ) (ˆλ k ˆλ, k =,..., ( ). (34) r )(ˆλ k ˆµ r ) According to (5), for signal subspace eigenvalues (r ), we get ˆλ r ˆµ r = T ˆλ G r, and therefore we have ˆλ R k = ˆλ G k ˆλ k T (ˆλ G k ˆλ G r )ˆλ G r (ˆλ k ˆλ r )(ˆλ k ˆµ r ). (35) Note that for ˆλ r ˆµ r < ˆλ r and ˆµ r T ˆλ T r when As4 is strongly et. Therefore the difference between (35) and the expression (see () and (32)) ˆλ R k = ˆλ G k ˆλ G k T + (ˆλ G k ˆλ G r )ˆλ G r (ˆλ G k ˆλ G r )(ˆλ G k ˆλ G r ) (36) that strictly tends to γ L k in () under asyptotic condition (), is only a difference of the second order (in T ) agnitude. Therefore, the reconstructed noise subspace eigenvalues in ˆR G (3) G-asyptoatically are not strictly identical, but the fluctuations around its asyptotic value γ L is quite negligible. For signal subspace eigenvalues, these fluctuations are even less significant, which ay be deonstrated by derivations siilar to those above. While it would be ore satisfying if ˆλ R k was found to be strictly equal to the L version ˆγL k, this is not the case, as observed by other researchers in the field [2]. But practically speaking, it is not significant that non G-consistent estiates ˆγ k L are replaced in (29) by (slightly) different non G-consistent estiates ˆλ R k. The ain point is that the reconstruction in (29) using the G-consistent estiates of eigenvalues and eigenvectors (5) should be at best as efficient as the conventional accurate LE DOA estiate. Therefore, the conducted analysis has deonstrated how and why the G-consistent individual estiates of eigenvalues and eigenvectors help to iprove USIC threshold perforance, but still lead to the traditional L perforance. This property is proven rigorously for direct G-consistent estiation of tr (R (Ω )R 0 ), and in first order (in T ) approxiation for the reconstruction approach in (29). Let us now support these theoretical findings by epirical results. 3 Nuerical Evaluation Here, we seek by siulation to deonstrate, using direct exhaustive search for the global axiu of the likelihood function and its reconstructed G-odification, that their DOA estiation perforance in the threshold region is identical. For this analysis, we have selected the sae scenario as in [] with = 8 sensor ULA, T = 0 i.i.d. training saples and two equipower sources (p = p 2 ) with DOAs {0 o, 0.4 o } in white noise power p 0 =. The nuber of sources = 2 is liited by the coputational load associated with the exhaustive -diensional search for the LF global axiu, but still deonstrates the relevant LE threshold behavior. For this scenario, the CRB resolution liit (as predicted by the CRB RSE of 0.2 o, derived in ore detail in []) occurs at an SNR of approxiately 40dB (ε CRB = 40dB).

7 24 Can the Threshold Perforance of axiu Likelihood DOA Estiation be Iproved by Tools fro Rando atrix Theory? Prob ITC under est or USIC failure USIC AIC DL AP 8 ULA, θ = [0, 0.4] deg, T = 0, 000 trials SNR (db) Figure. Saple USIC breakdown for closely spaced sources. Note the SNR gap between reliable detection using ITC and reliable estiation using USIC. Fig. plots the probability of underestiating the correct nuber of sources as a function of SNR for the AIC, DL, and AP inforation theoretic criteria (ITC). The sae figure shows the saple probability of conventional USIC breakdown, which illustrates the well-docuented fact that USIC breaks at SNR values significantly larger than the actual LE threshold [8]. In this paper, we exaine the perforance at ε ITC = 30dB, where Fig. shows the reliable detection of the correct nuber of sources = 2, but the L DOA estiate itself is questionable. It can be separately confired, following the ethodology in [0], that for this scenario, the eigenvalue splitting condition As4, required for G-consistency of the eigenvalue estiation, are et for the 30dB case in this scenario. The fact that this condition is et here coes with no surprise. Indeed, the reliable detection of the correct nuber of sources = 2 at this SNR already iplies that the cluster of noise subspace saple eigenvalues (ˆλ to ˆλ 6 ) is reliably separated fro the cluster of the sallest signal eigenvalue ˆλ 7. Since λ 8 λ 7 for our scenario with closely spaced sources, it is really the separation between the λ 7 cluster and the noise subspace one (λ = = λ 6 = ), which is of concern for evaluation of the As4 condition ULA, θ = [0, 0.40] deg, T = 0, 30dB SNR, G, {U#30} G asy Eig Standard Eig saple pdf Eigenvalue Figure 2. Saple p.d.f.s for the eigenvalues of ˆR G We therefore exaine the eigendecoposition of the G-reconstructed covariance atrix ˆR G. As predicted, the eigenvectors of ˆR and ˆR G are exactly the sae, and only the eigenvalues differ slightly (and then only in the noise subspace). The two subspace eigenvalues are also well separated both fro each other and the noise subspace eigenvalues. While in ˆR the noise subspace is described by a single value (= close but not precisely equal to ˆλ (i.e. eig ( ˆR ) G < ˆλ G ˆλ j ), in ˆR G they reain < eig 6 ( ˆR )). G This is deonstrated in Fig. 2, where we introduce saple p.d.f.s for eig 7 ( ˆR ), G eig 8 ( ˆR ), G and the noise eigenvalues eig n ( ˆR ) G = (eig ( ˆR ), G..., eig 6 ( ˆR )), G where here we treat as the noise subspace eigenvalue in ˆR G any one of its six sallest eigenvalues. The p.d.f.s for ˆλ and eig n ( ˆR ) G practically coincide. While Fig. 2 does deonstrate inor variations in the noise subspace eigenvalues in ˆR G around the single noise subspace eigenvalue ˆλ G in ˆR, caused by finite T and values, these variations are very unlikely to result in any noticeable changes in DOA estiation perforance when ˆR G is used instead of ˆR or ˆR in the likelihood function, siply because even larger variations of noise subspace eigenvalues in ˆR around a single L estiate (= ˆλ j ) in ˆR are also not associated with any DOA L estiation perforance degradation. Yet, since all the conditions (As-As4) of the Theore 3 [0] are et, we can expect that these inor variations do result in iproved individual eigenvalue ˆλ G j and eigensubspace ˆP j G estiation accuracy. To confir this, we repeat the analysis conducted in [0], where to assess the quality of the eigensubspace estiates, estre introduced an orthogonality factor: O(j) = tr [P ˆP j j ] tr [I n P j ] ˆP j. (37) Here P j, j =, 2, 3 are the eigenvectors/subspaces of the true covariance atrix R 0 and ˆP j are the estiates

8 Advances in Signal Processing 2(): 8-28, (either standard L ones or the G-consistent ˆP j G ones). This orthogonality factor is the ratio of the total power of the estiated subspace ˆP j that resides in the true subspace P j to the power of its leakage into the true orthogonal subspace [I P j ] ULA, θ = [0, 0.40] deg, T = 0, 30dB SNR, G, 500 {U#30} Rhat Ghat ULA, θ = [0, 0.40] deg, T = 0, 30dB SNR, G, 500 {U#30} Rhat Ghat saple pdf 0. saple pdf orthogonality orthogonality 3 Figure 3. Saple p.d.f.s for LE and GLE orthogonality factors for the two signal eigenvectors. The estiate ˆP G j (as defined in (30)) is better than the LE ˆP j, defined as ˆP = 6 ÛjÛ H j ; ˆP2 = Û7Û H 7 ; ˆP 3 = Û8Û H 8 if, with high probability, the respective orthogonality factor is greater. In Fig. 3, we introduce saple p.d.f.s for the orthogonality factor O(j) and O(j) G, averaged over the sae 500 onte-carlo trials for the SNR=30dB case. Siilarly to [0], we observe that the G-consistent subspace estiate achieves a uch higher orthogonality factor than the traditional L saple estiator, indicating a better capacity to estiate subspaces, and therefore better accuracy for subspace-based DOA estiation techniques such as USIC. To deonstrate, we copare the results of exhaustive global axiu search for the conventional (noralized) likelihood function (ratio): LR[R(Ω )] = det R (Ω ) ˆRe e tr R (Ω ) ˆR and its G-asyptotic counterpart GLR[R(Ω )] constructed using ˆR G instead of ˆR. (38) 8 ULA, θ = [0, 0.40] deg, T = 0, 30dB SNR, R, 500 {U#30} ULA, θ = [0, 0.40] deg, T = 0, 30dB SNR, G, 500 {U#30} 0 0 saple pdf saple pdf θ (deg) θ (deg) Figure 4. Saple p.d.f.s for LE and GLE DOA estiation at 30dB SNR; a- ˆR, b- ˆR G. This global optiization is perfored in two steps. In step, we find the global extreu over a fine grid on a half-plane (θ < θ 2 ), and in step 2, we conduct local optiization over the paraeters θ < θ 2 as well as the two source powers using the ATLAB finco routine. The white noise power is assued known a priori and set to unity. ore details on the optiization routine and LE results can be found in [, 2, 4]. Due to our convention that ˆθ < ˆθ 2, the DOA saple distributions illustrated by Fig. 4 overlap, since for scenarios with very closely spaced sources, over the 500 trials, the iniu ˆθ 2 was soeties less than the axiu ˆθ, even though on each individual trial θ was less than θ 2. One can see that the distribution for the DOAs derived using the standard LR (38) and the G-asyptotic counterpart are practically indistinguishable, as are any oents of these distributions. oreover, in addition to the identical statistical perforance of the DOA estiates, identical individual estiates were registered in a trial-by-trial basis. In addition to the illustrated results, the sae results were observed at several higher SNRs, and were also the sae when the L covariance atrix estiate ˆR = was used. Finally, in a copletely ad-hoc ˆγL ix and atch approach, we also assigned the G-consistent eigenvalue estiates ˆλ G (the G-consistent noise power estiate), ˆλ G 2 and ˆλ G 3 (5) to the axiu likelihood (utually orthogonal) eigensubspaces ˆP, ˆP2, and ˆP 3 and once again did not record any DOA estiation iproveents, showing that the inor variations in noise subspace eigenvalues in ˆR G with respect to ˆR was not significant in LE. P L

9 26 Can the Threshold Perforance of axiu Likelihood DOA Estiation be Iproved by Tools fro Rando atrix Theory? 4 Concluding Rearks The theoretical results fro RT and their exaination by direct onte-carlo siulations has confired that for Gaussian sources in Gaussian noise, L DOA estiation in the so-called threshold region is not iproved by use of recent results fro Rando atrix Theory. ore specifically, we deonstrated that the Girko Lea 3. fro [6] iplies that for the ulti-variate (coplex) Gaussian data case, the stochastic (unconditional) likelihood function is both a T -consistent and G-consistent estiate of the function / tr [R (Ω )R 0 ], where R 0 is represented by a set of T i.i.d. training vectors x j. We deonstrated that in full accordance with this Lea, an alternative reconstruction of the log-likelihood function using individually G-consistent estiates of eigenvalues and eigenvectors does not iprove DOA estiation threshold perforance, despite the fact that these individual G-consistent estiates are ore accurate than their corresponding L estiates. Thus, the deonstrated in [9 ] iproveent in USIC perforance delivered by the G-consistent estiates of the individual noise eigensubspace is not at odds with the lack of iproveent in LE perforance. While applicable to arbitrary paraetric description of the covariance atrix R 0, these results should not be extended beyond the ultivariate (coplex) i.i.d. Gaussian case without detailed exaination. Finally, it is iportant to stress that our results are confined to the currently (in [0]) unique set of G-consistent eigenvalue and eigenvector estiates, and therefore the broader question on whether the threshold perforance of L DOA estiation can be iproved by soe other technique reains open. Acknowledgeents The authors express their gratitude to Dr. Xavier estre of the Telecounications Technological Center of Catalonia (CTTC) for his guidance on RT results. Appendix Proof of Theore The G-consistent eigenvalue estiators defined in Theore 3 in [0] ((5) in the ain body of the paper) can be restructured as K ˆγ G = T ( ˆλ k ˆµ k ), (4.) = k k and therefore we have to prove that T ˆµ k = (T ) ˆλ k. (4.2) Given eqn. (7) in the ain body of the paper, we ay define T ˆµ k = = ˆλ ˆµ k ˆλ ˆµ k. (4.3) Substitution shows that instead of the for in eqn. (7), ˆµ,..., ˆµ can be introduced as the roots of the following polynoial: Q(µ) = (ˆµ k µ) = (ˆλ k µ) T ˆλ r (ˆλ j µ) j r (4.4) where the right-hand side expression is obtained fro eqn. (7). Evaluating Q(µ) at µ = λ, we obtain (ˆλ ˆµ k ) = T ˆλ j (ˆλ ˆλ j ). (4.5)

10 Advances in Signal Processing 2(): 8-28, On the other hand, observing the for of the derivative of Q (µ): Q (µ) = = and evaluating it at µ = ˆλ, we obtain l= (ˆµ k µ) = k l (ˆλ j µ) + T l= j l ˆλ r (ˆλ j µ) l= l r j r j l (4.6) (ˆµ k ˆλ ) = (ˆλ j ˆλ ) T ˆλ l= k l j l= l j j l (ˆλ j ˆλ ) T ˆλ r r j r j (ˆλ j ˆλ ) (4.7) Dividing both sides of (4.7) by (ˆλ j ˆλ ) and using (4.5) to siplify the left-hand side, we obtain j r, T l= ˆλ ˆλ ˆµ l = T r ˆλ + ˆλ r ˆλ r ˆλ (4.8) (Note that (4.4)-(4.8) follow Appendix IV in in [0]). On the other hand, fro (4.3) we get ˆλ k ˆµ j ˆλ k ˆµ j = ˆλ k ˆλ k (4.9) ˆλ k ˆµ j Fro (4.8), it then follows that ˆλ k ˆλ k ˆµ j = T ˆλ k + ˆλ r ˆλ r ˆλ k (4.0) Since ˆλ k + ˆλ r ˆλ r ˆλ k + = ( 2ˆλ r ˆλ r ˆλ k ) (4.) we get ˆλ k ˆµ j ˆλ k ˆµ j = (T + ) ˆλ k 2ˆλ kˆλr ˆλ r ˆλ k. (4.2) But for ˆλ k > 0, the following identity is true (see [0]) ˆλ kˆλr ˆλ r ˆλ k = 0. (4.3) Therefore, we get T ˆµ k = (T ) ˆλ k (4.4) as required by (4.2).

11 28 Can the Threshold Perforance of axiu Likelihood DOA Estiation be Iproved by Tools fro Rando atrix Theory? REFERENCES [] Y.I. Abraovich, B.A. Johnson, and N.K. Spencer. Statistical nonidentifiability of close eitters: axiu-likelihood estiation breakdown. In Proc. EUSIPCO-2009, pages , Glasgow, UK, [2] Y.I. Abraovich, B.A. Johnson, and N.K. Spencer. Statistical nonidentifiability of close eitters: axiu-likelihood estiation breakdown and its GSA analysis. In Proc. ICASSP-2009, pages , Taipei, Taiwan, [3] F. Athley. Threshold region perforance of axiu likelihood direction of arrival estiators. IEEE Trans. Signal Processing, 53(4): , Apr [4] P. Forster, P. Larzabal, and E. Boyer. Threshold perforance analysis of axiu likelihood DOA estiation. IEEE Trans. Sig. Proc., 52():383 39, Nov [5] A.B. Gershan. Pseudo-randoly generated estiator banks: a new tool for iproving the threshold perforance of direction finding. IEEE Trans. Sig. Proc., 46(5):35 364, ay 998. [6] V.L. Girko. An Introduction to Statistical Analysis of Rando Arrays. VSP, Utrecht, Netherlands, 998. [7] R.A. Janik and.a. Nowak. Wishart and anti-wishart rando atrices. J. Phys. A, 36: , [8] B.A. Johnson, Y.I. Abraovich, and X. estre. USIC, G-USIC, and axiu-likelihood perforance breakdown. IEEE Trans. Sig. Proc., 56(8): , August [9] X. estre. An iproved subspace based algorith for sall saple size regie. In Proc. ICASSP-06, Toulouse, [0] X. estre. Iproved estiation of eigenvalues and eigenvectors of covariance atrices using their saple estiates. IEEE Trans. Info. Theory, 54():53 529, Noveber [] X. estre. On the asyptotic behaviour of the saple estiates of eigenvalues and eigenvectors of covariance atrices. IEEE Trans. Sig. Proc., 56(): , Noveber [2] Xavier estre. E-ail counications. arch 05, 200. [3] R.J. uirhead. Aspects of ultivariate Statistical Theory. Wiley, New York, 982. [4] P. Parvazi, A.B. Gershan, and Y. Abraovich. Detecting outliers in the estiator bank-based direction finding techniques using the likelihood ratio quality assessent. In Proc. ICASSP, volue II, pages , Honolulu, Hawaii, April IEEE. [5] Christ D. Richond. ean-squared error perforance prediction of axiu-likelihood signal paraeter estiation. In Proceedings of the Adaptive Sensor Array Processing (ASAP) Workshop, Lexington, A, -3 ar 998. IT-LL. DTIC Ascension ADA [6] D. C. Rife and R. R. Boorstyn. ultiple tone paraeter estiation fro discrete-tie observations. Bell Syste Technical Journal, 55(9):389 40, Nov 976. [7] S. Zacks. The Theory of Statistical Inference. Wiley, New York, 97. [8] K. Zarifi and A.B. Gershan. Asyptotic perforance analysis of blind iniu output energy receivers for large DS-CDA systes. IEEE Trans. Sig. Proc., 56(2): , Feb 2008.