Asymptotic analysis of a GLRT for detection with large sensor arrays

Transcription

1 Asymptotic analysis of a GLRT for detection with large sensor arrays Sonja Hiltunen, P Loubaton, P Chevalier To cite this version: Sonja Hiltunen, P Loubaton, P Chevalier. Asymptotic analysis of a GLRT for detection with large sensor arrays. EUSIPCO, Sep 204, Lisbonne, Portugal nd European Signal Processing Conference EUSIPCO). <hal-06645> HAL Id: hal Submitted on 3 Oct 207 HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

2 ASYMPTOTIC AALYSIS OF A GLRT FOR DETECTIO WITH LARGE SESOR ARRAYS S. Hiltunen ),2) P.Loubaton 2) P. Chevalier 3),) ) Thales-Communications-Security, AMS/TCP, 4 Av. des Louvresses, Gennevilliers Cédex, France 2) Université Paris-Est Marne-la-Vallée, LIGM, UMR CRS 8049, 5 Boulevard Descartes, Marne la Vallée Cédex 2, France 3) CAM, Laboratoire CEDRIC, 292 rue Saint-Martin, 754 Paris Cédex 3, France. ABSTRACT This paper addresses the performance analysis of two GLRT receivers in the case where the number of sensors M is of the same order of magnitude as the sample size. In the asymptotic regime where M and converge towards at the same rate, the corresponding asymptotic means and variances are characterized using large random matrix theory, and compared to the standard situation where + and M is fixed. This asymptotic analysis allows to understand the behavior of the considered receivers, even for relatively small values of and M. Index Terms Multichannel detection, asymptotic analysis, large random matrices. ITRODUCTIO Due to the spectacular development of sensor networks, it is common to be faced with multivariate signals of high dimension. Very often, the sample size that can be used in practice in order to perform statistical inference cannot be much larger than the signal dimension. In this context, it is well established that a number of fundamental existing statistical signal processing methods fail. It is therefore of crucial importance to revisit certain classical problems in the highdimensional signals setting. Previous related works include e.g. [] and [2] in the source localization using subspace method context, or [3], [4] in the context of unsupervised detection. In the present paper, we address the problem of detecting a known signal corrupted by a temporally white, but possibly spatially correlated, additive complex Gaussian noise using a large sensor array. This is an important question that has been extensively addressed in the past, in the low dimensional signal context [5], [6], [7]). Although our results can be used in more general situations, we focus on the detection of a synchronization sequence transmitted by a single transmitter in a multipath propagation channel. Using random matrix methods, we study the behaviour of the classical generalized likelihood test GLRT) when the number of sensors M and the number of samples of the training sequence are large and of the same order of magnitude. It is shown that the asymptotic behaviour of the relevant statistics is quite different from the standard situation where + while M remains fixed, and that the performance predicted by the regime where M and are large provide results that are reliable for realistic values of M and. The large random matrix technics used in this paper are more or less classical see e.g. [8], [9]), but our results are of interest in the field of statistical signal processing. 2. PRESETATIO OF THE PROBLEM In the following, we assume that a single transmitter sends a synchronization sequence s n) n=,..., through a channel with L paths, and that the corresponding signal is received on a receiver with M sensors. The received M-dimensional signal is denoted by y n ) n=,...,. When the transmitter and the receiver are perfectly synchronized, y n is assumed to be given for each n =,..., by L y n = h l s n l + v n ) l=0 where v n is an additive complex Gaussian noise. Denoting by H the M L matrix H = h 0,..., h L ), Y = y,..., y ) can be written as Y = HS + V 2) s... s where V = v,..., v ) and where S =... s 2 L... s L+ represents the signal matrix. We assume from now on that the size of the training sequence satisfies > M + L. In order to simplify the calculations and notations, we assume that the signal matrix is orthogonal, meaning that SS = IL. In this paper, we study the classical problem of testing the hypothesis H characterized by 2) against the hypothesis H 0 defined by Y = V 3) When the additive noise is complex Gaussian, temporally white, but possibly spatially correlated with unknown covariance matrix, it is well established see e.g. [5]) that the generalized maximum likelihood test consists in comparing the statistics η c, defined by η c, = log det [ I L SY /)YY /) YS /) ] 4) to a threshold. If the noise is known to be also spatially white, the synchronization statistics, normalized by M, becomes )[ η w, = Tr SY /)YS /) M Tr YY /)] 5) As the size of the training sequence should be as small as possible, it is of practical interest to evaluate the behaviour of the two statistics when M and are of the same order of magnitude. To simplify the analysis made with random matrix theory, we study the limit distribution of both statistics with the assumption that the noise v n) n=,..., is temporally and spatially white, i.e. Ev mv n) = σ 2 I M δn m). We want to be clear with that this assumption is only made when calculating the limit distributions. The two GLRT statistics remain optimal for their respective noise type.

3 We also mention that the resulting limit distribution of the statistics under each hypothesis does not, in general, allow us to obtain a reliable approximation of the performance of the test for small false alarm probabilities. For this, it is recommended to use large deviations theory see e.g. [0]), which, in the large dimensional context, is not well developed. However, our results may be used to predict the detection probability corresponding to false alarm probabilities of the order of 0.. This point is not developed in this paper. In the following, we denote by W a L ) matrix for which the matrix Θ = W T, ST ) T is unitary and define the M L) and the M L) matrices V and V 2 by V, V 2) = VΘ = ) VW, V S It is clear that V and V 2 are complex Gaussian random matrices with independent identically distributed entries of variance σ 2, and that the entries of V and V 2 are mutually independent. Moreover, VV = VV + V2V 2 ote that, as > M + L, then matrix V V is invertible a.s. 3. STADARD ASYMPTOTIC AALYSIS In order to get a better understanding of the similarities and differences with the more complicated case where M and converge towards + at the same rate, to be discussed in section 4, we first recall some standard results concerning the asymptotic distribution of the statistics under H 0 and H when + but when M remains fixed see e.g. [7] where similar calculations are presented). 3.. Asymptotic behaviour under hypothesis H 0 Under hypothesis H 0, η w, can be written as 6) 7) η w, = Tr V 2V 2/) /Tr VV /M) 8) TrVV /M) converges a.s. towards σ 2. Therefore, η w, = Tr V2V 2/) σ 2 + o P ) = Tr V2V 2/) σ 2 + o P /) 9) and since V 2 is a M L matrix, it appears immediately that the limit distribution of η w, is a χ 2 distribution with 2ML degrees of freedom. This implies that Eη w, ) L M and Varηw, ). Similarly, ηc, can be written as L M η c, = log det I V2/ ) VV /) V ) 2/ ) 0) When + and M remains fixed, VV / converges towards σ 2 I and V 2 VV /) V 2 converges towards the zero matrix. Therefore, η c, converges a.s. towards 0. In order to characterize the limit distribution of η c,, we use the classical linearization procedure η c, = σ 2 Tr V 2V 2/) + o P /) from which it is clear that η c, has the same limit distribution as η w, Asymptotic behaviour under hypothesis H When M is fixed while, the behaviour of η w, under hypothesis H is given by Theorem η w, converges almost surely towards η w, defined by TrHH ) η w, = M TrHH + σ 2 I) Moreover, the asymptotic distribution of η w, η w, ) is a zero-mean Gaussian distribution with variance κ, where κ = 2σ6 TrHH ) + TrHH )) 2 σ 4 /M M 4 TrHH + σ 2 I)) 4 Under hypothesis H, η w, is given by [ Tr V ) 2/ + H V ) ] 2/ + H η w, = ) ) YY Tr M / The denominator becomes YY Tr M = M TrHH + σ 2 I) + M TrHV 2)/ + + M TrV2H )/ + M Tr VV / σ 2 I) + M TrV2V 2)/ When + and M remains fixed, TrYY )/M can be written as TrHH + σ 2 I))/M + ξ where ξ = O P / ). ote that TrV 2V 2)/ is O P /) and does not enter into the calculations in this asymptotic regime. Similarly, the numerator can be written as TrHH ) + β where β = O P / ). This gives us our expected value. To calculate the variance, elementary manipulations show that TrHH ) η w, = TrHH + σ 2 I)/M + TrHH + σ 2 I)/M [ ] TrHH ) β TrHH + σ 2 I)/M ξ + O P /) from which the variance is easily calculated to justify the Theorem. We now specify the behaviour of η c, in this asymptotic regime. Theorem 2 η c, converges almost surely towards η c, defined by ) η c, = log det I + H H σ 2 Moreover, the asymptotic distribution of η c, η c, ) is a zeromean Gaussian distribution with variance κ 2, where [ ) ] 2 κ 2 = Tr I I + H H 2) σ 2 It is easily seen that η c, = log det ) I + H H + o σ 2 P ) Using the linearization procedure, we obtain immediately that ) [ ) ] η c, = log det I + H H σ 2 + Tr I + H H σ 2 + O P /)

4 where matrix is given by = H HH + σ 2 I) Γ HH + σ 2 I) H+ V 2/ )HH + σ 2 I) H + H HH + σ 2 I) V 2/ ) with Γ = YY / HH + σ 2 I ). The Theorem follows from the observation that YY / HH + σ 2 I) 0 a.s. when and M is fixed, and from standard calculations. 4. ASYMPTOTIC AALYSIS WHE M AD COVERGE TOWARDS AT THE SAME RATE In this section, we assume that M and converge towards + in such a way that c = M converges towards c, where 0 < c < while the number of paths L remains fixed. This hypothesis is of course consistent with the condition > M + L. This asymptotic analysis differs deeply from the analysis in section 3. In particular, it is no longer true that the empirical covariance matrix YY / converges in the spectral norm sense towards its mathematical expectation. This, of course, is due to the fact that the number of entries of this M M matrix is of the same order of magnitude than the M available scalar observations. We also note that for any deterministic M M matrix A, the L L matrix V 2AV 2 converges towards 0 when + and M remains fixed, while this does not hold when M and are of the same order of magnitude. As M is growing, we have to specify how the power of the useful signal component HS is normalized. In the following, we assume that the norms of vectors h l ) l=0,...,l remain bounded when the number of antennas M increases. This implies that the signal to noise ratio at the output of the matched filter S Y/, i.e. L TrH H)/σ 2, is a O) term in our asymptotic regime. We however mention that the received signal to noise ratio TrHSS H /Mσ 2 ) converges towards 0 at rate when M M increases. Before studying the behaviour of the statistics, we first review some useful results [8],[],[9]. 4.. Background material. Proposition We consider an M ) random matrix X with variance σ 2 i.i.d. complex Gaussian entries. Then, λ min XX /) σ 2 c) 2 a.s. 3) Moreover, for each pair of unit norm deterministic M dimensional vectors u and v it holds that, for i =, 2, u XX /) i u v v = + σ 2i OP ) c ) 2i 4) Tr XX /) i c = σ 2i c ) + 2i OP ) 5) We note that if c 0, the conventional results obtained when M remains fixed are recovered. We finally mention the following result. Proposition 2 We consider a M L random matrix Z = z,..., z L) with variance σ 2 i.i.d. complex Gaussian entries. Then, for each random M M matrix A independent from Z for which sup M A < +, it holds that z kaz l δ k l σ 2 z E kaz l Z δ k l σ 2 TrA) TrA) 0 a.s 6) 2 = σ4 TrAA ) 7) where E Z represents the mathematical expectation w.r.t. the entries of Z Hypothesis H 0. The behaviour of η w, under H 0 is given by the following theorem Theorem 3 η w, Lc converges almost surely towards 0, and the asymptotic distribution of η w, Lc ) is a zero mean normal distribution with variance Lc. We take 8) as a starting point. TrV 2V 2/) converges a.s. to σ 2 ML/ whereas TrVV /M) converges to σ 2, which gives us our expected value. When M,, TrVV /M) σ 2 = O P / M) = O P /). Therefore, η w, = TrV2V 2/) σ 2 + O P /) which justifies the Theorem. We now specify the behaviour of η c, in our asymptotic regime. Theorem 4 η c, converges almost surely towards η c, = L log c ), and the asymptotic distribution of ) ηc, η c, is a zero mean normal distribution with variance Lc c. We use 7), 0) and the matrix inversion lemma, and obtain that η c, = log det I L + V2/ V V/) V ) 2/ In the following, we denote by F the L L matrix defined by F = V 2/ V V /) V 2/ 8) We provide only a sketch of proof. We first evaluate the almost sure behaviour of η c,. We use Proposition ) and Proposition 2) for Z = V 2/ and A = V V/). By 3), matrix A satisfies sup M A < + almost surely for M, large enough, and we deduce immediately that L L matrix F converges almost surely towards the matrix c c I L. We have used that, for a fixed L, the M L ratio behaves as c. This justifies the first statement of the Theorem. We do not prove the asymptotic Gaussianity of the η c,, but provide only an informal justification of the expression for the asymptotic variance. Using a standard first order expansion of η c,, η c, = L log c + Tr c ) ) + O P /) 9) where represents the difference between matrix F defined by 8) and matrix c c I L. Tr c ) ) can be written as Tr c ) ) = c ) L l= ɛ l where ɛ l = V2,l/ V V/) V 2,l / c and V 2,l represents column l of matrix V 2. By Proposition, ɛ l can be written as ɛ l =V 2,l/ V V /) V 2,l / σ2 Tr VV /) + O P /) By 7), it holds that the expectation of ɛ l 2 w.r.t. V 2,l is given by E V2,l ɛ l 2 = σ4 c Tr VV /) 2 + o P /)

5 Taking the expectation w.r.t. V and using 5), we obtain that the c variance of ɛ l is equal to. It is clear that the random variables ɛ l ) l=,...,l are decorrelated. Therefore, the asymptotic vari- c ) 3 ance of c ) L l= ɛ l coincides with Lc c as expected. ) We recall that if M is fixed, η w, and η c, both behave like a χ 2 distribution with 2ML degrees of freedom. Therefore, the behaviour of η c, in the two asymptotic regimes deeply differ. However, if c 0, log c ) c, and the asymptotic means and variances of η c, tend to coincide. For η w,, the asymptotic expected value and variance in the two regimes are the same, but when M, the asymptotic distribution is a normal distribution instead of a χ 2 distribution Asymptotic behaviour under hypothesis H. The limit distribution of the statistics η w, under hypothesis H, defined by ), is given by the following result. Theorem 5 η w, converges almost surely towards η w, defined by TrHH ) η w, = M TrHH + σ 2 I) + Lc Moreover, the asymptotic distribution of η w, η w, ) is a zero-mean Gaussian distribution with variance κ 3, where κ 3 = 2σ2 Tr HH + σ 4 Lc M 2 [TrHH + σ 2 I)] 2 = κ + Lc + O P /) 20) Since M is of the same order of magnitude as, the term TrV 2V2/) in the numerator of ) is no longer negligeable. In the denominator, TrV V + V 2V2)/M still converges towards σ 2, but TrHH /M) is now an O P /) term. For sake of comparison, however, we will keep the TrHH /M) term in the expression of the expected value and variance. This justifies the first statement of the Theorem. To calculate the variance, we note that the fluctuations of the denominator of ) are faster than those of the numerator. In the numerator, VarTrV 2H / )) = VarTrHV2/ )), = σ2 TrHH ), and VarTrV 2V2/)) = σ2 ML. These are all O 2 P /) terms, whereas the corresponding terms, scaled by M, in the denominator, are O P / 2 ) terms. The asymptotic variance can thus be expressed as VarTrHV 2/ + V 2H / + V 2V 2/)) M 2 [TrHH + σ 2 I)] 2 Since the three terms in the above expression are decorrelated, the variance of η w, equals the sum of the variances of the three terms. Elementary calculations lead to 20). The behaviour of η c, under hypothesis H is given by the following result. Theorem 6 η c, converges almost surely towards η, defined by η c, = L log c + log det I + H H/σ 2) Moreover, the asymptotic distribution of η c, η c, ) is a zeromean Gaussian distribution with variance Lc c + κ 2, where κ 2 is defined by 2). We again just give a sketch of proof. Defining the L L matrix G = H + V 2/ ) VV/) H + V ) 2/ 2) we obtain after some standard algebra that, under hypothesis H, η c, can be written as η c, = log det I L + G ) In order to evaluate its almost sure behaviour, we expand G. Using 4), it is easy to check that the matrix H V V/) H converges towards σ 2 c ) H H and that the matrix V 2/ ) V V/) H converges towards the zero matrix. c As we already showed that matrix F converges towards c I L, it holds that I L+G behaves as c I L+H H/σ 2 ) as expected. In order to prove the asymptotic Gaussianity of η c,, we again use the linearization trick, and express η c, η c, as η c, η c, = Tr c )I L + H H/σ 2 ) ) +OP /) where matrix is defined by ) c H H = G I + c σ 2 c ) η c η c, can be expressed as the sum of four terms that are asymptotically decorrelated. The asymptotic Gaussianity and the evaluation of the variance of three of these terms can be addressed as before using Propositions and 2. The proof of the asymptotic Gaussianity of the contribution of the term H V V/) H to η c η c, needs to establish a central limit theorem for L l= a l V V/) b l where a l, b l ) l=,...,l are bounded deterministic vectors. For this, we use an approach developed by Pastur and his colleagues [2] and [9], see also [3] where the approach is used to study the mutual information of large MIMO systems). Interestingly, it is seen that the asymptotic variances of both η w, and η c, coincide with the sum of their asymptotic variance in the standard regime + and M fixed, and an extra term which coincides with their asymptotic variance under H UMERICAL RESULTS In this section, we validate the relevance of the Gaussian approximation of section 4 and compare it with the asymptotic distributions obtained by the standard asymptotic analysis of section 3. In our numerical experiments, we have calculated the theoretical expected values and variances as well as their empirical values, evaluated by Monte Carlo simulations with trials. In the experiments, L has been fixed to 5, M to 40, while is increased, starting from = 80. Moreover, to avoid variance problems due to channel variations, the channel matrix H is fixed through all trials. Figures and 2 compare the theoretical variances with the empirical variance for η w, and η c, respectively. When c is small, the two theoretical variances coincide, as expected, and are very close to the true variance. When c is large, the assumption that M is small compared to is no longer valid, and the classical asymptotic analysis fails. It is interesting to note that even for small values of M,, the theoretical variance obtained by considering M, the same order of magnitude is close to the empirical variance. If, M are increased while keeping c constant, the empirical results are even closer to the theoretical values, since the number of samples is larger.

6 theoretical variance theoretical variance fixed M empirical variance Variance under H Variance under H c Fig.. Variance of η w, under hypothesis H theoretical variance theoretical variance fixed M empirical variance c Fig. 2. Variance of η c, under hypothesis H ote that despite the white noise assumption for calculating the limit distribution, detailed in section 2, we are able to characterize the limit distribution of both statistics in a satisfactory way. Due to lack of space, we omit the results for the expected values and the variance under H 0. We mention however that for both statistics, the theoretical expected values and variances calculated with the assumption that M, are of the same order of magnitude, are very close to their empirical counterparts. To validate the asymptotic Gaussianity of the synchronisation statistics, Figure 3 contains a histogram of the empirical values of η c, under H, along with a normal distribution of the theoretical expected value and variance, for the assumption, M at the same rate. Similar results also apply for the other statistics, with the same assumption, and for hypothesis H COCLUSIO In this paper, the asymptotic distributions of two generalized likelihood ratios are studied. The corresponding asymptotic expected values and variances are characterized when the number of samples in the training sequence increases at the same rate as the number of sensors M. It is seen that the asymptotic means and variances are close to their empirical counterparts for realistic values of M and. REFERECES [] X. Mestre and M.A Lagunas, Modified subspace algorithms for doa estimation with large arrays, IEEE Trans. Signal Process., vol. 56, no. 2, pp , empirical mean η c, Fig. 3. Empirical η c, superposed with a Gaussian distribution with its theoretical expected value and variance. =80, M=0, L=5. [2] P. Vallet, P. Loubaton, and X. Mestre, Improved subspace estimation for multivariate observations of high dimension: the deterministic signals case, IEEE Trans. Inf. Theory, vol. 58, no. 2, pp , 202. [3] P. Bianchi, M. Debbah, M. Maeda, and J. ajim, Performance of statistical tests for single source detection using random matrix theory, IEEE Trans. Inf. Theory, vol. 57, no. 4, pp , 20. [4] R.R. adakuditi and J.W Silverstein, Fundamental limit of sample generalized eigenvalue based detection of signals in noise using relatively few signal-bearing and noise-only samples, IEEE Journal on Selected Topics in Signal Process., vol. 4, no. 3, pp , 200. [5] D. W. Bliss and P. A. Parker, Temporal synchronization of mimo wireless communication in the presence of interference, IEEE Trans. Signal Process., vol. 58, no. 3, pp , 200. [6] Y. Jiang, P. Stoica, and J. Li, Array signal processing in the known waveform and steering vector case, IEEE Trans. Signal Process., vol. 52, no., pp , [7] M. Viberg, P. Stoica, and B. Ottersten, Maximum likelihood array processing in spatially correlated noise fields using parameterized signals, IEEE Trans. Signal Process., vol. 45, no. 4, pp , 997. [8] Z. Bai and J.W. Silverstein, Spectral analysis of large dimensional random matrices, Springer Series in Statistics, 2nd edition, 200. [9] L.A. Pastur and M. Shcherbina, Eigenvalue Distribution of Large Random Matrices, Mathematical Surveys and Monographs, Providence: American Mathematical Society, 20. [0] A. Dembo and O. Zeitouni, Large deviations techniques and applications, Springer, 2nd edition, [] V.L. Girko, An introduction to statistical analysis of random arrays, VSP, 998. [2] L.A. Pastur, A simple approach for the study of the global regime of large random matrices, Ukrainian Math. J., vol. 57, no. 6, pp , [3] W. Hachem, O. Khorunzhiy, P. Loubaton, J. ajim, and L. Pastur, A new approach for capacity analysis of large dimensional multi-antenna channels, IEEE Trans. Inf. Theory, vol. 54, no. 9, pp , 2008.