Analysis of Optimal Diagonal Loading for MPDR-based Spatial Power Estimators in the Snapshot Deficient Regime

Transcription

1 MITSUBISHI ELECTRIC RESEARCH LABORATORIES Analysis of Optimal Diagonal Loading for MPDR-based Spatial Power Estimators in the Snapshot Deficient Regime Pajovic, M.; Preisig, J.C.; Baggeroer, A.B. TR March 218 Abstract The Minimum Power Distortionless Response (MPDR) beamformer minimizes the output power while passing the look direction signal with unity gain. To alleviate the performance degradation caused by estimating the spatial correlation matrix with a relatively small number of snapshots of the received signal compared to the number of sensors, a regularization implemented via diagonal loading of the estimated correlation matrix is used. This paper presents a study for the optimal diagonal loading that minimizes the estimation mean square error of two diagonally loaded MPDR beamformer-based spatial power estimators in the snapshot deficient regime. First, the asymptotic behavior of the power estimators for fixed diagonal loading is analyzed and the approximate characterization of their expectations is derived. Second, it is conjectured that because of the snapshot deficient sample support the squared bias is the factor that primarily controls the optimal diagonal loading. Finally, the respective performances of the two power estimators are compared using MSE as the metric and it is shown that one outperforms the other. The analytical results are validated using simulation data IEEE Journal of Oceanic Engineering This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research Laboratories, Inc.; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Research Laboratories, Inc. All rights reserved. Copyright c Mitsubishi Electric Research Laboratories, Inc., Broadway, Cambridge, Massachusetts 2139

2

3 1 Analysis of Optimal Diagonal Loading for MPDR-based Spatial Power Estimators in the Snapshot Deficient Regime Milutin Pajovic and James C. Preisig and Arthur B. Baggeroer Abstract The Minimum Power Distortionless Response (MPDR) beamformer minimizes the output power while passing the look direction signal with unity gain. To alleviate the performance degradation caused by estimating the spatial correlation matrix with a relatively small number of snapshots of the received signal compared to the number of sensors, a regularization implemented via diagonal loading of the estimated correlation matrix is used. This paper presents a study for the optimal diagonal loading that minimizes the estimation mean square error of two diagonally loaded MPDR beamformer-based spatial power estimators in the snapshot deficient regime. First, the asymptotic behavior of the power estimators for fixed diagonal loading is analyzed and the approximate characterization of their expectations is derived. Second, it is conjectured that because of the snapshot deficient sample support the squared bias is the factor that primarily controls the optimal diagonal loading. Finally, the respective performances of the two power estimators are compared using MSE as the metric and it is shown that one outperforms the other. The analytical results are validated using simulation data. Index Terms Capon beamformer, Minimum Power Distortionless Response (MPDR) beamformer, diagonal loading, regularization, spacial spectrum estimation, random matrix theory M. Pajovic was with the Massachusetts Institute of Technology and Woods Hole Oceanographic Institution Joint Program, Cambridge/Woods Hole, MA 2139 USA. He is currently with the Mitsubishi Electric Research Laboratories, Cambridge, MA 2139 USA ( pajovic@merl.com). J. C. Preisig is with JPAnalytics LLC, Falmouth, MA 254 USA and also with the Department of Applied Ocean Physics and Engineering, Woods Hole Oceanographic Institution, Woods Hole, MA 2543 USA ( jpreisig@jpanalytics.com). A. B. Baggeroer is with the Departments of Mechanical Engineering and Electrical Engineering & Computer Science at the Massachusetts Institute of Technology, Cambridge, MA 2139 USA ( abb@boreas.mit.edu). This work was supported in part by the U.S. Office of Naval Research (ONR) under Grants N , N , N , N , N and N , Contract N14-14C-23, and the funds from the Academic Program Office of the Woods Hole Oceanographic Institution.

4 2 I. INTRODUCTION The power of a signal propagating in an environment can be estimated based upon the signal received at an array of sensors. The spatial spectrum is an estimate of the power as a function of the direction of propagation of the signal. A common approach to spectral estimation is to implement a linear beamformer (spatial filter) using appropriate filter weights, point the beamformer in the directions of interest, and calculate the power in the signal at the output of the beamformer. Capon s Minimum Variance Distortionless Response (MVDR) beamformer [1] is an adaptive beamformer whose array weights depend on the spatial correlation matrix of the noise and interference, as received at the array of sensors. On the other hand, the array weights of the Minimum Power Distortionless Response (MPDR) beamformer depend on the spatial correlation matrix of the signal of interest, noise and interference [2, Section 6.2.4]. In most applications, the spatial correlation matrix is unknown and must be estimated from the received data to form a sample correlation matrix (SCM). Due to the time-varying nature of the environment and the fact that an array can contain large number of sensors, the number of snapshots that can be collected over the approximate stationarity interval of the environment might be insufficient to accurately estimate the correlation matrix. Diagonal loading, which is one form of Tikhonov regularization [3, p. 45], is extensively used to address the problem of insufficient snapshot support. This approach consists of adding a small regularization matrix, usually a scaled identity matrix, to the estimated spatial correlation matrix with the goal of reducing the L2 norm of the resulting array weights and thus, the sensitivity of the beamformer to the model mismatch. Gilbert and Morgan [4] rigorously show that the diagonal loading approach improves the beamformer s robustness when the array elements are displaced with zero-mean, independent and identically distributed random perturbations. In addition, the practice and numerous references, such as Cox et al. [5], have indicated that the diagonal loading approach also reduces the sensitivity to the model mismatch caused by the insufficient sample support. Moreover, the diagonally loaded MPDR beamformer (e.g., [6], [7]) is the solution to a robust beamformer optimization problem where the uncertainty in the effective steering vector is caused by modeling mismatch or low snapshot support. A problem that arises is how to choose an optimal regularization such that a certain performance metric, such as mean square error (MSE), is optimized. The statistical characterization of the MVDR/MPDR beamformer output has received con-

5 3 siderable attention in the literature. Reed et al. [8] derive the probability distribution of the output signal-to-interference-plus-noise ratio (SINR) corresponding to the diagonally unloaded MVDR in the finite snapshot support regime. Abramovich [9] has concluded that diagonal loading improves convergence speed of the MVDR beamformer. Gabriel [1] and Carlson [11] explain Abramovich s result, while Cheremisin [12] analytically validates it under certain assumptions. The asymptotic SINR at the MPDR output in the large snapshot support regime is analyzed in Fertig s work [13] and, using matrix perturbation techniques in Ganz et al. [14] and Dilsavor et al. [15]. Abramovich [16] derives the approximate expression for the probability distribution of the SINR loss due to diagonal loading and steering vector mismatch. Capon and Goodman [17] derive the probability distribution of the power estimate made at the output of the MPDR beamformer by assuming a complex Gaussian received signal and zero diagonal loading in the computation of array weights. Raghunath and Reddy [18] obtain the expressions for the mean values of the power gain in a direction of interest, the output power and the norm of the weight error vector using the first-order perturbation analysis. Nadakuditi and Edelman [19] study the probability density function of the diagonally loaded MPDR output when the received signal is complex Gaussian and at most two signals are present in the sensed field. The corresponding bias under the same arrival model has initially been reported by the same authors in [2]. Mestre and Lagunas [21] characterize the expected value of the SINR at the MPDR beamformer output. Pajovic et al. [22] report a closed form first order moment characterization of the diagonally loaded MPDR beamformer-based spatial power estimator in the snapshot deficient regime. Rubin at al. [23] report a study of the probability density function of the SNR of the diagonally loaded MPDR beamformer output using a Gaussian method. The literature also addresses a problem of determining optimal diagonal loading and in that sense, different ad-hoc methods have been proposed. Carlson [11] instructs that the diagonal loading should be 1 db above the noise level. Cheremisin [12] proposes using diagonal loading larger than the noise power but lower than the smallest interference eigenvalue. Ma and Goh [24] use diagonal loading equal to the standard deviation of the diagonal elements in the SCM. Kim et al. [25] express an optimal diagonal loading in terms of the eigenvalues of the sample correlation matrix, with the goal to maximize the output SINR. Liu and Ding [26] propose an optimization of diagonal loading suitable for communications when the transmitted signal comes from a constant modulus constellation. Gu and Wolfe [27] propose a variable loading of the SCM eigenvalues with the goal to alleviate problems caused by low snapshot support and steering

6 4 vector mismatch. Mestre and Lagunas [21] report a comprehensive study of diagonal loading problem and a method for determining optimal diagonal loading with the goal to maximize the output SINR in the snapshot deficient regime. While SINR has traditionally been used in the analysis, some authors consider the MSE between the estimated and true parameter as the performance metric and also as a guide to beamformer design. As such, an accurate prediction of the parameter estimation MSE performance above the estimation threshold SNR of the MVDR beamformer in the snapshot abundant regime is obtained using the Taylor s theorem and complex gradient methods in Stoica et al. [28], Vaidyanathan and Buckley [29], Hawkes and Nehorai [3]. The MSE corresponding to the angle of arrival estimate obtained from a diagonally loaded Capon beamformer in the deficient sample support regime is characterized in Richmond et al. [31]. Richmond [32] derives the MSE threshold SNR prediction pertaining to angle of arrival estimation using Capon beamformer and accounting for finite sample effects. Pajovic et al. [33] outline a theoretical study of the MSE of a diagonally loaded MPDR-based power estimator in a low sample support regime, wherein a complex Gaussian input process is assumed and Hachem et al. s Gaussian method [34] is used. Serra and Rubio [35] find optimal scaling of the MVDR filter weights which minimizes the MSE corresponding to the estimate of the amplitude of the signal waveform in the low snapshot support regime. The beamformer weights are found in Eldar et al. [36] by optimizing an objective function based on the MSE between the estimated and true amplitude of the signal waveform. This paper studies a diagonal loading problem for two commonly used spatial power spectrum estimators based on the MPDR beamformer [37]. The MSE between the estimated and true spatial power spectra is adopted as a performance metric and one of the main goals is to explore how the optimal diagonal loading changes with steering direction in a snapshot deficient regime. In doing so, we study how the power estimators behave with respect to diagonal loading, number of snapshots and number of sensors. More specifically, the paper explores best-case spectral estimation when the diagonal loading of the MPDR processor minimizes the MSE in the snapshot deficient regime. In that sense, the paper does not propose an explicit method for obtaining the optimal diagonal loading in practice, but rather yields theoretical results and aims for theoretical bounds. The contributions of the paper are as follows. First, the behavior of the power estimators and their expectations in the limit when the number of snapshots and sensors grow large at the same

7 5 rate is analyzed using random matrix theory. It is shown that both power estimators for a fixed diagonal loading and steering direction almost surely converge to non-random quantities in the limit when the numbers of snapshots and sensors grow large at the same rate. The expected value of the power estimators in a realistic scenario of finite number of snapshots and sensors is then approximated using the derived asymptotic characterization. Second, the interplay between the bias and variance in determining the optimal diagonal loading which minimizes the estimation MSE is studied. It is conjectured that the variance of the modeled power estimators does not significantly impact the value of the optimal diagonal loading. Third and finally, the MSE performances of the two power estimators are compared and it is shown that one of them performs better, i.e., has a lower MSE. The remainder of the paper is organized as follows. Section II presents the background on the MPDR-based power estimate, a suitable representation of the power estimators studied, an arrival model for the snapshots and the definition of the true spatial power. Section III studies the asymptotic behavior of power estimators. Section IV analyzes how the squared bias and variance impact the value of optimal diagonal loading. Section V compares the relative performance of the two power estimators. Section VI concludes the paper. Throughout the paper, boldface uppercase letters denote matrices and boldface lowercase letters denote vectors. The superscript () H denotes complex conjugate transpose, i.e., Hermitian. II. ROBUST POWER SPECTRAL ESTIMATION A. Background An array of m sensors receives at discrete time k a signal whose Fourier coefficients in a frequency bin of interest and across the array constitute an observation vector z(k) C m 1, known as a snapshot. Without loss of generality, the snapshots are assumed to have zero mean. 1 A snapshot contains signatures of the signal of interest, interferers and noise. The correlation matrix between snapshots i and j is E [ z(i)z H (j) ] = S δ D (i j), (1) where δ D (x) is the discrete impulse function. 1 Otherwise, non-zero mean is subtracted from the snapshots and is taken care of after obtaining estimates corresponding to the zero mean part.

8 6 The MPDR beamformer, parameterized by the weight vector w MPDR, passes a signal arriving from a look direction characterized by the replica vector v s undistorted and minimizes the output power, i.e., [2, p. 451] w MPDR (v s ) = arg min w w H Sw s.t. w H v s = 1. (2) In a simple model of a plane wave arriving from elevation angle θ on a linear, uniformly spaced array with inter-element spacing d, the signal replica vector v s is given by [ ] H v s (θ) = 1 e j 2π u d cos(θ)... e j 2π u (m 1)d cos(θ), (3) where u is the signal wavelength. Here, θ = indicates a signal propagating from the array s endfire (i.e., along the axis of the line array) and θ = π/2 indicates a signal propagating to the broadside of the array (i.e., perpendicular to the axis of the line array). While the model in (3) is for a plane wave received at a linear uniform array, the results developed in this paper are equally applicable to a general signal arising in the contexts of non-linear and non-uniform arrays as well as in matched field processing problems. The solution to (2) is given by w MPDR (v s ) = S 1 v s v H s S 1 v s. (4) The number of sources and their respective received powers can be estimated by steering the MPDR beamformer across all possible look directions. When the beamformer is steered in the direction corresponding to the replica vector v s, its output is given by y(k) = wmpdr(v H s )z(k), (5) and the power of the signal arriving from such a direction is estimated as ˆP (v s ) = E [y (k)y(k)] = w H MPDR(v s )Sw MPDR (v s ). (6) The correlation matrix S is usually unknown and is estimated from the observed data. The sample correlation matrix (SCM) Ŝ is evaluated from n observations via Ŝ = 1 n n z(i)z H (i). (7) i=1

9 7 1 1 Eigenvalues of the ensemble correlation matrix Eigenvalues of the sample correlation matrix 1 eigenvalue eigenvalue index Fig. 1: Eigenvalues of the ensemble correlation matrix and one realization of the corresponding SCM. The underlying process is uncorrelated noise of power 1, received at 3 half-wavelength separated sensors. The number of snapshots is 5. Note that the SCM (7) is the maximum likelihood (ML) estimate of the correlation matrix S when the noise process is zero-mean Gaussian, which is the main motivation behind the widespread use of this estimator in most applications. The number of snapshots that can be collected in the interval over which the environment can be considered stationary, n, might be insufficient to accurately estimate S, which leads to one type of model mismatch [2, p ]. To illustrate and motivate this study, assume that an array of 3 sensors receives 5 snapshots of uncorrelated noise of unit power. The plots of the eigenvalues corresponding to the ensemble correlation matrix, S, and one realization of the SCM Ŝ are shown in Fig. 1. The eigenvalues of the true correlation matrix and its estimate differ significantly even though the number of snapshots is 1.67 times larger than the number of sensors. Intuitively, the quality of the SCM estimate of the ensemble correlation matrix deteriorates further as the number of snapshots decreases. To combat the sensitivity to mismatch and/or to improve the condition number of Ŝ, a diagonally loaded SCM Ŝδ is introduced Ŝ δ = Ŝ + δi, (8) where δ is a diagonal loading parameter. The power of the signal arriving from direction v s is estimated from (6) where the MPDR

10 8 weights are evaluated using (4) with a diagonally loaded SCM Ŝδ, and the SCM instead of S in (6). That is [37], Ŝ is used ˆP a (δ) = vh s Ŝ 1 δ [ v H s Ŝ 1 ŜŜ 1 δ v s δ v s ] 2. (9) If diagonally loaded SCM Ŝδ is substituted in (6) instead of S, an alternative and more compact form is obtained [37] ˆP b (δ) = 1. (1) vs HŜ 1 δ v s The power estimators as defined in (9) and (1) depend, in general, on diagonal loading δ and steering direction v s. To keep notation uncluttered, we make the dependence on v s implicit. The performance of the power estimators (9) and (1) is measured via estimation mean square error (MSE). Given a steering direction v s, the MSE of the power estimator ˆP is viewed as a function of the loading δ and represented via the bias-variance decomposition as 2 [ ] ( ) MSE(δ) = E 2 ˆP (δ) P + var ˆP (δ), (11) where the first term is a squared bias, which we denote by bias 2 (δ), while P is the true power of the signal arriving from the considered direction v s. In the absence of a subscript, ˆP refers to either of the two estimators. The optimal diagonal loading δ opt is defined as δ opt = arg min δ MSE(δ). (12) In addition, a diagonal loading which minimizes the squared bias is denoted by δ opt and formally defined as δ opt = arg min δ bias 2 (δ). (13) This paper develops an understanding of how the MSE performance in a snapshot deficient regime depends on diagonal loading and steering direction. In achieving this, the impact of diagonal loading δ, number of sensors m and number of snapshots n on the squared bias and variance of the power estimators is studied. A standard theory on regularization shows that a 2 Since the true and estimated powers are real-valued, the expectation in the bias-related term is taken of the bias rather than the absolute value of the bias.

11 9 diagonal loading optimizes the MSE in a way that the variance of the estimator is decreased at the expense of an increased bias [38, Section 3.2, p. 147]. However, due to specifics associated with a small sample support regime, it is shown here that i) the bias of either of the power estimators (9) or (1) is minimized for non-zero diagonal loading δ and, more importantly, ii) the squared bias dominates the minimization (12) in such a way that the difference between the optimal MSE and the MSE evaluated at the minimizer of the squared bias is negligible. B. Alternative Expressions for Power Estimators The power estimators (9) and (1) are represented via quadratic forms Q k, defined as Q k (δ) = v H s Ŝ k δ v s, k = 1, 2 (14) and viewed as functions of diagonal loading δ for a fixed steering direction v s. Therefore, the power estimator ˆP b is expressed as ˆP b (δ) = 1 Q 1 (δ). (15) On the other hand, solving (8) for Ŝ and substituting it into (9) yields ˆP a (δ) = vs HŜ 1 δ ) Ŝ 1 δ (Ŝδ δi v s [ ] 2 = vs HŜ 1 δ v s Q 1 (δ) δq 2 (δ) Q 2 1(δ) (16) The analysis in the paper uses these two alternative expressions for the power estimators. C. Snapshot Model If the snapshots z(i) are collected in a data matrix Z C m n, [ Z = z(1) z(2)... z(n) ], (17) the SCM is compactly given by Ŝ = 1 n ZZH. It is assumed that the data matrix Z can be modeled via the true correlation matrix S as Z = S 1 2 X, (18) where matrix X contains complex i.i.d. entries of zero mean and unit variance. S 1 2 is the Hermitian positive definite square root of S. Note that (18) models the data matrix (17) of

12 1 the colored process whose ensemble correlation matrix is S. Therefore, the SCM Ŝ is using (18) modeled as Ŝ = 1 n S 1 2 XX H S 1 2. (19) The above model is used to derive the asymptotic behavior of the power estimators. D. Definition of True Power The true power P (v s ) corresponding to a specific arrival model is defined in a way usually used for the power spectrum estimation of time series data. Namely, if the received signal is composed of incoherent multiple signals each having power P i and impinging on the array in the direction described by a replica vector v i, the true power in the steering direction v s is defined as [39, Section 9, p ] σv 2(vs) P (v s ) =, if v m s v i P i + σ2 v (vs), if v m s = v i, where σ 2 v(v s ) is the level of the noise power spectral density in the direction described by v s. Since the noise is assumed to be white, σ 2 v(v s ) = σ 2 v. To avoid confusion, we emphasize here that the MPDR beamformer is normalized such that (9) and (1) are power spectrum estimators (and not power spectrum density estimators). Consequently, (2) is the true power spectrum for the considered arrival model. (2) III. BEHAVIOR OF POWER ESTIMATORS IN THE SNAPSHOT DEFICIENT REGIME In this section, the asymptotic behavior of the power estimators (9) and (1) is characterized for the case where the number of sensors, m, and number of snapshots, n, grow large at the same rate for a fixed diagonal loading δ. The asymptotic results are then used to approximate the statistical expectations of the power estimators for finite and realistic m and n. A. Asymptotic Analysis of Power Estimators The asymptotic analysis of power estimators ˆP a and ˆP b is performed in the limit when both the number of snapshots n and the number of sensors m grow large at the same rate so that m/n c, c. The assumption that m and n grow at the same rate captures the fact that we often do not have enough snapshots to estimate accurately the input correlation. On the other

13 11 hand, this assumption is a mathematical necessity which enables analytical derivations. It is practically justified by a rapid rate of convergence, observed in numerical simulations. Finally, exact non-asymptotic analyses of adaptive processors have shown the importance of the ratio m/n [8], [17] so that asymptotic analysis that preserves this ratio is more likely to provide insights about non-asymptotic cases. The subsequent asymptotic analysis relies on the following assumptions. 1) The eigenvalues of the ensemble correlation matrix S, denoted by λ 1, λ 2,..., λ m, are uniformly upper bounded for all m. More precisely, λ 1 λ 2... λ m = O(1/m) <. In addition, the ensemble correlation matrix S is positive semi-definite, i.e., λ 1, λ 2,..., λ m. 2) The eigenvalues of the SCM Ŝ, denoted by ˆλ 1, ˆλ 2,..., ˆλ m, are uniformly upper bounded for all m, i.e., ˆλ 1 ˆλ 2... ˆλ m ˆD m <. 3) The norm of the signal replica vector, v s, normalized with m is uniformly upper bounded for all m, that is, v s = O( m). Note that for any array in a plane wave environment where the magnitude of each element of the replica vector equals 1, v s = m. All of these assumptions are realistic in practice, i.e., for finite m. Note, however, that assumption 1 effectively requires the power of the signal impinging upon the array be scaled as m so that the largest eigenvalue of the ensemble correlation matrix is upper bounded. We emphasize that this is a technical condition needed to obtain the asymptotic results. The following lemma, proved in the Appendix, characterizes the asymptotic behavior of power estimators ˆP a and ˆP b. Lemma III.1. Under the snapshot model (18) and assumptions 1 and 3, in the limit when m, n at the same rate, i.e., m/n c, where c (, ) when δ >, and c (, 1) when δ =, ˆP a (δ) Q 1 δ Q 2 Q 2 1 a.s. (21) and ˆP b (δ) 1 Q1 a.s., (22) where Q 1 and Q 2 are given by Q 1 = m k=1 β k λ k ( 1 c + cδ M1 ) + δ (23)

14 12 and Q 2 = m k=1 β 2 k λ k c ( M1 δ M ) [ ( ) ] 2, (24) λk 1 c + cδ M1 + δ where β k = v H s q k 2, while λ 1,..., λ m and q 1,..., q m are the eigenvalues and eigenvectors of the ensemble correlation matrix S, and quantities M 1 and M 2 are given as the fixed point solutions to 3 M1 = 1 m m k=1 1 λ k ( 1 c + cδ M1 ) + δ. (25) and M 2 = 1 m m k=1 c ( M1 2δ M 2 ) λk + 1 [ λk (1 c + cδ M 1 ) + δ ] 2. (26) The existence and uniqueness of the solutions to fixed point equations (25) and (26) follows from random matrix theorem used to prove Lemma III.1 and stated in the Appendix. The fixed point equations (25) and (26) can be solved via the classical iterative method which converges if appropriately initialized. Given that the considered SCM is diagonally loaded with δ, we initialize the iterative methods with, respectively, 1/δ and 1/δ 2. B. Approximate Expectation of Power Estimators As has been shown, the difference between either power estimator and the corresponding deterministic equivalent almost surely converges to zero in the limit when m, n at the same rate such that m/n c (, ). According to the Dominated Convergence Theorem [4, Theorem 5.4, p. 77], if power estimators ˆP a and ˆP b are uniformly bounded (i.e., bounded for each realization of the SCM), the difference between the expectation of either power estimator and the corresponding deterministic quantity converges to zero as m, n. Indeed, under assumptions 1, 2 and 3, ˆPa is uniformly upper bounded for all m and n = n(m). Similarly, power estimator ˆP b is under same assumptions uniformly bounded for all m and n = n(m), provided that diagonal loading is finite 4. Therefore, [ ] E ˆPa (δ) [ ] E ˆPb (δ) Q 1 δ Q 2 Q 2 1 (27) 1 Q1 (28) 3 These quantities are called limiting moments corresponding to the diagonally loaded SCM. 4 These statements are clear from the results proved in Lemma IV.1.

15 13 Although the established convergence results hold when m, n at the same rate such that m/n c, due to rapid convergence the asymptotic expressions accurately approximate the expectations of power estimators for realistic and moderate-to-relatively small 5 n and m. Therefore, for finite n and m, [ ] E ˆPa (δ) [ ] E ˆPb (δ) Q 1 δ Q 2 Q 2 1 (29) 1 Q1, (3) where Q 1 and Q 2 are evaluated using (23) and (24) with given m, n and c = m/n. We provide a numerical validation of the above approximations. C. Diagonally Unloaded Power Estimator The case of unloaded estimators, i.e., when δ = is considered here. First, assuming that the number of snapshots n is greater than the number of sensors m, it is noted that the power estimators ˆP a and ˆP b are equal for diagonal loading δ =. Substituting δ = in (22) yields that in the limit m, n such that n > m and m/n c (, 1) ˆP () Using (29) and (3), it is concluded that 1 c v H s S 1 v s a.s. (31) [ ] lim E ˆP (δ) δ This result is exploited in the later parts of the paper. 1 c v H s S 1 v s (32) We point out that Capon and Goodman [17] showed that, for a Gaussian noise model, ˆP () is proportional to χ 2 statistics with mean given by [ ] E ˆP () = 1 c + 1/n (33) vs H S 1 v s Since we are here in the n > m regime, approximation (32) is accurate for any reasonable number of sensors m used in practice. More importantly, the approximation is not restricted to only Gaussian noise, but holds for any arrival model satisfying the conditions of the Lemma III.1. 5 The rapid convergence has been observed in a variety of results in the context of random matrix theory. In addition, some theoretical results showing the convergence rates have appeared relatively recently (e.g., [41] and the references therein).

16 14 TABLE I: Summary of simulation scenarios First source Second source Sensors m Snapshots n Scenario 1 9 o, 1 db 92 o, 1 db 3 5 Scenario 2 9 o, 1 db 94 o, 5 db 4 25 D. Numerical Validation of Derived Expressions Approximations (29) and (3) are validated using Monte-Carlo simulations. A uniformlyspaced linear array with half-wavelength separation between adjacent elements is considered. In addition, a spatially uncorrelated, zero-mean noise with a variance of one corrupts the signal snapshots. Table I summarizes the two simulation scenarios considered in the remainder of the paper. We point out that the derived characterizations hold for more general arrival models, ambient noise and array shapes. 1) Validation of (29) and (3) in Scenario 1: In this scenario, two signals are arriving at elevation angles 6 of 9 o and 92 o with respect to the broadside of the array and each has power 1 (i.e., the SNR is 1 db). The per-sensor noise level is 1, i.e., db. The array contains 3 sensors and 5 snapshots are used to estimate the SCM. Note that the ratio between the array aperture and wavelength is 15. Also, note that the ratio between the number of sensors and number of snapshots c =.6 and the normalized trace of the ensemble correlation matrix of the input process is 21. Finally, note that in this scenario both plane waves are within the mainlobe corresponding to the conventional beamformer. The comparison between the mean of the estimate of the input power ˆP a and the corresponding theoretical prediction (29) for the diagonal loading value of.3 is shown in the top part of Fig. 2. A similar agreement between the mean of power estimate ˆP b evaluated via simulations and theoretical prediction (3) for the same value of diagonal loading.3 is obtained in the bottom part of Fig. 2. The comparisons between the simulated means of power estimators (9) and (1) and theoretical predictions (29) and (3) for a steering angle of 87 o are shown in Fig. 3. A good agreement between the plots validate the accuracy of the asymptotic results in predicting the expected values of the power estimators for finite values of m and n. 6 The elevation angle is defined relative to the axis of the array with 9 o being broadside and o and 18 o being signals coming from endfire to each end of the array.

17 15 Power (db) simulations theory steering angle Power (db) simulations theory steering angle Fig. 2: Expectation of the power estimator ˆP a (top) and ˆP b (bottom) versus steering angle for δ =.3 in the scenario 1 (sources at 9 o and 92 o, both of powers 1 db, 3 sensors and 5 snapshots). 2) Validation of (29) and (3) in Scenario 2: In this scenario, two signals are arriving at elevation angles of 9 o and 94 o with respect to the broadside of the array. Their SNR s are respectively 1 db and 5 db. The per-sensor noise level is 1, i.e., db. The array has 4 sensors and the number of snapshots available to estimate the SCM is 25. Note that the ratio between the array aperture and wavelength is 2. Also note that the ratio between the number of sensors and number of snapshots c = 1.6 and the normalized trace of the ensemble correlation matrix is Finally, note that the arrival angle of the interference is in the range of the first sidelobe corresponding to the conventional beamformer. The comparisons between the expected values of ˆPa and ˆP b, obtained from Monte-Carlo simulations, and the corresponding theoretical predictions (29) and (3) for diagonal loading of 2 and varying steering direction are shown in Fig. 4. The comparisons for steering direction of 87 o and varying diagonal loading are shown in Fig. 3. The presented plots validate the accuracy of the theoretical predictions.

18 16 Power (db) Scenario 1: simulations Scenario 1: theory Scenario 2: simulations Scenario 2: theory diagonal loading Power (db) Scenario 1: simulations Scenario 1: theory Scenario 2: simulations Scenario 2: theory diagonal loading Fig. 3: Expectation of the power estimator ˆP a (top) and ˆP b (bottom) versus diagonal loading level for steering angle of 87 o in the scenario 1 (sources at 9 o and 92 o, both of powers 1 db, 3 sensors and 5 snapshots) and scenario 2 (sources at 9 o and 94 o, of respective powers 1 and 5 db, 4 sensors and 25 snapshots). IV. INSIGHTS ABOUT OPTIMAL DIAGONAL LOADING In this section, the functional dependence of power estimators ˆP a and ˆP b on diagonal loading is studied. Then, it is conjectured from simulation results that the variance of either power estimator has insignificant impact on the value of diagonal loading which optimizes its mean square error (MSE) performance. In addition, the dependence of optimal diagonal loading on steering direction is numerically explored for both power estimators. A. Functional Dependence on Diagonal Loading The functional dependence of power estimators on diagonal loading is studied using analysis techniques for single variable functions. The obtained results are summarized in the following Lemma and proved in the Appendix.

19 17 5 simulations theory Power (db) steering angle 6 4 simulations theory Power (db) steering angle Fig. 4: Expectation of the power estimator ˆP a (top) and ˆP b (bottom) versus steering angle for δ = 2 in the scenario 2 (sources at 9 o and 94 o, of respective powers 1 and 5 db, 4 sensors and 25 snapshots). Lemma IV.1. For any (Hermitian non-negative definite) SCM holds Ŝ and for δ, the following 1) ˆPb (δ) ˆP a (δ), with equality if and only if δ =. 2) ˆPa (δ) is monotonically increasing for < δ <, unless all the eigenvalues of Ŝ are equal. Its slope is zero at δ = + (i.e., when δ approaches from the positive side) and when δ. 3) ˆPb (δ) is monotonically increasing for all δ. [ ] Note that the above results carry over to E ˆPa (δ) [ ] and E ˆPb (δ) whenever the derivative and expectation can interchange the order. In addition, we note that since both estimators are monotonically non-decreasing and ˆP b ˆP a, δ (b) (a) opt δ opt. (34) As a remark, the expected value of the power estimator ˆP a when δ is obtained by

20 18 recalling that the MPDR beamformer becomes a matched filter (MF) and thus, [ ] lim E ˆPa (δ) = 1 δ m 2 vh s Sv s. (35) B. Conjecture on Optimal Diagonal Loading The dependence of squared bias, variance and MSE on diagonal loading is studied via simulations by considering arrival scenarios detailed in Table I. In each scenario, the dependence of the squared bias, variance and MSE on diagonal loading is simulated for a given set of steering directions. This dependence reveals a somewhat surprising result that the variance has negligible impact on the value of optimal diagonal loading. To further visualize this observation, the optimal MSE, evaluated at δ opt, and the MSE evaluated at loading δ opt that minimizes the squared bias are compared. Steering angles 87 o and 89 o are chosen in both scenarios to illustrate qualitatively different effects arising when steering close to and slightly away from source directions. A detailed analysis of the relative roles of bias and variance in the optimization problem is contained in [42, Section 3.7]. 1) Scenario 1: The simulated dependence of the squared bias, variance and MSE on diagonal loading is shown in Fig. 5 for a steering angle 87 o. The plots show that the variance has almost no influence on a diagonal loading which minimizes the MSE. As can be noted, the plots also show that the diagonal loading which optimizes the MSE corresponding to power estimator ˆP b is not greater than the loading which optimizes the MSE corresponding to ˆP a, i.e., δ (b) opt δ (a) opt. This is the direct result of (34) and the observation that the bias term controls the optimal diagonal loading. A similar set of plots corresponding to steering angle of 89 o are given in Fig. 6. As can be observed from Fig. 6, the optimal loading for both estimators is zero. This happens because the true power in such direction (and in general in directions sufficiently close to the directions of arrival) is below the expected smallest value of either power estimator, achieved for zero loading. Since the power estimators are non-decreasing functions of loading δ, an unloaded power estimator minimizes the bias which, according to our conjecture, also minimizes the MSE. Intuitively, as the steering direction gets closer to but is not pointed exactly at the source, the optimal diagonal loading is reduced as the estimator needs to maintain more adaptability to null the source.

21 log 1 () 4 6 squared bias variance 8 MSE true power diagonal loading 2 1 log 1 () 2 4 squared bias 6 variance 8 MSE true power diagonal loading Fig. 5: Squared bias, variance and MSE corresponding to estimators ˆP a (top) and ˆP b (bottom) for steering angle of 87 o in the scenario 1 (sources at 9 o and 92 o, both of powers 1 db, 3 sensors and 5 snapshots). The per-sensor noise level is 1 ( db). 1 log 1 () squared bias variance MSE true power diagonal loading 1 log 1 () squared bias variance MSE true power diagonal loading Fig. 6: Squared bias, variance and MSE corresponding to estimators ˆP a (top) and ˆP b (bottom) for steering angle of 89 o in the scenario 1 (sources at 9 o and 92 o, both of powers 1 db, 3 sensors and 5 snapshots). The per-sensor noise level is 1 ( db).

22 2 MSE (db) Optimal MSE MSE at δ minimizing squared bias steering angle 4 42 Optimal MSE MSE at δ minimizing squared bias MSE (db) steering angle Fig. 7: Optimal MSE and the MSE evaluated at loading minimizing the squared bias for power estimator ˆP a in the scenario 1 (sources at 9 o and 92 o, both of powers 1 db, 3 sensors and 5 snapshots) for the range of steering angles around the source directions (top) and away from the source directions (bottom). The optimal MSE and the MSE evaluated at a loading which minimizes the squared bias are compared in Fig. 7 for power estimator ˆP a. The plots highlight two different ranges of steering angles. As shown, the performance loss is larger when steering away from the directions of arrival, but remains within 1 db of the optimal MSE. A smaller performance loss is observed in Fig. 8, which corresponds to the estimator ˆP b. The simulated optimal diagonal loadings versus steering angle are plotted in Fig. 9. As can be noted, the diagonal loading optimizing the squared bias and estimation MSE are almost the same. In addition, there is a range of steering directions for which a nearly diagonally unloaded estimator achieves the lowest MSE. The smallest diagonal loading used in the simulation tests is 1 2 and is optimal when steering close to main beams. Finally, the estimator ˆP b is optimized at smaller values of diagonal loading which do not vary significantly across the steering directions. 2) Scenario 2: The simulated dependences of the squared bias, variance and MSE on diagonal loading for estimators ˆP a and ˆP b, and steering angle of 87 o are shown in Fig. 1. The plots confirm that the variance has almost no influence on a diagonal loading which minimizes the

23 21 2 MSE (db) 2 Optimal MSE MSE at δ minimizing squared bias steering angle MSE (db) Optimal MSE MSE at δ minimizing squared bias steering angle Fig. 8: Optimal MSE and the MSE evaluated at loading minimizing the squared bias for power estimator ˆP b in the scenario 1 (sources at 9 o and 92 o, both of powers 1 db, 3 sensors and 5 snapshots) for the range of steering angles around the source directions (top) and away from the source directions (bottom). MSE. Note that as in Scenario 1, δ (b) opt δ (a) opt. The optimal MSE and the MSE evaluated at the minimizer δ opt of the squared bias are compared for estimators ˆP a and ˆP b in Figures 11 and 12. As can be observed from the plots, the largest performance loss appears with estimator ˆP a when steering away from the main beams. However, this loss is still within 1 db. The performance loss corresponding to estimator ˆP b is negligible. As a side note, the observation from Figures 5, 6 and 1 that the variance of ˆP b monotonically increases might be counterintuitive because ˆP b behaves as δ/v H s v s when δ gets large, and one may expect the variance to decrease. As a matter of fact, the simulations over broader δ range show the variance of ˆP b is monotonically non-decreasing and converges to a non-zero value. We emphasize that care must be taken when drawing conclusions about variance. As an example, in a single sensor case, ˆPb = 1/n n k=1 z k 2 + δ, and hence the variance is non-zero in any reasonable arrival model even though δ becomes large. The variance convergence properties are out of the scope of this paper; some initial results can be found in [42, Sections and 3.6].

24 22 diagonal loading level MSE minimizer Squared bias minimizer steering angle diagonal loading level MSE minimizer Squared bias minimizer steering angle Fig. 9: Diagonal loadings minimizing the MSE and squared bias corresponding to the power estimator ˆP a (top) and ˆP b (bottom) in the scenario 1 (sources at 9 o and 92 o, both of powers 1 db, 3 sensors and 5 snapshots). The per-sensor noise level is 1. The plots of the simulated optimal diagonal loading versus steering angle for two power estimators are shown in Figure 13. As expected, the optimal diagonal loading is always positive because in this scenario, c > 1 (i.e., the SCM is rank deficient). However, even though the SCM is rank deficient in this case, the optimal diagonal loading when steering close to source directions is small. This is in accordance with our earlier observation that when steering close to main beams, the optimal estimator tends to perform adaptation as much as possible, fully relying on the available data. Further, the diagonal loadings optimizing the squared bias and estimation MSE are almost the same. Finally, the estimator ˆP b is optimized at smaller values of diagonal loading which do not vary significantly across the steering directions. V. COMPARISON BETWEEN ˆP a AND ˆP b The optimized mean square errors of the power estimators ˆP a and ˆP b are compared in this section. The theoretical result concerning the estimation performance is stated and proved in the first part. The result is then validated using simulations in the second part.

25 log 1 () 4 6 squared bias variance 8 MSE true power diagonal loading 2 1 log 1 () squared bias variance 8 MSE true power diagonal loading Fig. 1: Squared bias, variance and MSE corresponding to estimators ˆP a (top) and ˆP b (bottom) for steering angle of 87 o in the scenario 2 (sources at 9 o and 94 o, of respective powers 1 and 5 db, 4 sensors and 25 snapshots). The per-sensor noise level is 1 ( db). Relying on the conjecture that the difference between the MSE s evaluated at the loading δ opt and at the loading δ opt is negligible for both estimators, the MSE s of the two estimators evaluated at respectively δ (a) opt and δ (b) opt are compared. The following result establishes that estimator ˆP a has larger variance than the estimator ˆP b when both variances are measured at loadings which minimize the corresponding squared biases. Theorem V.1. Denoting by δ (a) opt and δ (b) opt the loadings which minimize the squared biases of respectively ˆP a and ˆP b for some steering direction described by v s and true power P [ ] lim δ E ˆPa (δ), the corresponding variances are related as ( ) ( ) var ˆPa ( δ (a) opt ) var ˆPb ( δ opt) (b). (36) Proof: To simplify the exposition, the notation Q k,a = Q k ( δ (a) opt ) and Q k,b = Q k ( δ (b) opt) is introduced. Due to assumption P lim δ E[ ˆP a (δ)], it follows that the biases of the power estimators

26 24 MSE (db) Optimal MSE MSE at δ minimizing squared bias steering angle Optimal MSE MSE at δ minimizing squared bias MSE (db) steering angle Fig. 11: Optimal MSE and the MSE evaluated at loading minimizing the squared bias for power estimator ˆP a in the scenario 2 (sources at 9 o and 94 o, of respective powers 1 and 5 db, 4 sensors and 25 snapshots) for the range of steering angles around the source directions (top) and away from the source directions (bottom). ˆP a and ˆP b are equal to zero at, respectively, In addition, [ 1 E Q 1,a δ (a) opt δ (a) opt Q 2,a Q 2 1,a δ (b) and opt such that ] [ ] 1 = E. (37) (a) (b) δ opt δ opt, as noted in (34). Therefore, since ˆP b (δ) = 1/Q 1 (δ) is a monotonically increasing function, it follows that Q 1,b 1 Q 1,a 1 Q 1,b. (38) Thus, for some positive constant K, it holds ( ) 1 1 K Q 1,a Q 1,a 1 ( ) 1 K Q 1,b Q 1,b Taking the expectation of both sides of (39) and rearranging the terms yields [ ] [ 1 1 E KE 1 ] [ ] 1 E Q 2 1,a Q 1,a Q 1,b Q 2 1,b (39) (4)

27 25 MSE (db) Optimal MSE MSE at δ minimizing squared bias steering angle MSE (db) Optimal MSE MSE at δ minimizing squared bias steering angle Fig. 12: Optimal MSE and the MSE evaluated at loading minimizing the squared bias for power estimator ˆP b in the scenario 2 (sources at 9 o and 94 o, of respective powers 1 and 5 db, 4 sensors and 25 snapshots) for the range of steering angles around the source directions (top) and away from the source directions (bottom). The second term on the left hand side of (4) is evaluated using (37) such that [ ] [ ] [ ] 1 δ(a) opt Q 2,a 1 E KE E Q 2 1,a Q 2 1,a Q 2 1,b (41) Since ( δ(a) opt Q 2,a /Q 2 1,a) 2 >, adding its expectation to the left hand side of (41) does not change the inequality. Hence, [ ] 1 E KE Q 2 1,a [ ] [ δ(a) (a) opt Q 2,a δ + E ( Q 2 1,a opt Q 2,a Q 2 1,a ) 2 ] E [ 1 Q 2 1,b ] (42) To proceed further, note that the quadratic form Q 1 can be written in terms of the eigendecomposition of the SCM Ŝ as Q 1 (δ) = m i=1 v H s q i 2 ˆλ i + δ, where ˆλ i s and q i s are the eigenvalues and eigenvectors of the SCM Ŝ. Due to assumption 2,

28 26 diagonal loading level MSE minimizer Squared bias minimizer steering angle diagonal loading level MSE minimizer Squared bias minimizer steering angle Fig. 13: Diagonal loadings minimizing the MSE and squared bias corresponding to the power estimator ˆP a (top) and ˆP b (bottom) in the scenario 2 (sources at 9 o and 94 o, of respective powers 1 and 5 db, 4 sensors and 25 snapshots). The per-sensor noise level is 1. ˆλ i ˆD m, and thus Q 1 (δ) m ˆD m + δ. (43) Setting K = 2( ˆD (a) m + δ opt )/m >, note that (43) implies K 2/Q 1,a, which is used to upper bound the left hand side of (42) such that [ ] [ (a) (a) 1 δ opt Q 2,a δ opt Q 2,a E 2 + ( ) 2 E Q 2 1,a Q 3 1,a Q 2 1,a 1 Q 2 1,a ] (a) (a) δ opt Q 2,a δ opt Q 2,a K + ( ) 2 Q 2 1,a Q 2 1,a (44) Using the upper bound (44) in (42) yields [ 1 E Q 2 1,a (a) (a) δ opt Q 2,a δ 2 + ( Q 3 1,a opt Q 2,a Q 2 1,a ) 2 ] E [ 1 Q 2 1,b ] (45) Recognizing that the left and right sides of (45) are respectively the quadratic of ˆP a ( δ (a) opt ) and ˆP b ( δ (b) opt) yields E[ ˆP 2 a ( δ (a) opt )] E[ ˆP 2 b ( δ (b) opt)]. (46)

29 27 On the other hand, taking the square of both sides of (37) yields E 2 [ ˆP a ( δ (a) opt )] = E 2 [ ˆP b ( δ (b) opt)] (47) Finally, (46) and (47) yield the inequality (36) between the variances of power estimators. [ ] If the true power P lim δ E ˆPa (δ), the squared biases corresponding to ˆP a and ˆP b at, respectively, δ (a) opt and δ (b) opt are equal to zero. Given the inequality between their variances (36), it is concluded that their MSE s are related as MSE ( δ(a) ) ( δ(b) ) opt MSE opt. (48) Invoking the conjecture that the MSE loss made by using δ opt instead of δ opt is negligible, the optimal MSE s are approximately related as ( MSE δ (a) opt ) ( MSE δ (b) opt ). (49) A. Simulation Results The result of Theorem V.1 is illustrated with simulations using the arrival scenarios described in Table I. 1) Scenario 1: The comparison between the optimized MSE s of the two power estimators is for simulation Scenario 1 shown in the top plot of Fig. 14. The simulation results validate that the optimized MSE corresponding to estimator ˆP a is lower bounded by the optimized MSE corresponding to the estimator ˆP b. A qualitatively identical result is also obtained for the MSE s evaluated at the diagonal loadings which optimize biases. Note that the MSE s of ˆP a and ˆP b, compared in Fig. 14, are equal when steering close to the source directions. These are the ranges within which the optimal loading is close to zero and the two estimators are nearly identical. 2) Scenario 2: The comparison between the optimized MSE s of the two power estimators is for Scenario 2 shown in the bottom plot of Fig. 14. In comparison to Scenario 1, a larger difference between the MSE s of the estimators ˆP a and ˆP b is observed when the number of snapshots is smaller than the number of sensors. A qualitatively identical result is also obtained for the MSE s evaluated at the diagonal loadings which optimize their biases.