INPUT-TO-STATE COVARIANCES FOR SPECTRAL ANALYSIS: THE BIASED ESTIMATE

Transcription

1 INPUT-TO-STATE COVARIANCES FOR SPECTRAL ANALYSIS: THE BIASED ESTIMATE JOHAN KARLSSON AND PER ENQVIST Abstract. In many practical applications second order moments are used for estimation of power spectrum. This can be cast as an inverse problem where we seek a spectrum consistent with a given state-covariance matrix. Such a solution exists if and only if the covariance matrix is positive and belong to a low dimensional subspace. The sample covariance estimate of such a matrix will typically fall outside this class. Several approaches have been proposed, determining a covariance matrix within the correct structure minimizing the distance to the estimated sample covariance. Here we take another approach, akin to the biased estimate in the classical autocovariance case, where the original estimate is guaranteed to belong to the correct class. The proposed procedure is computationally efficient compared to previous approximation approaches and numerical examples suggest increased reliability compared to the ordinary sample covariance. Key words. Input-to-State Covariance Estimation, THREE, Spectral Analysis, Biased Covariance Estimate. AMS subject classifications. 62M15, 3E5, 93E1. 1. Introduction. In many practical spectral estimation applications the maximum entropy and linear prediction based covariance matching methods are used to determine low order rational models. The estimation method can then be considered as a two-step method, where in the first step the covariances are determined from the observed data sequence, and in the second step a method-of-moments approach is applied to fit a model that matches the covariance estimate. In this paper the first step will be covered, which takes into account that the covariance estimate has to have certain properties in order that the second step should be well posed. For the classical autocovariance case, the covariance matrix is constrained to be of the form of a non-negative definite Toeplitz matrix. The most common way to obtain such estimates is to use biased covariance estimates, since it efficiently provides an estimate of the correct form. These covariances coincide with the ones of the periodogram [27, 21]. Other possibilities include the use of iterative approaches, such as the maximum likelihood approach to estimate the covariance matrix of the right structure [3], and the Iterated Toeplitz Approximation Method (ITAM) [28] which aims to minimize the Hilbert-Schmidt norm to a nominal sample covariance estimate. Non-iterative approaches with good asymptotic properties are given in [24, 22]. Here, we will consider a more general setup where the Toeplitz covariance matrix is replaced with an input-to-state covariance matrix. The use of such covariances forms a framework for spectral analysis which allows for tuning of the desired resolution. It was developed in [4, 5, 11, 12, 13, 14, 19] and generalizes ideas used in beamspace processing which enables the user to improve the resolution in power spectral estimation over selected frequency bands. See [5, 12, 1, 2, 7, 9, 23, 26] for examples where high resolution methods and techniques, based on the input to state covariances, have been used and explored. As in the classical covariance matrix estimation described above where the covariance need to be Toeplitz, the set of admissible input-to-state covariance are also a low Supported by Swedish Research Council and the ACCESS Linaeus center. Johan Karlsson is with the department of Electrical and Computer Engineering, University of Florida, Gainesville, Florida, jkarlsson@ufl.edu. Per Enqvist is with the division of Optimization and Systems Theory, Royal Institute of Technology (KTH), Sweden, penqvist@kth.se. 1

2 dimensional subset of the Hermitian matrices. The sample state-covariance estimates generically does not belong to the correct structure for the practical cases of finite data sequences. In [17, Problem 9.1], this issue was raised and it was suggested that a suitable distance measure was minimized to obtain the admissible matrix closest to a samples covariance. Recently several approaches have been proposed along these lines, studying distance measures such as the Hellinger distance, the transportation distance, and the Kullback-Leibler distance [25], [1]. Another approach to dealing with non-admissible state-covariances is to apply a non-exact moment matching method. The nearness to the supplied state-covariance estimate then has to be incorporated in the optimization problem determining the best spectral measure, as in [8, 6]. A merit with the method described here is that it can straightforwardly be used with available implementations of other spectral estimation methods, such as MUSIC, ESPRIT [27] and THREE [5]. In this paper we take a different approach, akin to the biased covariance in the Toeplitz case, directly determining an estimate that is admissible. One benefit of this is that we get an estimate without resorting to intermediate optimization methods. Interestingly, the expected value of this estimate coincides with the state covariance corresponding to the convolution of the true spectra with the Fejér kernel, exactly as in the case with the classical biased covariance estimate. In Section 2 some basic properties of covariances and state-covariances are described. In Section 3 the biased state-covariance estimate is defined, and an algorithm for determining the estimate is given. In Section 4 an example is used to illustrate the method, and finally the conclusions in Section Background. Consider the discrete time stochastic process y(t), t Z which is m dimensional, zero-mean, and second-order stationary with power spectrum dµ. The covariance (or, equivalently, autocorrelation) samples are defined as c k := E{y(t)y(t k) }, for k =, ±1, ±2,..., where E{ } denotes the expectation operator. These are the Fourier coefficients of the power spectrum dµ of the process: c k = 1 2π e ikθ dµ(θ) for k =, ±1, ±2..., (2.1) and consequently c k = c k. The power spectrum is thought of as a non-negative Hermitian measure on the unit circle T = {z = e iθ : θ (, π]} which, for simplicity, is identified with the interval (, π]. Non-negativity of the power spectrum can be characterized in terms of the covariances by the non-negativity of the Hermitian block-toeplitz matrices [2, 21] c c 1 c p c 1 c c p+1 T =......, (2.2) c p c p 1 c for p. The inverse problem of determining a spectrum dµ from a set of finite covariances is known as the trigonometric moment problem. Whenever the corresponding Toeplitz matrix T is positive definite there is an infinite family of spectra satisfying the covariance constraints. 2

3 G 1 (z) x 1 y G 2 (z) x 2 G n (z) x n Fig Bank of filters. A generalization of this framework is the state covariance framework. Where instead of ordinary covariances, generalized statistics in the setting of filter-banks, i.e., we consider the stationary stochastic process y(t) as driving a set of state equations x(t) := Ax(t 1) + By(t), where A C n n and B C n m, x() = x C n and x(t) := (x 1 (t), x 2 (t),..., x n (t)) T. The matrix pair (A, B) is assumed to be controllable, A stable (i.e. A has all eigenvalues in the open unit disc) and B has full rank (= m). The filter dynamics is given by G(z) = (I za) 1 B, (2.3) where G(z) = (G 1 (z), G 2 (z),..., G n (z)) T, as depicted in Figure 2.1, and then the joint covariance matrix of the filter-bank outputs, Σ = E{x(t)x(t) }, takes the place of the Toeplitz matrix in the standard covariance framework (see [5, Equations (2.8), (2.1)] and [15, page 783, Equation (7)]). The matrix Σ provide moment constraints for the power spectrum dµ of the process according to the mapping Γ : dµ Σ = G(e iθ ) dµ 2π G(eiθ ), (2.4) which replaces the covariance constraints (2.1). As in the Toeplitz covariance case, the state covariance belong to a subspace, Range(Γ), of real dimension m(2n m) [1, Prop 3.1] of the Hermitian n n matrices. This set is characterized by the following theorem. Theorem 1 (Georgiou [13]). The matrix Σ is an admissible state covariance if and only if Σ, and Σ AΣA = BH + H B for some H C m n. Furthermore, from the proof of the theorem, H can be represented as H = 1 2 c B + c k B A k (2.5) where the sequence c k are the Fourier coefficients of any power spectrum dµ(θ) k= c ke ikθ satisfying Γ(dµ) = Σ. 3 k=1

4 A rather complete theory has been developed to characterize power spectra for the input process that are consistent with state covariances. This theory provides among other things a construction of the input spectrum of maximal entropy, spectral envelopes that are reminiscent of the Capon pseudo-spectra, and the identification of spectral lines with techniques analogous to the theory of the Pisarenko Harmonic Decomposition, MUSIC, and ESPRIT. These results apply for matrix-valued power spectra as well (see e.g., [14, 15, 16, 18]). 3. The biased state covariance estimator. The standard sample state covariance Σ sample = 1 x(t)x(t) (3.1) N suffers from the fact that it almost always does not belong to the subspace of admissible state covariances. Consider how this is handled in the Toeplitz covariance framework, using the biased state covariance estimator [27, 21] c b k := 1 N t=k+1 y(t)y(t k), k =, ±1, ±2,..., ±N. (3.2) This estimate is widely used and has the desirable property that it produces a positive Toeplitz matrix T bias. The price to pay for this is that the estimator is biased and the expected value of the estimate, E(c b k ) = N k N c k, does not correspond to the true covariance, but of the covariance corresponding to the smoothed measure f N dµ, where f N is the Fejér kernel ( ) N k f N (θ) = N eikθ = 1 sin (N+1)θ 2 2 N + 1 sin θ. 2 k= N For the canonical filter, A C m(p+1) m(p+1), B C m(p+1) m, defined by I... y(t n). A =.... I, B =., x(t) =. y(t 1), I y(t) the state covariance coincides with the Toeplitz covariance (2.2). Note that if the initial state is x() = and only y(1),..., y(n) are used as inputs, then the biased covariances coincides with T bias = 1 N N+p x(t)x(t) = 1 N x(t)x(t), the latter equality since A is (p + 1) nilpotent and hence x(t) = for t > N + p. In fact, applying this for the general input to state covariance A, B gives an estimate which is guaranteed to belong to the set of admissible state covariances, and the expected value of it corresponds to the state covariance of the spectrum f N dµ. Theorem 2. Let y(t) C m, where t Z, be a sequence with y(t) = for t > N. Let x(t) = Ax(t 1) + By(t) and let x() =. (3.3) 4

5 Then Σ bias := 1 N x(t)x(t) (3.4) is non-negative semidefinite and Σ bias Range(Γ). Furthermore if y(1),..., y(n) is a sample from a zero mean second order stationary process with spectrum dµ, then E(Σ bias ) = G(e iθ ) f N dµ G(e iθ ), (3.5) 2π where f N is the Fejér kernel. Proof. Non-negative semidefiniteness is trivial since every term is nonnegative. We will show that Σ bias belongs to Range(Γ) by considering the sum (3.4) and then showing that Σ bias AΣ bias A = BH + H B for some H. First note that Inserting (3.6) in (3.4), Σ bias = 1 N = 1 N x(t) = min(t,n) l=1 min(t,n) l=1 A t l By(l) for t 1. (3.6) min(t,n) k=1 l=1 k=1 t=max(l,k) A t l By(l)y(k) B A t k A t l By(l)y(k) B A t k. Using that this is a geometric sum, we get Σ bias AΣ bias A = 1 N N N l=1 k=1 Amax(l,k) l By(l)y(k) B A max(l,k) k where H bias = 1 N = BH bias + H bias B. l,k=1 l>k y(l)y(k) B A l k + 1 2N Hence, using Theorem 1, Σ bias Range(Γ). The expected value of H bias in this estimate is E(H bias ) = 1 2 c B + k=1 y(l)y(l) B. l=1 N k N c kb A k. (3.7) By noting that the coefficients in (2.5) are given by the Fourier coefficients of the convolution Φ of f N and the measure dµ, i.e., Φ(θ) = (f N dµ)(θ) = k= N N k N c ke ikθ, it follows from uniqueness of the solution to the Lyaponov equation that (3.5) holds. Σ AΣA = BE(H bias ) + E(H bias ) B 5

6 3.1. Algorithm for biased state covariance estimate. The biased state covariance may be calculated using the state equation and noting that the tail is a geometric sum which may be obtained solving a Lyapunov equation. Algorithm 1. Calculate the states x(t) using for t = 1,..., N, x() =. 2. Solve to get Z = t=n x(t)x(t). 3. Take x(t) = Ax(t 1) + By(t) Z AZA = x(n)x(n) Σ bias = 1 N ( Z + N 1 x(t)x(t) ) 4. Example. In the following academic example we study the reliability of spectral estimates corresponding to the biased and sample state covariance estimates. For this, consider the signal y(t) which consists of two closely spaced sinusoids in noise y(t) = sin(.3πt) + sin(.35πt) + w(t), t = 1,..., 2, (4.1) where w(t) N(, 1). We will compare the maximum entropy solution for the two covariance estimates Σ bias and Σ sample. The maximum entropy solution dµ ME is the spectrum maximizing max dµ and is given by [14] log( dµ(θ) )dθ subject to Σ = dθ. G(e iθ ) dµ 2π G(eiθ ), dµ ME = (B Σ 1 G) 1 B Σ 1 B(B Σ 1 G) dθ. (4.2) For reference, we also compare the results with the periodogram. Seven realizations of the signal (4.1) are considered and the methods are applied to each of them. Figure 4.1 shows one realization. Figure 4.2 shows the periodogram. The width of the periodogram main lobe equals twice the spacing between the sinusoids, and the two peaks are not resolved for any of the realizations. Figure 4.3 and Figure 4.4 shows the maximum entropy solutions, dµ bias and dµ unbias, obtained from (4.2) for the biased (3.4) and the sample (3.1) state covariance estimates, respectively. The input to state filters are selected, focusing resolution at the area where the spectral lines are, with 1 poles in the point.9e i.3π. When creating the filters, numerical errors cause the eigenvalues to differ slightly from the specified ones. The x s in the bottom of the respective figure shows the actual angles of the input to state filter. From the figures it can be seen that the biased estimate gives rise to more robust spectral estimates that varies less depending on the particular realization. The maximum of dµ bias identifies the frequency.3 with good accuracy, whereas the maxima 6

7 4 6 Periodogram Fig One realization of y(t). Fig The periodogram. 6 THREE, Biased, deg 1,.9*exp(i*.3*pi) 6 THREE, Unbiased, deg 1,.9*exp(i*.3*pi) Fig dµ ME based on 1th order biased state covariance. Fig dµ ME based on 1th order sample state covariance. of dµ unbias has larger variability. For the frequency point.35 there seem to be a bias, probably due to the fact that the resolution is lower further away from the poles of the filter and possible due to interference of the closely located peak at.3. Also for this spectral line, the variance of the peak is lower for the spectrum corresponding to the biased estimate, compared to the sample estimate. 5. Conclusions. We have derived an efficient state covariance estimator which gives an estimate which is positive and belong to the set of admissible state covariances. This is done in a fashion motivated by the biased covariance estimate for the Toeplitz case. It is shown that, similar to the standard biased covariances, the expected value of the estimate equals the (state) covariance on the convolution of the true spectrum and the Fejér kernel. Simulations suggest that this estimate also provides more reliable estimates than the sample covariance estimate. REFERENCES [1] A. N. Amini and T.T. Georgiou, Tunable Line Spectral Estimators Based on State-Covariance Subspace Analysis, IEEE Trans. on Signal Processing, 54(7): , July 26. 7

8 [2] A. N. Amini, E. Ebbini, and T. T. Georgiou, Noninvasive estimation of tissue temperature via high-resolution spectral analysis techniques, IEEE Trans. Biomed. Eng., 52(2): , Feb. 25. [3] Burg, J.P.; Luenberger, D.G.; Wenger, D.L.;, Estimation of structured covariance matrices, Proceedings of the IEEE, vol.7, no.9, pp , Sept [4] C. Byrnes, T.T. Georgiou, and A. Lindquist, A generalized entropy criterion for Nevanlinna- Pick interpolation: A convex optimization approach to certain problems in systems and control, IEEE Trans. on Automatic Control 45(6): , June 21. [5] C. I. Byrnes, T. T. Georgiou, and A. Lindquist, A new approach to Spectral Estimation: A tunable high-resolution spectral estimator, IEEE Trans. Signal Processing, vol. 48, pp , Nov. 2. [6] P. Enqvist, E. Avventi, Approximative covariance interpolation with a quadratic penalty, Proceedings of the IEEE Intern. Conf. on Decision and Control, pp , December 27. [7] P. Enqvist, Generalizing the Markov and covariance interpolation problem using input-to-state filters, preprint arxiv: [8] P. Enqvist, Approximative Covariance Interpolation, preprint arxiv: [9] A. Ferrante, M. Pavon, and F. Ramponi, Hellinger vs. Kullback- Leibler multivariable spectrum approximation, IEEE Trans. Autom. Control, bf 53(5): , May 28. [1] A. Ferrante, M. Pavon, and M. Zorzi, A Maximum Entropy Enhancement for a Family of High-Resolution Spectral Estimators, Automatic Control, IEEE Transactions on, vol.57, no.2, pp , Feb [11] T.T. Georgiou, Spectral Estimation via Selective Harmonic Amplification, IEEE Trans. on Automatic Contr., January 21, 46(1): [12] T.T. Georgiou, Spectral Estimation via Selective Harmonic Amplification: MUSIC, Redux, IEEE Trans. on Signal Proc., March 2, 48(3): [13] T.T. Georgiou, The structure of state covariances and its relation to the power spectrum of the input, IEEE Trans. on Automatic Control, 47(7): , July 22. [14] T.T. Georgiou, Spectral analysis based on the state covariance: the maximum entropy spectrum and linear fractional parameterization, IEEE Trans. on Automatic Control, November 22, 47(11): [15] T.T. Georgiou, Spectral Estimation via Selective Harmonic Amplification: MUSIC, Redux, IEEE Trans. on Signal Proc., March 2, 48(3): [16] T.T. Georgiou, The Carathéodory-Fejér-Pisarenko decomposition and its multivariable counterpart. IEEE Trans. on Automatic Control, 52(2): , February, 27. [17] T.T. Georgiou, Structured Covariances and Related Approximation Questions, In: A. Rantzer, C.I. Byrnes (eds) Directions in Mathematical Systems Theory and Optimization, LNCIS 296, pp , 23. Springer, Berlin. [18] T.T. Georgiou, Relative Entropy and the multi-variable multi-dimensional Moment Problem, IEEE Trans. on Information Theory, 52(3): , March 26. [19] T.T. Georgiou and A. Lindquist, Kullback-Leibler approximation of spectral Density functions, IEEE Trans. on Information Theory, 49(11), November 23. [2] U. Grenander and G. Szegö, Toeplitz Forms and their Applications, Chelsea, [21] S. Haykin, Nonlinear Methods of Spectral Analysis, Springer-Verlag, New York, 247 pages, [22] M. Jansson and B. Ottersten, Structured covariance matrix estimation: a parametric approach, Acoustics, Speech, and Signal Processing, 2. ICASSP. Proceedings. 2 IEEE International Conference on, vol.5, no., pp vol.5, 2. [23] A. H. Khanshan, H. Amindavar and H. Bakhshi, High-Resolution ARMA Estimation of Mixed Spectra IEEE Trans. Signal Processing, 58(1), January 21. [24] H. Li, P. Stoica and J. Li, Computationally efficient maximum likelihood estimation of structured covariance matrices, Signal Processing, IEEE Transactions on, vol.47, no.5, pp , May [25] L. Ning, X- Jiang and T.T. Georgiou, Geometric methods for estimation of structured covariances, arxiv v1 [math.oc] 17 Oct [26] F. Ramponi, A. Ferrante, and M. Pavon, A Globally Convergent Matricial Algorithm for Multivariate Spectral Estimation, IEEE Trans. Autom. Control, bf 54(1): , October 29. [27] P. Stoica and R. Moses, Introduction to Spectral Analysis, Prentice Hall, [28] H. Wang and G. Wakefield, Signal-subspace approximation for line spectrum estimation, Acoustics, Speech, and Signal Processing, IEEE International Conference on ICASSP 87., vol.12, no., pp , Apr