Lecture 4 - Spectral Estimation

Transcription

1 Lecture 4 - Spectral Estimation The Discrete Fourier Transform The Discrete Fourier Transform (DFT) is the equivalent of the continuous Fourier Transform for signals known only at N instants separated by sample times T (i.e. a finite sequence of data). Let f (t) be the continuous signal which is the source of the data. Let N samples be denoted f [],f[1],f[2],...,f[k],...,f[n 1]. The Fourier Transform of the original signal, f (t), would be F (jω) = f (t)e jωt dt We could regard each sample f [k] as an impulse having area f [k]. Then, since 74

2 the integrand exists only at the sample points: F (jω) = (N 1)T o f (t)e jωt dt = f []e j + f [1]e jωt f [k]e jωkt +...f(n 1)e jω(n 1)T ie. F (jω)= N 1 k= f [k]e jωkt We could in principle evaluate this for any ω, butwithonlyn data points to start with, only N final outputs will be significant. You may remember that the continuous Fourier transform could be evaluated over a finite interval (usually the fundamental period T o ) rather than from to + if the waveform was periodic. Similarly,since there are only a finite number of input data points, the DFT treats the data as if it were periodic (i.e. f (N) to f (2N 1) is the same as f () to f (N 1).) Hence the sequence shown below in Fig. 4.1(a) is considered to be one period of the periodic sequence in plot (b). 75

3 1 (a) (b) Figure 4.1: (a) Sequence of N = 1 samples. (b) implicit periodicity in DFT. 76

4 Since the operation treats the data as if it were periodic, we evaluate the 1 DFT equation for the fundamental frequency (one cycle per sequence, NT Hz, 2π NT rad/sec.) and its harmonics (not forgetting the d.c. component (or average) at ω =). i.e. set ω =, 2π NT, 2π 2π 2π 2,... n,... (N 1) NT NT NT or, in general F [n] = N 1 k= f [k]e j 2π N nk (n =:N 1) F [n] is the Discrete Fourier Transform of the sequence f [k]. We may write this equation in matrix form as: F [] F [1] 1 W W 2 W 3... W N 1 f [] 1 W F [2] 2 W 4 W 6... W N 2 f [1] = 1 W. 3 W 6 W 9... W N 3 f [2].. F [N 1] 1 W N 1 W N 2 W N 3 f [N 1]... W where W =exp( j2π/n) and W = W 2N etc. = 1. 77

5 DFT example Let the continuous signal be f (t) = }{{} 5 +2cos(2πt 9 o ) }{{} dc 1Hz +3cos4πt }{{} 2Hz Figure 4.2: Example signal for DFT. Let us sample f (t) at 4 times per second (ie. f s =4Hz)fromt =tot =

6 The values of the discrete samples are given by: f [k] =5+2cos( π 2 k 9o )+3cosπk by putting t = kt s = k 4 i.e. f [] = 8, f [1] = 4, f [2] = 8, f [3] =, (N =4) Therefore F [n] = F [] F [1] F [2] F [3] = 3 f [k]e j π 2 nk = j 1 j j 1 j f [] f [1] f [2] f [3] 3 f [k]( j) nk k= = 2 j4 12 j4 The magnitude of the DFT coefficients is shown below in Fig

7 2 15 F[n] f (Hz) Figure 4.3: DFT of four point sequence. 8

8 Inverse Discrete Fourier Transform The inverse transform of is times the complex conjugate of the original (symmet- i.e. the inverse matrix is 1 N ric) matrix. F [n] = f [k] = 1 N N 1 k= N 1 n= f [k]e j 2π N nk F [n]e +j 2π N nk Note that the F [n]coefficientsarecomplex. We can assume that the f [k] values are real (this is the simplest case; there are situations (e.g. radar) in which two inputs, at each k, are treated as a complex pair, sincetheyaretheoutputsfrom o and 9 o demodulators). In the process of taking the inverse transform the terms F [n] and F [N n] 81

9 (remember that the spectrum is symmetrical about N 2 )combinetoproduce2 frequency components, only one of which is considered to be valid (the one at the lower of the two frequencies, n 2π T Hz where n N 2 ;thehigherfrequency component is at an aliasing frequency (n > N 2 )). From the inverse transform formula, the contribution to f [k] off [n] and F [N n] is: f n [k] = 1 N {F [n]ej 2π N nk + F [N n]e j 2π N (N n)k } (4.2) For allf [k] real, F[N n] = But e j 2π N (N n)k = e} j2πk {{} 1 for all k i.e. F [N n] =F (n) N 1 k= Substituting into the Equation for f n [k] above gives, f [k]e j 2π N (N n)k +j 2πn e N k = e +j 2π N nk (i.e. the complex conjugate) f n [k] = 1 N {F [n]ej 2π N nk + F (n)e j 2π N nk } since e j2πk =1 82

10 ie. f n [k] = 2 N {Re{F [n]} cos 2π N nk Im{F [n]} sin 2π N nk} or f n [k] = 2 2π F [n] cos{( n)kt +arg(f [n])} N NT i.e. a sampled sinewave at 2πn NT Hz, of magnitude 2 N F [n]. For the special case of n =,F [] = f [k] (i.e. sum of all samples) and the contribution of F [] to f [k] isf [k] = 1 NF [] = average of f [k] =d.c. component. 83

11 Interpretation of example 1. F [] = 2 implies a d.c. value of 1 N F [] = 2 4 = 5 (as expected) 2. F [1] = j4 =F [3] implies a fundamental component of peak amplitude 2 N F [1] = = 2 with phase given by arg F [1] = 9 o i.e. 2 cos( 2π NT kt 9 o )=2cos( π 2 k 9 o ) (as expected) 3. F [2] = 12 (n = N 2 component noothern n component here) and this implies a f 2 [k] = 1 N F [2]ej 2π N 2k = 1 4 F [2]ejπk =3cosπk (as expected) since sin πk = for all k Thus, the conventional way of displaying a spectrum is not as shown in Fig. 4.3 but as shown in Fig. 4.4 (obviously, the information content is the same): 84

12 F[n] /sqrt(2) 2 sqrt(2) f (Hz) Figure 4.4: DFT of four point signal. 85

13 1 124 F [512] 2 In typical applications, N is much greater than 4; for example, for N = 124, F [n] has 124 components, but are the complex conjugates of 511 1, leaving F [] 124 as the d.c. component, 2 F [1] 2 F [511] to as complete a.c. components and as the cosine-only component at the highest distinguishable frequency (n = N 2 ). F [n] N F [n] 2 Most computer programmes evaluate (or N for the power spectral density) which gives the correct shape for the spectrum, except for the values at n = and N 2. 86

14 4.1 Discrete Fourier Transform Errors To what degree does the DFT approximate the Fourier transform of the function underlying the data? Clearly the DFT is only an approximation since it provides only for a finite set of frequencies. But how correct are these discrete values themselves? There are two main types of DFT errors: aliasing and leakage : Aliasing This is another manifestation of the phenomenon which we have now encountered several times. If the initial samples are not sufficiently closely spaced to represent high-frequency components present in the underlying function, then the DFT values will be corrupted by aliasing. As before, the solution is either to increase the sampling rate (if possible) or to pre-filter the signal in order to minimise its high-frequency spectral content Leakage Recall that the continuous Fourier transform of a periodic waveform requires the integration to be performed over the interval - to + or over an integer 87

15 number of cycles of the waveform. If we attempt to complete the DFT over a non-integer number of cycles of the input signal, then we might expect the transform to be corrupted in some way. This is indeed the case, as will now be shown. Consider the case of an input signal which is a sinusoid with a fractional number of cycles in the N data samples. The DFT for this case (for n =ton = N 2 )is shown below in F[n] freq Figure 4.5: Leakage. 88

16 We might have expected the DFT to give an output at just the quantised frequencies either side of the true frequency. This certainly does happen but we also find non-zero outputs at all other frequencies. This smearing effect, which is known as leakage, arises because we are effectively calculating the Fourier series for the waveform in Fig. 4.6, which has major discontinuities, hence other frequency components Figure 4.6: Leakage. The repeating waveform has discontinuities. Most sequences of real data are much more complicated than the sinusoidal sequences that we have so far considered and so it will not be possible to avoid introducing discontinuities when using a finite number of points from the sequence in order to calculate the DFT. The solution is to use one of the window functions which we encountered in the 89

17 design of FIR filters (e.g. the Hamming or Hanning windows). These window functions taper the samples towards zero values at both endpoints, and so there is no discontinuity (or very little, in the case of the Hanning window) with a hypothetical next period. Hence the leakage of spectral content away from its correct location is much reduced, as in Fig (a) 5 (b) Figure 4.7: Leakage is reduced using a Hanning window. 9

18 Stochastic Models 4.2 Introduction We now discuss autocorrelation and autoregressive processes; that is, the correlation between successive values of a time series and the linear relations between them. We also show how these models can be used for spectral estimation. 4.3 Autocorrelation Given a time series x t we can produce a lagged version of the time series x t T which lags the original by T samples. We can then calculate the covariance between the two signals σ xx (T )= 1 N 1 N (x t T µ x )(x t µ x ) (4.3) t=1 where µ x is the signal mean and there are N samples. We can then plot σ xx (T ) as a function of T. This is known as the autocovariance function. The autocor- 91

19 relation function is a normalised version of the autocovariance r xx (T )= σ xx(t ) σ xx () (4.4) Note that σ xx () = σ 2 x. We also have r xx () = 1. Also, because σ xy = σ yx we have r xx (T )=r xx ( T ); the autocorrelation (and autocovariance) are symmetric functions or even functions. Figure 4.8 shows a signal and a lagged version of it and Figure 4.9 shows the autocorrelation function. 92

20 t Figure 4.8: Signal x t (top) and x t+5 (bottom). The bottom trace leads the top trace by 5 samples. Or we may say it lags the top by -5 samples. 93

21 1.5 r (a) Lag Figure 4.9: Autocorrelation function for x t.noticethenegativecorrelationatlag2andpositivecorrelation at lag 4. Can you see from Figure 4.8 why these should occur? 94

22 4.4 Autoregressive models An autoregressive (AR) model predicts the value of a time series from previous values. A pth order AR model is defined as p x t = x t i a i + e t (4.5) i=1 where a i are the AR coefficients and e t is the prediction error. These errors are assumed to be Gaussian with zero-mean and variance σe. 2 It is also possible to include an extra parameter a to soak up the mean value of the time series. Alternatively, we can first subtract the mean from the data and then apply the zero-mean AR model described above. We would also subtract any trend from the data (such as a linear or exponential increase) as the AR model assumes stationarity. The above expression shows the relation for a single time step. To show the relation for all time steps we can use matrix notation. We can write the AR model in matrix form by making use of the embedding matrix, M, and by writing the signal and AR coefficients as vectors. We now 95

23 illustrate this for p =4. Thisgives x 4 x 3 x 2 x 1 x M = 5 x 4 x 3 x x N 1 x N 2 x N 3 x N 4 (4.6) We can also write the AR coefficients as a vector a = [a 1,a 2,a 3,a 4 ] T,the errors as a vector e =[e 5,e 6,...,e N ] T and the signal itself as a vector X = [x 5,x 6,...,x N ] T.Thisgives x 5 x 6.. x N = which can be compactly written as x 4 x 3 x 2 x 1 x 5 x 4 x 3 x x N 1 x N 2 x N 3 x N 4 a 1 a 2 a 3 a 4 + e 5 e 6.. e N (4.7) X = Ma + e (4.8) The AR model is therefore a special case of the multivariate regression model. The AR coefficients can therefore be computed from the equation â =(M T M) 1 M T X (4.9) 96

24 The AR predictions can then be computed as the vector ˆX = Mâ (4.1) and the error vector is then e = X ˆX. The variance of the noise is then calculated as the variance of the error vector. To illustrate this process we analyse our data set using an AR(4) model. The AR coefficients were estimated to be â =[1.46, 1.8,.6,.186] T (4.11) and the AR predictions are shown in Figure 4.1. The noise variance was estimated to be σe 2 =.79which corresponds to a standard deviation of.28. The variance of the original time series was.3882 giving a signal to noise ratio of ( )/.79 =

25 (a) t (b) t Figure 4.1: (a) Original signal (solid line), X, andpredictions(dottedline),ˆx, fromanar(4)modeland(b) the prediction errors, e. Notice that the variance of the errors is much less than that of the original signal.

26 4.4.1 Relation to autocorrelation The autoregressive model can be written as x t = a 1 x t 1 + a 2 x t a p x t p + e t (4.12) If we multiply both sides by x t k we get x t x t k = a 1 x t 1 x t k + a 2 x t 2 x t k a p x t p x t k + e t x t k (4.13) If we now sum over t and divide by N 1 and assume that the signal is zero mean (if it isn t we can easily make it so, just by subtracting the mean value from every sample) the above equation can be re-written in terms of covariances at different lags σ xx (k) =a 1 σ xx (k 1) + a 2 σ xx (k 2) a p σ xx (k p)+σ e,x (4.14) where the last term σ e,x is the covariance between the noise and the signal. But as the noise is assumed to be independent from the signal σ e,x =. Ifwenow divide every term by the signal variance we get a relation between the correlations at different lags r xx (k) =a 1 r xx (k 1) + a 2 r xx (k 2) a p r xx (k p) (4.15) 99

27 This holds for all lags. For an AR(p) model we can write this relation outfor the first p lags. For p =4 r xx (1) r xx () r xx ( 1) r xx ( 2) r xx ( 3) a 1 r xx (2) r xx (3) = r xx (1) r xx () r xx ( 1) r xx ( 2) a 2 (4.16) r xx (2) r xx (1) r xx () r xx ( 1) a 3 r xx (4) r xx (3) r xx (2) r xx (1) r xx () a 4 which can be compactly written as r = Ra (4.17) where r is the autocorrelation vector and R is the autocorrelation matrix. The above equations are known, after their discoverers, as the Yule-Walker relations. They provide another way to estimate AR coefficients a = R 1 r (4.18) This leads to a more efficient algorithm than the general method for multivariate linear regression (equation 4.9) because we can exploit the structure in the autocorrelation matrix. By noting that r xx (k) =r xx ( k) we can rewrite the 1

28 correlation matrix as R = 1 r xx (1) r xx (2) r xx (3) r xx (1) 1 r xx (1) r xx (2) r xx (2) r xx (1) 1 r xx (1) r xx (3) r xx (2) r xx (1) 1 (4.19) Because this matrix is both symmetric and a Toeplitz matrix (the terms along any diagonal are the same) we can use a recursive estimation technique known as the Levinson-Durbin algorithm. 4.5 Moving Average Models A Moving Average (MA) model of order q is defined as q x t = b i e t i (4.2) i= where e t is Gaussian random noise with zero mean and variance σ 2 e.theyarea type of FIR filter. These can be combined with AR models to get Autoregressive 11

29 Moving Average (ARMA) models p x t = a i x t i + i=1 q b i e t i (4.21) which can be described as an ARMA(p,q) model. They are a type of IIR filter. Usually, however, FIR and IIR filters have a set of fixed coefficients which have been chosen to give the filter particular frequency characteristics. In MA or ARMA modelling the coefficients are tuned to a particular time series so as to capture the spectral characteristics of the underlying process. i= 4.6 Spectral Estimation using AR models Autoregressive models can also be used for spectral estimation. An AR(p)model predicts the next value in a time series as a linear combination of thep previous values p x t = a k x t k + e t (4.22) k=1 where a k are the AR coefficients and e t is IID Gaussian noise with zero mean and variance σ 2. Note the sudden -ve, this is arbitrary and only added to make terms in the spectral equations (as below) +ve. 12

30 The above equation can be solved by using the z-transform. This allows the equation to be written as ( ) p X(z) 1+ a k z k = E(z) (4.23) It can then be rewritten for X(z) as k=1 E(z) X(z) = 1+ p k=1 a (4.24) kz k Taking z =exp(iωt s )whereω is frequency and T s is the sampling period we can see that the frequency domain characteristics of an AR model are givenby (and noting that the power in the noise process is σet 2 s ) σ 2 P (ω) = et s 1+ p k=1 a (4.25) k exp( ikωt s ) 2 An AR(p) model can provide spectral estimates with p/2 peaks; therefore if you know how many peaks you re looking for in the spectrum you can define the AR model order. Alternatively, AR model order estimation methods should automatically provide the appropriate level of smoothing of the estimated spectrum. AR spectral estimation has two distinct advantages over methods based on the Fourier transform (i) power can be estimated over a continuous range of 13

31 (a) (b) Figure 4.11: Power spectral estimates of two sinwaves in additive noise using (a) Discrete Fourier transform method and (b) autoregressive spectral estimation. frequencies (not just at fixed intervals) and (ii) the power estimates have less variance. 14