II. onparametric Spectrum Estimation for Stationary Random Signals - on-parametric Methods - - [p. 3] Periodogram - [p. 12] Periodogram properties - [p. 23] Modified periodogram - [p. 25] Bartlett s method - [p. 30] Blackman-Tukey procedure - [p. 33] Welch s procedure - [p. 36] Minimum Variance MV) Method - [p. 42] MultiTaper Method MTM) 04/01/09 EC4440.SpFY09/MPF -Section II 1
-Goal: To estimate the Power Spectral Density PSD) of a wss process -Two main types of approaches eist: -on parametric methods: do not rely on data model -based on FT and FFT, or -Parametric methods: -based on different parametric models of the data: -Moving Average: MA, -AutoRegressive: AR, -AutoRegressive & Moving Average: ARMA -based on subspace methods 04/01/09 EC4440.SpFY09/MPF -Section II 2
I. Classical methods use FT of the data sequence or its correlation sequence 1) Periodogram Easy to compute Limited ability to produce accurate PS estimate Recall PSD defined as: jω ) ) P e = k = R k e jωk In practice: R k) is approimated with estimated correlation sequence R k ) 04/01/09 EC4440.SpFY09/MPF -Section II 3
Recall: Biased correlation estimate of n), n=0, -1, defined as 1 k 1 * Rˆ k) = + k) ) 0 k< 1 k= 0 ˆ j jωk P e = R k e L ω ) ˆ ) L k = L L 10% PSD estimate sometimes known as correlogram Assume L=-1 1 j jωk Pˆ e = R k e ω ) ˆ ) k= + 1 periodogram 04/01/09 EC4440.SpFY09/MPF -Section II 4
Recall 1 * Rˆ k) = k )* k) for k ) 0 0 k< 1 Define: DFT ) = ) ) ) n n w n X k ˆ 1 P e = FT k * k 1 jω) jω) * = X e X e jω ) * ) ) Periodogram defined in terms of data directly 1 P e X e ˆ j j ω) ω = ) 2 04/01/09 EC4440.SpFY09/MPF -Section II 5
How to Compute the Periodogram Rectangular window of length DFT ) = ) ) ) n n w n X k j πk ) ) ) 2 1 2 ˆ =, = 0,... 1 P e X k k Zero padding n) in the time-domain increases the number of frequency PSD points available 04/01/09 EC4440.SpFY09/MPF -Section II 6
Eample: n) white noise ) Rk = j ) Pe ω = [Hayes] 04/01/09 EC4440.SpFY09/MPF -Section II 7
Average behavior of the periodogram: 1 { ˆ jω )} ˆ jkω E P e = E R k) e + 1 = 1 k= + 1 { ˆ )} E R k e 1 k 1 ER k = E + k { )} * ) ) = 0 jω k k = R ˆ k ) k k = R k) wb k) wb k) = 0 ow 04/01/09 EC4440.SpFY09/MPF -Section II 8
1 j jω k { ˆ ω )} ˆ ) k= + 1 { } E P e = E R k e k { )} k ER k = R k) wb k), wb k) = 0 ow 1 { ˆ jω )} ) ) B k= + 1 ) E P e = R k w k e jωk 1 E P e P e W e 2π { ˆ j ω )} j )* j ) = ω ω B 1 sin ω /2) sin ω/ 2) 2 04/01/09 EC4440.SpFY09/MPF -Section II 9
Eample: n) 1 tone in white noise ) = sin ω + φ) + ) n A n v n 0 { jω } 1 jω jω EP e ) = Pe ) We B ) 2π 2 2 A j 0) j 0) v W B e ωω + ) W B e ωω = σ + + ) 4 [Hayes] 04/01/09 EC4440.SpFY09/MPF -Section II 10
Eample: set of tones in white noise n) n) = sin2π*150*t) + 2*sin2*π*140*t) + 0.1*randnsizet)); f s =1KHz Obtained using the MATLAB function periodogram.m periodogramn,[],'twosided',1024,fs); plots the two sided periodogram note: better to stick to one-sided when dealing with real-data) [Mathworks] 04/01/09 EC4440.SpFY09/MPF -Section II 11
Properties of the Periodogram 1) Periodogram is biased when dealing with finite windows 2) Periodogram is asymptotically unbiased i.e., lim j j E ω P e ) ω + = P e ) 3) Periodogram has limited ability to resolve closely spaced narrowband components limiting factor due to the rectangular window, the shorter the data length, the worse off) not good) 4) Values of the periodogram spaced in frequency by integer multiples of 2π/ are approimately uncorrelated. As data length increases, rate of fluctuation in the periodogram increases not good). 5) Variance of the periodogram does not decrease as data length increases not good). In statistical terms, it means the periodogram is not a consistent estimator of the PSD. evertheless, the periodogram can be a useful tool for spectral estimation in situations where SR is high, and especially if the data is long. 04/01/09 EC4440.SpFY09/MPF -Section II 12
n = sin2*pi*150*t) + 2*sin2*pi*140*t) + 0.1*randnsizet));; fs=1000; nfft=1024; 100 samples Comments Periodogram has limited ability to resolve closely spaced narrowband components limiting factor due to the rectangular window, the shorter the data length, the worse off) not good) 67 samples [MathWorks] 04/01/09 EC4440.SpFY09/MPF -Section II 13
Comments: Variance of the periodogram does not decrease as the data length increases not good). For white noise n) one can show [Therrien, p.590] jω P e ) lim var 0 + jω 4 var P e ) = σ [Hayes] 04/01/09 EC4440.SpFY09/MPF -Section II 14
Comments: As data length increases, rate of fluctuation in the periodogram increases not good). [Kay] 04/01/09 EC4440.SpFY09/MPF -Section II 15
Connections between Periodogram and filterbank Periodogram can be viewed as obtained by using a filterbank structure. Recall periodogram defined for n), n=0, -1, as: 1 ) ) ) DFT ) j2 πkp/ = = ) p= 0 n n w n X k pe j 2πk ) ) 1 P e X ) k k 2 ˆ =, = 0,... 1 04/01/09 EC4440.SpFY09/MPF -Section II 16
For n) defined over a finite time window {n-+1), n)} 1 ) ) ) DFT ) = = j2 πkp/ ) p= 0 n n w n X k n pe 1 p= 0 h p) ) ) ) DFT n = n w n X k) = n ph ) p) k=0, -1 04/01/09 EC4440.SpFY09/MPF -Section II 17
For n) defined over a finite time window {n-+1), n)} 1 ) ) ) DFT n = n w n X k) = n ph ) p) p= 0 j πk ) ) ) 2 Pˆ e = X k 2 1 k=0, -1 n) Filter H z) 1 hp ) = e - j2 π kp / 2 2 π )) ˆ j k P e Periodogram DFT filterbank 04/01/09 EC4440.SpFY09/MPF -Section II 18
Filter H z) 1 h0 p) =, k = 0 2 2 π 0 )) ˆ j P e n) Filter H z) 1 h p = e k = - j2 π p/ 1 ), 1 2 2 π )) ˆ j P e Filter H z) 1 h p = e k = - j2 π p 1)/ 1 ), 1 04/01/09 EC4440.SpFY09/MPF -Section II 19 2 2 1) )) ˆ j P e π j πk ) ) ) 2 Pˆ e = X k 2 1
n) Filter H z) 1 h0 p) =, k = 0 Filter H z) 1 h p = e k = - j2 π p/ 1 ), 1 Filter H z) 1 h p = e k = 2 2 π 0 )) ˆ j P e - j2 π p 1)/ 1 ), 1 04/01/09 EC4440.SpFY09/MPF -Section II 20 2 Lowpass filter 2 π )) ˆ j P e Modulation term 2 2 1) )) ˆ j P e π j πk ) ) ) 2 Pˆ e = X k 2 1
Filter H z) 1 h0 p) =, k = 0 2 2 π 0 )) ˆ j P e Lowpass filter n) Filter H z) h p h p e k - j2 π p/ 1 ) = 0 ), = 1 2 2 π )) ˆ j P e Filter H z) h p = h p e k = - j2 π p 1)/ 1 ) 0 ), 1 Modulation term 2 2 1) )) ˆ j P e π j πk ) ) ) 2 Pˆ e = X k 2 1 04/01/09 EC4440.SpFY09/MPF -Section II 21
Periodogram DFT matri formulation X 1 1 1 0) j2 / j2 1)/ X 1) π π 1 e e = 2 j2 π 1)/ j2 π 1) / X 1) 1 e e 1/ n ) 1/ n 1) 1/ n + 1) j πk ) ) ) 2 Pˆ e = X k 2 1 May be viewed as a rectangular window applied to data before FT operation 04/01/09 EC4440.SpFY09/MPF -Section II 22
2. Modified Periodogram can apply a general window to n) prior to computing the periodogram periodogram etension) + ω 1 P e ) = n) w R n) e ˆ j jnω n = 2 ˆ 1 E P e = P e * WR e 2π jω ) jω ) jω ) 2 ote: amount of smoothing is determined by the window type Matlab implementation: Hs = spectrum.periodogram'hamming'); create modified periodogram object psdhs,n,'fs',fs,'fft',1024,'spectrumtype', onesided') plot 04/01/09 EC4440.SpFY09/MPF -Section II 23
Eample: n = sin2*pi*150*t) + 2*sin2*pi*140*t) + 0.1*randnsizet));; fs=1000; nfft=1024; data length=100. Hamming window Rectangular window ote: lower Hamming window sidelobes, but wider mainlobes with the Hamming window than the rect. window [MathWorks] 04/01/09 EC4440.SpFY09/MPF -Section II 24
3. Bartlett s Method average several periodograms rectangular window applied to each) leads to consistent estimate of the PSD Define: K ˆ 1 P ˆ B e = P e K jω ) k ) jω ) k = 1 where K: number of segments in the data. K L 1 jω 1 jnω PB e ) = n+ k 1) L) e KL k= 1 n= 0 2 n) L points L points L points n 1 n) 2 n) K n) n n n 04/01/09 EC4440.SpFY09/MPF -Section II 25
var ote: ˆ jω ) 1 ˆ k var ) jω P ) B e P e K 1 jω P ) e K Eamples: white noise [Hayes] two sinusoids in noise 04/01/09 EC4440.SpFY09/MPF -Section II 26
Compare: basic periodograms Periodogram of white noise with variance 1. a) Overlay 50 plots, =64 & b) average c) Overlay 50 plots, =128 & d) average e) Overlay 50 plots, =256 & f) average [Hayes, Fig. 8.9] 04/01/09 EC4440.SpFY09/MPF -Section II 27
Bartlett estimates Spectrum estimation of white noise with unit variance a) 50-plot overlay, =512, b) ensemble average. c) Overlay of 50 Bartlett estimates with K=4 & L=128, =512, d) ensemble average. e) Overlay of 50 Bartlett estimates with K=8 & L=64, =512, f) ensemble average. K: number of segments L: segment length [Hayes, Fig. 8.14] 04/01/09 EC4440.SpFY09/MPF -Section II 28
Spectrum estimation of 2 tones in white noise a) 50-plot overlay, =512, b) ensemble average. c) Overlay of 50 Bartlett estimates with K=4 & L=128, =512, d) ensemble average. e) Overlay of 50 Bartlett estimates with K=8 & L=64, =512, f) ensemble average. ote: Spectral peaks broadening, why? π φ) π φ ) n) = Asin 0.2 n+ + sin 0.25 n+ + vn ) 1 2 [Hayes, Fig. 8.15] 04/01/09 EC4440.SpFY09/MPF -Section II 29
4. Blackman-Tukey BT) Procedure apply windowing to correlation samples prior to transformation to reduce variance of the estimator M ˆ jω P ) ) ˆ ) BT e = w k R k e k= M jωk where the window wk) length is much smaller than data length, wk) = 0 for k > M where M << Maimum recommended window length M</5 04/01/09 EC4440.SpFY09/MPF -Section II 30
n ) = 10 ep2 π j0.15) n) + 20 ep2 π j0.2) n) + wn ) wn )~ 0,0.1) Data length=20 Periodogram BT PSD estimate Triangular window, length=4 [Kay, Fig.4.9] 04/01/09 EC4440.SpFY09/MPF -Section II 31
n ) = 10 ep2 π j0.15) n) + 20 ep2 π j0.2) n) + wn ) Periodogram Data length=100 wn )~ 0,0.1) BT PSD estimate Triangular window, length=20 [Kay, Fig.4.10] 04/01/09 EC4440.SpFY09/MPF -Section II 32
5. Welch s Procedure Modification to the Bartlett procedure: Averaging modified periodograms Dataset split into K possibly overlapping segments of length L window applied to each segment all K periodograms are averaged where K ˆ 1 P ˆ w e = P e K jω ) k ) jω ) k = 1 ˆ 1 P w n n e L 1 ) ) ) 2 k k jωn = ) n= 0 window data n) located in K th segment MATLAB Hs = spectrum.welch'rectangular', segmentlength, overlap %); define Welch spectrum object psdhs,n,'fs',fs,'fft',512); plot 04/01/09 EC4440.SpFY09/MPF -Section II 33
[Hayes] ote: what do we gain with the Welch s method? Reduction in spectral leakage that takes place thru the sidelobes of the data window 04/01/09 EC4440.SpFY09/MPF -Section II 34
Performance comparison variability resolution Figure of merit Periodogram 1 0.89 2π/) 0.89 2π/) Bartlett 1/K 0.89 K2π/) 0.89 2π/) Welch 50% overlap, triang. window) [Table 8.7, Hayes] 9/8K 1.28 2π/L) 0.72 2π/) BT 2M/2 0.64 2π/M) 0.43 2π/) K: number of data segments : overall data length L: data section length M: BT window length All schemes limited by data length Definitions: { jω var P )} e variability normalized variance) 2 jω E P e ) { } figure of merit variability resolution resolution should be as small as possible. mainlobe width at its 1/2 power 6dB) down level 04/01/09 EC4440.SpFY09/MPF -Section II 35
6. Minimum Variance MV) Spectrum Estimate Capon PSD PSD estimated by a filterbank where each filter of length P<= data length) adapts itself to data characteristics, i.e., - has center frequency f i spread between 0 and f s ) - has bandwidth Δ P filters spread over [0,f s ] Δ 1/f s ) - depends on R - designed so components at f i are passed without distortion and rejects maimum amount of out-of-band power ˆ jω ) P PMV e =, H 1 e R e e= e e jω j P 1) ω T [1,,..., ] [Manolakis, p.474] 04/01/09 EC4440.SpFY09/MPF -Section II 36
MATLAB implementation ˆ ω) P, [1, ω,..., 1) ω P ] MV e = e= e e H 1 e R e ote: j j j P T H 1 H H 1 H 1 H e R e= e UΛ U ) e= e UΛ U e H 1 H H = ) ) e U Λ e U H jω j P 1) ω eu= [1, e,..., e ] u1 u [ u u ] = FT ),, FT ) 1 P P 04/01/09 EC4440.SpFY09/MPF -Section II 37
H 1 H 1 H H = ) Λ ) e R e e U e U [ FT u ),, FT u )] 1 1 [ FT u ),, FT u )] 1 = 1/ λ ) FT u ) + +1/ λ ) FT u ) H = Λ e R e 2 2 1 1 P P P 1 2 = 1/ λk ) FT uk ) k=1 P P H Pˆ MV jω e ) = P k=1 1/ λ ) FT u ) k P k 2 04/01/09 EC4440.SpFY09/MPF -Section II 38
Application: n e e wn wn j2 0.1) 2 0.12 ) 10 π n j π n = + ) + ), )~ 0,1) Filter order M=P Data length =500 100 averaged trials MV: solid line Bartlett method: small dashed line All-pole: large dashed line Δ f = 0.02 Wind. Length ~ 50 [Manolakis, e. 9.5.1] 04/01/09 EC4440.SpFY09/MPF -Section II 39
MATLAB Implementation R=covar,P); [v,d]=eig ; UI=diaginvd)+eps); VI=absfftv,nfft)).^2; P=10log10P)-10log10VI*UI); In practice, - pick P < - zeropad FFT to insure accurate measurements 04/01/09 EC4440.SpFY09/MPF -Section II 40
7. MultiTaper Method MTM) Spectrum Estimate Thompson) Periodogram can be viewed as obtained by using a filterbank structure, using rectangular windows). Multitaper method MTM) can be viewed the same way, with different filters, i.e., different windows. These optimal FIR filters are derived from a set of sequences known as discrete prolate spheroidal sequences. A set of independent estimates of the power spectrum is computed, by premultiplying the data by orthogonal tapered windows designed to minimize spectral leakage. [Victorin] [Mathworks, statistical signal processing, spectral analysis, non parametric methods] 04/01/09 EC4440.SpFY09/MPF -Section II 41
7. MultiTaper Method MTM) cont PSD variance is reduced by averaging over different tapered windows using the full data. Since the data length is not shortened, the overall PSD bias is smaller than that obtained when splitting data in segments. MTM method provides a time-bandwidth parameter, W, with which to balance the variance and resolution. W is directly related to the number of tapers used to compute the spectrum. There are always 2*W-1 tapers used to form the estimate. As W increases, - there are more estimates of the power spectrum, and the variance of the estimate decreases, - each estimate ehibits more spectral leakage i.e., wider peaks) and the overall spectral estimate is more biased. For each data set, there is usually a value for W that allows an optimal trade-off between bias and variance. [Mathworks, statistical signal processing, spectral analysis, non parametric methods] 04/01/09 EC4440.SpFY09/MPF -Section II 42
Periodogram fs = 1000Hz; =1000; 2 sines, Amplitude = [1 2]; Sinusoid frequencies f = [150;140]; Hs1 = spectrum.mtm4,'adapt'); psdhs1,n,'fs',fs,'fft',1024) MTM, W=4 MTM, W=3/2 04/01/09 EC4440.SpFY09/MPF -Section II 43
References: [Manolakis] [Hayes] Statistical digital signal processing and modeling, M. M. Hayes, 1996, Wiley. [Kay] Modern spectral estimation, S. M. Kay, 1988, Prentice Hall. [Victorin] A. Victorin, Multi-Taper Method for Spectral Analysis and Signal Reconstruction of Solar Wind Data, MS Plasma Physics, Sept. 2007, Royal Institute of Technology, Sweeden, http://www.ee.kth.se/php/modules/publications/reports/2007/trita- EE_2007_054.pdf 04/01/09 EC4440.SpFY09/MPF -Section II 44