Introduction. Spectral Estimation Overview Periodogram Bias, variance, and distribution Blackman-Tukey Method Welch-Bartlett Method Others

Transcription

1 Spectral Estimation Overview Periodogram Bias, variance, and distribution Blackman-Tukey Method Welch-Bartlett Method Others Introduction R x (e jω ) r x (l)e jωl l= Most stationary random processes have continuous spectra If we have time, will discuss line spectra at end of term Recall the definition of above The estimation problem is to find a ˆR x (e jω ) given only a finite data record {x(n)} N If the autocorrelation can be estimated with great accuracy at long lags, can treat as a deterministic problem With short records (or equivalently, local stationarity), is more difficult J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. Example : Data Windowed Autocorrelation Solve for the expected value of the biased estimate of autocorrelation applied to a windowed data segment. How should w(n) be scaled such that the estimate is unbiased at l =. ˆr x (l) = x(n + l )w(n + l )x (n)w (n) N n=n l Periodogram ˆR x (e jω ) N v(n)e jωn = N N n= V (e jω ) where v(n) x(n)w(n). If w(n) =c (a constant), is called the Periodogram If w(n) is not constant, is called the Modified Periodogram and w(n) is called the data window Can be estimated quickly using the FFT Is related to the biased autocorrelation estimate we discussed last time N ˆR x (e jω )= ˆr v (l)e jωl l= (N ) J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. J. McNames Portland State University ECE 58/68 Spectral Estimation Ver..

2 Example : Periodogram and Biased Autocorrelation Let x(n) be a finite duration sequence, to N. Show that the biased estimate of autocorrelation is given by Example : Workspace ˆr x (l) = x(l) x( l) N and that the DTFT of ˆr x (l) is equal to the Periodogram of x(n) when w(n) =. J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 5 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 6 Periodogram ˆR x (e jω ) N v(n)e jωn = V (e jω ) = F{ˆr v (l)} N N n= Remember that we must use the biased estimate of autocorrelation Otherwise the estimated may be negative at some frequencies The equality N V (e jω ) = F{ˆr v (l)} no longer holds This relationship to ˆr(l) means we can use the FFT to calculate ˆr(l) very efficiently Take FFT of signal, magnitude square, and inverse FFT Requires O(N log N) operations instead of O(N )! Must take care to zero pad sufficiently Example : Periodogram as a Filter Bank ˆR x (e jω ) N v(n)e jωn N n= How is the periodogram a filter bank? What is the transfer function of the filter? How good are the filters? How close are they to ideal filters? J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 7 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 8

3 Example : Work Space Example : Peridogram Generate the periodogram for a signal from a known LTI system with two sets of complex conjugate poles close to one another and near the unit circle. Repeat for various signal lengths. J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 9 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. Example : MATLAB Code M = 5; % Padding to eliminate transient N = [5,5,,5,5,]; % Length of signal segment NA = 5; % No. averages cb = 95; % NZ = ; b = poly([-.8,.97*exp(j *pi/),.97*exp(-j *pi/),....97*exp(j *pi/6),.97*exp(-j *pi/6)]); % Numerator a = poly([.8,.95*exp(j**pi/),.95*exp(-j**pi/),....95*exp(j*.5*pi/),.95*exp(-j*.5*pi/)]); % Denominator b = b*sum(a)/sum(b); % Scale DC gain to Example : Pole Zero Map of ARMA Process J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. J. McNames Portland State University ECE 58/68 Spectral Estimation Ver..

4 Example : MATLAB Code Example : Signal figure h = circle; z = roots(b); p = roots(a); hold on; h = plot(real(z),imag(z), bo,real(p),imag(p), rx ); hold off; axis square; xlim([-..]); ylim([-..]); axislines; box off; Signal Length: Sample Index J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. Example : MATLAB Code n = ; w = randn(m+n,); x = filter(b,a,w); % System with known nx = length(x); x = x(nx-n+:nx); % Eliminate start-up transient (make stationary) figure; k = :n-; h = stem(k,x); set(h, Marker, none ); set(h, LineWidth,.5); box off; ylabel( Signal ); xlabel( Sample Index ); title(sprintf( Length:%d,n)); xlim([ n]); ylim([min(x) max(x)]); x 5 Example : True True J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 5 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 6

5 [R,w] = freqz(b,a,nz); R = abs(r).^; f = w/(*pi); subplot(,,); h = plot(f,r, r ); set(h, LineWidth,.); box off; ylabel( ); title( True ); xlim([.5]); ylim([ max(r)*.]); subplot(,,); h = semilogy(f,r, r ); set(h, LineWidth,.); box off; ylabel( ); xlim([.5]); ylim([.*min(r) max(r)*5]); xlabel( ); Example : MATLAB Code x 5 Example : Periodogram Estimate N:5 NA: :95% True J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 7 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 8 x 5 Example : Periodogram Estimate N:5 NA: :95% True x 5 Example : Periodogram Estimate N: NA: :95% True J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 9 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver..

6 x 5 Example : Periodogram Estimate N:5 NA: :95% True x 5 Example : Periodogram Estimate N:5 NA: :95% True J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. x 5 Example : Periodogram Estimate N: NA: :95% True Example : MATLAB Code for c = :length(n), nx = M+N(c); Rh = zeros(na,nz/+); kh = :NZ/+; fh = (kh-)/nz; for c = :NA, w = randn(m+n(c),); x = filter(b,a,w); % System with known x = x(nx-n(c)+:nx); % Eliminate start-up transient (make stationary) rh = (/N(c))*abs(fft(x,NZ)).^; Rh(c,:) = rh(kh). ; Rha = mean(rh); % Average Rhu = prctile(rh,-(-cb)/); % Upper confidence band Rhl = prctile(rh, (-cb)/); % Lower confidence band J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. J. McNames Portland State University ECE 58/68 Spectral Estimation Ver..

7 Example : MATLAB Code figure; subplot(,,); h = plot(fh,rh(,:), g,fh,rhl, b,fh,rhu, b,f,r, r,fh,rha, k ); set(h(), LineWidth,.); set(h(:), LineWidth,.5); set(h(), LineWidth,.5); set(h(5), LineWidth,.5); ylabel( ); title(sprintf( N:%d NA:%d :%d%%,n(c),na,cb)); xlim([.5]); ylim([ max(r)*.5]); legend(h([ 5]),,, Average, True,); subplot(,,); h = semilogy(fh,rh(,:), g,fh,rhl, b,fh,rhu, b,f,r, r,fh,rha, k ); set(h(), LineWidth,.); set(h(:), LineWidth,.5); set(h(), LineWidth,.5); set(h(5), LineWidth,.5); ylabel( ); xlabel( ); xlim([.5]); ylim([.*min(r) max(r)*5]); Periodogram Properties The main disadvantage of the Periodogram appears to be excessive variance As with any estimator, we would like to know the bias, variance, covariance, confidence intervals, and distribution We will consider these each in turn J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 5 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 6 [ ] E ˆRx (e jω ) = Periodogram Mean N l= (N ) E[ˆr v (l)] e jωl [ ] E[ˆr v (l)] = E x(n)w(n)x (n l)w (n l) N n= = r x (l)w(n)w (n l) N n= = r x (l) N n= = N r x(l)r w (l) w(n)w (n l) The expected value of ˆr v (l) is the true autocorrelation r x (l) windowed with r w (l) = N w(n) w( n). [ ] E ˆRx (e jω ) Periodogram Mean Continued = = N N l= (N ) N l= (N ) π E[ˆr x (l)] e jωl r x (l)r w (l)e jωl = R x (e ju ) ( π π N R w e j(ω u)) du [ ] Since E ˆRx (e jω ) R x (e jω ), the Periodogram is a biased estimator Here R w (e jω )= W(e jω ) is the ESD of the window Ideally, we would like R w (e jω )=πnδ(ω) The smearing limits our ability to distinguish sharp features (e.g. distinct peaks) in the J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 7 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 8

8 [ ] E ˆRx (e jω ) Periodogram Asymptotic Bias = N N l= (N ) r x (l)r w (l)e jωl Most windows are relatively constant over a range of samples proportional to the window length: { N l N r w (l) l >N Most stationary signals have an autocorrelation that approaches zero as l Thus, the periodogram can be considered to be an asymptotically unbiased estimator lim N E [ ˆRx (e jω ) ] = R x (e jω ) Periodogram Asymptotic Bias Continued [ ] lim E ˆRx (e jω ) = R x (e jω ) N The estimator is unbiased as long as N n= w(n) = π R w (e jω )dω = N π π and the mainlobe window width decreases as N Not of much relevance (asymptotic result) Practically, the bias is introduced by the sidelobes and smearing of the spectrum by the main lobe Can reduce with better windows, but can only trade main lobe width for decreased sidebands If signal is white noise (flat spectrum), the smearing of the convolution does not cause bias J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 9 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. Periodogram Covariance Theory ˆR w (e jω )= N w(n)e jωn N n= = N w(n)e jk π N n N n= The text lists the covariance of the Periodogram, but does not derive it Details of the derivation can be found in [] An outline of the derivation is provided here Suppose for now that no zero padding is used so we calculate the DFT only at the N points that we have samples of the signal Let w(n) be a WGN process with variance σw and zero mean Orthogonal Components Recall that when zero padding is not used, the exponentials are orthogonal (e jk π n)( N e jl π n) N = n=<n> = n=<n> Now consider W (e jω ) evaluated at ω = k π N. W (e jω )= N n= e j(k l) π N n { N k =, ±N,±N,... otherwise w(n)e jωn If w(n) is a WGN process, then W (e jω ) is also a Gaussian random variable, since it is a linear combination of Gaussian RVs. J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. J. McNames Portland State University ECE 58/68 Spectral Estimation Ver..

9 WGN Mean and Variance E [ W (e jω ) ] = N n= E[w(n)]e jωn = cov [ W (e jω ),W(e jω ) ] =E [ W (e jω )W (e jω ) ] [( N )( N ) ] =E w(k)e jω k w(l)e jω l = = N k= N k= N l= l= E[w(k)w(l) ]e jω k e jω l k= l= N e j(ω ω )k = σ w N k= σ wδ(k l)e j(ω k ω l) WGN DFT Mean and Variance Continued If we don t use zero padding, π ω = m N where m and m are integers. Then cov [ W (e jω ),W(e jω ) ] = σ w = σ w ω = m π N N k= N e j(m m ) π N k k= e j(ω ω )k = σ wnδ(m m ) Thus, for WGN and no zero padding, the random variables are uncorrelated and therefore independent! The variance of the RVs is σ w. Alternative, the number of uncorrelated estimates in the frequency domain scales with N. More data more uncorrelated samples. J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. J. McNames Portland State University ECE 58/68 Spectral Estimation Ver χ Random Variables v= v= v= v= v=5 v= v=.8 v= v=.6 v= Let x i be a set of IID RVs with a Normal distribution, x i N(, ) Then the RV χ where v χ = x i i= has a χ distribution with v degrees of freedom The PDF and CDF is available in MATLAB and equivalent x = linspace(,,) ; df = :5; % Degrees of freedom nd = length(df); nx = length(x); P = zeros(nx,nd); C = zeros(nx,nd); for c=:5, P(:,c) = chipdf(x,df(c)); C(:,c) = chicdf(x,df(c)); ls{c} = sprintf( v=%d,df(c)); figure h = plot(x,p); set(h, LineWidth,.5); box off; xlim([ x(end)]); ylim([.5]); legend(ls); figure h = plot(x,c); set(h, LineWidth,.5); box off; xlim([ x(end)]); ylim([.5]); legend(ls); MATLAB Code J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 5 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 6

10 WGN Mean and Variance Continued E [ W (e jω ) ] = N n= E[w(n)]e jωn = cov [ W (e jω ),W(e jω ) ] = σ wnδ(ω ω ) ˆR w (e jω )= N W (ejω ) If W (e jω ) is a complex Gaussian RV with zero mean and variance σ w, then what is the distribution of W (e jω )? E [ W (e jω ) ] ] =E [ Re{W (e jω )} +Im{W (e jω )} ] = σ w where the real and imaginary components are both independent Gaussian RVs each with variance σw/ Note that Im{W (e jω )} =if ω =or ω = π WGN Distribution Thus the distribution of W (e jω ) is given by { W (e jω ) σwn χ ()/ ω lπ χ () ω = lπ where χ () is a chi-squared distribution with one degree of freedom. Thus { ˆR w (e jω σ w χ ()/ ω lπ ) σ w χ () ω = lπ Thus, we know what the distribution of ˆR w (e jω ) is in the WGN case, but what about non-white processes? J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 7 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 8 Generalized Linear Processes x(n) =h(n) w(n) = R x (e jω )= H(e jω ) σ w k= h(k)w(n k) This is a standard and widely used model for wide-sense stationary processes Note that h(n) is two sided (non-causal) We require that h(n) has finite energy (a sufficient condition for stability) But how is ˆR x (e jω ) related to the known distribution of ˆR w (e jω )? Periodogram Distribution It s beyond the scope of this class to prove it (see [, pg. ]), but ˆR x (e jω )=R x (e jω ) ˆR σw w (e jω )+R ε (ω) where E[R ε (ω)] = O(/N ) Thus R ε (ω) as N If we assume N is large we can disregard it N must be large enough to make the bias due to windowing negligible This means that the periodogram estimate is χ distributed when the process is Gaussian We can then immediately write the distribution of ˆR x (e jω ) J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 9 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver..

11 Since Periodogram Distribution { ˆR x (e jω R x (e jω )χ ()/ ω lπ ) R x (e jω )χ () ω = lπ var [ χ ()/ ] = var [ χ () ] = The variance of the periodogram is given by { var{ ˆR x (e jω R )} x(e jω ) <ω<π Rx(e jω ) ω =,π which agrees with the text. Periodogram Covariance Depends on the true, unknown so is of limited value Can make some qualitative observations Estimates at two frequencies ω =(π/n)k and ω =(π/n)k where k and k are integers are approximately uncorrelated cov{ ˆR x (e jω ), ˆR x (e jω )} for k k The number of uncorrelated frequency pairs scales with N Thus, the periodogram at adjacent frequencies, say ω and ω + δ, becomes more variable as N increases J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. Periodogram Variance (Again) [ ( ) ] var{ ˆR sin(ωn) x (e jω )} Rx(e jω ) + N sin(ω) For large N this can be approximated as { [ ] var ˆRx (e jω R ) x(e jω ) Rx(e jω ) <ω<π ω =,π The variance depends on the true spectrum Perhaps not surprising since the variance of ˆr(l) depended on the true autocorrelation Note that the variance does not decrease as N Thus, the Periodogram is not a consistent estimator The estimate becomes more variable as a function of frequency, but the variance at a fixed frequency does not decrease var[] var[] x 5 Example : Periodogram Variance N:5 NA: :95% Estimated Variance Theoretical Variance J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. J. McNames Portland State University ECE 58/68 Spectral Estimation Ver..

12 Example : Periodogram Variance Example : Periodogram Variance var[] x 5 N:5 NA: :95% Estimated Variance Theoretical Variance var[] x 5 N: NA: :95% Estimated Variance Theoretical Variance var[] var[] J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 5 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 6 Example : Periodogram Variance Example : Periodogram Variance var[] x 5 N:5 NA: :95% Estimated Variance Theoretical Variance var[] x 5 N:5 NA: :95% Estimated Variance Theoretical Variance var[] var[] J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 7 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 8

13 var[] var[] x 5 Example : Periodogram Variance N: NA: :95% Estimated Variance Theoretical Variance Periodogram Variance ˆR x (e jω ) N v(n)e jωn = V (e jω ) = F{ˆr(l)} N N n= Since the ensemble variance of the Periodogram does not decrease as N increases, it is considered a poor estimator We cannot fix this by choosing a better window This is perhaps surprising, because it is the natural estimator To obtain a better estimator, we need to reduce the ensemble variance J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 9 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 5 Fixing Problems with the Periodogram As N increases, the ensemble variation stays constant The estimator is not consistent (statistically) As N increases, the covariation at neighboring frequencies, ω and ω + δ, decreases Thus we are essentially able to estimate the true at more frequencies We have better frequency resolution as N increases Key assumption: the true is smooth Clearly true for all ARMA processes with poles and zeros not too close to the unit circle Key concept: If the is smooth and our estimate consists of uncorrelated high-resolution estimates, we can obtain a better estimate by averaging estimates at adjacent frequencies Periodogram Smoothing Smoothing the Periodogram can be thought of in several ways It reduces the frequency resolution It blurs the spectrum It increases the bias of the estimator It reduces the variance of the estimator It could either increase or decrease the MSE depending on the degree of smoothing and the variability of the true Thus, we can tradeoff some of the excessive variance of the Periodogram for increases bias by smoothing There are several ways to achieve this Average contiguous values of the periodogram Average periodograms obtained from multiple segments Both approaches produce about the same quality of estimators J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 5 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 5

14 Smoothing the Periodogram ˆR x (PS) (e jω k ) M + M j= M ˆR x (e jω k j ) One approach to smoothing is to apply a moving average filter in the frequency domain In the definition above (borrowed from the book), only applies if ˆR x (e jω ) is known at discrete frequencies ω k (π/n)k This is usually the case because we estimate using the FFT Since the examples at these discrete frequencies are approximately uncorrelated var{ (PS) ˆR x (e jω k )} M + var{ ˆR x (e jω k )} However, we increase the bias, especially near sharp features in the frequency domain x 5 Example 5: Smoothed Periodogram Estimate M: N:5 NA: :95% True J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 5 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 5 x 5 Example 5: Smoothed Periodogram Estimate M: N:5 NA: :95% True x 5 Example 5: Smoothed Periodogram Estimate M:5 N:5 NA: :95% True J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 55 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 56

15 x 5 Example 5: Smoothed Periodogram Estimate M: N:5 NA: :95% True Example 5: MATLAB Code Q = [,,5,]; % MA filter order N = 5; % Length of signal segment NA = ; % No. averages P = 5; % Padding to eliminate transient cb = 95; % NZ = 8; b = poly([-.8,.97*exp(j *pi/),.97*exp(-j *pi/),....97*exp(j *pi/6),.97*exp(-j *pi/6)]); % Numerator a = poly([.8,.95*exp(j**pi/),.95*exp(-j**pi/),....95*exp(j*.5*pi/),.95*exp(-j*.5*pi/)]); % Denominator b = b*sum(a)/sum(b); % Set DC gain to [R,w] = freqz(b,a,nz); R = abs(r).^; f = w/(*pi); J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 57 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 58 Example 5: MATLAB Code Continued for c = :length(q), nx = P+N; Rh = zeros(na,nz/+); kh = :NZ/+; fh = (kh-)/nz; for c = :NA, w = randn(p+n,); x = filter(b,a,w); % System with known x = x(nx-n+:nx); % Eliminate transient (make stationary) rh = (/N)*abs(fft(x,NZ)).^; rh = [rh;rh;rh]; % Eliminate edge effects rhs = filter(ones(q(c),)/q(c),,rh); rhs = rhs((q(c)-)/+(nz+:*nz)); % Account for filter delay Rh(c,:) = rhs(kh). ; Rha = mean(rh); % Average Rhu = prctile(rh,-(-cb)/); % Upper confidence band Rhl = prctile(rh, (-cb)/); % Lower confidence band Example 5: MATLAB Code Continued figure; FigureSet(, LTX ); subplot(,,); h = plot(fh,rh(,:), g,f,r, r,fh,rhl, b,fh,rhu, b,fh,rha, k ); set(h(), LineWidth,.); set(h(), LineWidth,.); set(h(:), LineWidth,.5); set(h(5), LineWidth,.5); ylabel( ); title(sprintf( Q:%d N:%d NA:%d :%d%%,q(c),n,na,cb)); xlim([.5]); ylim([ max(r)*.5]); legend(h([ 5]),,, Average, True,); subplot(,,); h = semilogy(fh,rh(,:), g,f,r, r,fh,rhl, b,fh,rhu, b,fh,rha, k ); set(h(), LineWidth,.); set(h(), LineWidth,.); set(h(:), LineWidth,.5); set(h(5), LineWidth,.5); ylabel( ); xlabel( ); xlim([.5]); ylim([.*min(r) max(r)*5]); J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 59 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 6

16 Periodogram Smoothing Comments ˆR x (PS) (e jω k ) { } var R x (PS) (e jω ) M + M j= M ˆR x (e jω k j ) M + var { R x (e jω ) } Smoothing the periodogram is a good idea Trades reduced frequency resolution and increased bias for reduced variance The moving average is just a lowpass filter There are many other lowpass filters that might perform better Examples: smoothing splines, local polynomial models, weighted averaging (kernel smoothing) Blackman-Tukey Spectral Estimation Idea [ ] E ˆRx (e jω ) = π R x (e ju ) π π N R w(e j(ω u) )du Recall that windowing the signal is equivalent to convolving the signal s with the window s, in expectation Despite the blurring and smoothing of this effect, the estimate is still has too high of variance We can increase the smoothing and reduce the variance by multiplying the autocorrelation by an even shorter window J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 6 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 6 Blackman-Tukey Spectral Estimation ˆR x (BT) (e jω ) L l= (L ) ˆr x (l)w a (l)e jωl Here the window is user specified and has a finite duration of L samples Called the correlation or lag window In order for the estimate to be nonnegative, W (e jω ) for all ω Not all of the windows we described have this property In the frequency domain, the product ˆr x (l)w(l) is equivalent to convolving with W (e jω ) Assuming the window s spectrum is bumped shape, this convolution smooths the spectrum Note that the FFT can be used to calculate both ˆr x (l) and (e jω ) efficiently ˆR (BT) x Blackman-Tukey Bias & Variance ˆR x (BT) (e jω ) L l= (L ) ˆr x (l)w a (l)e jωl Conceptually, this is using an even more biased estimate of the autocorrelation: ˆr x (l)w a (l) The additional bias is towards zero (BT) Thus, it reduces the variance of both ˆr x (l) and ˆR x (e jω ) The user controls this tradeoff by picking the correlation window w a (l) The window can be any positive definite window with even symmetry (why?) The length of the window L has a larger impact than the shape J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 6 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 6

17 x 5 Example 6: Blackman-Tukey Estimate N:5 L:5 NA: :95% d CIs True Coverage.8.6. Example 7: Blackman-Tukey Coverage N:5 L:5 NA: :95%. J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 65 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 66 Example 7: Blackman-Tukey Estimate Example 8: Blackman-Tukey Coverage x 5 N:5 L:5 NA: :95% d CIs True Coverage.8.6. N:5 L:5 NA: :95%. J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 67 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 68

18 x 5 Example 8: Blackman-Tukey Estimate N:5 L: NA: :95% d CIs True Coverage.8.6. Example 9: Blackman-Tukey Coverage N:5 L: NA: :95%. J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 69 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 7 Example 9: Blackman-Tukey Estimate Example : Blackman-Tukey Coverage x 5 N:5 L:5 NA: :95% d CIs True Coverage.8.6. N:5 L:5 NA: :95%. J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 7 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 7

19 x 5 Example : Blackman-Tukey Estimate N:5 L:5 NA: :95% d CIs True Coverage.8.6. Example : Blackman-Tukey Coverage N:5 L:5 NA: :95%. J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 7 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 7 Example : MATLAB Code L = [5,5,,5,5]; % MA filter order N = 5; % Length of signal segment NA = ; % No. averages P = 5; % Padding to eliminate transient cb = 95; % NZ = ; np = ^nextpow(*n-); % Figure out how much to pad the signal b = poly([-.8,.97*exp(j *pi/),.97*exp(-j *pi/),....97*exp(j *pi/6),.97*exp(-j *pi/6)]); % Numerator a = poly([.8,.95*exp(j**pi/),.95*exp(-j**pi/),....95*exp(j*.5*pi/),.95*exp(-j*.5*pi/)]); % Denominator b = b*sum(a)/sum(b); % Set DC gain to [R,w] = freqz(b,a,nz); R = abs(r).^; f = w/(*pi); Example : MATLAB Code Continued for c = :length(l), nx = P+N; Rh = zeros(na,nz/+); kh = :NZ/+; fh = (kh-)/nz; for c = :NA, w = randn(p+n,); x = filter(b,a,w); % System with known x = x(nx-n+:nx); % Eliminate transient (make stationary) xf = fft(x,np); % Calculate FFT of x r = ifft(xf.*conj(xf)); % Calculate biased autocorrelation r = real(r(:nx))/n; % Eliminate superfluous zeros no = (L(c)+)/; % Number of one-sided points to included r = [r(no:-:);r(:no)]; % Make two-sided wn = blackman(length(r)); % Create window wn = wn/max(wn); % Make maximum = rh = abs(fft(r.*wn,nz)); % Window autocorrelation Rh(c,:) = rh(kh). ; Rha = mean(rh); % Average Rhu = prctile(rh,-(-cb)/); % Upper confidence band Rhl = prctile(rh, (-cb)/); % Lower confidence band J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 75 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 76

20 Example : MATLAB Code Continued figure; subplot(,,); h = plot(fh,rh(,:), g,f,r, r,fh,rhl, b,fh,rhu, b,fh,rha, k ); set(h(), LineWidth,.); set(h(), LineWidth,.); set(h(:), LineWidth,.5); set(h(5), LineWidth,.5); ylabel( ); title(sprintf( N:%d L:%d NA:%d :%d%%,n,l(c),na,cb)); xlim([.5]); ylim([ max(r)*.5]); AxisSet(8); legend(h([ 5]),,, Average, True,); subplot(,,); h = semilogy(fh,rh(,:), g,f,r, r,fh,rhl, b,fh,rhu, b,fh,rha, k ); set(h(), LineWidth,.); set(h(), LineWidth,.); set(h(:), LineWidth,.5); set(h(5), LineWidth,.5); ylabel( ); xlabel( ); xlim([.5]); ylim([.*min(r) max(r)*5]); ˆR (BT) x (e jω ) [ ] E ˆR(BT) x (e jω ) Blackman-Tukey Mean = = L l= (L ) L l= (L ) L l= (L ) ˆr x (l)w a (l)e jωl E[ˆr x (l)] w a (l)e jωl [ r x (l) ] N r w(l) w a (l)e jωl = R x (e jω ) W (e jω ) W a (e jω ) Recall that the periodogram had an expected value given by the convolution of the Bartlett window s spectrum and the true Multiplication by another window causes another convolution Text assumes the data window w(l) is rectangular J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 77 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 78 Blackman-Tukey Mean [ ] E ˆR(BT) x (e jω ) = R x (e jω ) W (e jω ) W a (e jω ) Note that the convolution is circular since each term is a periodic function of ω The estimate is asymptotically unbiased if w a () = As L and N approach, both windows become impulse trains Blackman-Tukey Sampling Distribution ˆR x (BT) (e jω ) L l= (L ) ˆr x (l)w a (l)e jωl Sampling properties of the spectral estimates are extremely complicated Exact results can only be obtained in special circumstances Most expressions for mean, variance, covariance, and distributions are asymptotic and only apply when N is large The asymptotic expressions assume that L increases with N, but more slowly than N: L, N, L/N J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 79 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 8

21 Blackman-Tukey Covariance (BT) cov{ ˆR x (e jω (BT) ), ˆR x (e jω )} π R πn x(e ju )W a (e j(ω u) )W a (e j(ω u) )du π Assumptions N is large so that W B (e jω ) behaves like an impulse train L is sufficiently large that W a (e jω ) is narrow and the product W a (e jω )W a (e jω ) is negligible Thus, covariance increases as The window s spectrum, W a (e j(ω u) ), grows wider The overlap between windows centered at different frequencies, ω and ω, increases For estimates to be uncorrelated, they must be separated in frequency by an amount greater than the bandwidth of the window Blackman-Tukey Variance (BT) var{ ˆR x (e jω )} π R πn x(e ju )Wa (e j(ω u) )du π If Rx(e jω ) is smooth over the width of the window and can be approximated as a constant, then π (BT) var{ ˆR x (e jω )} Rx(e jω ) Wa (e j(ω u) )du πn π = Rx(e jω ) E w N <ω<π E w = L l= (L ) w a(l) E w is the energy of the correlation window is called the variance reduction factor or variance ratio E w N J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 8 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 8 log Blackman-Tukey Confidence Intervals (BT) ˆR x (e jω ) log χ ν( α/) ν log (BT) ˆR x (e jω ) < log ν = < log ˆR (BT) (BT) ˆR x (e jω ) + log N L l= (L ) w a(l) x (e jω ), ν χ ν( α/) Example : Blackman-Tukey Bias-Variance Tradeoff Plot the bias and variance at the frequencies of the poles and zeros of the LTI system for a range of window lengths ranging from 5 5 samples. These are only approximate confidence intervals Only works when Bias is small N L N such that L/N (this is why we don t obtain the Periodogram estimate when L = N) This estimate is a linear combination of the periodogram estimate, so it is also approximately has a χ distribution, but with more degrees of freedom J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 8 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 8

22 Example : Blackman-Tukey Bias-Variance Frequencies Example : Blackman-Tukey Bias-Variance Tradeoff x 5 True ω =.785 rad/sample N:5 NA: x ω =.5 rad/sample x 9 ω =.56 rad/sample Variance Bias Variance + Bias MSE ω =.96 rad/sample Autocorrelation Window Length, L (samples) 5 Autocorrelation Window Length, L (samples) J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 85 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 86 Example : Blackman-Tukey Bias-Variance Tradeoff Example : MATLAB Code L = round(linspace(,5,5))*+; % Autocorrelation window length ef = [pi/ pi/6 *pi/.5*pi/]; % Evaluation frequencies N = 5; % Length of signal segment NA = ; % No. averages P = 5; % Padding to eliminate transient NZ = ^9; % No. zeros np = ^nextpow(*n-); % Figure out how much to pad the signal b = poly([-.8,.97*exp(j *pi/),.97*exp(-j *pi/),....97*exp(j *pi/6),.97*exp(-j *pi/6)]); % Numerator coefficients a = poly([.8,.95*exp(j**pi/),.95*exp(-j**pi/),....95*exp(j*.5*pi/),.95*exp(-j*.5*pi/)]); % Denominator coefficients b = b*sum(a)/sum(b); % Scale DC gain to [R,w] = freqz(b,a,nz); R = abs(r).^; R = R ; f = w; nl = length(l); Bias = zeros(nl,nz); Var = zeros(nl,nz); kh = :NZ; nx = P+N; J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 87 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 88

23 Example : MATLAB Code Continued for c = :nl, Rh = zeros(na,nz); no = (L(c)+)/; % Number of one-sided points to included in estimate wn = blackman(no*-+); % Create window wn = wn(:end-); % Trim MATLAB s zeros wn = wn/max(wn); % Make maximum = for c = :NA, w = randn(p+n,); x = filter(b,a,w); % System with known x = x(nx-n+:nx); % Eliminate start-up transient (make stationary) xf = fft(x,np); % Calculate FFT of x r = ifft(xf.*conj(xf)); % Calculate biased autocorrelation r = real(r(:nx))/n; % Eliminate superfluous zeros and nearly zero imaginar r = [r(no:-:);r(:no)]; % Make two-sided rh = abs(fft(r.*wn,*nz)); % Window autocorrelation and calculate the estimate Rh (c,:) = rh(kh). ; Rha = mean(rh); % Average Bias(c,:) = R-Rha; Var (c,:) = mean((rh - ones(na,)*rha).^); MSE (c,:) = mean((rh - ones(na,)*r ).^); drawnow; Example : MATLAB Code Continued figure; for c=:length(ef), [jnk,id] = min(abs(ef(c)-f)); subplot(,,c); h = plot(l,var(:,id),l,bias(:,id).^,l,var(:,id)+bias(:,id).^,l,mse(:,id)); set(h(), LineWidth,); set(h(), LineWidth,); xlim([min(l) max(l)]); ylim([.5*max(mse(:,id))]); box off; ylabel(sprintf( \\omega = %5.f rad/sample,ef(c))); if c==, xlabel( Autocorrelation Window Length, L (samples) ); elseif c==, title(sprintf( N:%d NA:%d,N,NA)); elseif c==, xlabel( Autocorrelation Window Length, L (samples) ); legend( Variance, Bias^, Variance + Bias^, MSE ); figure; smse = zeros(size(mse)); for c=:length(f), smse(:,c) = MSE(:,c)/max(MSE(:,c)); J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 89 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 9 h = imagesc(f,l,smse,[ ]); [jnk,id] = min(smse); hold on; plot(f,l(id), w. ); hold off; Periodogram Averaging ˆR x (PA) (e jω ) K ˆR x (e jω ) K i= Consider K IID random variables The variance of the mean is /K times the variance of the random variables If we had different independent realizations of the process we could average them and obtain a similar reduction in the variance of the In most situations, only a single realization is available Can subdivide the record into K possibly overlapping segments of length L J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 9 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 9

24 ˆR x (WB) (e jω k ) K K L i= Welch-Bartlett Spectral Estimation L v i (n)e jωn = KL n= K i= V i (e jω ) where v(n) x(n)w(n). Where w(n) is called the data window Window does not need to be positive definite Window trades sidelobe leakage (ripples) for mainlobe width (blurring) If no overlap between windows, called the Bartlett estimate If overlap, called Welch s method Example : Welch-Bartlett Spectral Estimation Generate the Welch-Bartlett estimated for a signal from a known LTI system with two sets of complex conjugate poles close to one another and near the unit circle. Repeat for various segment lengths. J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 9 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 9 x 5 Example : Welch-Bartlett Estimate N:5 L: OL:5% K:99 NA: :95% True x 5 Example : Welch-Bartlett Estimate N:5 L:5 OL:5% K:9 NA: :95% True J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 95 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 96

25 x 5 Example : Welch-Bartlett Estimate N:5 L: OL:5% K:9 NA: :95% True x 5 Example : Welch-Bartlett Estimate N:5 L: OL:5% K: NA: :95% True J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 97 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 98 Example : MATLAB Code L = [,5,,]; % Segment lengths O =.5; % Overlap M = 5; % Padding to eliminate transient N = 5; % Length of signal segment NA = ; % No. averages cb = 95; % NZ = ; b = poly([-.8,.97*exp(j *pi/),.97*exp(-j *pi/),....97*exp(j *pi/6),.97*exp(-j *pi/6)]); % Numerator coefficients a = poly([.8,.95*exp(j**pi/),.95*exp(-j**pi/),....95*exp(j*.5*pi/),.95*exp(-j*.5*pi/)]); % Denominator coefficients b = b*sum(a)/sum(b); % Scale DC gain to [R,w] = freqz(b,a,nz); R = abs(r).^; f = w/(*pi); Example : MATLAB Code Continued for c = :length(l), K = floor((n-o*l(c))./(l(c)-o*l(c))); % No. segments ss = floor(l(c)-o*l(c)); % Step size nx = M+N; Rh = zeros(na,nz/+); kh = :NZ/+; fh = (kh-)/nz; for c = :NA, w = randn(m+n,); x = filter(b,a,w); % System with known x = x(nx-n+:nx); % Eliminate start-up transient (make stationary) rh = zeros(nz,); for c = :K, id = (:L(c)) + (c-)*ss; rh = rh + abs(fft(x(id),nz)).^; rh = rh/(k*l(c)); Rh(c,:) = rh(kh). ; Rha = mean(rh); % Average Rhu = prctile(rh,-(-cb)/); % Upper confidence band Rhl = prctile(rh, (-cb)/); % Lower confidence band J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 99 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver..

26 Example : MATLAB Code Continued figure; subplot(,,); h = plot(fh,rh(,:), g,f,r, r,fh,rhl, b,fh,rhu, b,fh,rha, k ); set(h(), LineWidth,.5); set(h(), LineWidth,.); set(h(:), LineWidth,.5); set(h(5), LineWidth,.8); ylabel( ); title(sprintf( N:%d L:%d OL:%d%% K:%d NA:%d :%d%%,n,l(c),o* xlim([.5]); ylim([ max(r)*.5]); legend(h([ 5]),, True,, Average Estimat subplot(,,); h = semilogy(fh,rh(,:), g,f,r, r,fh,rhl, b,fh,rhu, b,fh,rha, k ); set(h(), LineWidth,.5); set(h(), LineWidth,.); set(h(:), LineWidth,.5); set(h(5), LineWidth,.8); ylabel( ); xlabel( ); xlim([.5]); ylim([.*min(r) max(r)*5]); [ ] E ˆR(WB) x (e jω k ) Welch-Bartlett Mean = K = E K i= [ L [ E V i (e jω ) ] L Vi (e jω ) ] = R x (e jω ) L R w(e jω ) = πl π π R x (e jω )R w (e j(ω θ) )dθ As with the periodogram, if the window is normalized such that L w (n) = π R w (e jω )dω = L π π n= then the estimate is asymptotically unbiased J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. Welch-Bartlett Variance (WB) var{ ˆR x (e jω k )} K var{ ˆR x (e jω k )} (WB) var{ ˆR x (e jω k )} K Assumes segments are independent Not true, so estimate is optimistic Book fails to mention this ˆR x (WB) (e jω k ) K K L i= Welch-Bartlett Concepts L v i (n)e jωn n= = KL K i= Vi (e jω ) Thus, the user can once again trade variance for bias If N = KL is fixed, then can reduce variance at the expense of increased variance by increasing K and decreasing L The variance is insensitive to the shape of the window Increasing overlap up to 5% reduces variance without substantially impacting the bias Further overlap increases the computational requirements, but does not help bias or variance Confidence intervals listed in the text J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. J. McNames Portland State University ECE 58/68 Spectral Estimation Ver..

27 Welch-Bartlett Comments ˆR (WB) x (e jω k )= KL K i= V i (e jω ) Consider the case when the overlap is L samples This would reduce variance the most In this case it becomes equivalent to periodogram smoothing [, pp ]! Bartlett originally proposed averaging periodograms of separate segments, but showed that this was essentially equivalent to the Blackman-Tukey method with a Bartlett autocorrelation window [, pp. 9-] When the overlap is less than L It is only approximately equivalent to periodogram smoothing It requires L FFTs Variance is increased Summary We discussed several different nonparametric methods of spectral estimation The natural estimate was the periodogram This estimate has unacceptable variance for most applications There are two fundamental approaches to reduce variance Periodogram smoothing Periodogram averaging They have comparable performance and ultimately are approximately equivalent Each enables the user to control the bias-variance tradeoff Typically the smoothing parameter is chosen to minimize bias Only approximate confidence intervals are available for each J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 5 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 6 MATLAB Resources The MATLAB implementation of these functions is constantly changing (improving?) The documentation is pretty good Open up the help window (helpwin) Signal Processing Toolbox Statistical Signal Processing Most of the techniques we have discussed are implemented Interface is not user-friendly References [] M. B. Priestley. Spectral Analysis and Time Series. Academic Press, 98. J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 7 J. McNames Portland State University ECE 58/68 Spectral Estimation Ver.. 8