System Identification

Transcription

1 System Identification Lecture 4: Transfer function averaging and smoothing Roy Smith Averaging Multiple estimates Multiple experiments: u r pk, y r pk, r,..., R, and k,..., K. Multiple estimates (ETFE): Ĝ r pe jω n Y rpe jω n U r pe jω n (drop the N from the Y N pe jω n notation) Averaging to improve the estimate. Ĝpe jω n α r Ĝ r pe jω n, r How to choose α r? with α r. r The average is calculated with α r {R

2 Averaging Optimal weighted average If the variance of Ĝrpe jω n is σrpe 2 jω n (with uncorrelated errors and identical means), then, jω variance Ĝpe n variance α r pe jω n Ĝrpe jω n This is minimized by r αrσ 2 rpe 2 jω n r α r pe jω n {σ2 rpe jω n. {σrpe 2 jω n r If variance Ĝr pe jω n φ vpe jω n N U rpe jω n 2 then α r pe jω n U rpe jω n 2. U r pe jω n 2 r Averaging Variance reduction Best result: ˇˇU r pe jω n ˇˇ is the same for all r,..., R. This gives, jω variance Ĝpe n variance Ĝr pe jω n. R If the estimates are biased we will not get as much variance reduction. (The method of splitting the data record and averaging the periodograms is attributed to Bartlett (948, 9).)

3 Averaging example (noisy data) Splitting the data record: K 72, R 3, N 24, upk ypk y pk y 2 pk y 3 pk Averaging example (noisy data) Estimates: Ĝ r pe jω n, r, 2, 3 and the weighted average, Ĝavgpe jω n. Magnitude Gpe jω Ĝ avg pe jω n Ĝ 3 pe jω n Ĝ pe jω n π log ω (rad/sample) Ĝ 2 pe jω n

4 Averaging example (noisy data) Estimates: Ĝ r pe jω n, r, 2, 3 and the weighted average, Ĝavgpe jω n. Errors: E r pe jω n Gpe jω n Ĝrpe jω n. Magnitude E 2 pe jω n E 3 pe jω n E pe jω n π log ω (rad/sample). E avg pe jω n Averaging with periodic signals Splitting the data record: K 72, N 24, R 3, upk has period M 24. upk ypk y pk y 2 pk y 3 pk (not really periodic: upk for k ă )

5 Averaging with periodic signals Estimates: Ĝ r pe jω n and Ĝavgpe jω n Ĝ2 pe jω n ` Ĝ3pe jω n {2 Magnitude Gpe jω Ĝ avg pe jω n Ĝ 2 pe jω n Ĝ 3 pe jω n π log ω (rad/sample) Averaging with periodic signals Estimates: Ĝ r pe jω n and Ĝavgpe jω n Ĝ2 pe jω n ` Ĝ3pe jω n {2 Errors: E r pe jω n Gpe jω n Ĝrpe jω n. Magnitude π log ω (rad/sample). E 2 pe jω n E 3 pe jω n E avg pe jω n

6 Averaging with periodic signals Estimates: Ĝ r pe jω n and Ĝavgpe jω n Ĝ2 pe jω n ` Ĝ3pe jω n {2 Errors: E r pe jω n Gpe jω n Ĝrpe jω n. Magnitude E avg pe jω n (non-periodic) π log ω (rad/sample). E 2 pe jω n E 3 pe jω n E avg pe jω n Bias-variance trade-offs in data record splitting Bartlett s procedure Divide a (long) data record into smaller parts for averaging. Data: tupk, ypku k,..., K with K very large. Choose R records, and calculation a length: N, NR ď K. u r pn uprn ` n, r,..., R, n,..., N. And average the resulting estimates: Gpe jω n R ÿ r α r Ĝ r pe jω n R ÿ r α r Ŷ r pe jω n Û r pe jω n. Bias and variance effects As R increases: The number of points calculated, N, decreases (so NR ď K); The variance decreases (by up to {R); The bias increases (due to non-periodicity transients)

7 Bias-variance trade-offs in data record splitting Mean-suare error MSE bias 2 ` variance. Transient bias grows linearly with the number of data splits. Variance decays with a rate of up to /(number of averages). Optimal bias-variance trade-off Error MSE Bias error Variance error 2 number of records, R Smoothing transfer function estimates What if we have no option of running periodic input experiments? The system will not tolerate periodic inputs; or The data has already been taken and given to us. More data just gives more freuencies (with the same error variance). Exploit the assumed smoothness of the underlying system Gpe jω is assumed to be a low order (smooth) system. E! ) pgpe jω n ĜN pe jω n pgpe jω s ĜN pe jω s ÝÑ pn s, (asymptotically at least)

8 Smoothing the ETFE Smooth transfer function assumption Assume the true system to be close to constant for a range of freuencies. Gpe jω n`r «Gpe jω n for r,, 2,..., R. Smooth minimum variance estimate The minimum variance smoothed estimate is, G N pe jω n α r Ĝ N pe jω n`r r R, α r α r r R N ˇ ˇU N pe jω n`r ˇˇ2. φ v pe jω n`r (Here we smooth over 2R ` points) Smoothing the ETFE If N is large (many closely spaced freuencies), the summation can be approximated by an integral, G N pe jω n α r Ĝ N pe jω n`r r R r R α r «ż ωn`r ω n r ż ωn`r ω n r αpe jξ ĜN pe jξ dξ αpe jξ dξ, with αpe jξ N ˇ ˇUN pe jξ ˇˇ2. φ v pe jξ

9 Smoothing the ETFE Smoothing window G N pe jω n ż π W γ pe jpξ ωn αpe jξ ĜN pe jξ dξ 2π π ż π, W γ pe jpξ ωn αpe jξ dξ 2π π with αpe jξ N ˇ ˇU N pe jξ ˇˇ2. φ v pe jξ The {2π scaling will make it easier to derived time-domain windows later Smoothing the ETFE Assumptions on φ v pe jω Assume φ v pe jω is also a smooth (and flat) function of freuency. ż π W γ pe jpξ ωn 2π ˇ φ v pe jξ φ v pe jω n ˇ dξ «. Then use, π αpe jξ N ˇ ˇUN pe jξ ˇˇ2, φ v pe jω n to get, ż π W γ pe jpξ ωn ˇ ˇU N pe jξ ˇ ˇ2 Ĝ N pe jξ dξ G N pe jω n 2π π N ż π W γ pe jpξ ωn. ˇ ˇU N pe jξ ˇ ˇ2 dξ 2π π N

10 Weighting functions Typical window (Hann) as a function of the width parameter, γ. W γ pe jωn 2 γ 2 N 26 γ γ γ π π{2 π{2 π Discrete freuency: ω Weighting functions Freuency smoothing window characteristics: width (specified by γ parameter) The wider the freuency window (i.e. decreasing γ)... the more adjacent freuencies included in the smoothed estimate, the smoother the result, the lower the noise induced variance, the higher the bias

11 Weighting functions Window characteristics: shape Some common choices: Bartlett: W γ pe jω γ ˆ 2 sin γω{2 sin ω{2 Hann: W γ pe jω D 2 γpω ` D 4 γpω π{γ ` D 4 γpω ` π{γ sin ωpγ `. where D γ pω sin ω{2 Others include: Hamming, Parzen, Kaiser,... The differences are mostly in the leakage properties of the energy to adjacent freuencies. And the ability to resolve close freuency peaks Weighting functions Example freuency domain windows. W γ pe jωn γ N 26 Welch Hann Hamming Bartlett π π{2 π{2 π Discrete freuency: ω

12 ETFE smoothing example: Matlab calculations U = fft(u); Y = fft(y); Gest = Y./U; Gs = *Gest; [omega,wg] = WfHann(gamma,N); zidx = find(omega==); omega = [omega(zidx:n);omega(:zidx-)]; Wg = [Wg(zidx:N);Wg(:zidx-)]; a = U.*conj(U); % calculate N point FFTs % ETFE estimate % smoothed estimate % window (centered) % shift to start at zero % freuency grid % variance weighting for wn = :N, Wnorm = ; for xi = :N, widx = mod(xi-wn,n)+; Gs(wn) = Gs(wn) +... Wg(widx) * Gest(xi) * a(xi); Wnorm = Wnorm + Wg(widx) * a(xi); end Gs(wn) = Gs(wn)/Wnorm; end % reset normalisation % wrap window index % weight normalisation Window properties Properties and characteristic values of window functions 2π ż π π ż π π W γ pe jξ dξ (Normalised) ξw γ pe jξ dξ ( Even sort of) Mpγ : ż π W pγ : 2π π ż π ξ 2 W γ pe jξ dξ (bias effect) π Wγ 2 pe jξ dξ (variance effect) Bartlett: Mpγ 2.78 γ, W pγ «.67γ (for γ ą ) Hamming: Mpγ π2 2γ 2, W pγ «.7γ (for γ ą )

13 Smoothed estimate properties Asymptotic bias properties!! )) E Gpe jω n E Gpe jω n Mpγ! ) E Gpe jω n Gpe jω n ˆ 2 G2 pe jω n ` G pe jω n φ upe jωn φ u pe jω n Increasing γ: makes the freuency window narrower; averages over fewer freuency values; makes M pγ smaller; and ` OpC 3 pγ loooomoooon ÝÑ as γ ÝÑ 8 reduces the bias of the smoothed estimate, Gpe jω n. ` O {? N looooomooooon ÝÑ as N ÝÑ Smoothed estimate properties Asymptotic variance properties "! ) 2* E Gpe jω n E Gpe jω n N W pγ φ vpe jω n φ u pe jω n Increasing γ: makes the freuency window narrower; averages over fewer freuency values; makes W pγ larger; and increases the variance of the smoothed estimate, Gpe jω n. ` o ` W pγ{n loooooomoooooon ÝÑ as γ ÝÑ 8 N ÝÑ 8 γ{n ÝÑ

14 Smoothed estimate properties Asymptotic MSE properties jω E "ˇˇˇ Gpe n Gpe jω n ˇ ˇ2* «M 2 pγ F pe jω n 2 ` N W pγ φ vpe jωn, φ u pe jω n where F pe jω n 2 G2 pe jω n ` G pe jω n φ upe jω n φ u pe jω n If Mpγ M{γ 2 and W pγ W γ then MSE is minimised by, and γ optimal ˆ 4M 2 F pe jω n 2 φ u pe jω n { N W {. φ v pe jω n MSE at γ optimal «CN 4{ Bibliography Windowing and ETFE smoothing P. Stoica & R. Moses, Introduction to Spectral Analysis (see Chapters and 2), Prentice-Hall, 997. Lennart Ljung, System Identification; Theory for the User, (see Section 6.4) Prentice-Hall, 2nd Ed., 999. M.S. Bartlett, Smoothing Periodograms from Time-Series with Continuous Spectra, Nature, vol. 6(496), pp , 948. M.S. Bartlett, Periodogram analysis and continuous spectra, Biometrika, vol. 37, pp. 6,