Autoregressive (AR) Modelling

Transcription

1 Autoregressive (AR) Modelling A. Uses of AR Modelling () Alications (a) Seech recognition and coding (storage) (b) System identification (c) Modelling and recognition of sonar, radar, geohysical signals (d) Sectral analysis (2) Reasons Why it is used (rather than MA or ARMA) (a) Each tye of rocess (MA, AR, ARMA) can be converted to the other tyes. (b) An AR model can be found by solving linear set of equations, unlike the others. (c) An AR sectrum, calculate from a signal of length N@T, can have much better frequency resolution than the /(N@T) of classical estimators. (d) Under certain circumstances, an AR model for P ss (w) can maximize entroy. (e) An AR model can have far fewer coefficients than the corresonding MA model, ust as a Butterworth IIR filter has far fewer coefficients than an FIR filter of similar erformance. B. Maximum Entroy Sectral Estimation In this section, we resent the classical ustification for using AR models in sectral analysis. This material is Robinson s [] exlanation of Burg s work [2]. For a given P ss (w), there exist an infinite number of ensemble sequences s(n). Given +2 coefficients ss(m) for - # m #, there exist an infinite number of extensions of the autocorrelation ^ ( ss (m) for *m* > ) and therefore an infinite number of estimates P ss( ω) of P ss (w). The maximum entroy method of sectral analysis (MEM) involves choosing the extension of ss(m) which maximizes the entroy (and randomness) of s(n). The resulting MEM estimate is the DTFT of the truncate autocorrelation lus its infinite length extension.. Assumtions (B) The random rocess s(n) is Gaussian and stationary. (B2). We know the ensemble autocorrelation ss (m) exactly for *m* #. (This is rarely a good, ractical assumtion) Also, the PSD estimate of P ss (w) must satisfy the correlation matching constraint; ss (m) ' m & ( )@e m d for *m*#

2 (B3). The entroy of s(n) is the average information er samle, calculated for ositive, log integrable PSD s as H ' m & ln( ( ))@d () (B4). We want to ick the extended ss(m) values, those for *m* >, to maximize H. ^ (B5). Since P ss (w) is assumed ositive we can find the DTFT of its inverse as ( ) ' 4 n'&4 (n)@e n (2) 2. Maximization Process MH M ss (m) ' m & Mln( ( M ss (m) Now, maximize H with resect to the unknown ss(m) s for *m* > as m d ' 0 m& (3) ( ) Substituting (2) into (3) we get (&m) ' 0 for *m*> (4) Combining (2) and (4) we get the MEM ower sectral density estimate as ( ) ' n'& (n)@e m (5) ^ But /P ss (w) can be factored as n'& (n)@e & n ' )A(e & ), 2

3 A(e ) ' n ' 0 a(n)e & n, a(0) =, (6) ( ) ' 2 A(e & )A(e ) (7) Given ss(m) for *m* #, the MEM estimate of P ss (w) uses an AR model of order ; s(n) ' e(n) & a(k)s(n&k) (8) where e(n) denotes white Gaussian noise with variance 2. Using, H(z) ' /A(z) (9a) h(0) ' (9b) we multily both sides of (8) by s(n+m) and take the exected value, yielding ss (m) ' & & a(k) ss (m&k) for m >0, (0a) ss (&m) for m <0 (0c) a(k) ss (m&k)% 2 for m ' 0, (0b) These are the AR Yule-Walker normal equations for a(n). We use them to find the a(n) s as follows. First, we aly the Toelitz recursion to the equations in unknowns reresented by (0a), using (0c) whenever necessary. Then we solve (0b) for Basis for High Resolution of MEM The MEM PSD estimate of equation (7) may be rewritten as ( ) ' 4 m'&4 ss (m)e & m () where ss(m) satisfies the correlation matching constraint of (B2) for *m* # and equals the extraolated values of (3) above for *m* >. Therefore, () is the DTFT of an infinite length, un-windowed, auto- 3

4 correlation sequence. () can therefore have much higher resolution than classical eriodogram-based aroaches. 4. Summary of MEM Estimation Process () Given the 2+ exact autocorrelation coefficients as in (B2), we solve the Yule-Walker equations for the unknown values of a(n) and the inut white noise variance 2, using the Toelitz recursion or Levinson recursion. (2) We form P ss ( ^ω ) using (7). (3) If ss(m) is desired for *m* >, they are obtained from the Yule-Walker difference equations, (0a) C. AR Modelling From Data In the revious section we see how an AR model can be ustified for sectral analysis, when we have a few samles of the ensemble autocorrelation of a WSS random rocess. In this section we show how to find the AR model from a finite-length segment of the WSS random rocess, when the ensemble autocorrelation is not known. The model can then be used in sectral analysis as in the revious section, in seech recognition, in linear redictive coding for seech, etc. The ower sectra formed from the AR models in this section are not true MEM sectra, even though they are sometimes called that in the literature. This material is from [3] and [4]. There are many different ways to find the AR model from the data segment. We describe the easiest aroach (autocorrelation) and the best aroach (modified autocovariance).. Basic Model and Assumtions The basic assumtions are ; (C) We have samles s(n), for 0 # n # N-, where s(n) is WSS. (C2) We don t have any samles of the ensemble autocorrelation function. (C3) We want to find a good set of AR model coefficients for high resolution sectral estimation, linear redictive coding (LPC), or for some other urose. 2. AR Difference Equations and Transfer Function The forward and backward AR difference equations are resectively s(n) ' e(n)& a(k)s(n&k), for 0 # n # N& (2a) 4

5 s(n) ' e(n)& a(k)s(n%k), for N& # n # 0 (2b) where e(n) denotes white noise of variance 2 and is the AR filter order. Clearly a(0) =. (3) The AR filter transfer function is H(z) ' % a(k)z &k (4) 3. AR Modelling Procedure In order to find the a(k) s, we first construct forward and backward linear redictors as s f (n) '& a(k)s(n&k), (5a) s b (n) '& a(k)s(n%k) (5b) The arguments for s in the forward and backward linear redictors are (n-) to n and n to (n+). Before measuring the rediction errors, we shift the indexes for the backward redictor by as s b (n&) '& a(k)s(n%k&) (5c) This shifts the backward redictor s s arguments to be (n-) to n. The forward and backward rediction errors are resectively r f (n) ' s(n)&s f (n) ' a(k)s(n&k), r b (n) ' s(n&)&s b (n&) 5

6 ' a(k)s(n%k&), he forward and backward error energies are n2 V f (a) ' [r f (n)] 2, V b (a) ' n2 [r b (n)] 2, In order to attemt to average out differences between the forward and backward models, we will minimize the combined rediction error function V(a) ' V f (a)%v b (a) ' n2 [r f (n)] 2 %[r b (n)] 2, (6) with resect to the unknowns a() through a(). There are two imortant cases, deending on how the limits n and n2 are chosen. 4. Autocorrelation Method For the autocorrelation method, we use the largest ossible range for the limits, which is (n,n2) = (0,N-+). Minimizing V(a) wrt a(m), we get n2 MV(a) Ma(m) ' 2@ [ a(k)s(n&k)]s(n&m) % [ a(k)s(n%k&)]s(n%m&) ' 0 ' 4@ a(k)@ ss (m&k) ' 0 for # m # (7) Using (3), the mth equation from (7) is a(k)@ ss (m&k) '& ss (m), for # m # (8) The rediction error energy is the minimize value of V f (a), denoted by. It is calculate using (6) and (7) as 6

7 ' a(k) n ' 0 ss (k&n) ' a(n)@ ss (&n) (9) n ' 0 Combining (7) and (9) we get a new set of equations, a(k)@ ss (m&k) (m) for 0 # m # (20) Since is unknown before the a(m) s are found, (20) reresents (+) equations in (+) unknowns. Several comments can be made about the autocorrelation aroach. () The time-average autocorrelations, ss(m), generated by the forward and backward error energy derivatives are identical. Therefore, for the autocorrelation aroach, it is sufficient to minimize V f (a) or V b (a) alone. (2) Note that a(0) is known to be, so the equations above are not homogeneous. (3) The system of equations in (8) is Toelitz and can be solved using the Toelitz recursion. Alternately, the equations in (20) can be solved using the Levinson recursion. (4) The time autocorrelations ss(m) are biased. (5) The AR sectrum is generated from the a(n) s using (7). (6) The AR filter H(z) is always stable. (7) Note that if the a(k) s are found with comlete accuracy, then the rediction error will be white. 5. Modified Autocovariance Method For the modified autocovariance method, we use the largest ossible range for the limits, such that no zero-value terms can occur in V(a). This range is (n,n2) = (,N-). Minimizing V(a) wrt a, we get n2 MV(a) Ma(m) ' 2@ [ a(k)s(n&k)]s(n&m) % [ a(k)s(n%k&)]s(n%m&) ' 0 ' 2@ a(k)@ N& n ' [s(n&k)s(n&m) % s(n%k&)s(n%m&)] ' 0 ' 2@ a(k)[ ss (&k,&m) % ss (k&,m&)] ' 0 (2) 7

8 here ss (i,) ' N& n ' s(n%i)s(n%) (22) Combining the two autocorrelations in (2) as C(k,m) ' ss (&k,&m) % ss (k&,m&) (23) the mth equation in (2) is a(k)@c(k,m) ' C(0,m) (24) Several comments can be made about the modified autocovariance aroach. () The time-average autocorrelations, ss(i,), generated by the forward and backward error energy derivatives are not identical. (2) Note that a(0) is known to be, so the equations above are not homogeneous. (3) From (2) the system of equations is symmetric and ersymmetric but not Toelitz. It can be solved efficiently, using an algorithm given in Aendix 8.D. of [4]. (4) The time autocorrelations ss(i,) are not biased. (5) The AR sectrum is generated from the a(n) s using (7). (6) The AR filter H(z) is not guaranteed to be stable. (7) Note that if the a(k) s are found with comlete accuracy, then the rediction error will be white. D. References [] Enders A. Robinson, "A Historical Persective of Sectrum Estimation," in Proceedings of the IEEE, vol. 70, no. 9, Setember 982. (See section XV.) [2] J.P. Burg, "Maximum Entroy Sectral Analysis," Proceedings of the 37th Meeting of the Society of Exloration Geohysicists, Oklahoma City, Oklahoma, 967. [3] R.A. Roberts and C.T. Mullis, "Digital Signal Processing," Addison-Wesley Publishing Comany, Reading Mass., 987. [4] S. Lawrence Marle Jr., Digital Sectral Analysis With Alications," Prentice-Hall, Inc., Englewood Cliffs, New Jersey,