Recursive Least-Squares Fixed-Interval Smoother Using Covariance Information based on Innovation Approach in Linear Continuous Stochastic Systems

Transcription

1 8 Froniers in Signal Processing, Vol. 1, No. 1, July 217 hps://dx.doi.org/1.2266/fsp Recursive Leas-Squares Fixed-Inerval Smooher Using Covariance Informaion based on Innovaion Approach in Linear Coninuous Sochasic Sysems Seiichi Nakamori Professor Emerius, Kagoshima Universiy, Kourimoo, Kagoshima, Japan Absrac This paper newly proposes he recursive leas-squares (RLS) fixed-inerval smooher and filer, based on he innovaion heory, in linear coninuous-ime sochasic sysems. I is assumed ha he signal is observed wih addiive whie noise and he signal process is uncorrelaed wih he observaion noise. I is a characerisic ha he esimaors use he covariance funcion of he signal, in he form of he semi-degenerae kernel, and he variance of he observaion noise. Also, he algorihm for he esimaion error variance funcion of he RLS fixed-inerval smooher is developed o validae he sabiliy of he proposed fixed-inerval smooher. The numerical simulaion example shows ha he esimaion accuracy of he proposed fixed-inerval smooher is superior o ha of he exising fixed-inerval smooher using he covariance informaion. Keywords: RLS Wiener esimaors, fixed-inerval smooher, innovaion approach, covariance informaion, linear coninuous-ime sysems. 1 Inroducion Hihero, here have been a lo of researches on he esimaion problems using he sae-space model [1], [2]. Recursive leas-squares (RLS) Wiener fixed-poin smooher [3] is also sudied in linear coninuousime sochasic sysems. I is ineresing ha he RLS Wiener esimaors use he informaion of he auo-variance funcion of he sae vecor besides he observaion marix and he sysem marix in he sae-space model. In [4], [5], recursive fixed-lag smooher, using he covariance informaion, is sudied in linear discreeime sysems. I is a characerisic ha he auo-covariance funcion of he signal is expressed in he semi-degenerae kernel form. In [6], he RLS Wiener fixed-lag smooher is developed in linear discree-ime sochasic sysems. In [7], assuming ha he signal process is modelled in erms of he auoregressive model, he RLS Wiener fixed-lag smooher is proposed in linear discree-ime sochasic sysems. In [8], in erms of he sae-space models, he smoohing algorihms are developed based on he innovaion heory. In [9], he opimal fixed-lag smooher and is error variance marix are derived for he coninuous linear sysems wih he whie measuremen noise. In [1], he RLS fixed-inerval smooher, using he covariance informaion, is designed based on he innovaion approach in linear coninuous-ime sochasic sysems. Here, i is assumed ha he observaion noise is whie. In [11], wih he sae-space model, he RLS fixed-inerval smooher is proposed based on he innovaion approach. In [12], based on he innovaion heory, he RLS fixed-inerval smooher, using he covariance informaion, is proposed in linear discree-ime sochasic sysems. Also, in [13], based on he innovaion heory, he RLS Wiener fixed-inerval smoohing algorihm is presened for he measuremen wih colored observaion noise. This paper, by correcing he expression of he impulse response funcion in [1], newly proposes he RLS fixed-inerval smooher and filer, based on he innovaion heory, in linear coninuous-ime sochasic sysems. I is assumed ha he signal is observed wih addiive whie noise and he signal process is uncorrelaed wih he observaion noise. I is a characerisic ha he esimaors use he covariance funcion of he signal, in he form of he semi-degenerae kernel, and he variance of he observaion noise. The fixed-inerval smoohing and filering algorihms are presened in Theorem 1. The correced poin in he expression of he impulse response funcion from ha in he fixed-inerval in [1] is clarified. The deails are discussed afer he proof of Theorem 1. The fixed-inerval smoohing algorihm, proposed Copyrigh 217 Isaac Scienific Publishing

2 Froniers in Signal Processing, Vol. 1, No. 1, July in his paper, is regarded as he coninuous-ime version of he fixed-inerval smooher [12] in linear discree-ime sochasic sysems. Also, he algorihm for he esimaion error variance funcion of he RLS fixed-inerval smooher is developed o validae he sabiliy of he proposed fixed-inerval smooher. In he numerical example of secion 5, he esimaion accuracy of he proposed fixed-inerval smooher is compared wih ha in [1]. 2 Fixed-inerval smoohing problem Le an observaion equaion be given by y () = z () + v () (1) in linear coninuous-ime sochasic sysems, where z() is an m 1 signal vecor and v() is whie observaion noise. I is assumed ha he signal and he observaion noise are muually independen sochasic processes wih zero means. Le he auo-covariance funcion of v() be given by E[v()v T (s)] = Rδ( s), R >. (2) Here, δ( ) denoes he Dirac δ funcion. Le K(, s) represen he auo-covariance funcion of he signal process and le K(, s) be expressed in he semi-degenerae kernel form [9], [1] of K(, s) = { A()B T (s), s, B()A T (s), s. (3) Here, A() and B(s) are bounded m p marices. Le a fixed-inerval smoohing esimae ẑ( T ) of z() be given by ẑ ( T ) = T g (, s) υ (s) ds (4) as a linear inegral ransformaion of he innovaion process υ (s) = y (s) ẑ (s s), s T, where g(, s), ẑ(s s) and T are referred o as he impulse response funcion, he filering esimae of z(s) and he fixed inerval respecively. The impulse response funcion, which minimizes he mean-square value of he fixed-inerval smoohing error z() ẑ( T ), J = E[ z() ẑ( T ) 2 ], (5) saisfies by an orhogonal projecion lemma [14] E[z()υ T (s)] = T g (, τ) E[υ(τ)υT (s)]dτ = g(, s)r (6) z() ẑ( T ) υ(s), s, T. (7) Here, denoes he noaion of he orhogonaliy. From (1), (2) and (6), he impulse response funcion, for he linear leas-squares fixed-inerval smoohing esimaes, saisfies g (, s) R = E[z () ( y (s) ẑ (s s)) ] T = K (, s) E [ z () υ T (τ) ] g T (s, τ) dτ = K (, s) (8) s g (, τ) RgT (s, τ) dτ. Based on he preliminary formulaion on he fixed-inerval smoohing problem, he fixed-inerval smoohing and filering algorihms are proposed in Theorem 1. Copyrigh 217 Isaac Scienific Publishing

3 1 Froniers in Signal Processing, Vol. 1, No. 1, July Fixed-inerval smoohing and filering algorihms The filering and fixed-inerval smoohing algorihms based on he innovaion heory is presened in Theorem 1. The algorihms are derived wih he invarian imbedding mehod [1] in linear coninuous sochasic sysem. Theorem 1 Le he observaion equaion be given by (1). Le he auo-covariance funcion of he signal z() be given by (3) in he semi-degenerae kernel form in linear coninuous-ime sochasic sysems. Then he RLS algorihms for he fixed-inerval smoohing esimae ẑ( T ) and he filering esimae ẑ ( ) of z() consis of (9)-(15). Fixed-inerval smoohing esimae ẑ( T ) of z(): de ( T ) d ẑ ( T ) = ẑ ( ) + (B () A () q ()) e( T ) (9) = J (, ) (y () ẑ ( )) + J (, ) RJ T () e( T ), e (T T) = (1) Filering esimae ẑ( ) of z(): J (, ) = A T () R 1 (11) ẑ ( ) = A () e () (12) de () =J () ν () =J () (y () ẑ ( )), e () = (13) d dq () d = J () RJ T (), q () = (14) J () = (B T () q () A T ())R 1 (15) Proof. The expression for he fixed-inerval smoohing esimae in (4) is wrien as ẑ( T ) = T g(, s)υ(s)ds + g(, s)υ(s)ds. (16) The firs erm on he righ hand side of (16) represens he filering esimae ẑ( ) of z() and he second erm he correcion quaniy of he fixed-inerval smoohing esimae o he filering esimae. A firs, for s, from (3) and (8), he impulse response funcion g(, s) for he filering esimae ẑ( ) saisfies g (, s) R = A()B T (s) Inroducing an auxiliary funcion J (s), which saisfies g(, τ)r(τ)g T (s, τ) dτ. (17) we ge J (s)r = B T (s) J (τ)r(τ)g T (s, τ) dτ, (18) Subsiuing (19) ino (18) and inroducing a funcion g(, s) = A()J (s), s T. (19) Copyrigh 217 Isaac Scienific Publishing

4 Froniers in Signal Processing, Vol. 1, No. 1, July we have q (s) = J (τ) RJ T (τ) dτ, (2) J (s) R = B T (s) Differeniaing (2) wih respec o s, we obain J (τ) R (τ) J T (τ) A T (s) dτ = B T (s) q (s) A T (s). (21) Le us inroduce a funcion dq (s) ds = J (s) RJ T (s), q () =. (22) e () = From (16) and (19), he filering esimae ẑ(, ) is given by ẑ (, ) = g (, τ) ν (τ) dτ = Differeniaing (23) wih respec o, we have J (τ) ν (τ) dτ. (23) A () J (τ)ν (τ) dτ = A () e (). (24) de () =J () ν () =J () (y () ẑ (, )), e () =. (25) d Secondly, for s T, from (3), (8), (19) and (2), g(, s) saisfies g (, s) R = B()A T (s) g (, τ) RgT (s, τ) dτ g (, τ) RgT (s, τ) dτ = B()A T (s) A()J (τ) RJ T (τ) A T (τ)dτ g (, τ) RJ T (τ) A T (τ) dτ = (B() A()q ())A T (s) g (, τ) RJ T (τ) A T (τ) dτ. (26) Inroducing an auxiliary funcion J (, s), which saisfies we obain J (, s) R = A T (s) J(, τ)rj T (τ) dτa T (s), (27) g (, s) = (B () A () q ()) J (, s), s T. (28) From (16) and (28), we have an expression for he fixed-inerval smoohing esimae as follows. ẑ ( T ) = ẑ ( ) + T g (, s) υ (s) ds = ẑ ( ) + (B() A()q ()) T J(, s) υ (s) ds = ẑ ( ) + (B() A()q ())e( T ) Here, we inroduced he funcion e( T ) expressed by (29) Differeniaing (27) wih respec o, we have T e ( T ) = J (, s)υ (s) ds. (3) Copyrigh 217 Isaac Scienific Publishing

5 12 Froniers in Signal Processing, Vol. 1, No. 1, July 217 From (27) and (31), we obain J (, s) s R = J (, ) RJ T () A T (s) J (, s) Differeniaing (3) wih respec o and using (32), we ge J(, τ) RJ T (τ) dτa T (s). (31) = J (, ) RJ T () J (, s). (32) de( T ) d = J(, )υ () + T J(,s) υ (s) ds = J(, )υ () + J(, )RJ T () T J(, s)υ (s) ds = J(, )(y() ẑ ( )) + J(, )RJ T () e( T ), e(t T ) =. (33) From (27), J(, ) is given by J (, ) = A T () R 1. (Q.E.D.) In (9), he fixed-inerval smoohing esimae ẑ ( T ) is given as a sum of he filering esimae ẑ ( ) and he quaniy (B () A () q ()) e( T ). e( T ) is calculaed recursively by (1) in he reverse direcion of ime, saring wih he condiion e (T T) = a = T of e( T). The filering esimae is calculaed recursively by (12) wih (13)-(15), saring wih he iniial condiions e () = and q () =. As shown in (19) and (28), in he proof of Theorem 1, he impulse response funcion g (, s) has differen expressions for s T and s T. However, in [1], g (, s) has he same expression for boh s T and s T. In his paper, his erroneous expression for g (, s) in [1] is correced. Secion 4 proposes he fixed-inerval smoohing error variance funcion of he fixed-inerval smooher proposed in Theorem 1. 4 Fixed-inerval smoohing error variance funcion Concerning g(, s) of he firs erm in (16), g (, s) R represens he filering error covariance funcion E[(z () ẑ ( ))(z (s) ẑ(s s)) T ]. Similarly, regarding g(, s),, s T, in (8), g (, s) R represens he smoohing error covariance funcion E[(z () ẑ ( T ))(z (s) ẑ(s T )) T ]. Referring o Theorem 1, he fixed-inerval smoohing error variance funcion P( T) is formulaed as P ( T) = E[(z () ẑ( T))(z () ẑ( T)) T ] = K (, ) E[ẑ( T)ẑ T ( T)] = K (, ) E[(ẑ ( ) + (B () A () q ()) e ( T ))((ẑ ( ) + (B () A () q ()) e ( T )) T ] = K (, ) E [ ẑ ( ) ẑ T ( ) ] (B () A () q ()) E[e ( T ) e T ( T )](B () A () q ()) T. Here, from (3) and he independence propery of he filering esimae wih he innovaion process, we see ha ẑ ( ) is independen from e ( T ) as E [ ẑ ( ) e T ( T) ] T = E[ẑ ( ) ( J (, s)υ (s) ds) T ] T = E [ ẑ ( ) υ T (s) ] J T (, s) ds =. Also, from (3), E[e ( T ) e T ( T )] is formulaed as follows. E [ e ( T ) e T ( T ) ] = T J (, s)e [ υ (s) υ T (s) ] J T (, τ) dsdτ T = J(, s)rj T (, s) ds Le P( ) be he filering error variance funcion, which is given by Copyrigh 217 Isaac Scienific Publishing

6 Froniers in Signal Processing, Vol. 1, No. 1, July Puing P ( ) = K (, ) E [ ẑ ( ) ẑ T ( ) ]. (34) T U (, T) = J(, s)rj T (, s) ds, he fixed-inerval smoohing error variance funcion is given by P ( T) = P ( ) (B () A () q ()) U(, T )(B () A () q ()) T. (35) The auo-variance funcion of he filering esimae ẑ ( ), E [ ẑ ( ) ẑ T ( ) ], is, from (23) and (24), From (2), we ge E [ ẑ ( ) ẑ T ( ) ] =A () E[e () e T ()]A T () = A() J (s) RJ T (s) dsa T (). E [ ẑ ( ) ẑ T ( ) ] = A () q ()A T (). (36) Differenial equaion for q () is given by (14). Differeniaing U(,T) wih respec o and using (32), we obain du (, T ) d = J (, ) RJ T (, ) J (, ) RJ T () U (, T ) U(, T )J ()J T (, ), U (T, T) =. (37) In summary, he algorihm for he fixed-inerval smoohing error variance funcion P( T) consiss of (11), (14), (15) wih (34) - (37). Since (B () A () q ()) U(, T )(B () A () q ()) T and he filering error variance funcion P( ) are posiive semidefinie marices, he following relaionship holds. P( T) P( ) K(, ) (38) (38) indicaes ha he filering error variance funcion P( ) is lower bounded by he fixed-inerval smoohing error variance funcion P( T) and upper bounded by he auo-variance funcion K(, ) of he signal z(). This fac indicaes ha he esimaion accuracy of he proposed fixed-inerval smooher is preferable or equal o ha of he filer in Theorem 1 and he proposed fixed-inerval smooher and filer are sable for he bounded auo-variance funcion K(, ). 5 A Numerical Simulaion Example Le a scalar observaion equaion be given by y() = z() + v(). (39) Le he observaion noise v() be a zero-mean whie Gaussian process wih he variance R, N(, R). Le he auo-covariance funcion of he signal z() be given by K(, s) = 3 16 e s e 3 s. (4) From (4), he funcions A() and B(s) in (3) are expressed as follows: A() = [ 3 16 e 5 48 e 3 ], B(s) = [ e s e 3s ]. (41) If we subsiue (41) ino he fixed-inerval smoohing algorihm of Theorem 1, we can calculae he fixed-inerval smoohing esimae recursively. Fig. 1 illusraes he signal z() and he filering esimae Copyrigh 217 Isaac Scienific Publishing

7 14 Froniers in Signal Processing, Vol. 1, No. 1, July 217 ẑ( ) vs., 2, for he whie Gaussian observaion noise N(,.1 2 ). The filering esimae, saring wih he iniial value ẑ ( ) = a ime =, approaches he signal gradually as ime advances. Fig. 2 illusraes he signal z() and esimaes by he proposed fixed-inerval smooher in Theorem 1 and he fixed-inerval smooher in [1] vs. for he observaion noise N(,.1 2 ). The fixed-inerval smooher, saring from he fixed inerval T = 2, calculaes he fixed-inerval smoohing esimae backward o he ime =. In comparison wih he filering esimae in Fig. 1, he fixed-inerval smooher esimaes he signal beer, in Fig.2, han he filer. Also, from Fig. 2, i is seen ha he esimaion accuracy of he proposed fixed-inerval smooher is preferable o he fixed-inerval smooher in [1]. Table 1 shows he mean-square values (MSVs) of esimaion errors by he proposed filer, he fier in [1], he proposed fixed-inerval smooher and he fixed-inerval smooher in [1] for he observaion noises N(,.1 2 ), N (,.3 2) and N(,.5 2 ). Here, he MSV of filering errors is calculaed by 1 2 (z (i ) ẑ (i i )) 2, =.1. 2 Also, he MSV of fixed-inerval smoohing errors is calculaed by 1 2 i=1 2 i=1 (z(i ) ẑ (i 2 )) 2. Table 1 indicaes, for boh he filering and fixed-inerval smoohing esimaes, as he variance of he observaion noise becomes small, he esimaion accuracies are improved. From he MSVs of he esimaion errors, i should be noed ha he esimaion accuracy of he curren fixed-inerval smooher is preferable o ha of he filer. Also, from Table 1, i is seen ha he esimaion accuracy of he proposed fixed-inerval smooher is superior o ha in [1]. The MSVs of he filering errors by he proposed filer and he filer in [1] are same. This is based on he fac ha he proposed filering algorihm in Theorem 1 is same as ha in [1]. In he calculaions of he filering and fixed-inerval smoohing esimaes, as he numerical inegraion, he fourh-order Runge-Kua mehod wih he inegraion sep size =.1 is adoped. For references, he sae-space model, which generaes he signal process, is specified by z() = x 1 (), dx 1() d = x 2 (), dx2() d = 3x 1 () 4x 2 () 2w (), E [w () w (s)] = δ ( s). Figure 1. Signal and filering esimae vs. for he observaion noise N(,.1 2 ). Copyrigh 217 Isaac Scienific Publishing

8 Froniers in Signal Processing, Vol. 1, No. 1, July Figure 2. Signal and esimaes by he proposed fixed-inerval smooher and he fixed-inerval smooher in [1] vs. for he observaion noise N(,.1 2 ). Table 1. Mean-square values of esimaion errors by he proposed filer, he filer in [1], he proposed fixedinerval smooher and he fixed-inerval smooher in [1] for he observaion noises N(,.1 2 ), N (,.3 2) and N(,.5 2 ). Observaion noise MSV of filering errors MSV of esimaion errors MSV of esimaion errors by he proposed filer by he proposed by he fixed-inerval and he filer in [1] fixed-inerval smooher smooher in [1] N(,.1 2 ) N(,.3 2 ) N(,.5 2 ) Conclusions This paper, based on he innovaion heory, proposed he new fixed-inerval smooher using he covariance informaion in linear coninuous-ime sochasic sysems. From he numerical simulaion example in secion 5, he fixed-inerval smooher, proposed in his paper, improves he esimaion accuracy of he filer and is esimaion accuracy is superior o ha of he fixed-inerval smooher in [1]. This indicaes ha he curren fixed-inerval smoohing algorihm in Theorem 1 shows he feasible smoohing characerisic. Also, from he variance of he fixed-inerval smoohing errors and he variance of he filering errors in secion 4, i is shown ha he esimaion accuracy of he proposed fixed-inerval smooher is preferable or equal o ha of he filer in Theorem 1. References 1. T. Kailah, Lecures on Wiener and Kalman Filering. Springer-Verlag, D. Simon, Opimal Sae Esimaion Kalman, H, and Nonlinear Approaches. Wiley-Inerscience, S. Nakamori, Design of recursive Wiener smooher given covariance informaion, IEICE Trans. Fundamenals of Elecronics, Communicaion and Compuer Sciences, vol. E79-A, no. 6, pp , S. Nakamori, A. H. Carazo, and J. L. Pérez, Design of RLS fixed-lag smooher using covariance informaion in linear discree sochasic sysems, Applied Mahemaical Modelling, vol. 34, no. 4, pp , S. Nakamori, A. H. Carazo, and J. L. Pérez, Design of fixed-lag smooher using covariance informaion based on innovaions approach in linear discree-ime sochasic sysems, Applied Mahemaics and Compuaion, vol. 193, no. 1, pp , 27. Copyrigh 217 Isaac Scienific Publishing

9 16 Froniers in Signal Processing, Vol. 1, No. 1, July S. Nakamori, A. H. Carazo, and J. L. Pérez, Design of RLS Wiener fixed-lag smooher using covariance informaion in linear discree sochasic sysems, Applied Mahemaical Modelling, vol. 32, no. 7, pp , S. Nakamori, Recursive leas squares fixed-lag Wiener smoohing using auoregressive signal models for linear discree-ime sysems, Applied Mahemaical Modelling, vol. 39, no. 21, pp , T. Kailah and P. Fros, An innovaion approach o leas-squares esimaion par II: Linear smoohing in addiive whie noise, IEEE Trans.Auomaic Conrol, vol. AC-13, no. 6, pp , S. Nakamori, RLS fixed-lag smooher using covariance informaion based on innovaion approach in linear coninuous sochasic sysems, Journal of Informaion, vol. 1, no. 2, pp , S. Nakamori, New design of fixed-inerval smooher using covariance informaion in linear sochasic coninuous-ime sysems, Applied Mahemaics and Compuaion, vol. 144, no. 2, pp , S. Fujia and T. Fukao, Opimal linear fixed-inerval smoohing for colored noise, Informaion and Conrol, vol. 17, pp , S. Nakamori, A. H. Carazo, and J. L. Pérez, Design of a fixed-inerval smooher using covariance informaion based on he innovaions approach in linear discree-ime sochasic sysems, Applied Mahemaical Modelling, vol. 3, pp , S. Nakamori, RLS Wiener smooher for colored observaion noise wih relaion o innovaion heory in linear discree-ime sochasic sysems, I.J. Informaion Technology and Compuer Science, vol. 5, no. 3, pp. 1 12, A. P. Sage and J. L. Melsa, Esimaion Theory wih Applicaions o Communicaions and Conrol. McGraw- Hill, Copyrigh 217 Isaac Scienific Publishing