State Estimation with ARMarkov Models

Transcription

1 Deartment of Mechanical and Aerosace Engineering Technical Reort No. 3046, October Princeton University, Princeton, NJ. State Estimation with ARMarkov Models Ryoung K. Lim 1 Columbia University, New York, NY Minh Q. Phan 2 Princeton University, Princeton, NJ Richard W. Longman 3 Columbia University, New York, NY Abstract The ARMarkov models were originally develoed for adative neural control, and later for redictive control, and state-sace identification. Recently, an interaction matrix formulation has been develoed that exlains the internal structure of the ARMarkov models and their connection to the state-sace reresentation. Using the interaction matrix formulation, we show in this aer how a state estimator can be identified directly from inut-outut data. The conventional aroach is to design such a state estimator from knowledge of the lant, and the difficult-to-obtain rocess and measurement noise statistics. A numerical examle comares the identified state estimator with an otimal Kalman filter derived with erfect knowledge of the lant and noise statistics. 1 Graduate Student, Deartment of Mechanical Engineering. 2 Assistant Professor, Deartment of Mechanical and Aerosace Engineering, Dissertation Advisor. 3 Professor, Deartment of Mechanical Engineering, Dissertation Co-Advisor.

2 1. Introduction State estimation is an imortant element of modern control theory. Given a known model of the system under the influence of rocess and measurement noise with known statistics secified in terms of their covariances, it is well known that the Kalman filter is an otimal state estimator in the sense that its state estimation error is minimized. In ractice, it is difficult to design such an otimal estimator because neither the system nor the noise statistics can be known exactly. From the oint of view of system identification, information about the system and the noise statistics are embedded in a sufficiently long set of inut-outut data. Thus it would be advantageous to be able to obtain such an estimator directly from inut-outut data without having to identify the system and the noise statistics searately. This is the roblem of observer identification. Recently, a class of models known as ARMarkov models has been develoed in the context of adative neural control, Ref. 1. The term ARMarkov refers to Auto- Regressive model with exlicit Markov arameter coefficients. ARMarkov models form a bridge between the common ARX model (Auto-Regressive model with exogenous inuts) where the Markov arameters are imlicit, and the non-auto-regressive ulse resonse model where every coefficient is a Markov arameter. Later the ARMarkov models are used for state-sace system identification, Refs In articular, it was found that when the ARMarkov models are used to identify the system Hankel matrix, the true or effective order of the system can be detected more effectively than with an ARX model. This issue has been investigated extensively in Ref. 4. In fact, with an ARX 2

3 model it is also ossible to identify a state-sace model together with an observer gain as shown in Refs. 5 and 6. But with this technique it is not always ossible to determine the order of the state sace realization by Hankel singular value truncation alone and a searate ost-identification model reduction rocedure must be used. With ARMarkov models, we have the oortunity to identify state estimators with true or effective orders without having to invoke a searate model reduction ste as normally required. This is one motivation for the resent aer. It is clear that for state estimation, efficient detection of the dimension of the effective state sace model is imortant because it is comutationally a burden to have a state estimator with unnecessarily large dimensions. The ARMarkov model used in Ref. 4 is based on the develoment of the interaction matrix in Ref. 7 that can be exlained in terms of a generalization of the wellknown Cayley-Hamilton theorem. In all of these develoments the role of the interaction matrix has been to justify the structures of various inut-outut models and the relationshi among the coefficients of these models, but there has been no need to identify the interaction matrix itself from inut-outut data. The second motivation of this aer is an investigation of the role of this interaction matrix in the context of state estimation as oosed to treating it as a convenient mathematical construct. In this work, we establish that fact that identifying a state estimator amounts to identifying this interaction matrix. Another interesting asect of this formulation is that this interaction matrix based state estimator has a non-standard form, different from the usual form of a Luenberger observer or a Kalman filter. However, it will be shown that this new form is very convenient both from the oint of view of state estimator design as well as from the oint of view of its 3

4 identification from inut-outut data. This oint makes u the third motivation for this work. In this aer we will quickly derive such a state estimator using the interaction matrix. We then show how this state estimator can be identified from inut-outut data. This identification will first be derived in the deterministic (noise-free) setting. Then a stochastic analysis will be carried out that shows why the calculations involved in the deterministic case are indeed justified in the stochastic case. Following the theoretical justification, a numerical examle illustrates how an identified state estimator using the technique develoed here comares to that of an otimal Kalman filter designed with erfect knowledge of the system and erfect knowledge of the noise statistics. In articular, we show that the outut residuals obtained with this identified state estimator indeed match the residuals of the otimal Kalman filter. 2. State Estimation by Interaction Matrix In the following we briefly derive a state estimator via an interaction matrix. Consider an n-th order, r-inut, m-outut discrete-time model of a system in state-sace format xk ( + 1) = Axk ( ) + Buk ( ) y( k) = Cx( k) + Du( k) (1) 4

5 By reeated substitution, we have for some 0, x( k + ) = A x( k) + Cu ( k) y ( k) = Ox( k) + Tu ( k) (2) where u ( k ) and y ( k ) are defined as column vectors of inut and outut data going stes into the future starting with u( k) and yk ( ), resectively, u uk ( ) uk ( + 1) ( k) = M uk ( + 1) yk ( ) yk ( + 1), y( k) = M yk ( + 1) (3) For a sufficiently large, C in Eq. (2) is an n r controllability matrix, O is a m n observability matrix, T is a m r Toelitz matrix of the system Markov arameters, C = [ A 1 B, K, AB, B], O = C CA M CA 1 D 0 0 L 0 CB D O O M, T = CAB CB D O 0 M O O O 0 CA 2 B K CAB CB D (4) As long as m n, it is guaranteed for an observable system that an interaction matrix M exists such that A + MO = 0. The existence of M ensures that for k 0 an exression 5

6 for xk ( + ) exists where the state variable is comletely eliminated from the right hand side of Eq. (2), x(k + ) = A x(k) + Cu (k) = A x(k) + Cu (k) + M[ Ox(k) + T u (k)] My (k) = ( A + MO)x(k) + ( C + MT )u (k) My (k) = ( C + MT )u (k) My (k) (5) Shifting the time indices back by time stes, we have for k an exression that relates the current state of the system in terms of ast inut and ast outut measurements, x(k) = α i u(k i) + β i y(k i) (6) i=1 i=1 where [ α, K, α 2, α 1 ]= C+ MT, and [ β, K, β 2, β 1 ]= M. Note that these formulas are alicable to both the single-inut single-outut and multile-inut multile-outut cases. Equation (6) is a state estimator, but not in the standard form of a Luenberger observer or a Kalman filter. In fact, this non-standard form is quite convenient from both ersectives of design and identification. If an observable state-sace model of the system is known, to design this state estimator, one simly forms the controllability matrix C, the Toelitz matrix T, comute an interaction matrix M from A + MO = 0, 6

7 m n. When is chosen such that m > n then M is no longer unique, in which case the solution M = A + O where O + denotes the seudo-inverse of O, roduces a minimum-norm solution for M, whose elements are recisely the outut gains of this state estimator. We have focused on state estimation, but outut estimation can be similarly derived. It is simly, ( ) + y( k) = C C+ MT u ( k ) CMy ( k ) Du( k) (7) + and the minimum norm solution for CM is CA O, whose elements are the outut gains for the outut estimator is CM = CA + O. On the other hand, if the system is unknown but inut-outut data is available, then the arameter combinations C + MT and M can be comuted for the state estimator. This is shown in the next section. 3. Identification of State Estimator with ARMarkov Models We must first derive an inut-outut exression that involves C + MT and M by combining Eq. (5) with the outut exression in Eq. (2), y ( k) = Ox( k) + Tu ( k) ( ) + = OC+ MTu ( k ) OMy ( k ) Tu ( k) (8) 7

8 Equation (8) is a family of ARMarkov models. The first member of the family is an ARX model for yk ( ), which is a secial case of an ARMarkov model. All remaining members are exressions for yk ( + 1 ),, yk ( + 1 ) which are true ARMarkov models. Each ARMarkov model is different from another in that it has an increasing number of Markov arameters aearing exlicitly as coefficients. We need not one but such a family of ARMarkov models to solve for C + MT and M. Furthermore, we assume to have only outut measurements and not the full state, therefore the above inut-outut exression does not have C + MT and M aear exlicitly as coefficients, but the combinations O( C+ MT ) and OM (and T ). Since O is not known, the identification of C + MT and M from inut-outut data is in fact a non-linear roblem. Fortunately, an exact solution can be found without any kind of iterations as shown below. For simlicity define A = OC+ ( MT ), B = OM, then A, B, and T can be identified from inut-outut data as T T [ A T B ]= YV ( VV ) + (9) where u ( 2 k ) combines u ( k) with u ( k) + for convenience, [ ] Y = y ( ) y ( + 1) L y ( + l ) (10) u u u V = 2 ( 0) 2 ( 1) L 2 ( l) y( 0) y( 1) L y ( l) (11) 8

9 To find ( C + MT ) and M from A and B we must first find O. Taking advantage of the internal structure of the coefficients A, B as revealed by the interaction matrix, the combination H = OC can be comuted from H = A + BT (12) The observability matrix O is obtained by a singular value decomosition of H = U Σ V T n n n, where n is the order of the system, O = U n Σ 1 2 n / (13) Since the state-sace reresentation is uncertain u to a similarity transformation of the state vector, we generally have O in a different set of coordinates from O but they are related to each other by a similarity transformation, O = OT. The identified arameter combinations, however, are invariant with resect to such a transformation, ( ) 1 1 A = OC ( + MT)= OT( T C+ T MT)= OC+ MT 1 B = OM = OTT M = OM (14) 9

10 Hence, as long as is chosen to be sufficiently large such that the observability matrix O or O has rank n, the needed arameters ( C + M T) and M to construct a state estimator can be found from C + T T [( MT) M]= ( O O) 1 O [ A B ] (15) We thus see that it is not necessary to roduce a realization of A, B, C in the above stes of extracting a state estimator from inut-outut data. However, this ste can be easily done as well, and would be necessary if we desire to ut the state estimator in a different set of coordinates than that chosen by the above realization, such as the modal coordinates. Any realization algorithm can be used for that urose. Here we review the realization rovided by the Eigensystem Realization Algorithm (ERA), Ref. 8. The rocedure calls for the extraction of two Hankel matrices H( 0 ) and H( 1 ) from H, H( 0) = CB L M M M n1 CA B n2 n1+ n2 CA B L CA B, H( 1) = n1 + 1 CAB L CA B M M M n2 + 1 n1+ n2 + 1 CA B L CA B (16) The matrix H has the Markov arameters of increasing order going from left to right, whereas H( 0 ) and H( 1 ) are tyically defined with the higher order Markov arameters going from right to left. To maintain the standard notation, a trivial rearrangement of the Markov arameters is needed in forming H( 0 ) and H( 1 ). A s-th order state-sace model T is A = 1/ 2 Σ U VΣ 1/ 2 H () 1, B is the first r columns of Σ 1 / 2 T s V s, C is the first m rows of s s s s 10

11 U s Σ 1 2 s ( ) /, and s min rn ( + 1), mn ( + 1 ). The matrix U s and V s are made u of s left and 1 2 right singular vectors of H( 0 ), resectively. The diagonal matrix Σ s is made u of s corresonding singular values of H( 0 ). With erfect data H( 0 ) has exactly n ositive singular values (all remaining singular values are identically zero), where n is the true minimum order of the system, s = n. Otherwise, the user can secify the order of the state-sace model by s, the number of Hankel singular values to retain. The articular set of coordinates of the realization in O has a secial roerty that it becomes internally balanced when is large and the system is stable, Ref. 9. A realization is said to be internally balanced if and only if the controllability and observability grammians are equal to each other, and both equal to Σ n. As mentioned, the realization can also be ut in another user-secified set of coordinates. We simly need to comute the corresonding observability matrix O in that set of coordinates and use Eq. (15) to obtain the corresonding state estimator. Stochastic Analysis In the revious section the identification of the state estimator is derived in the deterministic setting. Now we will consider the situation where data is corruted by rocess and measurement noise, and show how the same deterministic calculations are justified in the stochastic case. Consider the case where rocess and measurement noise are resent in Eq. (1). The corresonding version of Eq. (2) is 11

12 x( k + ) = A x( k) + Cu ( k) + v( k) y ( k) = Ox( k) + Tu ( k) + w ( k) (17) Although Eq. (17) is -ste ahead, i.e., it relates xk ( + ) to xk ( ) with samling interval t, inut uk ( ), and outut yk ( ), it can be thought of as one-ste ahead with samling interval t, inut u with some gain K, ( k), and outut y ( k ). Hence it admits an estimator of the form + x ( k) = x ( k) + K[ y ( k) y ( k) x ]= ( k) + Ke( k) + x ( k) = A x ( k ) + C u ( k ) y ( k) = Ox ( k) + Tu ( k) (18) The quality of the estimation certainly deends on the gain K. For a choice of K, ek ( ) is the corresonding outut residual defined to be the difference between estimated outut yk ( ) and measured outut yk ( ), ek ( ) = yk ( ) yk ( ). The exression for the state in Eq. (18) can be written as x ( k) = A ( x ( k ) + K( y ( k ) y( k ) ))+ Cu( k ) ( ) + = A x ( k ) + A K Ox ( k ) + Tu ( k ) y ( k ) Cu ( k ) ( ) ( ) = A + A KO x ( k ) + C+ AKT u( k ) AKy( k ) (19) The above equation is somewhat subtle for the following reason. Due to the resence of the term ( A + A KO ), it is a -ste ahead state estimator, i.e., it estimates x ( ) from 12

13 ( ) x ( 0 ), then x ( 2 ) from x ( ), and so on. However, if A + A KO is zero (which will be justified later), then this equation can be used to rovide state estimation every single ste beginning with x ( ), then x ( + 1 ), x ( + 2 ), and so on. When combined with the outut exression we have y ( k) = OAx ƒ ( k ) + O C+ A KT u ( k ) OA Ky ( k ) T u ( k) e( k) (20) ( ) + + where A ƒ = A + A KO. From a given set of inut-outut data of sufficient length we can form the data matrices Y and V as defined in Eqs. (10) and (11), then Y = [ A T B ] V + O AX ƒ + E (21) ( ) [ ] where A = OC+ AK T, B = OA K, X = x ( 0) x () 1 K x () l, and e() 1 ( ) e() 1 ( + 1) K e() 1 ( + l) e e + e + ( 2) ( ) ( 2) ( 1) K ( 2) ( l) E = [ e ( ) e ( + 1) K e ( + l) ]= M M K M e ( ) e ( + ) e ( + ) ( ) ( ) 1 K ( ) l (22) The residual matrix E has the following interretation when A ƒ = 0. The first residual sequence e ( () k ) 1, k =, + 1,.., + l is associated with the ARX model derived from the first m rows of Eq. (20). The second residual sequence e ( ( ) k ) 2 is associated with the first ARMarkov model derived from the second m rows of Eq. (20). Similarly, the 13

14 remaining residual sequences e ( j) ( k), j = 3 through, are associated with the remaining ARMarkov models. Because not a single but a family of models are being used in the inut-outut ma of Eq. (20), we do not have a single but rather a family of residuals. Pre-multilying Eq. (21) by V T and re-arranging it yields, T T T T YV [ A T B ] VV = OAX ƒ V + EV (23) Now let us imose conditions on K so that the observer in Eq. (19) ossesses desirable roerties. If K is chosen such that ƒ A = 0 then by choosing T T [ A T B ]= YV ( VV ) + (24) the left hand side of Eq. (23) vanishes. Referring back to Eq. (21), with A ƒ = 0, the solution given in Eq. (24) is exactly the one that minimizes the Euclidean norm of E which is the sum of the squares of the residuals for the entire data record. Furthermore, from Eq. (23), this solution also results in EV T = 0, which can be written exlicitly as e e e ut ut ut ut () 1 ( ) () 1 ( + 1) K () 1 ( + l) ( 0) K ( 1) ( ) K ( 2 1) e( 2) ( ) e( 2) ( + 1) K e( 2) ( + l) ut() 1 K ut( ) ut( + 1) K ut( 2) M M K M = 0 M K M M K M e( ) ( ) e( ) ( + 1) e( ) ( + ) K l ut ut + ut + ut ( l) K ( 1 l) ( l) K ( l) (25) 14

15 e e e T T () 1 ( ) () 1 ( + 1) K () 1 ( + l) y ( 0) K y ( 1) e e e T T ( 2) ( ) ( 2) ( + 1) K ( 2) ( + l) y () 1 y ( K 1) = 0 M M K M M K M e( ) ( ) e( ) ( + 1) e( ) ( + ) K l T T y ( l) K y ( 1 + l) (26) Let us now examine the imlications of Eqs. (25) and (26) for the j-th member of the residual sequences e ( j) ( k). When the length of the data record tends to infinity, the above equations imly for a stationary random rocess, + l T 1 T E{ e( j) ( k) u ( k i) }= lim e( j) ( k) u ( k i) = 0, i = + 1,..., 0, 1, 2,.., (27) l l k= + l T 1 T E{ e( j) ( k) y ( k i) }= lim e( j) ( k) y ( k i) = 0, i = 1, 2,.., (28) l l k= where E {}. denotes the exectation oerator. Thus, if the data record is sufficiently long, the identified family of ARMarkov models has its combined residuals minimized, and in articular, Eqs. (27) and (28) state that the residual for each member of the family becomes uncorrelated with inut and outut data. Recall these results are obtained while imosing the condition on K such that A ƒ = A + A KO = 0. We note here that this condition can also be satisfied if is large and the system is stable so that A 0. By referring back to the deterministic formulation, it is clear that the interaction matrix M lays the same role as AK at every ste of the derivation, including the condition A + MO = 0. Hence, the deterministic calculations are indeed justified in the stochastic case. 15

16 Illustration Consider a chain of three masses connected by srings and damers with force inut to the first mass and osition measurements of the last two masses. The statesace matrices for this dynamical system are 03 1 I A = M K 1 M C, B = , C = [ ], D = 0 The state vector is made u of ositions and velocities of the three masses in the following order, x x, x, x, x«, x«, x«t, and the mass, stiffness, and daming matrices are = [ ] m1 0 0 M = 0 m m 3 c1 + c2 c2 0 k1 + k2 k2 0, C = c c + c c , K = k k + k k c c k k 3 3 where m1 = 05. Kg, m2 = m3 = 1Kg, k1 = k2 = k3 = 10 N m, c1 = c2 = c3 = 035. Nsec m. The samling interval is 0.1 sec. The system is excited by random inut, and the outut data is corruted by significant rocess and measurement noise. In modal coordinates, each of the modal states is corruted by about 3-5% rocess noise (measured in terms of standard deviation ratios of noise to signal), and the oututs are corruted by about 15% measurement noise. Because this is a simulation, we can actually comute the noise 16

17 statistics given in terms of their covariances. From exact knowledge of the system and the comuted rocess and measurement noise covariances, a Kalman filter is designed. The Kalman filter reresents the best or otimal estimation that can be achieved for the given system with the known noise statistics. Next we use the above set of noise corruted inut-outut data to identify a state estimator with the rocedure described in this aer, and this is done without knowledge of the system and without knowledge of the embedded rocess and measurement noise statistics. As mentioned in the introduction, ARMarkov models are effective in caturing the true or effective system order. Order determination is achieved by examining the quality of the identified state sace model in reroducing the identification data for various model order selection by Hankel singular value truncation. This is shown in Fig. 1 which indicates that the system order is six, which is indeed the case. Figure 2 shows the actual measured (noise-corruted) oututs together with an overlay of the results of the otimal Kalman filter estimation and the identified 6-th order state estimator. Let us examine the first outut. The jagged curve is the measured noisecorruted outut. The smooth curve reresents the otimal filtering by the Kalman filter. Note that result obtained with the identified state estimator closely follows the Kalman filter result. The same attern is observed for the second outut. Figure 3 shows a comarison of the residuals itself, for each of the two oututs. Recall that the Kalman filter results are derived with exact knowledge of the system and noise statistics, whereas 17

18 the identified state estimator is derived from inut-outut data alone. Figure 4 shows the auto-correlation of the Kalman filter residual and our identified state estimator residual. In addition to comaring filtered oututs, we comare the filtered modal states and this is shown in Fig. 5. Since the Kalman filter minimizes the state estimation error, it is interesting to see how the identified state estimator comares to this otimal result. Keeing in mind that in the resence of noise, identification can never erfectly extract the system model and noise statistics with finite data records but with increasing data length and, imrovement in the identification should be exected. Recall that increasing is beneficial because it hels making the residuals more and more uncorrelated with identification data as shown in the theoretical section. Indeed, it is shown in Table 1 that the norm of the state estimation error (and of outut rediction error) of the identified state estimator aroaches that of the Kalman filter with increasing data length and. These illustrations indicate that the roosed state estimator identified from inut-outut data does indeed aroach the otimal Kalman filter designed with erfect knowledge of the system and erfect knowledge of rocess and measurement noise statistics. Conclusions In this aer we have shown how ARMarkov models can be used to identify a state estimator from inut-outut data. This work extends revious develoment of ARMarkov models for system identification and control alications. Being able to identify a state estimator from data is significant in view of the standard aroach of 18

19 designing such a state estimator from an assumed model of the system and the noise statistics. A key ingredient that makes this ossible is the recent derivation of ARMarkov models through the use of an interaction matrix. In our revious work, the role of the interaction matrix is to justify the existence of various inut-outut models and to establish various relationshi among the identified coefficients. For control and system identification roblems, there is no need to recover the interaction matrix itself. In this work, we roceed one ste further by actually recovering the interaction matrix, and exlaining it in the context of a state estimator. This state estimator is not in the standard form of a Luenberger observer or a Kalman filter but in this new form, this state estimator is easy to design from a known model, and if the model is not known, it can be identified from inut-outut data. References 1. Hyland, D.C., Adative Neural Control (ANC) Architecture - a Tutorial, Proceedings of the Industry, Government, and University Forum on Adative Neural Control for Aerosace Structural Systems, Harris Cor., Melbourne, FL, Akers, J.C., and Bernstein, D.S., ARMARKOV Least-Squares Identification, Proceedings of the American Control Conference, Albuquerque, NM, 1997,

20 3. Akers, J.C., and Bernstein, D.S., Time-Domain Identification Using ARMARKOV/Toelitz Models, Proceedings of the American Control Conference, Albuquerque, NM, 1997, Lim, R.K., Phan, M.Q., and Longman, R.W., State-Sace System Identification with Identified Hankel Matrix, Deartment of Mechanical and Aerosace Engineering Technical Reort No. 3045, Princeton University, Set Juang, J.-N., Phan, M., Horta, L.G., and Longman, R.W., Identification of Observer/Kalman Filter Markov Parameters: Theory and Exeriments, Journal of Guidance, Control, and Dynamics, Vol. 16, No. 2, 1993, Juang, J.-N., Alied System Identification, Prentice-Hall, Englewood Cliffs, NJ, 1994, Phan, M.Q., Lim, R.K., and Longman, R.W., Unifying Inut-Outut and State- Sace Persectives of Predictive Control, Deartment of Mechanical and Aerosace Engineering Technical Reort No. 3044, Princeton University, Set Juang, J.-N., and Paa, R.S., An Eigensystem Realization Algorithm for Model Parameter Identification and Model Reduction, Journal of Guidance, Control, and Dynamics, Vol. 8, No.5, 1985, Juang, J.-N., and Lew, J.-S., Integration of System Identification and Robust Controller Designs for Flexible Structures in Sace, Proceedings of the AIAA Guidance, Control, and Navigation Conference, Portland, OR, 1990,

21 1.8 Prediction error Selected state sace model order Fig. 1. Order determination 21

22 0.05 ARMarkov Kalman Measured Outut ARMarkov Kalman Measured Outut Time (sec.) Fig 2. Measured and estimated oututs by ARMarkov and Kalman filter. 22

23 Residual outut ARMarkov Kalman Residual outut Time (sec.) Fig. 3. ARMarkov and Kalman filter outut residuals. 23

24 Auto-correlation (residual 1) Auto-correlation (residual 2) ARMarkov Kalman Number of time shifts Fig. 4. Auto-correlation of outut residuals. 24

25 4 ARMarkov Kalman Actual State State 4 0 State Time (sec.) Fig. 5. True and estimated modal states by ARMarkov and Kalman filter. 25

26 Data Length State Residual ARMarkov Kalman Outut Residual ARMarkov Kalman Table 1. Comarison of state and outut residuals by ARMarkov and Kalman filter. 26