= 0 otherwise. Eu(n) = 0 and Eu(n)u(m) = δ n m

Transcription

1 A-AE 567 Final Homework Spring 212 You will need Matlab and Simulink. You work must be neat and easy to read. Clearly, identify your answers in a box. You will loose points for poorly written work. You must work alone. Exam is due in a box outside my office ARMS 321 by 12: pm Friday May 4. NAME: 1

2 1 Problem Consider the joint density function f x,y (x, y) determined by f x,y (x, y) = cx if x 2 y 1 and x 1 = otherwise where c is a constant. Find: The constant c. The conditional expectation ĝ(y) = E(x y = y). Compute the error E(x ĝ(y)) 2. Find a polynomial p(y) = α + βy + γy 2 of degree at most two which is the best approximation of x, that is, E(x p(y)) 2 = inf{e(x p(y)) 2 : p is a polynomial of degree at most 2} Compute the error E(x p(y)) 2. 2 Problem Let u(n) be a discrete time white noise process, that is, Eu(n) = and Eu(n)u(m) = δ n m for all integers n and m where δ n is the Kronecker delta. Let θ be a uniform random variable over the interval [, 2π]. Consider the random process defined by y(n) = 2 sin(n + θ) + u(n) 2

3 where n is an integer. Moreover, assume that θ and u(n) are independent random variables for all integers n. Consider the prediction problem: σ 2 = min{e y(n) α 1 y(n 1) α 2 y(n 2) α 3 y(n 3) α 4 y(n 4) 2 : α j R}. The minimum is taken over all constants {α j } 4 1. (i) Find Ey(n)y(m) where n and m are integers. (ii) Find the optimal constants { α j } 4 1 and the error σ such that 4 σ 2 = E y(n) α j y(n j) 2. (iii) What is your guess for the error σ in the following optimization problem: σ 2 = min{e y(n) α j y(n j) 2 }. j=1 The theory behind this problem is beyond the scope of the course. However, you should be able to guess at σ. j=1 3 Problem Consider the discrete time system x(n + 1) = Ax(n) + Bu(n) and y(n) = Cx(n) + Dv(n) where u and v are independent Gaussian white noise processes and the initial condition is x() =. The system is third order. Moreover, the matrices {A, B, C, D} and the output {y(j)} 1 are given on the course webpage in the file a567 data12.mat. Let M n = span{y(j)} n. Find the following estimates which are vectors in R 3. P M1 x(9). 3

4 P M1 x(1) P M1 x(11) P M1 x(13) 4 Problem Consider the state space system x(n + 1) = Ax(n) + u(n) and y(n) = Cx(n) + v(n) (4.1) where u(n) and v(n) are mean zero Gaussian random process which are independent to the initial condition x(). Moreover, assume that E u(n) [ ] u(m) v(m) = R 11 R 12 δ n m. v(n) R 21 R 22 Here δ = 1 and for k nonzero, δ k =. Let M n = span{y(k)} n and x(n) = P Mn 1 x(n) denote the optimal state estimate. Let Q n = E x(n) x(n) = E (x(n) x(n)) (x(n) x(n)) be the error covariance matrix. Find the Kalman filter for the state space system in (4.1). To be precise, find a recursive estimate for the optimal state x(n) and a recursive formula for the error covariance Q n. (i) The recursion for the optimal state is x(n + 1) = (ii) The recursion for the error covariance is Q n+1 = (iii) Find x(n n) = P Mn x(n) in term of x(n), Q n and the data. (iv) Find x(n 1 n) = P Mn x(n 1) in term of x(n), Q n and the data. 4

5 5 Problem: The inverted pendulum problem Part of this problem is taken from the web site: Control tutorials for Matlab, Consider the classical inverted pendulum problem, consisting of a cart of mass m 1 with an inverted pendulum with inertia J on the cart. The linearized equations of motion in state space form are given by ẋ 1 x ẍ (J+m 2l 2 )ζ m 2 2 gl2 ẋ = q q + φ 1 φ φ m 2lζ m 2 gl(m 1 +m 2 ) q q φ y = x 1 ẋ v 1 1 φ 1 v 2 2 φ q = J(m 1 + m 2 ) + m 1 m 2 l 2. J+m 2 l 2 q m 2 l q 1 w + 4 u 1 u Here x is the position of the cart and φ is the angle of the inverted pendulum from the vertical position. The control force on the cart is denoted by w. Moreover, u 1, u 2, v 1 and v 2 are all independent white noise processes. Notice that u 1 is a random force on the cart and u 2 is a random torque on the pendulum. The notation is m 1 =.5kg is the mass of the cart m 2 =.2kg is the mass of the pendulum ζ =.1N/m/sec is the friction of the cart l =.3m is the distance to the pendulum s center of mass 5

6 J =.6kgm 2 is the inertia of the pendulum w is the force applied to the cart x is the position of the cart φ is the angle in radians of the pendulum from the vertical position. Assume that all the initial conditions are zero. Design a feedback controller w = K x based on the steady state Kalman filter such that the cart moves not more that one meter from the origin ( x 1) and the angle moves no more than ±2 degrees from ] the vertical position ( φ.35). Your state feedback gain K = [k 1 k 2 k 3 k 4 must satisfy k j 3 for j = 1, 2, 3, 4. Simulate your controller in Simulink for 3 seconds. Hand the graphs from your Simulink program for: (i) The position x and the estimated position of the cart x on the same graph. (ii) The velocity ẋ and the estimated velocity of the cart x on the same graph. (iii) The angle φ and the estimated angle of the pendulum φ on the same graph. (iv) The angular velocity φ and the estimated angular velocity of the pendulum φ on the same graph. (v) Hand in your gain K. On the band limited white nose generators, set both the noise power and sample time equal to.1. The seed for u 1, u 2, v 1 and v 2 is respectively, 23341, 23342, and

7 6 Problem Consider the third order state space system ẋ = Ax and y = Cx 1 [ ] A = 1 and C = Find the error σ and the optimal initial condition x to solve the following optimization problem σ 2 = e 2t Ce At x { } 2 dt = inf e 2t Ce At x 2 dt : x C 3 7 A review of steady state Kalman filtering Let us review the continuous time steady state Kalman filter. Throughout we assume that the system is time invariant, that is, ẋ = Ax + Bu and y = Cx + Dv (7.1) where A is an operator on X and B maps U into X while C maps X into Y and D maps V into Y. The operators {A, B, C, D} are all fixed and do not depend upon time t. We also assume that the operator D is onto Y, or equivalently, DD is invertible. Let Q(t) be the solution to the Riccati differential equation: Q = AQ + QA + BB QC (DD ) 1 CQ subject to Q() =. (7.2) The following is a classical result in linear quadratic regulator theory and Riccati differential equations. THEOREM 7.1 Consider the linear time invariant system {A, B, C, D} where D is onto. Then there exists a solution Q(t) to the Riccati differential equation in (7.2) 7

8 for all t. In other words, the Riccati differential equation in (7.2) does not have a finite escape time. Moreover, the following holds. (i) The solution {Q(t) : t } forms an increasing sequence of positive operators, that is, Q(σ) Q(t) for all σ t. (ii) If the pair {C, A} is observable, then Q(t) converges to a positive operator P as t tends to infinity, that is, P = lim t Q(t). (7.3) In this case, P is a positive solution for the algebraic Riccati equation = AP + P A + BB P C (DD ) 1 CP. (7.4) (iii) If {A, B, C, D} is controllable and observable, then P is strictly positive. (iv) If {A, B, C, D} is controllable and observable, then P is the only strictly positive solution to the algebraic Riccati equation (7.4). To be precise, if Ξ is a strictly positive solution to (7.4), then Ξ = P = lim t Q(t). (v) If {A, B, C, D} is controllable and observable, then A P C (DD ) 1 C is a continuous time stable operator, that is, all the eigenvalues of A P C (DD ) 1 C are contained in the open left half plane. 7.1 The steady state Kalman filter with control Assume that {A, B, C, D} is a controllable and observable system. Let P be the unique positive solution to the algebraic Riccati equation in (7.4). By passing limits in the Kalman filter, we arrive at the steady state Kalman filter defined by x = (A LC) x + Ly L = P C (DD ) 1. (7.5) 8

9 Theorem 7.1 guarantees that A LC is a continuous time stable operator. Moreover, L is called the Kalman gain or Kalman observer. The steady state Kalman filter provides an estimate x(t) for the state x(t) given the past {y(σ) : σ < t}. The Kalman filter converges to the steady state Kalman filter. In other words, the steady state Kalman filter converges to the optimal state estimator as t tends to infinity. Assume that {A, B, C, D} is a controllable and observable system. Consider the Kalman filtering problem with a control added to the state equation, that is, ẋ = Ax + Bu + B 1 w and y = Cx + Dv. (7.6) Here w is a known input or control with values in W, and B 1 is an operator mapping W into X. As before, u and v are independent white noise processes, which are also independent to the initial condition x(). In this case, the steady state Kalman filter is given by x = (A LC) x + Ly + B 1 w L = P C (DD ) 1. (7.7) As expected, P is the unique positive solution to the algebraic Riccati equation (7.4) in Theorem 7.1. Finally, it is noted that (7.7) reduces to the steady state Kalman filter in (7.5) when B 1 =. 9

10 simouty To Workspace3 output y simoutx1 To Workspace1 Band Limited White Noise2 x = Ax+Bu y = Cx+Du State Space em x = Ax+Bu y = Cx+Du em x1 c* u Kalman Filter d Matrix Gain1 x2 Band Limited White Noise1 Gain simoutx2 To Workspace2 Sine Wave Figure 1: The Simulink block diagram for w(t) = sin(t). 8 An example of a sinusoid input Consider the stable state space system given by ẋ1 = 1 x 1 + u + 1 sin(t) ẋ 2 x 2 y = x 1 x 2 + v. (8.1) Notice that { 1, 2} are the eigenvalues for the 2 2 state space matrix in (8.1). As expected, u and v are independent white noise processes. For simplicity we have assumed that all the initial conditions are zero. The known input w = sin(t). Using [ ] tr [ ] P = are(a, C C/(d 2 ), B B ) in Matlab with d = 1, B = 1 and C = 1 1 (the transpose of a matrix is denoted by tr), we arrived at.818 P =.31 and L = The eigenvalues for P are {.817,.166}. Since the eigenvalues for P are small, we expect the Kalman filter to perform rather well. The Simulink block diagram to 1

11 5 The plot of the output y 2 The plot of x 1 and hatx time time (a) The output y feed into the Kalman filter. (b) The state space x 1 and x The plot of x 2 and hatx time (c) The state space x 2 and x 2 Figure 2: The plots of y, x 1, x 1, x 2 and x 2. implement this Kalman filter is given in Figure 1. In this Simulink program, we used.1 for both the noise power and sample time for our white noise generators. The seed for the band limited white noise generator 1 is 23342, while the seed for the band limited white noise generator 2 is The output y for the state space model (8.1) is given in Figure 2, Part (a). The graphs for x 1 and its state estimate x 1 are presented in Figure 2, Part (b). Finally, the graphs for x 2 and its corresponding state estimate x 2 are given in Figure 2, Part (c). 11

12 9 State feedback and the Kalman filter As before, assume that {A, B, C, D} is a controllable and observable system. The state equations for the Kalman filtering problem with a control w are given by ẋ = Ax + Bu + B 1 w and y = Cx + Dv. (9.1) Here w is a known input or control with values in W, and B 1 is an operator mapping W into X. As before, u and v are independent white noise processes. Recall that the steady state Kalman filter is given by x = (A LC) x + Ly + B 1 w L = P C (DD ) 1 = AP + P A + BB P C (DD ) 1 CP where P >. (9.2) Now let us use the estimated state x to design a state feedback controller K. To this end, compute a state feedback controller K mapping X into W such that A B 1 K is stable. One can use pole placement or the linear quadratic regulator to determine a state feedback gain K satisfying a specified performance criteria. Since x is the estimated state, the feedback control is given by w = K x. Substituting w = K x into ẋ = Ax + Bu + B 1 w and (9.2) with y = Cx + Dv yields ẋ = A B 1K x x + B u. (9.3) x LC A LC B 1 K LD v So the feedback system is precisely a state space system driven by the white noise [ tr. process u v] Recall that two state space systems {A 2, B 2 } and {A 3, B 3 } are called similar if there exists an invertible transformation Ξ such that A 3 = ΞA 2 Ξ 1 and B 3 = ΞB 2. If {A 2, B 2 } and {A 3, B 3 } are similar, then A 2 and A 3 have the same eigenvalues. In particular, A 2 is stable if and only if A 3 is stable. 12

13 We claim that the state space system (9.3) is similar to the following state space system ẋ = A B 1K B 1 K x x + B u. (9.4) x A LC B LD v The error in estimation is denoted by x = x x. By construction A B 1 K and A LC are both stable. Since the state space matrix in (9.4) is upper triangular with A B 1 K and A LC on the diagonal, the state space system in (9.4) is stable. Because the two state space matrices in (9.3) and (9.4) are similar, the state space operator T = A LC B 1K on X (9.5) A LC B 1 K X in (9.3) must also be stable. In fact, the eigenvalues of T are precisely the eigenvalues of A B 1 K union the eigenvalues of A LC. To show that (9.3) and (9.4) are similar, consider the operator determined by Ξ = I on X. I I X Recall that x = x x is the error in estimation. Now observe that Ξ x x = I x x = x x. I I In other words, Ξ maps [ x ] tr [ tr. x into x x] The operator Ξ is invertible. In fact, Ξ 1 = Ξ. To verify that (9.3) is similar to (9.4), notice that ΞT Ξ 1 = I A B 1K Ξ I I LC A LC B 1 K = A B 1K I = A B 1K B 1 K A LC LC A I I A LC Ξ B = I B = B. LD I I LD B LD 13

14 This yields the state space system in (9.4) and proves our claim. 1 An example of state feedback control Consider the state space system ẋ1 = 1 x 1 + u + 2 w(t) ẋ 2 x 2 y = x 1 + x 2 + v. (1.1) The eigenvalues for the 2 2 state space matrix in (1.1) are {1, 2}. The state space system in (1.1) is unstable. Using P = are(a, C C/(d 2 ), B B ) in Matlab with [ ] tr [ ] d = 1, B = 1 and C = 1 1, we arrived at.5 P =.5 and L = The eigenvalues for the matrix P are {.4418, }. By employing the linear [ tr, quadratic regulator in Matlab K = lqr(a, B 1, eye(2), 1/4) with B 1 = 2 1] we obtained K = [ ] In this case, the matrix T in (9.5) is given by T = A B 1K =. LC A LC B 1 K The eigenvalues for T are { 1, 3.568±.484i, }. As expected, T is stable. The Simulink block diagram to implement this state feedback control w = K x with the Kalman filter to estimate x is given in Figure 3. In this Simulink program, we 14

15 used.1 for both the noise power and sample time for our white noise generators. Moreover, the seed for the band limited white noise generator 1 is while the seed for the band limited white noise generator 2 is The output y for the state space model (1.1) is given in Figure 4, Part (a). Figure 4, Part (b) graphs x 1 and its state estimate x 1, while Figure 4, Part (c) graphs x 2 and its state estimate x 2. Finally, it is noted that our estimate x 1 for the state space x 1 is very good, that is, the graphs for x 1 and x 1 are practically identical. 15

16 simout1 To Workspace1 output y simout2 To Workspace2 Band Limited White Noise2 x = Ax+Bu y = Cx+Du State Space em x = Ax+Bu y = Cx+Du em x1 c* u Kalman FIlter d Matrix Gain1 x2 Band Limited White Noise1 Gain simout k* u To Workspace Feedback Figure 3: The Simulink block diagram for w = K x. 16

17 3 The plot of the output y 2 The plot of x 1 and hatx time time (a) The output y feed into the Kalman filter. (b) The state space x 1 and x The plot of x 2 and hatx time (c) The state space x 2 and x 2 Figure 4: The plots of y, x 1, x 1, x 2 and x State feedback and the discrete Kalman filter Assume that {A, B, C, D} is a controllable and observable discrete time system. The state equations for the discrete time Kalman filtering problem with a control w are given by x(n + 1) = Ax(n) + Bu(n) + B 1 w(n) and y(n) = Cx(n) + Dv(n). (11.1) 17

18 Here w is a known input or control with values in W, and B 1 is an operator mapping W into X. As before, u and v are independent white noise processes. Recall that the steady state Kalman filter is given by x(n + 1) = (A LC) x(n) + Ly(n) + B 1 w(n) L = AP C (CP C + DD ) 1 (11.2) P = AP A + BB AP C (CP C + DD ) 1 CP A where P >. Finally, A LC is stable, that is all the eigenvalues of A LC are contained in the open unit disc. Now let us use the estimated state x to design a state feedback controller K. To this end, compute a state feedback controller K mapping X into W such that A B 1 K is stable. One can use pole placement or the discrete time linear quadratic regulator to determine a state feedback gain K satisfying a specified performance criteria. Since x is the steady state estimate, the feedback control is given by w = K x. Substituting w(n) = K x(n) into x(n + 1) = Ax(n) + Bu(n) + B 1 w(n) and (11.2) with y = Cx + Dv yields x(n + 1) = A B 1K x(n) + B u(n). (11.3) x(n + 1) LC A LC B 1 K x(n) LD v(n) So the feedback system is precisely a state space system driven by the white noise [ tr. process u v] Recall that two state space systems {A 2, B 2 } and {A 3, B 3 } are called similar if there exists an invertible transformation Ξ such that A 3 = ΞA 2 Ξ 1 and B 3 = ΞB 2. If {A 2, B 2 } and {A 3, B 3 } are similar, then A 2 and A 3 have the same eigenvalues. In particular, A 2 is stable if and only if A 3 is stable. 18

19 We claim that the state space system (11.3) is similar to the following state space system: x(n + 1) = A B 1K B 1 K x(n) + B u(n). (11.4) x(n + 1) A LC x(n) B LD v(n) The error in estimation is denoted by x = x x. By construction A B 1 K and A LC are both stable. Since the state space matrix in (11.4) is upper triangular with A B 1 K and A LC on the diagonal, the state space system in (11.4) is stable. Because the two state space matrices in (11.3) and (11.4) are similar, the state space operator T = A LC B 1K on X (11.5) A LC B 1 K X in (11.3) must also be stable. In fact, the eigenvalues of T are precisely the eigenvalues of A B 1 K union the eigenvalues of A LC. To show that (11.3) and (11.4) are similar, consider the operator determined by Ξ = I on X. I I X Recall that x = x x is the error in estimation. Now observe that Ξ x x = I x x = x x. I I [ ] tr [ tr. In other words, Ξ maps x x into x x] The operator Ξ is invertible. In fact, Ξ 1 = Ξ. To verify that (11.3) is similar to (11.4), notice that ΞT Ξ 1 = I A B 1K Ξ I I LC A LC B 1 K = A B 1K I = A B 1K B 1 K A LC LC A I I A LC Ξ B = I B = B. LD I I LD B LD 19

20 This yields the state space system in (11.4) and proves our claim. 12 An example of state feedback control Consider the state space system: x 1(n + 1) = 1 x 1(n) + u + 2 w(t) x 2 (n + 1) 2 3 x 2 (n) 1 1 y = x 1 + x 2 + v. (12.1) The eigenvalues for the 2 2 state space matrix in (12.1) are {.5616, }. The discrete state space system in (12.1) is unstable. Using P = dare(a, C, B B, d d ) [ ] tr [ ] in Matlab with d = 1, B = 1 and C = 1 1, we arrived at.649 P = and L = The eigenvalues for the matrix P are {.121, }. By employing the discrete time [ tr, linear quadratic regulator in Matlab K = dlqr(a, B 1, eye(2), 1/8) with B 1 = 2 1] we obtained K = [ ] In this case, the matrix T in (11.5) is given by T = A B 1K =. LC A LC B 1 K The eigenvalues for T are {.5738,.344,.1545,.2456}. As expected, T is stable. The Simulink block diagram to implement this state feedback control w = K x with the steady state Kalman filter to estimate x is given in Figure 5. In this Simulink 2

21 program, we used.1 for the sample time of the white noise or random number generators. Moreover, we set the seed equal to 2 and 22. Furthermore, we presented a Simulink program based on the equations in (11.3). The output y for the state space model (12.1) is given in Figure 6, Part (a). Figure 6, Part (b) graphs x 1 and its steady state estimate x 1, while Figure 6, Part (c) graphs x 2 and its steady state estimate x 2. Finally, it is noted that our steady state estimate x 1 for the state space x 1 is very good, that is, the graphs for x 1 and x 1 are practically identical. Let M n = span{y(j) : j n}. Recall that x(n) = P Mn 1 x(n) is the optimal state estimate given the past {y(j)} n 1. Now consider the state estimate x(n n) = P Mn x(n). Clearly, x(n n) is a better estimate of the state x(n) than x(n). According to the class notes in steady state x(n n) ( I P C (CP C + DD ) 1 C ) x(n) + P C (CP C + DD ) 1 y(n). This steady state estimate of x(n n) is also included in our Simulink program as the third variable going into the respective scopes. As expected, x(n n) is a slightly better estimate of the state x(n) than x(n); see Figure 6. 21

22 K*u Gain y output K*u Gain2 Random Number Random Number1 y(n)=cx(n)+du(n) x(n+1)=ax(n)+bu(n) Discrete State Space K*u Gain1 first state second state Figure 5: The Simulink block diagram for w = K x. 22

23 Time offset: Time offset: (a) The output y. (b) The state x 1, x 1 and x 1 (n n) Time offset: (c) The state x 2, x 2 and x 2 (n n) Figure 6: The plots of y, x 1, x 1, x 2 and x 2. 23

24 13 The Kalman filter and state feedback control Assume that {A, B, C, D} is a controllable and observable discrete time system. The state equations for the discrete time Kalman filtering problem with a control w are given by x(n + 1) = Ax(n) + Bu(n) + B 1 w(n) and y(n) = Cx(n) + Dv(n). (13.1) Here w is a known input or control with values in W, and B 1 is an operator mapping W into X. As before, u and v are independent white noise processes. Recall that the Kalman filter is given by x(n + 1) = (A L n C) x(n) + L n y(n) + B 1 w(n) L n = AQ n C (CQ n C + DD ) 1 (13.2) Q n+1 = AQ n A + BB A n C (C n C + DD ) 1 C n A. The initial condition Q = Ex()x(). Finally, if Q =, then Q n is increasing (Q n Q n+1 for all n), and Q n converges to P, the unique positive solution to the algebraic Riccati equation in (11.2). Now let us use the estimated state x to design a state feedback controller K. To this end, compute a state feedback controller K mapping X into W such that A B 1 K is stable. One can use pole placement or the discrete time linear quadratic regulator to determine a state feedback gain K satisfying a specified performance criteria. Since x is the estimated state, the feedback control is given by w = K x. Substituting w(n) = K x(n) into x(n + 1) = Ax(n) + Bu(n) + B 1 w(n) and (13.2) with y = Cx + Dv yields x(n + 1) = A B 1K x(n) + B u(n). (13.3) x(n + 1) L n C A L n C B 1 K x(n) LD v(n) 24

25 So the feedback system is precisely a state space system driven by the white noise [ tr. process u v] By repeating our previous arguments, with x = x x, we see that the system in (13.3) is similar to the following time varying state space system x(n + 1) = A B 1K B 1 K x(n) + B u(n). (13.4) x(n + 1) A L n C x(n) B L n D v(n) The example in (12.1) revisited. system in (12.1), that is, x 1(n + 1) = x 2 (n + 1) Now let us apply the Kalman filter to the 1 x 1(n) + u + 2 w(t) 2 3 x 2 (n) 1 1 y = x 1 + x 2 + v. (13.5) The Matlab commands we used to simulate the state space system in (13.3) are given by a = [, 1; 2, 3]; b = [; 1]; b1 = [2; 1]; c = [1, 1]; d = 1; q{1} = zeros(2, 2); for j = 1 : 1; q{j+1} = a q{j} a +b b a q{j} c inv(c q{j} c +d d ) c q{j} a ; end for j = 1 : 1; L{j} = a q{j} c inv(c q{j} c + d d ); end p = dare(a, c, b b, d d ); norm(p q{1}) k = dlqr(a, b1, eye(2), 1/8) for j = 1 : 1; t{j} = [a, b1 k; L{j} c, a L{j} c b1 k]; B{j} = [b, [; ]; [; ], L{j} d]; end for j = 1 : 1; u1{j} = randn(2, 1); end x1 = zeros(4, 1); for j = 1 : 1; x{j + 1} = t{j} x{j} + B{j} u1{j}; end 25

26 for j = 1 : 1; x1(j) = x{j}(1); x2(j) = x{j}(2); x3(j) = x{j}(3); x4(j) = x{j}(4); end Next we compute the steady state response in (11.3). L s = a p c inv(c p c + d d ); t s = [a, b1 k; L s c, a L s c b1 k]; B s = [b, [; ]; [; ], L s d]; x s {1} = zeros(4, 1); for j = 1 : 1; x s {j + 1} = t s x s {j} + B s u1{j}; end for j = 1 : 1; x s1 (j) = x s {j}(1); x s2 (j) = x s {j}(2); x s3 (j) = x s {j}(3); x s4 (j) = x s {j}(4); end The stairs command was used to generate the graphs. Notice that we computed the state x which comes from the system using Q n, and the steady state x s using P. In this example, P Q 1 where Q =. Figure 7 Parts (a) and (b) compare x 1 with the steady state x 1s, and respectively, x 2 with the steady state x 2s. Because Q n yields a more accurate estimate of the state than the steady state, in the beginning x 1 x 1s and x 2 x 2s. Since the feedback controller is trying to make the state as small as possible, using Q n yields a better result for about 1 steps. Then the system converges to the steady state response. Finally, Figure 7, Parts (c) and (d) compares x and x. The estimates are quite good. 26

27 The first state with Q n and P The second state with Q n and P (a) states x 1 and x 1s. (b) The states x 2 and x 2s. 8 The first state and its estimate 6 The second state and its estimate (c) The state x 1 and x 1. (d) The state x 2 and x 2 Figure 7: The plots of y, x 1, x 1, x 2 and x 2. 27