Lecture 2 and 3: Controllability of DT-LTI systems

1 Lecture 2 and 3: Controllability of DT-LTI systems Spring 2013 - EE 194, Advanced Control (Prof Khan) January 23 (Wed) and 28 (Mon), 2013 I LTI SYSTEMS Recall that continuous-time LTI systems can be represented by constant-coefficient, linear differential equations A simple example in the series RLC circuit where we get two differential equations, one each for the voltage across the capacitor, v c (t), and the current through the inductor, i L (t) The equations have the following form: v c (t) = a 11 v c (t) + a 12 i L (t) + b 1 v s (t), (1) i L (t) = a 21 v c (t) + a 22 i L (t) + b 2 v s (t), (2) where v s (t) is the input (source voltage) to the series RLC circuit and the coefficients, a ij and b i, depend on the circuit parameters, R, L, C Note that ẋ(t) = d x(t) (3) dt The above set of equations can be compactly written in the matrix form as follows: v c (t) a 11 a 12 v c (t) b 1 = + v s (t), (4) i L (t) a 21 a 22 i L (t) b 2 ẋ(t) = Ax(t) + Bu(t) (5) The fact that the above equation represents continuous-time dynamics is easily identified by the dot above x(t) and the (t) s are often ignored Note that any number of differential equations can be compactly written in the matrix form first-order, linear, matrix differential equation Furthermore, if continuous-time dynamics are given by an nth order differential equation, eg: ẋ (n) = a n 1 ẋ (n 1) + a n 2 ẋ (n 2) + a 1 ẋ + a 0 x + bu, (6) where ẋ (n) = dn x(t); (7) d n t The lecture material is largely based on: Fundamentals of Linear State Space Systems, John S Bay, McGraw-Hill Series in Electrical Engineering, 1998

2 then the matrix representation can be verified to be x (n 1) a n 1 a n 2 a 0 x (n 2) = 1 0 x 1 0 x (n 1) x (n 2) x + b 0 0 u, (8) which is again in the form of Eq (5) The discrete-time dynamics can now be obtained from Eq (5) by only specifying a sampling time, T s In particular, the discrete-time dynamics (sampled) are given by x k+1 = e ATs x k + A 1 (e ATs I)Bu k (9) It is not uncommon to use the first-order approximation of the matrix exponential, e( ), to get the following approximation of the discrete-time dynamics: x k+1 = (I + T s A)x k + T s Bu k (10) The above approximation can be equivalently realized from Eq (5) by using the Euler approximation of the derivative the approximations are valid for small enough T s The above discretizes continuous-time dynamics to obtain an equivalent discrete-time representation (A-D conversion); however, there do exist LTI systems that are by nature discrete-time Examples are: Linear prediction in speech processing Time-series modeling for stocks Remark 1 The matrix exponential, e A, is defined as: e A = I + A + A2 2! + A3 3! + = It can be shown that when A = V DV 1 where D is a diagonal matrix, then m=0 A m m! (11) e A = V e D V 1 (12) Remark 2 The matrix differential equation: ẋ = Ax + Bu is a compact representation of an nth order continuous-time LTI system The measurements of this system are often represented by y = Cx Remark 3 The solution to ẋ = Ax + Bu is given by x(t) = e A(t t 0) x(t 0 ) + t t 0 e A(t τ) Bu(τ)dτ (13)

3 Remark 4 The matrix difference equation: x k+1 = Ax k + Bu k is a compact representation of an nth order discrete-time LTI system The measurements of this system are often represented by y k = Cx k Remark 5 The solution to x k+1 = Ax k + Bu k is given by k x k+1 = A k+1 x 0 + A m Bu k m (14) m=0 II DISCRETE-TIME LTI DYNAMICS For a large part of this course, we will deal with DT LTI systems The results are very similar to the CT LTI systems, if not exactly the same; however, the analysis is more involved in the CT case and requires a functional knowledge of functional calculus In the (almost) most general form, the matrix representation of DT LTI dynamics are x k+1 = Ax k + Bu k + v k, (15) y k = Cx k + Du k + r k (16) Since u k is a known sequence of inputs, it is not uncommon to assume D = 0 we will use this assumption The list of the variables involved are defined as follows: State vector: x k R n, k 0 Initial condition: x 0 System matrix: A R n n Input vector: u k R m Input matrix: B R n m System/process noise: v k N(0, Q) Observation/measurement/output vector: y k R p Observation/measurement/output matrix: C R p n Observation/measurement/output noise: r k N(0, R) A Optimal control problem Find a sequence of control inputs, u 0, u 1, u 2, of the form State-feedback control: u k = K k x k, (17) such that x N = 0, while minimizing the following cost: Finite horizon LQR (ignore the process noise): N ( ) J = x T τ Qx τ + u T τ Ru τ, (18) τ=0 where Q 0, R 0 are given and they are symmetric

4 Infinite horizon LQR (ignore the process noise): J = N= τ=0 ( x T τ Qx τ + u T τ Ru τ ), (19) where Q 0, R 0 are given and they are symmetric LQG: The noise covariances, Q and R, are used in the cost N ( ) J = E x T τ Qx τ + u T τ Ru τ (20) τ=0 Remark 6 The symbol represents positive-definiteness A positive-definite matrix, A, is such that z T Az > 0 for all z 0 For positive-semidefinite matrices, we use the symbol and z T Az 0 Remark 7 It can be shown that a matrix is positive-definite if and only if all of its eigenvalues are > 0 Positive-semidefiniteness requires eigenvalues to be non-negative B Optimal estimation problem Let x k m to be the estimate of the state vector, x k, from the observations given up to time m, m k The optimal estimation problem is to find x k k of the form: x k k = L k y 0:k, (21) from all of the observations up to time k, such that the following estimation cost, is minimized It is also known as LQE J = E (xk x k k ) T ( xk x k k ), (22)

5 III CONTROLLABILITY OF DT-LTI SYSTEMS The DT-LTI system is given by the following n-dimensional state-space: where: x k+1 = Ax k + Bu k, (23) The state vector, x k, lies in R n, ie, there are n state variables; The input vector, u k, lies in R m, ie, there are m inputs; The above naturally lead to A R n n and B R n m ; The matrix A is called the system matrix and the matrix B is called the input matrix Definition 1 (Controllability) A DT-LTI system is said to be controllable in n time-steps when there exists a sequence of inputs, u 0, u 1,, u n 1, such that x n = 0 regardless of the initial condition, x 0, where 0 is a vector with n zeros In other words, using the control inputs, u 0, u 1,, u n 1, we can force an arbitrary initial condition, x 0, to go to 0 in n time-steps In the following, let us assume the extreme case when there is only 1 control input: u k R (an italic represents a scalar) Naturally this forces the input matrix, B, to lie in R n 1 A Controllability in n time-steps The n-dimensional DT-LTI state-space is given by: x k+1 = Ax k + Bu k, k = A k+1 x 0 + A m Bu k m (24) We are interested in studying the controllability in n time-steps, ie, we would like x n = 0, where 0 is a vector with n zeros From Eq (24), we have 0 = x n = A n x 0 + A n x 0 = n 1 m=0 m=0 n 1 m=0 A m Bu n 1 m A m Bu n 1 m,

6 Subsequently, A n x 0 = n 1 m=0 A m Bu n 1 m, = Bu n 1 + ABu n 2 + A 2 Bu n 3 + + A n 2 Bu 1 + A n 1 Bu 0, u n 1 u n 2 u = B AB A 2 B A n 2 B A n 1 B n 3, (25) }{{} C n u 0, }{{} u 0,,n 1 = C n u 0,,n 1 (26) Let us consider the controllability matrix, C n The first element B is n 1, the second element AB is also n 1, and the last (nth) element is also n 1 Hence, controllability matrix, C, is n n Similarly, the sequence of n control inputs, u 0, u 1,, u n 1, is compactly written as the vector, u 0,,n 1 is n 1 From the linear system of equations, it is clear that the following system of equations A n x 0 = C n u 0,,n 1, (27) has a solution when C n is invertible However, it is easy to verify that for an arbitrary input matrix, B R n m, ie, with m control inputs, the controllability matrix, C n has the dimensions n mn; being a square matrix only when m = 1 So the necessary condition for Eq (27) to have a solution is rank(c n ) = n (28) This is because an n mn matrix cannot have a rank greater than n The above exposition can be summarized as the following Conclusion: For any DT-LTI system with one control input, there exists some sequence of n control inputs, u 0, u 1,, u n 1, such that an arbitrary initial condition, x 0, can be forced to 0 in n time-steps; when ( ) rank(c n ) = rank B AB A n 1 B = n (29) Such a system is said to be controllable by Def 1 u 1

7 B Controllability in less than n time-steps For a DT-LTI system with one control input, recall from the previous section (Eq (25)) that u n 1 u n 2 u A n x 0 = B AB A 2 B A n 2 B A n 1 B n 3 u 1 u 0, If we want the system to be controllable in n 1 time steps, then the above equation leads to A n 1 x 0 = u n 2 u n 3 B AB A 2 B A n 2 B }{{} u C 1 n 1 u 0, }{{} u 0,,n 2 The new matrix, C n 1, has n rows but only n 1 columns So this system of linear equations does not have a solution since the rank cannot exceed that minimum of the number of rows and columns, ie, rank(c n 1 ) min(n, n 1) = n 1 n (30) Conclusion: Any DT-LTI system is not controllable with one control input in (strictly) less than n time-steps

8 C Controllability in more than n time-steps For a DT-LTI system with one control input, recall from the previous section (Eq (25)) that u n 1 u n 2 u A n x 0 = B AB A 2 B A n 2 B A n 1 B n 3 u 1 u 0, If we want the system to be controllable in n + 1 time steps, then the above equation leads to A n+1 x 0 = B AB A 2 B A n 2 B A n 1 B A n B }{{} C n+1 u n u n 2 u n 3 } u 0, {{ } u 0,,n Controllability in n + 1 time-steps is only desirable if the system is not controllable in n timesteps When a system is not controllable in n time-steps, we must have ( ) rank(c n ) = rank B AB A n 1 B < n In other words, in the hope of improving the rank of C n, we are appending an additional column, A n B, to C n ie, C n+1 = C n A n B We now claim that (See Section IV) In fact, the following is true rank(c n ) = rank(c n+1 ) rank(c n ) = rank(c n+1 ) = rank(c n+2 ) = Before justifying this claim, let us understand the consequence of the above statement In words, it is saying that adding additional columns of the form A n B, A n+1 B, A n+2 B,, to the matrix, C n, does not improve its rank Recall that adding additional such columns is equivalent to using more control inputs, u n, u n+1, u n+2, In short, if a system (with one control input) is not controllable in n time-steps, then it is not controllable in any number of time-steps greater than n Combining this with the previous section, we conclude that If an n-dimensional DT-LTI system with one control input is controllable, then any x 0 can be forced to 0 in exactly n time-steps, not less u 1

9 IV CAYLEY HAMILTON THEOREM The well-known Cayley Hamilton theorem states that 1 Every real-valued square matrix, A, satisfies its own characteristic polynomial The characteristic polynomial of a matrix, A R n n is defined as χ(λ) det(λi A), for some a i s From the theorem, we have or, = λ n + a n 1 λ n 1 + a n 2 λ n 2 + + a 1 λ + a 0, χ(a) = 0 A n + a n 1 A n 1 + a n 2 A n 2 + + a 1 A + a 0 I = 0 n n, (31) A n = a n 1 A n 1 a n 2 A n 2 a 1 A a 0 I, (32) ie, A n is linear combination of A 0, A 1,, A n 1 Multiplying both sides by A, we get A n+1 = a n 1 A n a n 2 A n 1 a 1 A 2 a 0 A, = a n 1 ( a n 1 A n 1 a n 2 A n 2 a 1 A a 0 I) a n 2 A n 1 a 1 A 2 a 0 A, ie, A n+1 is also linear combination of A 0, A 1,, A n 1, and so on for any integer power of A strictly greater than n 1 Since A n, A n+1, are each linear combinations of A 0, A 1,, A n 1, we can easily conclude that rank(c n ) = rank(c n+1 ) = rank(c n+2 ) = 1 The statement is more general but the following suffices

10 Example 1 (Cayley Hamilton Mechanics) Let a 3 3 square matrix be given by 2 1 2 A = 1 2 2 2 1 4 Verify that From CH theorem, we must have χ(λ) = det(λi A) = λ 3 8λ 2 + 13λ 6 A 3 8A 2 + 13A 6I = 0 3 3, A 3 = 8A 2 13A + 6I This can be verified in Matlab A = 2 1 2 1 2 2 2 1 4 eig(a) ans = 60000 10000 10000 poly(6 1 1) ans = 1-8 13-6 Aˆ3 ans = 52 35 86 51 36 86 78 51 130 8*Aˆ2-13*A + 6*eye(3) ans = 52 35 86 51 36 86 78 51 130

11 V STATE FEEDBACK CONTROL Let us consider single input systems and use the state-feedback control law, ie, With the state-feedback control, the system evolution is u k = Kx k (33) x k+1 = Ax k + Bu k = Ax k BKx k = (A BK)x k = (A BK) k+1 x 0 If a system is controllable then any arbitrary initial condition, x 0, can be forced to zero in n time steps Recall that x n = A n x 0 + C n u 0,,n 1, u n 1 u n 2 u = A n x 0 + C n n 3 = A n x 0 C n u 1 u 0, K(A BK) n 1 K(A BK) n 2 K(A BK) 0 = x n = A n x 0 C n n 3 x 0, K(A BK) K, K(A BK) n 1 K(A BK) n 2 K(A BK) A n x 0 = C n n 3 x 0, K(A BK) K, K(A BK) n 1 K(A BK) n 2 K(A BK) n 3 C 1 n A n x 0 = K(A BK) K, x 0 K(A BK) n 1 K(A BK) n 2 K(A BK) n 3 K(A BK) K, x 0,

12 Clearly, the above is only true when C n is invertible Since the above has to be true for all x 0, we have K = 0 0 0 1 Cn 1 A n, (34) where there are n 1 zeros in the above Conclusion: For a single input controllable system, ie, C n is invertible, Eq (34) takes any initial condition, x 0, to 0 in n time-steps