The generalised continuous algebraic Riccati equation and impulse-free continuous-time LQ optimal control

Size: px

Start display at page:

Download "The generalised continuous algebraic Riccati equation and impulse-free continuous-time LQ optimal control"

Marybeth Palmer
5 years ago
Views:

1 The generalised continuous algebraic iccati equation and impulse-free continuous-time LQ optimal control Augusto Ferrante Lorenzo Ntogramatzidis arxiv: v1 math.ds 23 May 213 Dipartimento di Ingegneria dell Informazione Università di Padova, via Gradenigo, 6/B Padova, Italy Department of Mathematics and Statistics Curtin University, Perth WA, Australia. October 9, 218 Abstract The purpose of this paper is to investigate the role that the continuous-time generalised iccati equation plays within the context of singular linear-quadratic optimal control. This equation has been ined following the analogy with the discrete-time generalised iccati equation, but, differently from the discrete case, to date the importance of this equation in the context of optimal control is yet to be understood. This note addresses this point. We show in particular that when the continuous-time generalised iccati equation admits a symmetric solution, the corresponding linear-quadratic (LQ) problem admits an impulse-free optimal control. Keywords: generalised iccati difference equation, finite-horizon LQ problem, generalised discrete algebraic iccati equation, extended symplectic pencil. Partially supported by the Italian Ministry for Education and esearch (MIU) under PIN grant n. 285FFJ2Z New Algorithms and Applications of System Identification and Adaptive Control and by the Australian esearch Council under the grant FT esearch carried out while the first author was visiting Curtin University, Perth (WA), Australia.

2 1 Introduction iccati equations are universally regarded as a cornerstone of modern control theory. In particular, it is well known that the solution of the classic finite and infinite-horizon LQ optimal control problem strongly depends on the matrix weighting the input in the cost function, traditionally denoted by. When is positive inite, the problem is said to be regular (see e.g. 1, 8), whereas when is positive semiinite, the problem is called singular. The singular cases have been treated within the framework of geometric control theory, see for example 6, 13, 1 and the references cited therein. In particular, in 6 and 13 it was proved that an optimal solution of the singular LQ problem exists for all initial conditions if the class of allowable controls is extended to include distributions. In the discrete time, the solution of finite and infinite-horizon LQ problems can be found resorting to the so-called generalised discrete algebraic iccati equation. In particular, the link between the solutions of LQ problems and the solutions of generalised discrete algebraic/difference equations have been investigated in 9, 4 for the finite horizon and in 3 for the infinite horizon. A similar generalisation has been carried out for the continuous-time algebraic iccati equation in 7, where the generalised iccati equation was ined in such a way that the inverse of appearing in the standard iccati equation is replaced by its pseudo-inverse. Some conditions under which this equation admits a stabilising solution were investigated in terms of the so-called lating subspaces of the extended Hamiltonian pencil. Some preliminary work on the continuous-time algebraic iccati equation within the context of spectral factorisation has been carried out in 2 and 12. Nevertheless, to date the role of this equation in relation to the solution of optimal control problems in the continuous time has not been fully explained. The goal of this paper is to fill this gap, by providing a counterpart of the results in 3 for the continuous case. In particular, we describe the role that the generalised continuous algebraic iccati equation plays in singular LQ optimal control. Such role does not trivially follow from the analogy with the discrete case. Indeed, in the continuous time, whenever the optimal control involves distributions, none of the solutions of the generalised iccati equation is optimising. The goal of this paper is to address this delicate issue. Thus, the first aim of this paper is to explain the connection of the generalised continuous-time algebraic iccati equation and of the generalised iccati differential equation which is also ined by substitution of the inverse of with the pseudo-inverse and the solution of the standard LQ optimal control problem with infinite and finite horizons, respectively. We will show that when the generalised iccati equation possesses a symmetric solution, both the finite and the infinite-horizon LQ problems admit an impulse-free solution. Moreover, such control can always be expressed as a state-feedback, where the gain can be obtained from the solution of the generalised continuous-time algebraic/differential iccati 1

3 equation. We also provide an insightful geometric characterisation of the situations in which the singular LQ problem admits impulse-free solutions, in terms of the largest output-nulling controllability subspace and of the smallest input-containing subspace of the underlying system. Notation. The image and the kernel of matrix M are denoted by im M and ker M, respectively, while the transpose and the Moore-Penrose pseudo-inverse of M are denoted by M T and M, respectively. 2 Generalised CAE We consider the following matrix equation X A+A T X (S+ X B) (S T + B T X)+Q=, (1) with Q,A n n, B,S n m, m m and we make the following standing assumption: Q S Π= = Π T. (2) S T Eq. (1) is often referred to as the generalised continuous algebraic iccati equation GCAE(Σ), and represents a generalisation of the classic continuous algebraic iccati equation arising in infinite-horizon LQ problems since here is allowed to be singular. Eq. (1) along with the additional condition ker ker(s+ X B), (3) will be referred to as constrained generalised continuous algebraic iccati equation, and is denoted by CGCAE(Σ). Observe that from (2) we have ker kers, which implies that (3) is equivalent to ker ker(x B). The following notation is used throughout the paper. First, let G= I m be the orthogonal projector that projects onto ker. Hence, is the orthogonal projector that projects onto im = im. In fact, ker = img. Moreover, we consider a non-singular matrix T = T 1 T 2 where imt 1 = im and imt 2 = img, and we ine B 1 = BT 1 and B 2 = BT 2. Finally, to any X = X T n n we associate Q X K X Π X = Q+A T X+ X A, S X = S+ X B, (4) = (S T + B T X)= SX, T A X = A BK X, (5) QX S X. (6) = S T X Lemma 2.1 Let X be a solution of CGCAE(Σ). Then, X B 2 =. 2

4 Proof: From (3) and from ker=img, we get (S+X B)G=. Moreover, since Π, kers ker. This means that K n m exists such that S = K. Therefore, S = K = K = S, so that SG=S S =. Hence, im(bg) kerx or, equivalently, XB 2 =. Lemma 2.2 Let F = A B S T and Λ=Q S S T. Then, Λ and GCAE(Σ) ined in (1) has the same set of symmetric solutions of the following equation: X F+ F T X X B B T X+ Λ= (7) Proof: Matrix Λ is the generalised Schur complement of in Π. Therefore, since Π is positive semiinite, such is also Λ. The rest of the proof is a matter of standard substitutions of F and Λ into (7) to verify that (1) is obtained. emark 2.1 The result established for GCAE(Σ) in Lemma 2.2 extends without further difficulties to the so-called generalised iccati differential equation GDE(Σ) Ṗ(t)+P(t)A+A T P(t) (S+ P(t)B) (S T + B T P(t))+Q=. (8) Indeed, we easily see that (8) has the same set of symmetric solutions of the equation: Ṗ(t)+P(t)F+ F T P(t) P(t)B B T P(t)+Λ=. (9) Lemma 2.3 Let X = X T be a solution of CGCAE(Σ). Let (F,B 2 ) be the reachable subspace of the pair(f,b 2 ). Then (1) kerx kerλ; (2) X (F,B 2 )={}; (3) Λ(F,B 2 )={}. Proof: (1). Let ξ kerx. Multiplying (7) to the left by ξ T and to the right by ξ, we get ξ T Λξ =. Since Λ, this implies Λξ =. Hence, kerx kerλ. (2). Let ξ kerx. Post-multiplying (7) by ξ we find X F ξ =. This implies that kerx is F-invariant. In view of Lemma 2.2, the subspace kerx contains imb 2. Hence, it contains (F,B 2 ) that is the smallest F-invariant subspace containing imb 2. This implies (F,B 2 ) kerx. (3). This follows directly from the chain of inclusions (F,B 2 ) kerx kerλ. 3

5 3 The finite-horizon LQ problem Lemma 3.1 Let H = H T be such that H (F,B 2 )={}. If CGCAE(Σ) (1-3) admits solutions, the generalised iccati differential equation Ṗ T (t)+p T (t)a+a T P T (t) (S+ P T (t)b) (S T + B T P T (t))+q=, (1) with the terminal condition P T (T)=H (11) admits a unique solution for all t T, and this solution satisfies P T (t)bg= for all t T. Proof: Consider a set of coordinates in the input space such that the first coordinates span im and the second set of coordinates spans img=ker. In this basis can be written as = with 1 being 1 invertible. In the same basis, matrix B can be partitioned accordingly as B =B 1 B 2 as shown above, so that imb 2 = im(bg). Let us now consider the change of basis matrix U =U 1 U 2 in the state space where imu 1 = (F,B 2 ), so that U 1 F11 F 12 F U =, U 1 B 1 = O F 22 and U T ΛU = O O O P 22 (t) O O B11 B 12, U 1 B 2 = B21 where we have used the fact that Λ(F,B O Λ 22 2 ) = {}. In this basis, since we are O O. Consider the following matrix function O H 22 assuming H (F,B 2 ) = {}, we can write U T H U = P T (t)=, where P 22 (t) satisfies Ṗ 22 (t)+p 22 (t)f 22 +F T 22 P 22(t) P 22 (t)vp 22 (t)+λ 22 = (12) P 22 (T)=H 22, (13) in which V is the sub-block 22 of the matrix B B T. O O O P 22 (t) O From 5, Corollary 2.4 we find that, since Π = Π T and H = H T, both (1) and (12) admit a unique solution ined in (,T. It is easy to see that P T (t)=, where P 22 (t), t (,T, is the solution of (12-13), solves (1) and (11). We can therefore conclude that P T (t) is the unique solution of (1-11). Moreover, this solution O O B21 satisfies P T (t)b 2 = for all t T since in the chosen basis P T (t)b 2 = =. O P 22 (t) O, 4

6 Now we consider the generalised iccati problem GDE(Σ) (1-11) in relation with the finite-horizon LQ problem, which consists in the minimisation of the performance index T Q S x(t) J T,H (x,u) = x T (t) u T (t), S T u(t) where we only assume Π= Q S S T +x T (T)H x(t) (14) = Π T subject to ẋ(t)= Ax(t)+Bu(t), (15) and the constraint on the initial state x()=x n. The following theorem is the first main result of this paper. It shows that when CGCAE(Σ) admit a solution, the finite-horizon LQ problem always admits an impulse-free solution. Theorem 3.1 Let CGCAE(Σ) admit a solution. The finite-horizon LQ problem (14-15) admits impulsefree optimal solutions. All such solutions are given by u(t)= (S T +B T P T (t)) x(t)+gv(t), (16) where v(t) is an arbitrary regular function, and P T (t) is the solution of (1) with the terminal condition (11). The optimal cost is x T P T()x. Proof: Let us first assume that H (F,B 2 )={}. The cost (14) can be written for any matrix-valued differentiable function P(t) as J T,H (x,u) = = T where we have used the fact that T x T (t) u T (t) Q S T S T x(t) u(t) d ( +x T (T)H x(t)+ x T (t)p(t)x(t) ) +x T ()P()x() x T (T)P(T)x(T) Q+A T P(t)+P(t)A+Ṗ(t) P(t)B+S x(t) x T (t) u T (t) S T +B T P(t) u(t) +x T (T)(H P(T))x(T)+x T ()P()x(), d ( x T (t)p(t)x(t) ) = ẋ T (t)p(t)x(t) +x T (t)ṗ(t)x(t)+x T (t)p(t)ẋ(t). 5

7 Let us now consider P(t)=P T (t) to be the solution of (1) with final condition P T (T) = H. Since, as proved in Lemma 3.1, the identity P T (t)bg= holds for all t T, we have also ker(p T (t)b) ker for all t T. Thus, ker(p T (t)b+s) ker for all t T, and we can write Q+AT P T (t)+p T (t)a+ P T (t) P T (t)b+s = = S T + B T P T (t) (PT (t)b+s) (S T + B T P T (t)) P T (t)b+s S T + B T P T (t) O (PT (t)b+s) 1 2 O 1 2 O O 1 2 (S T + B T P T (t)) 1 2 since ker(p T (t)b+s) ker gives(p T (t)b+s) =(P T (t)b+s). Hence, J T,H (x,u) = T 1 2 (S T + B T P T (t))x(t)+ 1 2 u(t) 2 2 +x T ()P T ()x(), since P T (T)=H. If there exists a control u(t) for which 1 2 (S T + B T P T (t))x(t)+ 1 2 u(t)= (17) for all t,t), then the cost function is minimal in correspondence with this control and all minimising controls satisfy (17). The set of controls satisfying (17) can be parameterised as u(t) = (S T + B T P T (t))x(t)+ Gv(t), where G = I m and v(t) is arbitrary. 1 Now, consider the case (F,B 2 ) kerh. Consider the change of basis U = U 1 U 2 where imu 1 = (F,B 2 ), and where imu 2 is the U T orthogonal complement of imu 1. Changing coordinates gives 1 H11 H 12 U2 T HU 1 U 2 = H12 T H. Let us now 22 I U21 perform a further change of coordinates with the matrix such that U O I 21 = H 11 H 12. There holds I O H 11 H 12 I U 21 = H 11 O, I H12 T H 22 O I O H 22 U T 21 where H 22 = H12 T U 21+ H 22. Thus, by writing the cost function in this new basis we get T Q S x(t) J T,H (x,u) = x T (t) u T (t) S T u(t) + x T 1 (T) xt 2 (T) H 11 O x1 (T). O H 22 x 2 (T) 1 Note that this has to be understood in a L 2 sense. 6

8 Clearly, J T,H (x,u) J T,H (x,u), where H O O = O H 22. On the other hand, in the optimal control ined by J T,H (x,u) there is a degree of freedom which is the component of the state trajectory on (F,B 2 ). In other words, in the minimisation of J T,H (x,u) we can decide to drive x 1 to the origin in,t without destroying optimality. As such, J T,H (x,u)=j T,H (x,u). The penalty matrix of the final state in this new performance index satisfies O O O H 22 (F,B 2 )={}. emark 3.1 Notice that when H =, we have P T () P T+δT () for all δt, because J t, (x,u) is a non-decreasing function in t. We are now interested in studying P T () when the terminal condition vanishes, i.e., when H =, and the time interval increases. To this end, we consider a generalised iccati differential equation where the time is reversed, and where the terminal condition becomes an initial condition, which is now equal to zero. More specifically, we consider the new matrix function X(t)= P t ()=P T (T t). We re-write GDE(Σ) as a differential equation to be solved forward: Ẋ(t)= X(t)A+A T X(t) (S+ X(t)B) (S T + B T X(t))+Q, (18) X()=. (19) In the following theorem, the second main result of this paper is introduced. This theorem determines when the infinite-horizon LQ problem admits an impulse-free solution, and the set of optimal controls minimising the infinite-horizon cost subject to the constraint (15). J (x,u) = x T (t) u T (t) Q S S T x(t) u(t) (2) Theorem 3.2 Suppose CGCAE(Σ) admits symmetric solutions, and that for every x there exists an input u(t) m, with t, such that J (x,u) in (2) is finite. Then we have: (1) A solution X = X T of CGCAE(Σ) is obtained as the limit of the time varying matrix generated by integrating (18) with the zero initial condition (19). (2) The value of the optimal cost is x T Xx. (3) X is the minimum positive semiinite solution of CGCAE(Σ). 7

9 (4) The set of all optimal controls minimising J in (2) can be parameterised as with arbitrary v(t). Proof: (1). Consider the problem of minimising J t, = t u(t)= S T X x(t)+gv(t), (21) x T (τ) u T (τ)π x(τ) subject to (15) with assigned initial state x n. From Theorem 3.1 the optimal control for this problem exists, and the optimal cost is equal to J (x,u)=x T P T()x = x T X(T)x. We have already observed that X(t)=P t () is an increasing flow of matrices in the sense of the positive semiiniteness of symmetric matrices, i.e., X(t + δt) X(t) for all δt. We now show that X(t) is bounded. Indeed, given the i-th canonical basis vector e i of n, we have e T i X(t)e i J (e i,ū i ), where ū i is a control that renders J (e i,ū i ) finite (which exists by assumption). Thus, u(τ) X(t) I n max{j (e i,ū i ) : i {1,...,n}} t. Therefore, X(t) is bounded. Taking the limit on both sides of (18) we immediately see that X is indeed a solution of CGCAE(Σ). (2). Let J (x ) = inf J (x,u). (22) u Clearly, J (x ) Jt (x )=x T X(t)x for all t. Then, by taking the limit, we get J (x ) x T Xx. We now show that the time-invariant feedback control ut = K X x t, where K X = (S T + B T X), yields the cost x T Xx, which is therefore the optimal value of the cost. Consider the cost index J T, X (x,u). The optimal cost for this index is achieved by using the controls satisfying (16), where P T (t) is constant and equal to X, since X is a stationary solution of (1-11) and H = X. Therefore, an optimal control for this index is given by the time-invariant feedback u (t)= K X x(t). The optimal cost does not depend on the length T of the time interval and is given by J T, X = Xx xt. Now we have x T Xx J (x ) J(x,u ) = x T (t) (u ) T (t) T = lim T x T (t) (u ) T (t) dτ x(t) Π u (t) Π x(t) u (t) = lim T J T, X xt T Xx T lim T xt Xx = x T Xx. (23) 8

10 Comparing the first and last term of the latter expression we see that all the inequalities are indeed equalities, so that the infimum in (22) is a minimum and its value is indeed x T Xx. (3). Suppose by contradiction that there exist another positive semiinite solution X of CGCAE(Σ) and a vector x n such that x T X x < x T X x. Take the time-invariant feedback ũ(t)= K X x(t). The same argument that led to (23) now gives J (x,ũ) x T Xx < x T X x, which is a contradiction because we have shown that x T X x is the optimal value of J (x,u). (4). Consider J = x T X x = inf u(t),t x T (t) u T (t) Q S S T x(t) u(t). This infimum is indeed a minimum because we know that the optimal cost x T X x can be obtained for some u. Hence, J = min u(t),t x T (t) u T Q S x(t) (t). For any control u(t), t, ), and for a given T >, let x u (t) be the state reached at time t = T starting from initial condition x() and using the control u(t), t, T. Then J S T T = min Q S x(t) x T (t) u T (t) u(t),t S T u(t) Q S x(t) + x T (t) u T (t) T S T u(t) T x(t) = min x T (t) u T (t) Π + x T u(t) X x u (T), u(t),t,t) u(t) }{{} J T, X (x,u) where the latter is due to the principle of optimality. Thus, u(t), t, ) minimises J if and only if u(t), t,t minimises J T, X (x,u) and u(t), t T, ) is such that T x T (t) u T (t) Q S S T x(t) u(t) u(t) = x T u (T) X x u (T). The set of controls that minimise J are those, and only those, that minimise J T, X. The optimal cost of the latter problem is independent of how big the value of T is selected. This optimal cost is achieved by using the controls given by (16), where P T (t) is constant and equal to X, since X is a stationary solution of (1-11) and H = X. 9

11 We conclude this section with a result that links the existence of solutions of the generalised iccati equation with a geometric identity involving the smallest input containing subspace S and the largest reachability output-nulling subspace of the underlying system, i.e., of the quadruple (A,B,C,D), where C and D are matrices of suitable sizes such that C T Π= C D D T For more details of the underlying geometric concepts of input-containing and output-nulling subspaces, we refer to the monograph 11. Proposition 3.1 Let CGCAE(Σ) admit a solution X = X T. Then, S =. Proof: Let X = X T be a solution of CGCAE(Σ). Observe also that CGCAE(Σ) can be re-written as { X A + A T X X B B T X+ Q = (24) ker kerx B where A = A B S T and Q = Q S S T. ecall that G = I m, so that B 2 = BG, and (24). becomes { X A + A T X X B B T X+ Q = X BG= (25) It is easy to see that kerx kerq. Indeed, by multiplying the first of (25) on the left by ξ T and on the right by ξ, where ξ kerx, we get ξ T Q ξ =. However, Q is positive semiinite, being the generalised Schur complement of Q in Π. Hence, Q ξ =, which implies kerx kerq. Since X BG =, we get also Q BG =. By post-multiplying the first of (25) by a vector ξ kerx we find X A ξ =, which says that kerx is A -invariant. This means that kerx is an A -invariant subspace containing the image of BG. Then, the reachable subspace of the pair (A,BG), denoted by (A,BG), which is the smallest A -invariant subspace containing the image of BG, is contained in kerx, i.e., (A,BG) kerx. Therefore also (A,BG) kerq. Notice that Q can be written as C T C, where C = C D S T. Indeed, C T C = C T C C T D S T S D T C+ S D T D S T = Q S S S S T + S S T = Q. Consider the two quadruples (A,B,C,D) and (A,B,C,D). We observe that the second is obtained directly from the first by applying the feedback input u(t)= Sx(t)+v(t). We denote by V, the 1

12 largest output-nulling and reachability subspace of(a,b,c,d), and by S the smallest input-containing subspace of (A,B,C,D). Likewise, we denote by V,, S the same subspaces relative to the quadruple(a,b,c,d). Thus, V = V, =, and S = S. The first two identities are obvious, as output-nulling subspaces can be made invariant under state-feedback transformations and reachability is invariant under the same transformation. The third follows from 11, Theorem There holds = (A,BG). Indeed, consider a state x 1 (A,BG). There exists a control function u driving the state from the origin to x 1, and we show that this control keeps the output at zero. Since im(bg)= B kerd, such control can be chosen to satisfy Du(t)= for all t. Moreover, as we have already seen, from Q = C T C and (A,BG)=(A,BG) we have C (A,BG)= since (A,BG) lies in kerq. Therefore, the output is identically zero. This implies that (A,BG). However, the reachability subspace of(a,b,c,d) cannot be greater than (A,BG), since D T C = D T (I m D(D T D) D T )C=. Therefore, such control must necessarily render the output non-zero. The same argument can be used to prove that S = (A,BG), where distributions can also be used in the allowed control, since (A,BG) represents also the set of states that are reachable from the origin using distributions in the control law 11, p Hence, S =. 4 Concluding remarks In this paper we established a new theory that showed that, when the CGCAE(Σ) admits solutions, the corresponding singular LQ problem admits an impulse-free solution, and the optimal control can be expressed in terms of a state feedback. A very interesting question, which is currently being investigated by the authors, is the converse implication of this statement: when the singular LQ problem admits a regular solution for all initial states x n, does the CGCAE(Σ) admit at least one symmetric positive semiinite solution? At this stage we can only conjecture that this is the case, on the basis of some preliminary work, but the issue is indeed an open and interesting one. eferences 1 B.D.O. Anderson and J.B. Moore. Optimal Control: Linear Quadratic Methods. Prentice Hall International, London, T. Chen and B.A. Francis. Spectral and inner-outer factorisations of rational matrices. SIAM J. Matrix Anal. Appl., 1(1):1 17,

13 3 A. Ferrante, L. Ntogramatzidis, The generalised discrete algebraic iccati equation in LQ optimal control. Automatica, 49(2): , A. Ferrante, L. Ntogramatzidis, The extended symplectic pencil and the finite-horizon LQ problem with two-sided boundary conditions. In press on IEEE Trans. Aut. Control. Manuscript available at 5 G. Freiling, G. Jank, and A. Sarychev. Non-blow-up conditions for iccati-type matrix differential and difference equations. esults of Mathematics, 37:84 13, 2. 6 M.L.J. Hautus and L.M. Silverman. System structure and singular control. Linear Algebra Appl., 5:369 42, V. Ionescu and C. Oarǎ. Generalized continuous-time iccati theory. Linear Algebra Appl., 232:111 13, F.L. Lewis and V. Syrmos. Optimal Control. John Wiley & Sons, New York, D. appaport and L.M. Silverman. Structure and stability of discrete-time optimal systems. IEEE Trans. Aut. Control, AC-16: , A. Saberi and P. Sannuti. Cheap and singular controls for linear quadratic regulators. IEEE Trans. Aut. Control, AC-32(3):28 219, March H.L. Trentelman, A.A. Stoorvogel, and M. Hautus. Control theory for linear systems. Springer, M. Weiss, Spectral and inner-outer factorisations through the constrained iccati equation. IEEE Trans. Aut. Control, AC-39(3): , J.C. Willems, A. Kìtapçi, and L.M. Silverman. Singular optimal control: a geometric approach. SIAM J. Control Optim., 24(2): , March

LQR and H 2 control problems

LQR and H 2 control problems Domenico Prattichizzo DII, University of Siena, Italy MTNS - July 5-9, 2010 D. Prattichizzo (Siena, Italy) MTNS - July 5-9, 2010 1 / 23 Geometric Control Theory for Linear