Lecture 6. Foundations of LMIs in System and Control Theory

Lecture 6. Foundations of LMIs in System and Control Theory Ivan Papusha CDS270 2: Mathematical Methods in Control and System Engineering May 4, 2015 1 / 22

Logistics hw5 due this Wed, May 6 do an easy problem or CYOA hw4 solutions posted online 2 / 22

Riccati differential equation Recall the Riccati boundary value problem, Ṗ = AT P +PA+Q PBR 1 B T P P(T) = Q T where P(t) = P(t) T 0 is the variable. Q,Q T 0 and R 0 given used in solving finite horizon LQR with terminal matrix Q T steady state solution as T is the solution to the ARE, 0 = A T P +PA+Q PBR 1 B T P question. how do we solve for P in the ARE? 3 / 22

Classical technique Write down the optimality conditions for finite horizon LQR T 1 minimize x(t) T Qx(t)+u(t) T Ru(t)dt + 1 2 0 2 x(t)t Q T x(t) subject to ẋ(t) = Ax(t)+Bu(t), t (0,T) x(0) = z dual variables ν : [0,T] R n Lagrangian: T L(x,u,ν) = 1 x(t) T Qx(t)+u(t) T Ru(t)dt+ 1 2 0 2 x(t)t Q T x(t) T + ν(t) T (Ax(t)+Bu(t) ẋ(t))dt +ν(0) T (x(0) z) 0 4 / 22

Optimality conditions Lagrangian: L(x,u,ν) = 1 2 + T 0 T interior. for t (0,T) we have: integration by parts: T 0 0 x(t) T Qx(t)+u(t) T Ru(t)dt + 1 2 x(t)t Q T x(t) ν(t) T (Ax(t)+Bu(t) ẋ(t))dt +ν(0) T (x(0) z) ν(t) T ẋ(t)dt = ν(t) T x(t) x(t) L = Qx(t)+A T ν(t)+ ν(t) = 0 u(t) L = Ru(t)+B T ν(t) = 0 ν(t) L = Ax(t)+Bu(t) ẋ(t) = 0 t=t t=0 T 0 ν(t) T x(t)dt 5 / 22

Optimality conditions boundaries. x(t) L = Q T x(t) ν(t) = 0 ν(0) L = x(0) z = 0 boundary value problem. ẋ(t) = Ax +Bu(t), x(0) = z u(t) = R 1 B T ν(t) ν(t) = Qx(t)+A T ν(t), ν(t) = Q T x(t) after substituting u(t) = R 1 B T ν(t), we obtain [ẋ ] [ ][ ] A BR = 1 B T x ν Q A T, x(0) = z, ν(t) = Q ν T x(t) }{{} H 6 / 22

Hamiltonian matrix Consider the Hamiltonian matrix differential equation, [Ẋ(t) ] [ ][ ] A BR = 1 B T X(t) Ẏ(t) Q A T, Y(t) }{{} H with matrix variables X(t),Y(t) R n n the matrix H R 2n 2n is a Hamiltonian matrix such matrices obey JH = H T J, where [ ] 0 I J =, (J 1 = J T = J) I 0 the eigenvalues of H are symmetric about real and imaginary axes proof. H is real, and H T v = λv implies HJv = λjv 7 / 22

Relationship between Riccati and Hamiltonian ODEs If X(t),Y(t) R n n obey the Hamiltonian ODE [Ẋ(t) ] [ ][ ] A BR = 1 B T X(t) Ẏ(t) Q A T, Y(t) then P(t) = Y(t)X(t) 1 obeys the Riccati ODE, Ṗ = AT P +PA+Q PBR 1 B T P. proof. Ṗ = (ẎX 1 +Y (X 1 )) = ẎX 1 +YX 1 ẊX 1 (using ẊX 1 +X (X 1 ) = 0) = (QX +A T Y)X 1 +YX 1 (AX BR 1 B T Y)X 1 = A T P +PA+Q PBR 1 B T P. 8 / 22

Spectrum of the Hamiltonian matrix The matrix P defines a similarity transformation, [ ][ ][ ] I 0 A BR 1 B T I 0 P I Q A T P I [ ] A BR = 1 B T P BR 1 B T (A T P +PA+Q PBR 1 B T P) (A BR 1 B T P) T and if P further satisfies the ARE, then with K = R 1 B T P, this equals [ ] A+BK BR 1 B T 0 (A+BK) T. Thus if (A,B) is controllable, the eigenvalues of H are related to the closed loop eigenvalues by spec(h) = spec(a+bk) spec( (A+BK)). 9 / 22

Solving the ARE If A + BK (stable) is diagonalizable, T 1 (A+BK)T = Λ, then hence [ ][ ] [ ][ ] T 1 0 I 0 I 0 T 0 0 T T H P I P I 0 T T [ ][ ][ ] T 1 0 A+BK BR = 1 B T T 0 0 T T 0 (A+BK) T 0 T T [ ] Λ T = 1 BR 1 B T T T, 0 Λ [ ] [ ] T T H = Λ. PT PT 10 / 22

Classical algorithm We wish to solve 1 the ARE A T P +PA+Q PBR 1 B T P = 0, P 0 1. form the Hamiltonian matrix [ ] A BR H = 1 B T Q A T 2. find eigenvectors v 1,...,v n of H corresponding to the n stable eigenvalues, and partition as [ ] [ ] [ ] X T v1 v n = = R 2n n Y PT 3. the unique positive semidefinite solution to the ARE is given by P := YX 1 1 for more general discussion, involving Jordan forms, see JE Potter, Matrix Quadratic Solutions, J. SIAM Appl. Math. 14(3):496 501, 1966 11 / 22

Single dimension In the n = 1 case, the ARE is 2ap +q p2 b 2 = 0 r where the variable is p R. We seek the positive solution, p +. 2ap +q p2 b 2 r q (a < 0) p p + p + = inf{p p 0,2ap +q p2 b 2 0} r = sup{p p 0,2ap +q p2 b 2 0} r p 2ap +q p2 b 2 r 0 12 / 22

Riccati inequality (nonstandard) In the full matrix case, change the equality to (matrix) inequality, A T P +PA+Q PBR 1 B T P 0 (1) fact. if (A,B) controllable and (A,Q 1/2 ) observable, and P are 0 with A T P are +P are A+Q P are BR 1 B T P = 0, then P are is minimal, in the sense that for any P satisfying (1) we have P are P. 13 / 22

Riccati equation: nonconvex approach This suggests that the ARE is a closed-form solution to the nonconvex problem minimize x(0) T Px(0) subject to A T P +PA+Q PBR 1 B T P 0 P 0 danger. we cannot use a Schur complement here, [ ] A T P +PA+Q PBR 1 B T A P 0 T P +PA+Q PB B T 0 P R (because R 0) 14 / 22

Riccati inequality (standard) The other Riccati inequality is much more common, A T P +PA+Q PBR 1 B T P 0 (2) fact. if (A,B) controllable and (A,Q 1/2 ) observable, and P are 0 with A T P are +P are A+Q P are BR 1 B T P = 0, then P are is maximal, in the sense that for any P satisfying (2) we have P P are. 15 / 22

Riccati equation: LMI approach The standard Riccati inequality leads to the convex problem: maximize x(0) T Px(0) subject to A T P +PA+Q PBR 1 B T P 0 P 0. Since R 0, use a Schur complement to obtain the equivalent SDP: maximize x(0) [ T Px(0) ] A subject to T P +PA+Q PB B T 0 P R P 0 16 / 22

Aside: SDP duality Consider the SDP in inequality form where x R n is the variable. dual variable Z = Z T Lagrangian: minimize c T x subject to x 1 F 1 + +x n F n +G 0 L(x,Z) = c T x +Tr((x 1 F 1 + +x n F n +G)Z) = x 1 (c 1 +Tr(F 1 Z))+ +x n (c n +Tr(F n Z))+Tr(GZ), which is affine in x R n dual function: g(z) = inf x L(x,Z) = { Tr(GZ), Tr(Fi Z)+c i = 0, for all i = 1,...,n, otherwise 17 / 22

Aside: SDP duality primal SDP: minimize c T x subject to x 1 F 1 + +x n F n +G 0 dual SDP: maximize Tr(GZ) subject to Tr(F i Z)+c i = 0, i = 1,...,n Z 0 Strong duality obtains if primal is strictly feasible, i.e., there is an x R n, x 1 F 1 + +x n F n +G 0. 18 / 22

Taking the dual We wish to find the dual of the SDP maximize x(0) [ T Px(0) ] A subject to T P +PA+Q PB B T 0 P R P 0 dual variables associated with the two constraints: ] ] [ Q Y T [ QT Y = T Y Z Y Z T 0, W = W T 0 Lagrangian (note the signs): L(P, Q,Y,Z,W) = x(0)px(0) [ ] A +Tr T P +PA+Q PB ][ Q Y T B T +Tr(PW) P R Y Z 19 / 22

Taking the dual Simpifying the Lagrangian, L(P, Q,Y,Z,W) = [ ] A = x(0)px(0)+tr T P +PA+Q PB ][ Q Y T B T +Tr(PW) P R Y Z = Tr { XP +(A T P +PA+Q) Q +PBY +B T PY T +RZ +PW } = Tr { (X + QA T +A Q +BY +Y T B T +W)P } +Tr(Q Q +RZ), where X = x(0)x(0) T and we used the cyclic property of Tr( ) The dual function is a supremum (when primal is maximize ), g( Q,Y,Z,W) = supl(p, Q,Y,Z,W) P { Tr(Q Q +RZ), X + QA = T +A Q +BY +Y T B T +W = 0, otherwise 20 / 22

Taking the dual Thus the dual is an SDP minimize Tr(Q Q +RZ) subject to X + QA T ] +A Q +BY +Y T B T +W = 0 [ Q Y T 0 Y Z objective is the LQR cost primal constraint P 0 is automatically satisfied, so W = 0 the dual variable turns out to be the state-input Gram matrix ] [ Q Y T [ ] x(t) [x = Y Z u(t) T (t) u T (t) ] dt (for technical considerations, see Seungil s thesis) 0 21 / 22

State-input Gram matrix Consider the quantity d dt (x(t)x(t)t ) = ẋ(t)x(t) T +x(t)ẋ(t) T Take the integral of both sides x( )x( ) T x(0)x(0) T = }{{}}{{} =0 =X which is the equality constraint = (Ax +Bu)x T +x(ax +Bu) T = xx T A T +Axx T +Bux T +xu T B T. 0 xx T A T +Axx T +Bux T +xu T B T dt, X = QA T +A Q +BY +Y T B T = 0, where Q = 0 xx T dt, Y = 0 ux T dt, and Z = 0 uu T dt. 22 / 22