Time-Invariant Linear Quadratic Regulators Robert Stengel Optimal Control and Estimation MAE 546 Princeton University, 17 Asymptotic approach from time-varying to constant gains Elimination of cross weighting in cost function Controllability and observability of an LTI system Requirements for closed-loop stability Algebraic Riccati equation Equilibrium response to commands Copyright 17 by Robert Stengel. All rights reserved. For educational use only. http://www.princeton.edu/~stengel/mae546.html http://www.princeton.edu/~stengel/optconest.html 1 Continuous-Time, Linear, Time-Invariant System Model x = Fx + Gu + Lw, x(t o ) given y = H x x + H u u + H w w Comment: (.) notation distinguishes linear-system variables from nonlinear-system variables
Linear-Quadratic Regulator: Finite Final Time x = Fx + Gu u = R 1 M T + G T P t x t = C( t)x( t) + 1 t ) f + * (,+ J = 1 xt (t f )P(t f )x(t f ) x T u T Q M M T R x u -+ dt. /+ P ( t) = F GR 1 M T T P( t) P( t) F GR 1 M T + P( t)gr 1 G T P( t) + MR 1 M T Q = P f P t f 3 Transformation of Variables to Eliminate Cost Function Cross Weighting Original LTI minimization problem min u 1 J 1 = 1 t f x T 1 Q 1 x 1 + x T 1 M 1 u 1 + u 1 R 1 u 1 dt subject to x 1 = F 1 x 1 + G 1 u 1 Can we find a transformation such that min u J = 1 t f x T Q x + u T R u dt subject to x = F x + G u = min u 1 J 1 4
Artful Manipulation Rewrite integrand of J 1 to eliminate cross weighting of state and control x T 1 Q 1 x 1 + x T 1 M 1 u 1 + u 1 R 1 u 1 = x T 1 Q 1 M 1 R 1 T ( 1 M 1 )x 1 + u 1 + R 1 1 M T T 1 x 1 R1 u 1 + R 1 1 M T 1 x 1 x 1 T Q x 1 + u T R 1 u The transformation produces the following equivalences x = x 1 u = u 1 + R 1 1 M 1 T x 1 Q = Q 1 M 1 R 1 1 M 1 T R = R 1 5 (Q,R) and (Q,M,R) LQ Problems are Equivalent x = x 1 x = x 1 u = u 1 + R 1 1 M 1 T x 1 Q = Q 1 M 1 R 1 T 1 M 1 R = R 1 x = F x + G u x = F x 1 + G u 1 + R 1 1 M T 1 x 1 = F + R 1 T ( 1 M 1 )x 1 + G u 1 = x 1 = F 1 x 1 + G 1 u 1 G = G 1 F = F 1 G R 1 T 1 M 1 = F 1 G 1 R 1 1 M 1 T Therefore, the forms are equivalent Whatever we prove for a (Q,R) cost function pertains to a (Q,M,R) cost function 6
x = x + u; x = 1 p ( t) = 1 p( t) + p ( t) = 1 p t f Recall: LQ Optimal Control of an Unstable First-Order System f = 1; g = 1 Control gain = p( t) u = p( t)x x = 1 p( t) x 7 Riccati Solution and Control Gain for Open-Loop Stable and Unstable 1 st -Order Systems P( t f ) = Variations in control gains are significant only in the last 1- of the illustrated time interval As time interval increases, percentage decreases 8
P() Approaches Steady State as t f -> P With M =, = Q F T P( t) P t t f P( t f ) = { F + P( t)gr 1 G T P( t) }dt from t f to Progression of initial Riccati matrix is monotonic with increasing final time for t f > t f1, J *( t f ) J *( t f1 ), P P 1 (see eq. 5.4-9 to 5.4-11, OCE) Rate of change approaches zero with increasing final time dp dt t f 9 Algebraic Riccati Equation and Constant Control Gain Matrix Steady-state Riccati solution (M = ) Q F T P P F + PGR 1 G T P Q F T P SS P SS F + P SS GR 1 G T P SS = Steady-state control gain matrix C ss = R 1 G T P t f = R 1 G T P ss = 1
Controllability of a LTI System Controllability: All elements of the state can be brought from arbitrary initial conditions to zero in finite time x = Fx + Gu x() = x ; x(t finite ) = System is Completely Controllable if Controllability Matrix = G FG F n1 G n nm has Rank n 11 Controllability Examples For non-zero coefficients F = G 1 n n ); G = () n FG ( = n ) 3 n n ) * Rank = ( ) () F = G FG 1 n n ( = n 4 n ); G = () n ) () ) ) * Rank = ( G F = 1 b ; G = b FG = b Rank = 1 G F = FG 1 b = b b b ; G = b Rank = 1
Requirements for Guaranteed Closed-Loop Stability 13 Optimal Cost with Feedback Control = 1 J *( t f ) = 1 t f With u( t) = C( t)x = R 1 G T P( t)x t f With terminal cost = x * T Qx * + u* T Ru* dt Substitute optimal control law in cost function x * T T Qx * + C( t)x * R C( t)x * dt = 1 t f x * T Qx * + x * T ( t)c T ( t)rc( t)x *( t) dt 14
J *( t f ) = 1 Optimal Cost with LQ Feedback Control t f Consolidate terms x * T Q + C T ( t)rc( t) x * t From eq. 5.4-9, OCE, optimal cost depends only on the initial state and J ( t f ) = 1 x *T ()Px *() dt 15 Optimal Quadratic Cost Function is Bounded J *() = lim J *( t f ) = 1 t f 1 t f ( t f x * T Q + C T ( t)rc( t) x *( t) dt x * T Q + C T ( t)rc( t) x *( t) dt 1 x * T Q + C T RC ( x * t dt = 1 x *T ()Px *() As final time goes to infinity J is bounded and positive provided that Because J is bounded, C is a stabilizing gain matrix Q > R > 16
Requirements for Guaranteeing Stability of the LQ Regulator x = Fx + Gu = [ F GC]x Closed-loop system is stable whether or not open-loop system is stable if... Q > R >... and (F,G) is a controllable pair Rank G FG F n1 G = n 17 Lyapunov Stability of the LQ Regulator x = [ F GC]x = F GR 1 G T P x V = x T ( t)px ( t) + x T ( t)px( t) = x T t Lyapunov function V x( t) = ( xt t)px t + F GR 1 G T T { P P } x t P F GR 1 G T P Rate of change of Lyapunov function 18
Lyapunov Stability of the LQ Regulator V = x T t Algebraic Riccati equation Q F T P PF + PGR 1 G T P = Substituting in rate equation + F GR 1 G T T { P P } x t P F GR 1 G T P = x T ( t) { Q + PGR 1 G T P}x( t) Defining matrix is positive definite Therefore, closed-loop system is stable 19 Less Restrictive Stability Requirements Q may be positive semi-definite if (F,D) is an observable pair, where Q D T D, where D may not be ( n n) Observability requirement Rank D T F T D T F T n1 D T = n
Observability Example x 1 x = 1 () n (*) n y = 1 x 1 x x 1 x = Hx t = Fx t For non-zero coefficients H T F T H T = ( n 1 ( n ) ) * Rank = 1 Even Less Restrictive Stability Requirements If F contains stable modes, closed-loop stability is guaranteed if (F,G) is a stabilizable pair (F,D) is a detectable pair
Stability Requirements with Cross Weighting, M, in Cost Function If F contains stable modes, closed-loop stability is guaranteed if [(F GR -1 M T ),G] is a stabilizable pair [(F GR -1 M T ),D] is a detectable pair (Q GR -1 M T ) R > 3 Example: LQ Optimal Control of a First-Order LTI System Cost Function J = 1 ( )x (t f ) + lim 1 t f t f t o ( qx + ru )dt Open-Loop System x = f x + gu Algebraic Riccati Equation q fp + g p r = p fr g p qr g = u = gp r x = cx Choose positive solution of p = fr g ± Control Law fr g + qr g = fr 1± 1+ g ) g ) fr ( * qr,, + 4
Example: LQ Optimal Control of a First-Order LTI System Closed-Loop System x = f g p r ( x = ( f c)x Stability requires that ( f c) < If f <, then system is stable with no control ( c = ) 5 Example: LQ Optimal Control of a First-Order LTI System If f > (unstable), and r >, then fr >, and g p = fr 1+ 1+ g ) g ) fr ( * qr,, + If q, and g, then p fr 1+ 1 q g ( = fr g and closed-loop system is, as q, f g r p ( = f g r fr g ( = f f = f Stable closed - loop system is mirror image of unstable open - loop system when q = 6
Solution of the Algebraic Riccati Equation 7 Solution Methods for the Continuous- Time Algebraic Riccati Equation Q F T P PF + PGR 1 G T P = 1) Integrate Riccati differential equation to steady state ) Explicit scalar equations for elements of P a) Difficult for n > 3 b) May use symbolic math (MATLAB Symbolic Math Toolbox, Mathematica,...) 8
Example: Scalar Solution for the Algebraic Riccati Equation Q F T P PF + PGR 1 G T P = Second-order example q 11 q f 11 f 1 f 1 f T p 11 p 1 g 11 g 1 + p 1 p g 1 g p 11 p 1 p 1 p p 11 p 1 p 1 p r 11 r 1 g 11 g 1 g 1 g T f 11 f 1 f 1 f p 11 p 1 p 1 p = Solve three scalar equations for p 11, p 1, and p 9 More Solutions for the Algebraic Riccati Equation Q F T P PF + PGR 1 G T P = See OCE, Section 6.1 for Kalman-Englar method Kleinman s method MacFarlane-Potter method Laub s method [used in MATLAB] 3
Equilibrium Response to a Command Input 31 Steady-State Response to Commands x = Fx + Gu + Lw, x(t o ) given y = H x x + H u u + H w w State equilibrium with constant inputs... = Fx *+Gu*+Lw * x* = F 1 Gu*+Lw *... constrained by requirement to satisfy command input y* = H x x * +H u u * +H w w * 3
Steady-State Response to Commands Equilibrium that satisfies a commanded input, y C = Fx *+Gu*+Lw * y* = H x x *+H u u*+h w w * Combine equations y C = F H x G H u x * u * + L H w w * (n + r) x (n + m) 33 Equilibrium Values of State and Control to Satisfy Commanded Input Equilibrium that satisfies a commanded input, y C x * u* = F H x G H u (1 (Lw * A (1 y C ( H w w * (Lw * y C ( H w w * A must be square for inverse to exist Then, number of commands = number of controls 34
Inverse of the Matrix F H x G H u 1 A 1 = B = B 11 B 1 B 1 B x * u* = B 11 B 1 B 1 B B ij have same dimensions as equivalent blocks of A Equilibrium that satisfies a commanded input, y C (Lw * y C ( H w w * x* = B 11 Lw * +B 1 y C H w w * u* = B 1 Lw * +B y C H w w * 35 Elements of Matrix Inverse and Solutions for Open-Loop Equilibrium Substitution and elimination (see Supplement) B 11 B 1 B 1 B = F 1 (GB 1 + I n ) F 1 GB B H x F 1 (H x F 1 G + H u ) 1 Solve for B, then B 1 and B 1, then B 11 x* = B 1 y C ( B 11 L + B 1 H w )w * u* = B y C ( B 1 L + B H w )w * 36
LQ Regulator with Command Input (Proportional Control Law) u = u C Cx( t) How do we define u C? 37 Non-Zero Steady-State Regulation with LQ Regulator Command input provides equivalent state and control values for the LQ regulator Control law with command input x *( t) B 1 y * u = u* C x t = B y *C x t = ( B + CB 1 )y *Cx t 38
LQ Regulator with Forward Gain Matrix x *( t) u = u* C x t = C F y *C B x t C F B + CB 1 C B C Disturbance affects the system, whether or not it is measured If measured, disturbance effect of can be countered by C D (analogous to C F ) 39 Next Time: Cost Functions and Controller Structures 4
Supplemental Material 41 M t Square-Root Solution for the Algebraic Riccati Equation Q F T P PF + PGR 1 G T P = Square root of P: P DD T ; D P Integrate D to steady state D T t f D ( t) = D T M LT ( t), D t f M LT ( t) + M UT ( t) = D 1 ( t)f T D( t) D T t F T D T t D 1 t u = R 1 G T T D SS D SS x t = C SS x t where d 11 d D = 11 d 11 d 11 d 11 d 11 = P t f t f QD T t + D T t and GR 1 G T D T t i < j 1 ( m ij ) LT ( t) = m ij, i = j i > j m ij 4
Matrix Inverse Identity OCE, eq..-57 to -67 B 11 B 1 B 1 B A 11 A 1 A 1 A I m+n = I n I m B 11 B 1 B 1 B A 11 A 1 A 1 A ( B 11 A 1 + B 1 A ) ( B 1 A 1 + B A ) = B A + B A 11 11 1 1 B 1 A 11 + B A 1 ( B 11 A 11 + B 1 A 1 ) = I n ( B 11 A 1 + B 1 A ) = = B 1 A 11 + B A 1 ( B 1 A 1 + B A ) = I m Solve for B, then B 1 and B 1, then B 11 43