Time-Invariant Linear Quadratic Regulators Robert Stengel Optimal Control and Estimation MAE 546 Princeton University, 15 Asymptotic approach from time-varying to constant gains Elimination of cross weighting in cost function Controllability and observability of an LTI system Requirements for closed-loop stability Algebraic Riccati equation Equilibrium response to commands Copyright 15 by Robert Stengel. All rights reserved. For educational use only. http://www.princeton.edu/~stengel/mae546.html http://www.princeton.edu/~stengel/optconest.html 1 Continuous-Time, Linear, Time-Invariant System Model x = Fx + Gu + Lw, x(t o ) given y = H x x + H u u + H w w
Linear-Quadratic Regulator: Finite Final Time x = Fx + Gu J = 1 xt (t f )P(t f )x(t f ) u = R 1 M T + G T P t x t = Cx t t + 1 t f x T u T Q M T M R x u dt P = t F GR 1 M T T P t PF t GR 1 M T + PGR t 1 G T P+ t MR 1 M T Q P( t f )= P f 3 Transformation of Variables to Eliminate Cost Function Cross Weighting Original LTI minimization problem min J 1 = 1 u 1 t f x T 1 Q 1 x 1 + x T 1 M 1 u 1 + u 1 R 1 u 1 dt subject to x 1 = F 1 x 1 + G 1 u 1 min u J = 1 t f x T Q x + u T R u dt subject to x = F x + G u = min u 1 J 1 4
Artful Manipulation Rewrite integrand of J 1 to eliminate cross weighting of state and control x T 1 Q 1 x 1 + x T 1 M 1 u 1 + u 1 R 1 u 1 = x T 1 Q 1 M 1 R 1 T ( 1 M 1 )x 1 + u 1 + R 1 1 M T T 1 x 1 R1 u 1 + R 1 1 M T 1 x 1 x T 1 Q x 1 + u T R 1 u The transformation produces the following equivalences x = x 1 Q = Q 1 M 1 R 1 T 1 M 1 u = u 1 + R 1 1 M T 1 x 1 R = R 1 5 (Q,R) and (Q,M,R) LQ Problems are Equivalent x = x 1 x = x 1 u = u 1 + R 1 1 M 1 T x 1 Q = Q 1 M 1 R 1 1 M 1 T R = R 1 x = F x + G u x = F x 1 + G u 1 + R 1 1 M T 1 x 1 = F + R 1 T ( 1 M 1 )x 1 + G u 1 = x 1 = F 1 x 1 + G 1 u 1 G = G 1 F = F 1 G R 1 1 M 1 T = F 1 G 1 R 1 1 M 1 T Therefore, the forms are equivalent Whatever we prove for a (Q,R) cost function pertains to a (Q,M,R) cost function 6
Recall: LQ Optimal Control of an Unstable First-Order System x = x + u; x( )= 1 p= t 1 pt + p t pt ( f )= 1 f = 1; g = 1 Control gain = p t u = pt x x = 1 pt x 7 Riccati Solution and Control Gain for Open-Loop Stable and Unstable 1 st -Order Systems P( t f )= last 1-% of the illustrated time interval As time interval increases, percentage decreases 8
P() Approaches Steady State as t f -> With M =, P( )= { Q F T P t PF t + PGR t 1 G T P t }dt t f from t f to Progression of initial Riccati matrix is monotonic with for t f > t f1 P ( ) P 1 ( ) Rate of change approaches zero with dp( ) dt t f 9 Algebraic Riccati Equation and Constant Control Gain Matrix Steady-state Riccati solution Q F T P( ) P( )F + P( )GR 1 G T P( )= Q F T P SS P SS F + P SS GR 1 G T P SS = Steady-state control gain matrix C ss = R 1 G T P( t f )= R 1 G T P ss 1
Controllability of a LTI System Controllability: All elements of the state can be brought from arbitrary initial conditions to zero in x = Fx + Gu x() = x x(t finite ) = System is Completely Controllable if Controllability Matrix = G FG F n1 G n nm has Rank n 11 Controllability Examples F = G 1 n n ; G = n FG = n Rank = 3 n n F = G FG 1 n n = n ; G = n 4 n Rank = G F = 1 b ; G = b FG = b Rank = 1 G F = FG 1 b = b b b ; G = b Rank = 1
Requirements for Guaranteed Closed-Loop Stability 13 Optimal Cost with Feedback Control = 1 J *( t f )= 1 t f = 1 With u= t Cx t = R 1 G T Px t t f x * T Qx * + u* T Ru* dt x * T T Qx * + Cx t * R Cx t * dt t f With terminal cost = Substitute optimal control law in cost function x * T Qx * + x * T C t T RC t x t * t dt 14
J *( t f )= 1 Optimal Cost with LQ Feedback Control t f Consolidate terms x * T Q + C T RC t t From eq. 5.4-9, OCE, optimal cost depends only on the initial condition J( t f )= 1 xt ()P( )x() x * t dt 15 Optimal Quadratic Cost Function is Bounded J *( t f )= 1 t f x * T Q + C T RC t t x * t dt J *( )= lim 1 t f 1 t f x * T Q + C T RC t t x * T Q + C T RC x * t J is bounded and positive provided that Q > R > Because J is bounded, C is a stabilizing gain matrix x * t dt dt = 1 xt ()Px() 16
Requirements for Guaranteeing Stability of the LQ Regulator x = Fx + Gu = [ F GC]x Closed-loop system is stable whether or not open-loop system is stable if... Q > R >... and (F,G) is a controllable pair Rank G FG F n1 G = n 17 Lyapunov Stability of the LQ Regulator = x T t x = [ F GC]x = F GR 1 G T P x Lyapunov function V x t = xt Px t t P F GR 1 G T P Rate of change of Lyapunov function Px t + t x T Px t t V = x T { + F GR 1 G T T P P } x t 18
Lyapunov Stability of the LQ Regulator Algebraic Riccati equation Q F T P PF + PGR 1 G T P = Substituting in rate equation { } x t V = x T P t F GR 1 G T P + F GR 1 G T T P P = x T Q t { + PGR 1 G T P}x t Therefore, closed-loop system is stable 19 Less Restrictive Stability Requirements Q may be if (F,D) is an observable pair, where Q D T D, where D may not be ( n n) Observability requirement Rank D T F T D T F T ( ) n1 D T = n
Observability Example x 1 x = 1 n n y = 1 x 1 x x 1 x = Hx t = Fx t H T F T H T = n 1 n Rank = 1 Even Less Restrictive Stability Requirements If F contains stable modes, closed-loop stability is guaranteed if (F,G) is a stabilizable pair (F,D) is a detectable pair
Stability Requirements with Cross Weighting If F contains stable modes, closed-loop stability is guaranteed if [(F GR -1 M T ),G] is a stabilizable pair [(F GR -1 M T ),D] is a detectable pair (Q GR -1 M T ) R > 3 Example: LQ Optimal Control of a First-Order LTI System Cost Function J = 1 ( )x (t f ) + lim 1 t f t f t o ( qx + ru )dt Open-Loop System x = f x + gu Algebraic Riccati Equation q fp + g p r = p fr qr p g g = u = gp r x = cx Choose positive solution of p = fr g ± Control Law fr g + qr g = fr 1± 1+ g g fr qr 4
Example: LQ Optimal Control of a First-Order LTI System Closed-Loop System x = f g p r x = ( f c)x Stability requires that ( f c)< If f <, then system is stable with no control ( c = ) 5 Example: LQ Optimal Control of a First-Order LTI System If f > (unstable), and r >, then fr >, and g p = fr 1+ 1+ g g fr qr If q, and g, then p fr 1+ 1 q g = fr g and closed-loop system is, as q, f g p r = g f r fr g = ( f f )= f Stable closed - loop system is "mirror image" of unstable open - loop system when q = 6
Solution of the Algebraic Riccati Equation 7 Solution Methods for the Continuous- Time Algebraic Riccati Equation Q F T P PF + PGR 1 G T P = 1) Integrate Riccati differential equation to steady state ) Explicit scalar equations for elements of P a)n > 3 b) May use symbolic math (MATLAB Symbolic Math Toolbox, Mathematica,...) 8
Example: Scalar Solution for the Algebraic Riccati Equation Q F T P PF + PGR 1 G T P = Second-order example q 11 q f 11 f 1 f 1 f T p 11 p 1 g 11 g 1 + p 1 p g 1 g p 11 p 1 p 1 p p 11 p 1 p 1 p r 11 r 1 g 11 g 1 g 1 g T f 11 f 1 f 1 f p 11 p 1 p 1 p = Solve three scalar equations for p 11, p 1, and p 9 More Solutions for the Algebraic Riccati Equation Q F T P PF + PGR 1 G T P = See OCE, Section 6.1 for Kalman-Englar method Kleinmans method MacFarlane-Potter method Laubs method [used in MATLAB] 3
Equilibrium Response to a Command Input 31 Steady-State Response to Commands x = Fx + Gu + Lw, x(t o ) given y = H x x + H u u + H w w State equilibrium with constant inputs... = Fx *+Gu*+Lw * ( ) x* = F 1 Gu*+Lw *... constrained by requirement to satisfy command input y* = H x x * +H u u * +H w w * 3
Steady-State Response to Commands y C = Fx * +Gu * +Lw * y* = H x x * +H u u * +H w w * Combine equations y C = F H x G H u x * u * + L H w w * (n + r) x (n + m) 33 Equilibrium Values of State and Control to Satisfy Commanded Input y C x * u* = F H x G H u 1 Lw * A 1 y C H w w * Lw * y C H w w * A must be square for inverse to exist Then, number of commands = number of controls 34
Inverse of the Matrix F H x G H u 1 A 1 = B = B 11 B 1 B 1 B x * u* = B 11 B 1 B 1 B B ij have same dimensions as equivalent blocks of A y C Lw * y C H w w * ( ) ( ) x* = B 11 Lw * +B 1 y C H w w * u* = B 1 Lw * +B y C H w w * 35 Elements of Matrix Inverse and Solutions for Open-Loop Equilibrium Substitution and elimination (see Supplement) B 11 B 1 B 1 B = F 1 ( GB 1 + I n ) F 1 GB B H x F 1 ( H x F 1 G + H u ) 1 Solve for B, then B 1 and B 1, then B 1 x* = B 1 y C ( B 11 L + B 1 H w )w * u* = B y C ( B 1 L + B H w )w * 36
LQ Regulator with Command Input (Proportional Control Law) u = u C Cx t u C? 37 Non-Zero Steady-State Regulation with LQ Regulator Command input provides equivalent state and control values for the LQ regulator Control law with command input u = u* C xx t * t B 1 y * = B y *C x t = ( B + CB 1 )y *Cx t 38
LQ Regulator with Forward Gain Matrix u = u * C xx t * t where = C F y * C B x t C F B + CB 1 C B C Disturbance affects the system, whether or not it is measured If measured, disturbance effect of can be countered by C D 39 Next Time: Cost Functions and Controller Structures 4
Supplemental Material 41 Square-Root Solution for the Algebraic Riccati Equation Q F T P PF + PGR 1 G T P = Square root of P: P DD T ; D P Integrate D to steady state ( )D T ( t f )= P t f t f D = t D T M LT, t D t f = D 1 F t T D t D T F t T D T u = R 1 where M t M LT + t M UT t G T T D SS D SS = C SS x t t D 1 QD t T x t where d 11 d D = 11 d 11 d 11 d 11 d 11 ( ) + t D T GR t 1 G T D T and ( m ij ) LT = t 1 m ij m ij, i < j i = j i > j 4 t
Matrix Inverse Identity OCE, eq..-57 to -67 B 11 B 1 B 1 B A 11 A 1 A 1 A I m+n = I n I m B 11 B 1 B 1 B A 11 A 1 A 1 A ( ) ( B 11 A 1 + B 1 A ) ( ) ( B 1 A 1 + B A ) = B A + B A 11 11 1 1 B 1 A 11 + B A 1 ( B 11 A 11 + B 1 A 1 ) = I n ( B 11 A 1 + B 1 A ) = ( ) = B 1 A 11 + B A 1 ( B 1 A 1 + B A ) = I m Solve for B, then B 1 and B 1, then B 1 43