Optimal Control, Guidance and Estimation Lecture Linear Quadratic Regulator (LQR) I Pro. Radhakant Padhi Dept. o Aerospace Engineering Indian Institute o Science - Bangalore Generic Optimal Control Problem Perormance Index (to minimize / maximize): Path Constraint: Xɺ = t X U Boundary Conditions: X = ϕ (, ) + (,, ) J t X L t X U dt (,, ) t t t = X : Speciied ( ) : Fixed, X t : Free Pro. Radhakant Padhi, AE Dept., IISc-Bangalore
LQR Design: Problem Objective o drive the state X o a linear (rather linearized) system Xɺ = A X + BU to the origin by minimizing the ollowing quadratic perormance index (cost unction) J = ( X S X ) + ( X Q X + U RU ) dt where t S, Q (psd), R > (pd) t Pro. Radhakant Padhi, AE Dept., IISc-Bangalore 3 LQR Design: Guideline or Selection o Weighting Matrices S (psd), Q (psd), R > (pd) hese are usually chosen as diagonal matrices, with s q r i i i = maximum expected/acceptable value o / ( xi ) = maximum expected/acceptable value o ( / xi ) = maximum expected/acceptable value o ( / u i ) Pro. Radhakant Padhi, AE Dept., IISc-Bangalore 4
LQR Design: Some Facts to Remember { } he pair A, B needs to be controllable and the pair A, Q needs to be detectable { } S (psd), Q (psd), R > (pd) (these are usually chosen as diagonal matrices) By deault, it is assumed that t Constrained problems (i.e. problems with state and control inequality constraints) are not considered here. hose will be considered later. Pro. Radhakant Padhi, AE Dept., IISc-Bangalore 5 LQR Design: Problem Statement Perormance Index (to minimize): Path Constraint: t J = ( X S X ) + X Q X + U RU dt t ϕ( X ) L( X, U ) Xɺ = A X + BU Boundary Conditions: X t = X : Speciied ( ) : Fixed, X t : Free Pro. Radhakant Padhi, AE Dept., IISc-Bangalore 6
LQR Design: Necessary Conditions o Optimality erminal penalty: ϕ ( X ) = ( X S X ) Hamiltonian: H = X Q X + U RU + AX + BU State Equation: Xɺ = AX + BU Costate Equation: Optimal Control Eq.: Boundary Condition: ( H / X ) ( QX A ) ɺ = = + ( / ) = = H U U R B ( ϕ / X ) = = S X Pro. Radhakant Padhi, AE Dept., IISc-Bangalore 7 LQR Design: Derivation o Riccati Equation Guess: Justiication: ( t) P ( t) X ( t) = From unctional analysis theory o normed linear space, lies in the "dual space" o X t o all continuous linear unctionals o X t. ( t), which is the space consisting Reerence: Optimization by Vector Space Methods D. G. Luenberger, John Wiley & Sons, 969. Pro. Radhakant Padhi, AE Dept., IISc-Bangalore 8
LQR Design: Derivation o Riccati Equation Guess ( t) = P ( t) X ( t) ɺ = PX ɺ + PXɺ = PX ɺ + P AX + BU = PX ɺ + P( AX BR B ) = PX ɺ + P( AX BR B PX ) ( QX + A PX ) = ( Pɺ + PA PBR B P) X ( Pɺ + PA + A P PBR B P + Q) X = Pro. Radhakant Padhi, AE Dept., IISc-Bangalore 9 LQR Design: Derivation o Riccati Equation Riccati equation ɺ + + + = P PA A P PBR B P Q Boundary condition ( ) = ( is ree) P t X S X X ( ) P t = S Pro. Radhakant Padhi, AE Dept., IISc-Bangalore
LQR Design: Solution Procedure Use the boundary condition P t = S and integrate the Riccati Equation backwards rom to t t Store the solution history or the Riccati matrix Compute the optimal control online U = R B P X = K X Pro. Radhakant Padhi, AE Dept., IISc-Bangalore LQR Design: Ininite ime Regulator Problem heorem (By Kalman) As t, or constant Q and R matrices, Pɺ t Algebraic Riccati Equation (ARE) Note: PA A P PBR B P Q + + = ARE is still a nonlinear equation or the Riccati matrix. It is not straightorward to solve. However, eicient numerical methods are now available. A positive deinite solution or the Riccati matrix is needed to obtain a stabilizing controller. Pro. Radhakant Padhi, AE Dept., IISc-Bangalore
Example : Stabilization o Inverted Pendulum Pro. Radhakant Padhi Dept. o Aerospace Engineering Indian Institute o Science - Bangalore A Motivating Example: Stabilization o Inverted Pendulum System dynamics: ɺɺ θ = ω θ u, ω = g / L n n ( Linearized about vertical equilibrium point) u θ L mg System dynamics (state space orm): Deine: x = θ, x = ɺ θ xɺ x u x = ωn + x ɺ Xɺ A X B Perormance Index (to minimize): J = θ u dt + c Q =, R = c Pro. Radhakant Padhi, AE Dept., IISc-Bangalore 4
A Motivating Example: Stabilization o Inverted Pendulum ARE: PA A P PBR B P Q Let + + = P p p = p p 3 (a symmetric matrix) pω n p pωn p3ω n c p c p p 3 + + p = 3ωn p p p c p p3 c p3 Equations: pωn c p + = p = 4 ω n ± ωn + c c p + p ω c p p = (repeated) 3 n 3 p c p = p = ± p c 3 3 u θ L mg Pro. Radhakant Padhi, AE Dept., IISc-Bangalore 5 A Motivating Example: Stabilization o Inverted Pendulum However, Hence, p p 3 4 n n 3 3 n 3 3 3 n is a diagonal term, which needs to be real and positive. needs to be positive. hereore p = ω + ω + c, p = p c c Moreover, p + p ω c p p = p = c p p p ω (not needed in this problem) u θ L mg Gain Matrix: K = R B P = c p c p 3 Control: = = + ɺ ( θ 3θ ) u K X c p p Pro. Radhakant Padhi, AE Dept., IISc-Bangalore 6
A Motivating Example: Stabilization o Inverted Pendulum Analysis Open-Loop System: I A = = ωn = ω = ± ω n n ( right hal pole: unstable system) u θ L mg Closed-Loop System: Deine: ACL = A BK = ωn c p c p 3 Closed-Loop Poles: I A = CL ω = ω + c 4 n p = + c ( ωn ω ) p3 = p = + c c ( ωn ω ) / Pro. Radhakant Padhi, AE Dept., IISc-Bangalore 7 A Motivating Example: Stabilization o Inverted Pendulum Analysis Closed-Loop Poles: ( n ) / + ω + ω + ω =, = ω + ω ± ω ω 4 ( Note: ω = ωn + c > ωn ) ( n ) j ( n ) / / u θ L mg Both o the closed-loop poles are strictly in the let-hal plane. Hence, the closed-loop is guaranteed to be asymptotically stable. Pro. Radhakant Padhi, AE Dept., IISc-Bangalore 8
Example : Finite-time time emperature Control Pro. Radhakant Padhi Dept. o Aerospace Engineering Indian Institute o Science - Bangalore Example: Finite ime emperature Control Problem System dynamics : ɺ θ = a θ θ + bu where a, b : Constants θ θ u a a : emperature : Ambient temperature (Constant = C) : Heat input Pro. Radhakant Padhi, AE Dept., IISc-Bangalore
Problem ormulations Case : Case : Cost Function: Cost Function: J θ = ( t ) t t u dt J = s ( θ 3) + u dt = θ = 3 C (Hard constraint) s > : Weightage ( t ) i.e. θ 3 C (Sot Constraint) Pro. Radhakant Padhi, AE Dept., IISc-Bangalore Solution: Solution: x ( θ θa ), θ ( ) = θa ( θ θ ) xɺ = ax + bu, x = = Η = + + ɺ Η = = a x Η = u = b u u ax bu a Pro. Radhakant Padhi, AE Dept., IISc-Bangalore a Necessary conditions xɺ = ax + bu ɺ = a u = b
Solution: Case (Hard constraint) a( t t ) a( t t) = e = e u = be a t ( t) a t xɺ = ax b e ( t) aking laplace transorm: sx ( s) x( ) at ax s b e = s a Pro. Radhakant Padhi, AE Dept., IISc-Bangalore 3 Solution: Case (Hard constraint) s a at = b e a s a s + a Hence = a at X s = b e at at at x t b e e e Unknown ( θ θ ) However, x( t ) = = C a Pro. Radhakant Padhi, AE Dept., IISc-Bangalore 4
Solution: Case (Hard constraint) at ( e ) at ( e ) at at at x( t ) = = b e e e b = a a = b x( t) = b a at at a e e at at at e ( e e ) = b ( e ) a e e at at at Pro. Radhakant Padhi, AE Dept., IISc-Bangalore 5 Solution: Case (Hard constraint) Note : x( t ) = at ( e at e ) at ( e at e ) = (i.e. he boundary condition is "exactly met"!) Controller : u( t) = be at ( ) ae = b ( at ) ( at at e b e e ) a t t a Pro. Radhakant Padhi, AE Dept., IISc-Bangalore 6
Solution: Case (Sot constraint) θ 3 C x C. Hence the cost unction is t J = s ( x ) + u dt = s ( x ) x = + s However, we have b x t e e e a at at at = ( ) Pro. Radhakant Padhi, AE Dept., IISc-Bangalore 7 Solution: Case (Sot constraint) At t = t, x t = e = + a s b at b at i. e. + ( e ) = s a s a i. e. = at a + s b ( e ) Hence a( t ) t a t t s a = e = e a + s b e at Pro. Radhakant Padhi, AE Dept., IISc-Bangalore 8
Solution: Case (Sot constraint) u t = b = be = be s a a + s b e a t t at at s abe s b ae + e e a t t at at at Pro. Radhakant Padhi, AE Dept., IISc-Bangalore 9 Correlation between hard and sot constraint results As s, at abe lim u( t) = lim b ae + e e s s ( e ) S. C. s at at at at ae = = u t b e at at H. C. i. e. he "sot constraint" problem behaves like the "hard constraint" problem when s. Pro. Radhakant Padhi, AE Dept., IISc-Bangalore 3
hanks or the Attention.!! Pro. Radhakant Padhi, AE Dept., IISc-Bangalore 3