Optimal Control, Guidance and Estimation Lecture 11 Linear Quadratic Regulator (LQR) II Pro. Radhakant Padhi Dept. o Aerospace Engineering Indian Institute o Science - Bangalore Outline Summary o LQR design Stability o closed-loop system in LQR Optimum value o the cost unction Extension o LQR design For cross-product term in cost unction Rate o state minimization Rate o control minimization LQR design with prescribed degree o stability LQR or command tracking LQR or inhomogeneous systems OPIMAL CONROL, GUIDANCE AND ESIMAION 2
Summary o LQR Design Pro. Radhakant Padhi Dept. o Aerospace Engineering Indian Institute o Science - Bangalore LQR Design: Problem Statement Perormance Index (to minimize): Path Constraint: t 1 1 J = ( S ) + 2 Q + U RU dt t 2 ϕ( ) L(, U ) ɺ = A + BU Boundary Conditions: t = : Speciied ( ) : Fixed, t : Free OPIMAL CONROL, GUIDANCE AND ESIMAION 4
LQR Design: Necessary Conditions o Optimality 1 erminal penalty: ϕ ( ) = ( S ) 2 1 Hamiltonian: H = Q + U RU + A + BU 2 State Equation: ɺ = A + BU Costate Equation: Optimal Control Eq.: Boundary Condition: λ ( H / ) ( Q A λ ) ɺ λ = = + ( / ) = = H U U R B λ ( ϕ / ) λ = = S OPIMAL CONROL, GUIDANCE AND ESIMAION 5 LQR Design: Derivation o Riccati Equation Guess λ ( t) = P ( t) ( t) ɺ λ = P ɺ + Pɺ = P ɺ + P A + BU = P ɺ + P( A BR B λ ) = P ɺ + P( A BR B P ) ( Q + A P ) = ( Pɺ + PA PBR B P) 1 ( Pɺ + PA + A P PBR B P + Q) = OPIMAL CONROL, GUIDANCE AND ESIMAION 6
LQR Design: Derivation o Riccati Equation Riccati equation ɺ + + + = P PA A P PBR B P Q Boundary condition ( ) = ( is ree) P t S ( ) P t = S OPIMAL CONROL, GUIDANCE AND ESIMAION 7 LQR Design: Ininite ime Regulator Problem heorem (By Kalman) As t, or constant Q and R matrices, Pɺ t Algebraic Riccati Equation (ARE) PA A P PBR B P Q Final Control Solution: + + = U = R B P = K OPIMAL CONROL, GUIDANCE AND ESIMAION 8
Stability o closed-loop loop system in LQR Pro. Radhakant Padhi Dept. o Aerospace Engineering Indian Institute o Science - Bangalore LQR Design: Stability o Closed Loop System Closed loop system Lyapunov unction Vɺ = ɺ P + Pɺ ɺ = A + BU = A BK V = P = A BK P + P A BK = + A BR B P P P A BR B P = + + 1 PA A P PBR B P Q Q PBR B P Q PBR B P = OPIMAL CONROL, GUIDANCE AND ESIMAION 1
LQR Design: Stability o Closed Loop System For R >, R >. Also P > -1 So PBR B P > Also Q. -1 ( PBR B P Q) Hence, + > V ɺ < Hence, the closed loop system is always asymptotically stable! OPIMAL CONROL, GUIDANCE AND ESIMAION 11 LQR Design: Minimum value o cost unction 1 J = ( Q + U RU ) dt 1 = Q + ( R B P ) R ( R B P ) dt 1 = ( Q + PBR B P) dt 1 1 = ( Vɺ ) dt [ V ] P t = = 2 2 1 1 = P P P 2 = 2 OPIMAL CONROL, GUIDANCE AND ESIMAION 12
Extensions o LQR Design Pro. Radhakant Padhi Dept. o Aerospace Engineering Indian Institute o Science - Bangalore LQR Extensions: 1. Cross Product erm in P.I. J = ( Q 2 WU U RU ) dt + + Let us consider the expression: ( ) + ( + ) ( + ) Q WR W U R W R U R W = Q + U RU + U W + WU = Q + 2 WU + U RU OPIMAL CONROL, GUIDANCE AND ESIMAION 14
LQR Extensions: 1. Cross Product erm in P.I. J = ( Q WR W ) ( U R W ) R( U R W ) + + + dt Q1 U1 = ( Q1 + U1 RU1) dt t ɺ = A + BU Control Solution = A + B U R W U = K ( 1 ) ( A BR W ) BU1 = + = A + BU 1 1 1 U = U1 R W = K + R W OPIMAL CONROL, GUIDANCE AND ESIMAION 15 LQR Extensions: 2. Weightage on Rate o State J = ( Q + U RU + ɺ Sɺ ) dt = Q U RU ( A BU ) S ( A BU ) + + + + dt Q + U RU + A S A + A SBU = dt + U B S A + U B SBU Q1 R1 W = ( Q A SA) U ( R B SB) U 2 ( A SB) U dt + + + + = ( Q1 + U RU 1 + 2 ) WU dt Leads to a cross product case OPIMAL CONROL, GUIDANCE AND ESIMAION 16
LQR Extensions: 3. Weightage on Rate o Control ( ɺ ɺ ) 1 J = Q + U RU + U RU ˆ dt Let =, V = Uɺ U 1 Q J V RV ˆ = dt + R ( ) 1 J = Qˆ + V RV ˆ dt OPIMAL CONROL, GUIDANCE AND ESIMAION 17 LQR Extensions: 3. Weightage on Rate o Control ɺ = A + BU, = Uɺ = V A B ɺ = V Aˆ BV ˆ + = + I Note : Aˆ { A B} Bˆ (1) he dimension o the problem has increased rom n to ( n + m) (2) I, is controllable, it can be shown that the new system is also controllable. OPIMAL CONROL, GUIDANCE AND ESIMAION 18
LQR Extensions: 3. Weightage on Rate o Control Solution : ˆ ˆ V = Uɺ = R B Pˆ where Pˆ is the solution o ˆ A P PA PBR B P Qˆ ˆ ˆ ˆ ˆ ˆ ˆ + ˆ + = Hence ˆ ˆ ˆ P 1 11 P 12 1 [ ] ˆ ˆ ˆ Uɺ = R I = R P12 P 22 ˆ P ˆ U 12 P 22 = Rˆ Pˆ Rˆ Pˆ U 12 22 OPIMAL CONROL, GUIDANCE AND ESIMAION 19 LQR Extensions: 3. Weightage on Rate o Control However, Uɺ = Rˆ Pˆ Rˆ Pˆ U is a dynamic 12 22 equation in U and hence is not easy or implementation. For this reason, we want an expression in the RHS only as a unction o and operations on it. State equation: ɺ = A + BU + his suggests: U = B ɺ A ( Note : his is only an approximate solution, unless m n) OPIMAL CONROL, GUIDANCE AND ESIMAION 2
LQR Extensions: 3. Weightage on Rate o Control Uɺ = Rˆ Pˆ Rˆ Pˆ B ɺ Rˆ Pˆ B Aɺ 1 + 1 + 12 22 22 ˆ 1 ˆ ˆ + ˆ 1 R P ˆ + = 12 + P22 B A R P22 B ɺ 1 2 K2 = K ɺ K K1 Integrating this expression both sides, U = K K z dz + U t 1 2 Proportional Initial condition Itegral Note : U can be obtained using a perormance index without the Uɺ term OPIMAL CONROL, GUIDANCE AND ESIMAION 21 LQR Extensions: 4. Prescribed Degree o Stability Condition: All the Eiganvalues o the closed loop system should lie to the let o line AB 2αt J = where, e t Q + U RU dt α αt αt αt αt = ( e Q e e U R e U ) dt t + A = ( ɶ Qɶ + Uɶ RUɶ ) dt t αt Let ɶ = e Co-ordinate α transormation B αt Uɶ = e U OPIMAL CONROL, GUIDANCE AND ESIMAION jω 22 σ
LQR Extensions: 4. Prescribed Degree o Stability ɺɶ ɺ αt αt = e + αe αt = + + αt e A BU αe αt αt αt α = A e + B e U + e ɺɶ = A + I ɶ + BUɶ ( α ) Control Solution: Uɶ = K ɶ αt e U U = = K αt K e OPIMAL CONROL, GUIDANCE AND ESIMAION 23 LQR Extensions: 4. Prescribed Degree o Stability Modiied System: Uɶ = K ɶ ɺɶ = ( ) + α ɶ ɺ = ( ) A BK I K Actual System: U = K A BK is designed in such a way that eigenvalues o A BK + α I Hence, eigenvalues o will lie in the let-hal plane. ( A BK ) will lie to the let o a line parallel to the imaginary axis, which is located away by distance α rom the imaginary axis. OPIMAL CONROL, GUIDANCE AND ESIMAION 24
LQR Design or Command racking Pro. Radhakant Padhi Dept. o Aerospace Engineering Indian Institute o Science - Bangalore LQR Design or Command racking Problem: o design U such that a part o the state vector o the linear system ɺ = A + BU tracks a commanded reerence signal. i.e. rc, where = N Solution: 1) Formulate a standard LQR problem. However, select the Q matrix properly. r c 2) Implement the controller as U = K N Q ypically Q = OPIMAL CONROL, GUIDANCE AND ESIMAION 26
LQR Design or Command racking U B + ɺ + A LQ Regulation - K U B + ɺ + LQ racking - K A + N rc ( t) - OPIMAL CONROL, GUIDANCE AND ESIMAION 27 LQR Design or Command racking with Integral Feedback Solution (with integral controller): 1) Augment the system dynamics with integral states ɺ A AN B ɺ N AN ANN N B = + N U ɺ I I I 2) Select the Q matrix properly (should penalize only and states) t 3) Control solution U = K ( rc ) N ( rc ) dt I OPIMAL CONROL, GUIDANCE AND ESIMAION 28
LQR Design or Inhomogeneous Systems Pro. Radhakant Padhi Dept. o Aerospace Engineering Indian Institute o Science - Bangalore LQR Design or Inhomogeneous Systems Reerence OPIMAL CONROL, GUIDANCE AND ESIMAION 3
LQR Design or Inhomogeneous Systems o derive the state o a linear (rather linearized) system ɺ = A + BU + C to the origin by minimizing the ollowing quadratic perormance index (cost unction) 1 1 J = ( S ) + ( Q + U RU ) dt 2 where S, Q (psd), R > (pd) t OPIMAL CONROL, GUIDANCE AND ESIMAION 31 LQR Design or Inhomogeneous Systems Perormance Index (to minimize): 1 1 J = ( S ) + ( Q + U RU ) dt 2 Path Constraint: t ɺ = A + BU + C Boundary Conditions: t = : Speciied ( ) : Fixed, t : Free OPIMAL CONROL, GUIDANCE AND ESIMAION 32
LQR Design or Inhomogeneous Systems 1 erminal penalty: ϕ ( ) = ( S ) 2 1 Hamiltonian: H = Q + U RU + A + BU + C 2 State Equation: ɺ = A + BU + C Costate Equation: Optimal Control Eq.: Boundary Condition: λ ( H / ) ( Q A λ ) ɺ λ = = + ( / ) = = H U U R B λ ( ϕ / ) λ = = S OPIMAL CONROL, GUIDANCE AND ESIMAION 33 LQR Design or Inhomogeneous Systems Guess λ ( t) = P( t) ( t) + K ( t) ɺ λ = P ɺ + Pɺ + Kɺ = P ɺ + P A + BU + C + Kɺ = ɺ + ( λ ) + + ( ) ɺ ( ) P P A BR B PC Kɺ Q + A P + K = P + P A BR B P + K + PC + Kɺ 1 ( Pɺ + PA + A P PBR B P + Q) 1 ( Kɺ A K PBR B P PC) + + + = OPIMAL CONROL, GUIDANCE AND ESIMAION 34
LQR Design or Inhomogeneous Systems Riccati equation ɺ + + + = P PA A P PBR B P Q Auxiliary equation Boundary conditions Kɺ + A PBR B K + PC = ( ) + ( ) = ( is ree) P t K t S ( ) P t = S K ( t ) = OPIMAL CONROL, GUIDANCE AND ESIMAION 35 LQR Design or Inhomogeneous Systems Control Solution: U = R B λ = + R B P K = R B P R B K Note: here is a residual controller even ater. his part o the controller osets the continuous disturbance. OPIMAL CONROL, GUIDANCE AND ESIMAION 36
hanks or the Attention.!! OPIMAL CONROL, GUIDANCE AND ESIMAION 37