Linear Quadratic Gausssian Control Design with Loop Transfer Recovery

Linear Quadratic Gausssian Control Design with Loop Transfer Recovery Leonid Freidovich Department of Mathematics Michigan State University MI 48824, USA e-mail:leonid@math.msu.edu http://www.math.msu.edu/ leonid/ December 4, 2003 Presentation for ME-891, Fall 2003

Outline 1

Outline 1 (a) Linear quadratic regulation [LQR]

Outline 1 (a) Linear quadratic regulation [LQR] (b) H 2 design [LQG]

Outline 1 (a) Linear quadratic regulation [LQR] (b) H 2 design [LQG] (c) Idea of loop transfer recovery [LTR]

Outline 1 (a) Linear quadratic regulation [LQR] (b) H 2 design [LQG] (c) Idea of loop transfer recovery [LTR] (d) Technical implementations of LQR/LTR

Outline 1 (a) Linear quadratic regulation [LQR] (b) H 2 design [LQG] (c) Idea of loop transfer recovery [LTR] (d) Technical implementations of LQR/LTR (e) References [LQR/LQG/LTR]

Linear Quadratic Regulation [LQR] (A, B 2 ) is controllable. ẋ = Ax + B 2 u, x(0) = x 0. (1) 2

Linear Quadratic Regulation [LQR] (A, B 2 ) is controllable. ẋ = Ax + B 2 u, x(0) = x 0. (1) 2 Goal: Design state feedback controller u = K(s)x, [ K(s) is a proper real rational transfer function] (2) depending on A and B 2 such that for arbitrary x 0 : 1. the closed-loop system (1), (2) is internally stable, 2. the solution x(t) and the control signal u(t) satisfy certain specifications. [LQR/LQG/LTR]

Let us choose C 1, D 12 and define performance index: 3 J lqr = z 2 2 = C 1 x + D 12 u 2 2 = [C 1 x(t) + D 12 u(t)] [C 1 x(t) + D 12 u(t)] dt (3) 0

Let us choose C 1, D 12 and define performance index: 3 J lqr = z 2 2 = C 1 x + D 12 u 2 2 = [C 1 x(t) + D 12 u(t)] [C 1 x(t) + D 12 u(t)] dt (3) 0 If C 1D 12 = 0 then J lqr = C 1 x 2 2 + D 12 u 2 2.

Let us choose C 1, D 12 and define performance index: 3 J lqr = z 2 2 = C 1 x + D 12 u 2 2 = [C 1 x(t) + D 12 u(t)] [C 1 x(t) + D 12 u(t)] dt (3) 0 If C 1D 12 = 0 then J lqr = C 1 x 2 2 + D 12 u 2 2. LQR problem: Design state feedback controller (2) depending on A, B 2, C 1, and D 12 such that for arbitrary x 0 : 1. the closed-loop system (1), (2) is internally stable, 2. the performance index (3) is the smallest. [LQR/LQG/LTR]

The solution exists if 4 D12D 12 > 0, (C 1, A) is detectable; [ ] A jωi B2 has full column rank for all ω. C 1 D 12 [LQR/LQG/LTR]

Properties of the solution: 1. optimal controller K(s) = K lqr is the constant gain: 5 K lqr = (D12D 12 ) 1 (B2X + D12C 1 ), where X 0 is the stabilizing solution of the Riccati equation: Ā X + XĀ XB 2(D12D 12 ) 1 B2X + C1[I D 12 (D12D 12 ) 1 D12]C 1 = 0, with Ā = A B 2(D12D 12 ) 1 D12C 1, 2. A + B 2 K lqr is Hurwitz and lim t x(t) = 0,

Linear Quadratic Gaussian [LQG or H 2 ] problem Model: ẋ = Ax + B 2 u, y = C 2 x + D 21 u, x(0) = x 0. (4) 6 Assumptions: (A, B 2 ) is controllable, (C 2, A) is detectable. [LQR/LQG/LTR]

Goal: Design output feedback controller 7 u = K(s)y, [ K(s) is a proper real rational transfer function] (5) depending on A, B 2, C 2, and D 21 such that for arbitrary x 0 : 1. the closed-loop system (6), (5) is internally stable, 2. the solution x(t) and the control signal u(t) satisfy certain specifications. Let us choose C 1, D 12, as before. It could be shown that if D 12 0, then the output controller minimizing J lqr = z 2 = C 1 x + D 12 u 2 2 cannot be proper (i.e. solution does not exist). [LQR/LQG/LTR]

Consider ẋ = Ax + B 1 w + B 2 u, y = C 2 x + D 21 u, x(0) = x 0. (6) 8

Consider ẋ = Ax + B 1 w + B 2 u, y = C 2 x + D 21 u, x(0) = x 0. (6) 8 Define performance index J lqg = T w z 2 2 = max w 2 1 { } z 2 w 2 2 = max w 2 1 { C1 x + D 12 u 2 } w 2. (7) 2

Consider ẋ = Ax + B 1 w + B 2 u, y = C 2 x + D 21 u, x(0) = x 0. (6) 8 Define performance index J lqg = T w z 2 2 = max w 2 1 { } z 2 w 2 2 = max w 2 1 { C1 x + D 12 u 2 } w 2. (7) 2 LQG problem: Design output feedback controller (5) depending on A, B 1, B 2, C 1, C 2, D 12, and D 21 such that for arbitrary x 0 : 1. the closed-loop system (6), (5) is internally stable, 2. the performance index (7) is the smallest. [LQR/LQG/LTR]

The solution exists if 9 D 12D 12 > 0, [ ] A jωi B2 C 1 D 12 D 21 D 21 > 0, [ ] A jωi B1 C 2 D 21 has full column rank for all ω. has full row rank for all ω. [LQR/LQG/LTR]

Properties of the solution: 1. optimal K(s) is the observer based controller: 10 u = K lqgˆx, ˆx = Aˆx + B 2 u + L lqg (C 2ˆx + D 21 u y), ˆx(0) = ˆx 0,

Properties of the solution: 1. optimal K(s) is the observer based controller: 10 u = K lqgˆx, ˆx = Aˆx + B 2 u + L lqg (C 2ˆx + D 21 u y), ˆx(0) = ˆx 0, where K lqg = K lqr and L lqg = (Y C2 + B 1 D21)(D 21 D21) 1, where Y 0 is the stabilizing solution of the Riccati equation: A Y + Y A Y C 2(D 21 D 21) 1 C 2 Y + B 1 [I D 21(D 21 D 21) 1 D 21 ]B 1 = 0, with A = A B 1 D 21(D 21 D 21) 1 C 2, [LQR/LQG/LTR]

2. observer cannot be too fast since transient with peaking is not compatible with small values of C 1 x + D 12 u, 11

2. observer cannot be too fast since transient with peaking is not compatible with small values of C 1 x + D 12 u, 11 3. if w(t) 0 then lim t x(t) = 0,

2. observer cannot be too fast since transient with peaking is not compatible with small values of C 1 x + D 12 u, 11 3. if w(t) 0 then lim t x(t) = 0, 4. in general, neither gain nor phase margins are guaranteed (have to check robustness for each particular design). [LQR/LQG/LTR]

LQG with loop transfer recovery Consider the observer based controller for the system (4) with D 21 = 0 ẋ = Ax + B 2 u, y = C 2 x, x(0) = x 0, 12

LQG with loop transfer recovery Consider the observer based controller for the system (4) with D 21 = 0 12 Assumptions: ẋ = Ax + B 2 u, y = C 2 x, x(0) = x 0, u = K lqrˆx, ˆx = Aˆx + B 2 u + L(C 2ˆx y), ˆx(0) = ˆx 0. (8) (A, B 2 ) is controllable, (C 2, A) is detectable, K lqr is the optimal gain from the LQR problem. [LQR/LQG/LTR]

Goal: Design the observer gain L so that the closed-loop system (8) recovers internal stability and some of the robustness properties (gain and phase margins) of the LQR design. 13

Goal: Design the observer gain L so that the closed-loop system (8) recovers internal stability and some of the robustness properties (gain and phase margins) of the LQR design. 13 Compare w u ẋ = Ax + B 2 u x û K lqr Figure 1: û = {L t (s)} u û = K lqr (si A) 1 B 2 u }{{} ẋ=ax+b 2 u def = {L t (s)} u [LQR/LQG/LTR]

and w u x y ẋ = Ax + B 2 u C 2 14 û K lqr ˆx observer Figure 2: û = {L o (s)} u û = K lqr [ (si A LC 2 B 2 K lqr ) 1 LC 2 ] (si A) }{{} 1 B 2 u }{{} observer: ˆx=Aˆx+B2 [K lqr ˆx]+LC 2 (ˆx x) ẋ=ax+b 2 u def = {L o (s)} u

LQG/LTR problem: Design the observer gain L so that 15 1. the closed-loop system (8) is internally stable, 2. for all ω [0, ω max ] : L o (jω) L t (jω), where ω max 1, target (under state feedback) open-loop transfer function is L t (s) = K lqr (si A) 1 B 2, achieved (under output feedback) open-loop transfer function is L o (s) = K lqr T x ˆx (s)(si A) 1 B 2, and excited observer transfer function is T x ˆx (s) = [ (si A LC 2 B 2 K lqr ) 1 LC 2 ]. [LQR/LQG/LTR]

Doyle Stein formula It could be shown that if L[I C 2 (jωi A) 1 L] 1 = B 2 [C 2 (jωi A) 1 B 2 ] 1 16 then L o (jω) = L t (jω).

Doyle Stein formula It could be shown that if L[I C 2 (jωi A) 1 L] 1 = B 2 [C 2 (jωi A) 1 B 2 ] 1 16 then L o (jω) = L t (jω). Suppose lim {εl(ε)} = B 2W, ε 0 where W is not singular and ε 1.

Doyle Stein formula It could be shown that if L[I C 2 (jωi A) 1 L] 1 = B 2 [C 2 (jωi A) 1 B 2 ] 1 16 then L o (jω) = L t (jω). Suppose lim {εl(ε)} = B 2W, ε 0 where W is not singular and ε 1. Then lim {B 2W [εi + C 2 (jωi A) 1 B 2 W ] 1 } = B 2 [C 2 (jωi A) 1 B 2 ] 1 ε 0 and hence lim {L o(jω)} = L t (jω). ε 0 [LQR/LQG/LTR]

Remarks on ( L(ε)[I C 2 (jωi A) 1 L(ε)] 1 B 2 [C 2 (jωi A) 1 B 2 ] 1) 17 In general, the choice L(ε) = B 2 W/ε does not guarantee internal stability of the closed-loop system.

Remarks on ( L(ε)[I C 2 (jωi A) 1 L(ε)] 1 B 2 [C 2 (jωi A) 1 B 2 ] 1) 17 In general, the choice L(ε) = B 2 W/ε does not guarantee internal stability of the closed-loop system. Since L(ε)[I C 2 (jωi A) 1 L(ε)] 1 = (jωi A)(jωI [A + L(ε)C 2 ]) 1 L(ε), if εl(ε) = B 2 W + O(ε), then some of the eigenvalues of [A + L(ε)C 2 ] (i.e. poles of the observer error dynamics) approach the zeros of [C 2 (jωi A) 1 B 2 ] (i.e. zeros of the plant) and the others approach infinity as ε 0.

LQR design of L(ε) Consider LQR problem for the system 18 x = Ā x + B 2 ū, x(0) = x 0, where Ā = A and B 2 = C 2.

LQR design of L(ε) Consider LQR problem for the system 18 x = Ā x + B 2 ū, x(0) = x 0, where Ā = A and B 2 = C 2. Take C 1 (ε) = B 2/ε (cheap control)

LQR design of L(ε) Consider LQR problem for the system 18 x = Ā x + B 2 ū, x(0) = x 0, where Ā = A and B 2 = C 2. Take C 1 (ε) = B 2/ε (cheap control) and D 12 independent of ε so that D 12 D 12 = R > 0 and C 1 D 12 = 0. Suppose K lqr (ε) is the solution gain of the problem above for each ε > 0. Take L(ε) = [ Klqr (ε) ]. [LQR/LQG/LTR]

Then [Ā + B2 Klqr (ε) ] = A + L(ε)C2 19 is Hurwitz

Then [Ā + B2 Klqr (ε) ] = A + L(ε)C2 19 is Hurwitz and L(ε) = X(ε)C 2R 1, where X(ε) is the stabilizing solution of the Riccati equation A X(ε) + X(ε)A X(ε)C 2R 1 C 2 X(ε) + B2 B2/ε 2 = 0.

Then [Ā + B2 Klqr (ε) ] = A + L(ε)C2 19 is Hurwitz and L(ε) = X(ε)C 2R 1, where X(ε) is the stabilizing solution of the Riccati equation A X(ε) + X(ε)A X(ε)C 2R 1 C 2 X(ε) + B2 B 2/ε 2 = 0. It could be shown that if the plant is of minimum phase (and left invertible) then lim{ε 2 [A X(ε) + X(ε)A ]} = 0 and ε 0 lim{εl(ε)} = lim { ε X(ε)C 2R 1 } = B 2 R 1/2 = B 2 W ε 0 ε 0 [LQR/LQG/LTR]

Alternative designs of L(ε) 20 For the broken closed-loop system ẋ = Ax + B 2 u, y = C 2 x + D 21 u, x(0) = x 0,

Alternative designs of L(ε) 20 For the broken closed-loop system ẋ = Ax + B 2 u, y = C 2 x + D 21 u, x(0) = x 0, û = K lqrˆx, ˆx = Aˆx + B 2 û + L(C 2ˆx + D 21 û y), ˆx(0) = ˆx 0. we have: L t (s) = K lqr (si A) 1 B 2 and L o (s) = K lqr [ (si A LC 2 B 2 K lqr +LD 21 K lqr ) 1 ]L[C 2 (si A) 1 B 2 +D 21 ]. [LQR/LQG/LTR]

It could be shown that 21 L t (s) L o (s) = M(s)[I + M(s)] 1 (I K lqr B 2 ), where M(s) = K lqr ([si A] 1 LC 2 ) 1 (B 2 + LD 21 ). Therefore it is reasonable to search for L that minimizes M(s) 2 or M(s). Also, a pole-placement technique can be used to design the observer gain as well. However, it is not trivial and the system need to be transformed into the special coordinate basis. [LQR/LQG/LTR]

References 22 [1] H. Kwakernaak and R. Sivan, Linear optimal control systems, N.Y.: Wiley-Interscience, 1972. [2] J.C. Doyle, Guaranteed margins for LQG regulators, IEEE Trans. on AC, Vol. 23, pp. 756 757, 1978. [3] J.C. Doyle and G. Stein, Robustness with observers, IEEE Trans. on AC, Vol. 24, pp. 607 611, 1979. [4] J.C. Doyle and G. Stein, Multivariable feedback design: concepts for a classical/modern synthesis, IEEE Trans. on AC, Vol. 26, pp. 4 16, 1981. [5] G. Stein and M. Athans, The LQG/LTR procedure for multivariable control design, IEEE Trans. on AC, Vol. 32, pp. 105 114, 1987. [LQR/LQG/LTR]

[6] J.C. Doyle, K. Glover, P.P. Khargonekar, and B.A. Francis, State-space solution to standard H 2 and H control problems, IEEE Trans. on AC, Vol. 34, pp. 831 847, 1989. 23 [7] A. Saberi, B.M. Chen, and P. Sanuti, Loop transfer recovery: analysis and design, London: Springer-Verlag, 1993. [8] K. Zhou and J.C. Doyle, Essentials of robust control, N.J.: Prentice-Hall, 1998. [9] J.S. Freudenberg, Robustness with observers, Lecture notes, University of Michigan, 1999. [10] C. Gökçek, ME-891: Robust and Optimal Control, Lecture notes, Michigan State University, 2003. [11] B.M. Chen, Loop transfer recovery, Lecture notes, 2003. [LQR/LQG/LTR]