Extensions and applications of LQ

Extensions and applications of LQ 1 Discrete time systems 2 Assigning closed loop pole location 3 Frequency shaping

LQ Regulator for Discrete Time Systems Consider the discrete time system: x(k + 1) = A(k)x(k) + B(k)u(k); x(0) = x 0. with the performance criteria given by: J(u( ), x 0 ) = 1 2 x T (k f )Sx(k f ) + 1 2 k f 1 k=k 0 [ ] x T (k)qx(k) + u T (k)ru(k).

Solution - via Dynamic Programming Cost-to-go function at time k, state x denoted by: J(x, k) At final stage k = k f, let P(k f ) = S: At stage k:j(x, k) = min u J(x, k f ) = 1 2 x T Sx = 1 2 x T P(k f )x [ 1 2 x T Q(k)x + 1 2 ut R(k)u + J(x(k + 1, u), k + 1) where x(k + 1, u) is the state at k + 1 if input u is applied x(k + 1, u) = A(k)x + B(k)u u (x, k) := optimal control for k-th stage optimization ]

Consider k = k f 1 and [ 1 min u 2 x T Q(k)x + 1 2 ut R(k)u + 1 ] 2 x T (k f, u)p(k f )x(k f, u) [ 1 = min u 2 x T Q(k)x + 1 2 ut R(k)u + 1 2 x T A T (k)p(k f )A(k)x + u T B T (k)p(k f )A(k)x + 1 ] 2 ut B T (k)p(k f )B(k)u [ ] 0 = R(k) + B T (k)p(k f )B(k) u + B T (k)p(k f )A(k)x Hence optimal control policy is: u (x, k) = [R(k) + B(k)P(k f )B(k)] 1 B T (k)p(k f )A(k)x

The optimal cost is: J(x, k) = 1 2 x T P(k)x where P(k) =Q(k) + A T (k)p(k f )A(k) + u T [R(k) + B(k)P(k f )B(k)] u + 2u T B T (k)p(k f )A(k)x =Q(k) + A T (k)p(k f )A(k) + u T [R(k) + B(k)P(k f )B(k)] u 2u T [R(k) + B(k)P(k f )B(k)] u =Q(k) + A T (k)p(k f )A(k) A T (k)p(k f )B(k) [R(k) + B(k)P(k f )B(k)] 1 B T (k)p(k f )A(k)

Riccati Difference Equation Writing k f = k + 1, we have a backwards recursion: P(k) = Q(k) + A T (k)p(k f )A(k) A T (k)p(k + 1)B(k) [R(k) + B(k)P(k + 1)B(k)] 1 B T (k)p(k + 1)A(k) with P(k f ) = S. This is the Discrete Time Riccati Difference Equation.

Discrete Time LQ - Solution The optimal control is given by: u o (k) = K (k) x(k) K (k) = [R(k) + B T (k)p(k + 1)B(k)] 1 B T (k)p(k + 1)A(k) where P(k) is solution to Discrete Time Riccati Difference Eqn: P(k) =Q(k) + A T (k)p(k + 1)A(k) [ 1 A(k) T P(k + 1)B(k) R + B T (k)p(k + 1)B(k)] B T (k)p(k + 1)A(k); P(k f ) = S. The optimal cost-to-go at time k is: J o (x(k), k) = 1 2 x T (k)p(k)x(k). Notice that positive definiteness condition on R(k) implies that: is invertible. R(k) + B T (k)p(k + 1)B(k)

Discrete time LTI LQR For A, B, Q and R being constants, we have: P(k ) P satisfies the Algebraic Discrete Time Riccatti Equation: A T P A A T P B[R + B T P B] 1 B T PA + Q = P If (A, B) is controllable and (A, C) (where Q = C T C) is observable, then P is the unique positive definite solution. If (A, C) is only detectable, then P is the positive semi-definite solution. The feedback gain is then: K = [R + B T P B] 1 B T P A The closed loop system is stable, meaning that all eigenvalues of A BK have magnitudes less than 1 (lie in the unit disk centered at the origin).

Extensions Infinite LQ design methodology ensures a set of gains such that the closed loop poles are on the open LHP for continuous time system, and inside a unit disk for a discrete time system. The above properties can be exploited to ensure that closed loop poles lie in a certain region (left of α or within a disk). Frequency weighting can be used to penalized control or performance. Approach is to design weighting filters, and then convert the Frequency Shaped LQ into a standard LQ problem. Model mataching can also be done using similar procedure (See Goodwin section 22.6)

Eigenvalue placements LQR can be thought of as a way of generating stabilizing feedback gains. However, exactly where the closed loop poles are in the LHP is not clear. We now propose a couple of ways in which we can exert some control over them. The idea is to transform the problem. In this section, we assume that (A, B) is controllable, and (A, C) is observable where Q = C T C.

Guaranteed convergence rate Problem 1: Move poles so that they are to the left of α This ensures that convergence rate is at lesat e αt. (i.e. if the eigenvalues of A BK are λ i, we want Re(λ i ) < α, hence more stable). Idea: LQ gain K gives: ẋ = A x + Bu Re ( eig(a BK ) ) < 0 eig(a + αi) = eig(a) + α

Design LQ control for new problem and get gain K. ẋ = (A + αi)x + Bu. This ensures that the Re(eig(A + αi) BK ) < 0. Notice that (A + αi) BK and A BK have the same eigenvectors. Thus, the eigenvalues of A BK, say λ i and those of A + αi BK, σ i, are related by λ i = σ i α Since Re(σ i ) < 0, Re(λ i ) < α.

Eigenvalues to lie in a disk A more interesting case is to ensure that the eigenvalues of the closed loop system lie in a disk centered at ( α, 0) and with radius ρ < α. This, in addition to specifying the convergence rate to be faster than α ρ, it also specifies limits for the damping ratio, so that the system will not be too oscillatory.

Eigenvalues to lie in a disk -2 Idea: Use discrete time LQ eigenvalues of A BK lie in a unit disk centered at the origin Scale the unit disk to one with radius ρ Shift the eigen values by α

Eigenvalues to lie in a disk -3 Let the continuous time plant be: ẋ = Ax + Bu Scaling: If we solve the discrete time LQ problem for the plant, x(k + 1) = 1 ρ A x(k) + 1 ρ Bu(k) then, the eigenvalues of 1 ρ (A BK ) would lie in the unit disk and the eigenvalues of (A BK ) would lie in the disk with radius ρ, both centered at the origin. Translation: Same trick as before, set A = A + αi.

Eigenvalues to lie in a disk - 3 In summary, if we use the discrete time LQ control design method for the plant x(k + 1) = 1 ρ (A + αi)x(k) + 1 ρ Bu(k) then, the eigenvalues of 1 ρ ((A + αi) BK ) would lie within the unit disk centered at the origin. This implies that the eigenvalues of ((A + αi) BK ) lie in a disk of radius ρ centered at the origin. Finally, this implies that the eigenvalues of A BK lie in a disk or radius ρ centered at ( α, 0). Note: Discrete time LQ design is used even as original problem is in continuous time!!!

Frequency Shaping Original LQ problem is specified in the time domain. The cost function is the L 2 norms of the weighted control and u = R 1 2 u, and of the weighted state z = Q 1 2 x. = 0 0 x T Qx + u T Rudt z T z + u T u dt = z( ) 2 2 + u ( ) 2 2 Frequency domain sometimes more useful. For example In dual stage actuator, one actuator prefers large amplitude low frequency, the other prefers high frequency small amplitude Disturbances lie within a narrow bandwidth Robustness are easier to specify in the frequency domain

Parseval theorem For a squared integrable function h(t) R p with ht (t)h(t)dt <, h T (t)h(t)dt = 1 H (jw)h(jw)dw (1) 2π H(jw) is the fourier transform or as H(s = jw), i.e. the Laplace transform of h(t) evaluated at s = jw. H (jw) denotes the conjugate transpose of H(jw). For H(s) with real coefficient, H (jw) = H( jw) T. Parseval theorem states that the energy (L 2 norm) in the signal can be evaluated either in the frequency or in the time domain.

Frequency Domain Cost Function So, suppose that we want to optimize the criteria in the frequency domain as: J(u) = 1 2π X (jw)q 1 (jw)q 1(jw)X(jw) + U (jw)r 1 (jw)r 1(jw)U(jw) dw (2) State and control weightings are given by Q(w) = Q 1 (jw)q 1(jw); R(w) = R 1 (jw)r 1(jw).

If we define the filtered version of x and u as: Cost function becomes: J(u) = 1 2π X 1 (s) = Q 1 (s)x(s) U 1 (s) = R 1 (s)u(s) X 1 (jw)x 1(jw) + U 1 (jw)u 1(jw) dw Now, apply Parseval Theorem in reverse, J(u) = x T 1 (t)x 1(t) + u T 1 (t)u 1(t) dt. (3) If we know the dynamics of x 1 and u 1 is the control input, then we can solve using the standard LQ technique. However, u 1 is not actually the input...

Q 1 (s) and R 1 (s) are filters (e.g. low/high pass, stop band etc.): Filter inputs: actual state x(t) and input u(t) Filter outputs: x 1 (t) and u 1 (t) Realization of the filters: Q 1 (s) = C Q (si A Q ) 1 B Q + D Q (4) R 1 (s) = C R (si A R ) 1 B R + D R (5) which says that in the time domain: ż 1 = A Q z 1 + B Q x (6) x 1 = C Q z 1 + D Q x (7) and similarly, ż 2 = A R z 2 + B R u (8) u 1 = C R z 2 + D R u. (9)

Example: Q 1 (s) = 5 s + 5 ż = 5z + 5u y = z so that Y (s)/u(s) = Q 1 (s). In general, we can realize" a transfer function using Matlab s» [A, B, C, D] = tf2ss(num, den)

Hence we can define an augmented plant: x A 0 0 x B d z 1 = B dt Q A Q 0 z 1 + 0 u(t) z 2 0 0 A R z 2 B R or with x = [x; z 1 ; z 2 ], etc. x = Ā x + Bu. u 1 = ( 0 0 C R ) x + DR u x 1 = ( D Q C Q 0 ) x Substituting above, the cost function Eq.(3) becomes: J(u) = ( x T u T )( Q e N T ) )( x dt (10) N u where DQ T D Q DQ T C Q 0 0 Q e = CQ T D Q CQ T C Q 0 ; N = 0 ; R e = D 0 0 CR T C R CR T D R T D R. R R e

Convert problem into standard form 1 Eq.(10) is still not in standard form yet because of the off diagonal block N. We can convert Eq.(10) into the standard form if we consider: u(t) = R 1 e N x + v (11) The integrand in Eq.(10) becomes: ( x T v T )( I N T R 1 )( Qe N T )( 0 I N R e = ( x T v T )( Q e N T Re 1 ) N 0 )( x 0 R e v = x T Q x + v T R e v e I 0 R 1 e N I ) )( x v where Q := Q e N T R T e N, R = Re

Convert problem into standard form 2 Define new state dynamics: 1 x = (Ā BR e N) x + Bv (12) and cost function, J(v) = x T Q x + v T Rv dt. (13) Eqs.(12)-(13) are in standard LQ format and can be solved!!! The stabilizability and detectability conditions are now needed for the the augmented system (what are they?).

Summary - various extensions Discrete time LQ can be formulated (and derived) similarly as in continuous time With (A, B) stabilizable and (A, C) detectable (where Q = C T C), LQ control gives procedures for control gains that result in stable closed loop systems (eigenvalues on LHP for cont. time or inside unit disk for discrete time). Tuning then involves tuning Q and R instead of tuning the gain directly

Summary - cont d We can place eigen values in a specified region by transforming systems. Note discrete time LQ procedure is used for continuous time problem!!! Frequency dependent cost function can be used by defining problem in terms of filtered input and state In general, LQ is a reliable first cut control design approach.