The Linear Quadratic Regulator on Time Scales

Transcription

1 Inernaional Journal of Difference Equaions ISSN , Volume 5, Number 2, pp (21) hp://campusmsedu/ijde The Linear Quadraic Regulaor on Time Scales Marin Bohner and Nick Winz Missouri Universiy of Science and Technology Deparmen of Mahemaics and Saisics 4 Wes 12h Sree, Rolla, MO , USA bohner@msedu and njwn7d@msedu Absrac In his paper, we unify and exend he linear quadraic regulaor o dynamic equaions on ime scales associaed wih a linear ime-invarian sysem Here we seek o find an opimal conrol ha minimizes a cos funcional We show ha when he final sae is fixed, he opimal (open-loop) inpu is in erms of a final sae difference On he oher hand, when he final sae is free, we have a closedloop opimal conrol AMS Subjec Classificaions: 34N5, 39A6, 39A1, 39A13, 49N5, 49N1, 49N4, 49N9 Keywords: Time scale, dynamic equaion, opimal conrol, regulaor problem, cos funcional, Riccai equaion 1 Inroducion In he lae 195s and early 196s, he emergence of sae-space mehods gave way o he formulaion and soluion of a new kind of opimal conrol problem The goal of such problems was o find an opimal conrol ha minimizes a quadraic cos funcional associaed wih a linear sysem In 1958, Kalman and Koepcke [8] firs used his mehod o find an opimal conrol for a sampled-daa sysem Shorly hereafer, Kalman exended hese resuls o coninuous ime (see [6, 7]) Since hen, wha is now called he linear quadraic regulaor (LQR) plays a cenral rôle in conrol engineering For more deails on he linear quadraic regulaor in he coninuous and discree cases, one may see he books by Lewis and Syrmos [1] and Kwakernaak and Sivan [9] Received Ocober 6, 21; Acceped November 3, 21 Communicaed by Mehme Ünal

2 15 Marin Bohner and Nick Winz In his work, we inroduce he concep of he linear quadraic regulaor on ime scales, hence unifying he discree and coninuous cases and exending hem o cases in beween We consider he linear ime-invarian sae equaion associaed wih he quadraic performance index x () = Ax() + Bu(), x( ) = x (11) J = 1 2 xt ( f )S( f )x( f ) (x T Qx + u T Ru)(τ) τ, (12) where S( f ), Q and R > In Secion 2, we offer a brief inroducion o calculus on ime scales We also consider he derivaive and inegral of funcions on an arbirary ime scale, along wih some basic properies of each Also, since we are mainly concerned wih linear sysems, we examine he marix exponenial and is properies In Secion 3, we consider he variaional properies needed such ha a minimum conrol exiss In Secion 4, we find he form of cos funcional (12) in he absence of an inpu In his seing, we use a generalized Lyapunov equaion o rewrie he cos funcional For Secions 5 and 6, we deermine an opimal conrol u ha minimizes (12) over he inerval [, f ] We assume ha x( ) is given and ha he final ime f T is a fixed known number Depending on he final sae, his opimal conrol can ake on wo compleely differen forms In Secion 5, he final sae is fixed resuling in an open-loop conrol This means ha he opimal inpu is no in erms of he curren sae, bu a final sae difference insead For Secion 6, he final sae is free resuling in a closed-loop conrol This means ha he opimal inpu is in erms of he curren sae (ie, sae feedback) Here, we use he simple useful formula o find a Riccai equaion whose soluion gives us an opimal conrol In Secion 7, we include scalar examples for boh cases This work is from he second auhor s disseraion [11] 2 Time Scales Preliminaries Before we presen our linear sysem, a brief inroducion o he heory of dynamic equaions on ime scales is in order For a more in-deph sudy of ime scales, see Bohner and Peerson s books [2, 3] Definiion 21 A ime scale T is an arbirary nonempy closed subse of he real numbers We le T κ = T \ {max T} if max T exiss; oherwise T κ = T Example 22 The mos common examples of ime scales are T = R, T = Z, T = hz for h >, and T = q N for q > 1 Definiion 23 We define he forward jump operaor σ : T T and he graininess funcion µ : T [, ) by σ() := inf {s T : s > } and µ() = σ()

3 The Linear Quadraic Regulaor on Time Scales 151 Definiion 24 For any funcion f : T R, we define he funcion f σ : T R by f σ = f σ Nex, we define he dela (or Hilger) derivaive as follows Definiion 25 Assume f : T R and le T κ The dela derivaive f () is he number (when i exiss) such ha given any ε >, here is a neighborhood U of such ha [f(σ()) f(s)] f ()[σ() s] ε σ() s for all s U In he nex wo heorems, we consider some properies of he dela derivaive Theorem 26 (See [2, Theorem 116]) Suppose f : T R is a funcion and le T κ Then we have he following: a If f is differeniable a, hen f is coninuous a b If f is coninuous a, where is righ-scaered, hen f is differeniable a and f () = f(σ()) f() µ() c If f is differeniable a, where is righ-dense, hen d If f is differeniable a, hen f f() f(s) () = lim s s f(σ()) = f() + µ()f () (21) Noe ha (21) is someimes called he simple useful formula Remark 27 Noe he following examples a When T = R, hen (if he limi exiss) b When T = Z, hen c When T = q Z for q > 1, hen f () = lim s f() f(s) s = f () f () = f( + 1) f() =: f() f () = f(q) f() (q 1) =: D q f()

4 152 Marin Bohner and Nick Winz Nex we consider he lineariy propery as well as he produc rules Theorem 28 (See [2, Theorem 12]) Le f, g : T R be differeniable a T κ Then we have he following: a For any consans α and β, he sum (αf + βg) : T R is differeniable a wih (αf + βg) () = αf () + βg () b The produc fg : T R is differeniable a wih (fg) () = f ()g() + f σ ()g () = f()g () + f ()g σ () Definiion 29 A funcion f : T R is said o be rd-coninuous on T when f is coninuous in poins T wih σ() = and i has finie lef-sided limis in poins T wih sup {s T : s < } = The class of rd-coninuous funcions f : T R is denoed by C rd = C rd (T) = C rd (T, R) The se of funcions f : T R ha are differeniable and whose derivaive is rd-coninuous is denoed by C 1 rd Theorem 21 (See [2, Theorem 174]) Any rd-coninuous funcion f : T R has an aniderivaive F, ie, F = f on T κ Definiion 211 Le f C rd and le F be any funcion such ha F () = f() for all T κ Then he Cauchy inegral of f is defined by b a f() = F (b) F (a) for all a, b T Example 212 Le a, b T wih a < b and assume ha f C rd When T = R, hen b a f() = b a f() d When T = Z, hen b a b 1 f() = f() =a Nex, we presen he marix exponenial and some of is properies Definiion 213 An m n marix-valued funcion A on T is rd-coninuous if each of is enries are rd-coninuous Furhermore, if m = n, A is said o be regressive (we wrie A R) if I + µ()a() is inverible for all T κ

5 The Linear Quadraic Regulaor on Time Scales 153 Theorem 214 (See [2, Theorem 58]) Suppose ha A is regressive and rd-coninuous Then he iniial value problem X () = A()X(), X( ) = I, where I is he ideniy marix, has a unique n n marix-valued soluion X Definiion 215 The soluion X from Theorem 214 is called he marix exponenial funcion on T and is denoed by e A (, ) Theorem 216 (See [2, Theorem 521]) Le A be regressive and rd-coninuous Then for r, s, T, a e A (, s)e A (s, r) = e A (, r), hence e A (, ) = I, b e A (σ(), s) = (I + µ()a())e A (, s), c e A (, σ(s)) = e A (, s)(i + µ(s)a(s)) 1, d (e A (, s)) = Ae A (, s), e (e A (, )) = e σ A(, )A(s) = e A (, )(I + µ(s)a(s)) 1 A(s) Nex we give he soluion (sae response) o our linear sysem using variaion of parameers Theorem 217 (See [2, Theorem 524]) Le A R be an n n marix-valued funcion on T and suppose ha f : T R n is rd-coninuous Le T and x R n Then he soluion of he iniial value problem x () = A()x() + f(), x( ) = x is given by x() = e A (, )x + e A (, σ(τ))f(τ) τ 3 Opimizaion of Linear Sysems on Time Scales Definiion 31 Le a, b T wih a < b and α, β R n A funcion ŷ C 1 rd wih ŷ(a) = α, ŷ(b) = β is said o be a (weak) local minimum o he variaional problem J (y) = b a L(, y σ (), y ()) min, y(a) = α, y(b) = β, (31) where L : T R 2n R, if here exiss δ > such ha y ŷ < δ and J (ŷ) J (y) for all y C 1 rd saisfying ŷ(a) = α and ŷ(b) = β If J (ŷ) < J (y) for all ŷ y, hen

6 154 Marin Bohner and Nick Winz ŷ is said o be proper An η C 1 rd is called an admissible variaion of (31) provided η(a) = η(b) = Le η C 1 rd be an admissible variaion We define he funcion Φ : R R by Φ(ε) = Φ(ε; y, η) = J (y + εη), ε R Then he firs variaion of J is defined by J 1 (y, η) = Φ(; y, η), while he second variaion of J is defined by J 2 (y, η) = Φ(; y, η) In he nex wo heorems, we provide necessary and sufficien condiions for a local minimum Theorem 32 (See [1, Theorem 32]) If ŷ C 1 rd is a local minimum of (31), hen J 1 (ŷ, η) = and J 2 (ŷ, η) for all admissible variaions η Theorem 33 (See [1, Theorem 33]) Le ŷ C 1 rd wih ŷ(a) = α and ŷ(b) = β If J 1 (ŷ, η) = and J 2 (ŷ, η) > for all nonrivial admissible variaions η, hen ŷ C 1 rd is a proper weak local minimum o (31) Now we consider he linear ime-invarian sysem (plan) x () = Ax() + Bu(), x( ) = x, (32) where x R n represens he sae and u R m represens he inpu Associaed wih (32) is he quadraic cos funcional J = 1 2 xt ( f )S( f )x( f ) (x T Qx + u T Ru)(τ) τ, (33) where S( f ), Q and R > Now we deermine he form of he inpu (conrol law) ha minimizes (33) To minimize (33), we inroduce he augmened cos funcional J + = 1 2 xt ( f )S( f )x( f ) [ ] (xt Qx + u T Ru) + (λ σ ) T (Ax + Bu x ) (τ) τ = 1 2 xt ( f )S( f )x( f ) + where he so-called Hamilonian H is given by [H(x, u, λ σ ) (λ σ ) T x ](τ) τ, H(x, u, λ) = 1 2 (xt Qx + u T Ru) + λ T (Ax + Bu), (34) and λ R n is a muliplier o be deermined in laer subsecions Nex, we provide necessary condiions for an opimal conrol We assume ha d dε f(τ, ε) τ = for all f : T R R such ha f(, ε), f(, ε)/ ε C rd (T) f(τ, ε) τ (35) ε

7 The Linear Quadraic Regulaor on Time Scales 155 Lemma 34 Assume (35) Then he firs variaion of J + is zero provided ha x, u, and λ saisfy Proof Firs noe ha Then x = Ax + Bu, λ = Qx + A T λ σ, Φ(ε) = J + ((x, u, λ) + ε(η 1, η 2, η 3 )) = 1 2 (x + εη 1) T ( f )S( f )(x + εη 1 )( f ) f f (36a) (36b) = Ru + B T λ σ (36c) [(x + εη 1 ) T Q(x + εη 1 ) + (u + εη 2 ) T R(u + εη 2 )](τ) τ [(λ σ + εη σ 3 ) T A(x + εη 1 ) + (λ σ + εη σ 3 ) T B(u + εη 2 )](τ) τ (λ σ + εη σ 3 ) T (x + εη 1 ) (τ) τ Φ(ε) = 1 2 ηt 1 ( f )S( f )(x + εη 1 )( f ) (x + εη 1) T ( f )S( f )η 1 ( f ) f f f f Thus he firs variaion can be wrien as [η T 1 Q(x + εη 1 ) + (x + εη 1 ) T Qη 1 ](τ) τ [η T 2 R(u + εη 2 ) + (u + εη 2 ) T Rη 2 ](τ) τ [(η σ 3 ) T A(x + εη 1 ) + (λ σ + εη σ 3 ) T Aη 1 ](τ) τ [(η σ 3 ) T B(u + εη 2 ) + (λ σ + εη σ 3 ) T Bη 2 ](τ) τ [(η σ 3 ) T (x + εη 1 ) + (λ σ + εη σ 3 ) T η 1 ](τ) τ Φ() = [S( f )x( f ) λ( f )] T η 1 ( f ) + λ T ( )η 1 ( ) + + f [(A T λ σ + Qx + λ ) T η 1 + (Ru + B T λ σ ) T η 2 ](τ) τ [(Ax + Bu x ) T η σ 3 ](τ) τ

8 156 Marin Bohner and Nick Winz Now in order for Φ() =, we se each coefficien of independen incremens η 1, η 2, η σ 3 equal o zero This yields he necessary condiions for a minimum of J + Using he Hamilonian (34), we have sae and cosae equaions and Similarly, we have he saionary condiion This concludes he proof x = H λ (x, u, λ σ ) = Ax + Bu λ = H x (x, u, λ σ ) = Qx + A T λ σ = H u (x, u, λ σ ) = Ru + B T λ σ Remark 35 We noe ha x, u, and λ solve (36) if and only if hey solve x = Ax BR 1 B T λ σ, λ = Qx + A T λ σ, u = R 1 B T λ σ (37a) (37b) (37c) Noe ha in order o find an opimal conrol, one mus deermine a value for he cosae In Figure 31, we presen he block diagram ha describes he formulaion of he sae and cosae equaions in our linear quadraic opimal conroller Finally, we give sufficien condiions for a local opimal conrol Lemma 36 Assume (35) Then he second variaion of J + is posiive provided ha η 1 and η 2 saisfy he consrains η 1 = Aη 1 + Bη 2 and η 2 Proof Taking he second derivaive of Φ(ε), we have Φ(ε) = 1 2 ηt 1 ( f )S( f )η 1 ( f ) ηt 1 ( f )S( f )η 1 ( f ) f f [η T 1 Qη 1 + η T 1 Qη 1 + η T 2 Rη 2 + η T 2 Rη 2 ](τ) τ [(η σ 3 ) T Aη 1 + (η σ 3 ) T Aη 1 + (η σ 3 ) T Bη 2 + (η σ 3 ) T Bη 2 ](τ) τ [(η σ 3 ) T η 1 + (η σ 3 ) T η 1 ](τ) τ = η T 1 ( f )S( f )η 1 ( f ) + +2 [η T 1 Qη 1 + η T 2 Rη 2 ](τ) τ [(Aη 1 + Bη 2 η 1 ) T η σ 3 ](τ) τ

9 The Linear Quadraic Regulaor on Time Scales 157 u() x () B + R 1 B T σ() A λ () + A T Q Figure 31: Relaionship of he sae and cosae equaions wih he opimal conrol in he LQR If we assume ha η 1 and η 2 saisfy he consrain hen he second variaion is given by Φ() = η T 1 ( f )S( f )η 1 ( f ) + η 1 = Aη 1 + Bη 2, [η T 1 Qη 1 + η T 2 Rη 2 ](τ) τ (38) Noe ha S( f ) and Q while R > Thus if η 2, hen (38) is guaraneed o be posiive 4 Zero Inpu and he Observabiliy Lyapunov Equaion In his secion, we find he form of our cos funcional (33) in he absence of a given inpu Here we make no assumpions as o wheher or no he final sae is fixed We use he noion of Lyapunov and Riccai equaions on ime scales o obain our resuls We now consider he auonomous dynamic equaion (36a) wih B =, ie, where A is assumed o be regressive x = Ax, (41)

10 158 Marin Bohner and Nick Winz Definiion 41 Le S C 1 rd(t, R n n ) Then V : T R n R defined by is called a ime scales Lyapunov funcion V (, x) = x T S()x Lemma 42 Le S : T R n be differeniable If x solves (41), hen (x T Sx) = x T [A T S + (I + µa T )SA + (I + µa T )S (I + µa)]x (42) Proof Using he produc rule, we have (x T Sx) = (x T S) x σ + (x T S)x Now using (21) and (41), we have = [(x T ) S + (x T ) σ S ]x σ + (x T S)Ax (x T Sx) = [x T A T S + (x + µx ) T S ](x + µx ) + x T SAx This gives (42) as desired = [x T A T S + x T (I + µa) T S ](I + µa)x + x T SAx = x T [A T S + (I + µa T )SA + (I + µa T )S (I + µa)]x Definiion 43 Le S C 1 rd(t, R n n ) and Q be symmeric Then a Lyapunov equaion is defined o be S = (I + µa T ) 1 [A T S + (I + µa T )SA + Q](I + µa) 1 (43) As (43) describes he ineracion beween he plan and cos in he absence of u, i can be called an observabiliy Lyapunov equaion Lemma 44 An S solves (43) if and only if i solves S = A T S σ + (I + µa T )S σ A + Q (44) Proof Muliplying (43) by (I + µa T ) on he lef and by (I + µa) on he righ, we arrive a (I + µa T )S (I + µa) = A T S + (I + µa T )SA + Q Now expanding he lef-hand side, we have S µa T S µ(i + µa T )S A = A T S + (I + µa T )SA + Q Moving he las wo erms over o he righ-hand side and using (21) yields This gives (44) as desired S = A T (S + µs ) + (I + µa T )(S + µs )A + Q = A T S σ + (I + µa T )S σ A + Q

11 The Linear Quadraic Regulaor on Time Scales 159 Theorem 45 A soluion o (43) is given by S() = e T A( f, )S( f )e A ( f, ) + e T A(τ, )Qe A (τ, ) τ Proof Firs noe ha Theorem 216(a) allows he represenaion S() = e T A( f, ) S()e A ( f, ), where S() = S( f ) + Now using produc rule, we have e T A(τ, f )Qe A (τ, f ) τ S () = (e A( f, )) T S(σ())eA ( f, σ()) + e T A( f, ) S ()e A ( f, ) +e T A( f, ) S(σ())e A( f, ) Since by Theorem 216(e) (e A( f, )) T S(σ())eA ( f, σ()) = A T e A ( f, σ()) S(σ())e A ( f, σ()) = A T S(σ()), by Theorem 216(a) e T A( f, ) S ()e A ( f, ) = e T A( f, )e T A(, f )Qe A (, f )e A ( f, ) = Q, and by Theorem 216(c) and (e) e T A( f, ) S(σ())e A( f, ) = (I + µ()a T )e T A( f, σ()) S(σ())e A ( f, σ())a = (I + µ()a T )S(σ())A, S solves (44) By Lemma 44, S also solves (43) This concludes he proof Now we rewrie he quadraic performance index (33) as follows Theorem 46 Suppose ha S solves (43) If x solves (41) and u =, hen he cos funcional (33) can be rewrien as J = 1 2 xt ( )S( )x( )

12 16 Marin Bohner and Nick Winz Proof We use Lemma 42 o rewrie he cos funcional (33) as J = 1 2 xt ( f )S( f )x( f ) = 1 2 xt ( f )S( f )x( f ) f (x T Qx)() (x T Qx)() xt ( f )S( f )x( f ) xt ( )S( )x( ) = 1 2 xt ( )S( )x( ) = 1 2 xt ( )S( )x( ) = 1 2 xt ( )S( )x( ), where (43) is used in he las sep [x T Qx + (x T Sx) ]() (x T Sx) () x T [Q + (I + µa T )S (I + µa) + A T S(I + µa) + SA]x() 5 Fixed-Final-Sae and Open-Loop Conrol In his secion, we seek an opimal conrol when he final sae is fixed We assume Q = and solve (37) subjec o x( ) = x and x( f ) = x f, (51) where x f is a given fixed reference value Since x( f ) is fixed, i is redundan o have a final-sae weighing in he performance index, so we le S( f ) = Then he cos funcional (33) becomes J = 1 2 (u T Ru)(τ) τ (52) Definiion 51 The final sae difference, d f, is he difference beween he zero inpu soluion and he desired final sae, ie, d f = x( f ) e A ( f, )x( ) (53) Nex, we find an opimal conrol proporional o (53) Noe ha he following heorem mirrors Kalman s generalized conrollabiliy crierion as found in [4, Theorem 32] Theorem 52 Le Q = and suppose ha x, u, and λ solve (37) such ha (51) hold If G C := e A ( f, σ(τ))br 1 B T e T A( f, σ(τ)) τ is inverible, (54)

13 The Linear Quadraic Regulaor on Time Scales 161 hen u() = R 1 B T e T A( f, σ())g 1 C d f (55) Proof Since Q =, (37b) becomes λ = A T λ σ, whose soluion, using Theorem 216(e), is Using (56), (37a) becomes λ() = e T A( f, )λ( f ) (56) x () = Ax() BR 1 B T e T A( f, σ())λ( f ) (57) Now solving (57) wih Theorem 217 a ime = f, we ge x( f ) = e A ( f, )x( ) = e A ( f, )x( ) G C λ( f ) Then solving for λ( f ) and using (51), we have Using his and (56), (37c) becomes Hence (55) holds e A ( f, σ(τ))br 1 B T e T A( f, σ(τ))λ( f ) τ λ( f ) = G 1 C [x( f) e A ( f, )x( )] = G 1 C d f u() = R 1 B T λ(σ()) = R 1 B T e T A( f, σ())λ( f ) = R 1 B T e T A( f, σ())g 1 C d f Noe ha (54) represens a weighed conrollabiliy Gramian The conrol (55) represens he minimum-energy conrol ha drives he given iniial sae x( ) o he desired final reference value x( f ) I is called an open-loop conrol since while i depends on boh he iniial and final saes, i does no rely on he curren sae Nex, we deermine he opimal cos Theorem 53 If u is given by (55), hen he cos funcional (52) can be rewrien as J = 1 2 dt f G 1 C d f (58)

14 162 Marin Bohner and Nick Winz Proof Le u be as given by (55) Then (52) can be rewrien as J = 1 2 = 1 2 f = 1 2 dt f G 1 C Hence (58) holds (u T Ru)(τ) τ d T f G 1 C e A( f, σ(τ))br 1 RR 1 B T e T A( f, σ(τ))g 1 = 1 2 dt f G 1 C G CG 1 C d f = 1 2 dt f G 1 C d f e A ( f, σ(τ))br 1 B T e T A( f, σ(τ)) τg 1 C d f C d f τ 6 Free-Final-Sae and Closed-Loop Conrol In his secion, we develop an opimal conrol law in he form of sae feedback In considering he boundary condiions, noe ha x( ) is known (meaning η 1 ( ) = ) while x( f ) is free (meaning η 1 ( f ) ) Thus he coefficien on η 1 ( f ) mus be zero This gives he erminal condiion on he cosae o be λ( f ) = S( f )x( f ) (61) Remark 61 Now in order o solve he wo-poin boundary value problem, we make he assumpion ha x and λ saisfy a linear relaionship similar o (61) for all [, f ], ie, λ() = S()x() (62) This condiion (62) is called a sweep condiion, a erm used by Bryson and Ho in [5] Since he erminal condiion S( f ), i is naural o assume ha S as well Theorem 62 Assume ha S solves S = Q + A T S σ + (I + µa T )S σ ( I + µbr 1 B T S σ) 1 ( A BR 1 B T S σ) (63) If x saisfies and λ is given by (62), hen x = ( I + µbr 1 B T S σ) 1 ( A BR 1 B T S σ) x (64) λ = Qx + A T λ σ (65)

15 The Linear Quadraic Regulaor on Time Scales 163 Proof Since λ is as given in (62), we may use he produc rule, (63), (64), and (21) o arrive a which gives (65) as desired λ = S x S σ x = Qx + A T S σ x + (I + µa T )S σ x S σ x = Qx + A T S σ x + µa T S σ x = Qx + A T S σ x σ = Qx + A T λ σ, In order o rewrie he righ-hand side of he Riccai equaion (63), we now show he following marix ideniy Lemma 63 If R + µb T S σ B is inverible, hen A BR 1 B T S σ = ( I + µbr 1 B T S σ) [A B(R + µb T S σ B) 1 B T S σ (I + µa)] (66) Proof The righ-hand side of (66) is equal o A + µbr 1 B T S σ A B(R + µb T S σ B) 1 B T S σ (I + µa) and he las erm of (67) is equal o µbr 1 B T S σ B(R + µb T S σ B) 1 B T S σ (I + µa), (67) BR 1 ( R + R + µb T S σ B)(R + µb T S σ B) 1 B T S σ (I + µa) = BR 1 R(R + µb T S σ B) 1 B T S σ (I + µa) BR 1 (R + µb T S σ B)(R + µb T S σ B) 1 B T S σ (I + µa) = B(R + µb T S σ B) 1 B T S σ (I + µa) BR 1 B T S σ (I + µa) Using his in (67) yields (66) Theorem 64 If boh R + µb T S σ B and I + µbr 1 B T S σ are inverible, hen S solves (63) if and only if i solves S = Q+A T S σ +(I +µa T )S σ A (I +µa T )S σ B(R+µB T S σ B) 1 B T S σ (I +µa) (68) Proof The saemen follows direcly from Lemma 63 Nex, we rewrie he righ-hand side of he Riccai equaion (68) in erms of he Kalman gain defined as follows

16 164 Marin Bohner and Nick Winz Definiion 65 Le R + µb T S σ B be inverible Then he marix-valued funcion K() = (R + µ()b T S σ ()B) 1 B T S σ ()(I + µ()a) (69) is called he sae feedback or Kalman gain Lemma 66 If R + µb T S σ B is inverible and S is symmeric, hen (I + µa T )S σ B(R + µb T S σ B) 1 B T S σ (I + µa) = K T (R + µb T S σ B)K (61) Proof Using (69) wice, we have K T (R + µb T S σ B)K = (I + µa T )S σ B(R + µb T S σ B) 1 (R + µb T S σ B)K This gives (61) as desired = (I + µa T )S σ BK = (I + µa T )S σ B(R + µb T S σ B) 1 B T S σ (I + µa) Theorem 67 If R + µb T S σ B is inverible and S is symmeric, hen S solves (68) if and only if i solves S = Q + A T S σ + (I + µa T )S σ A K T (R + µb T S σ B)K (611) Proof The saemen follows direcly from Lemma 66 In order o rewrie he righ-hand side of he Riccai equaion (611) in so-called (generalized) Joseph sabilized form (see [1]), we now show he following marix ideniy Lemma 68 If R + µb T S σ B is inverible and S is symmeric, hen (I + µa T )S σ A K T (R + µb T S σ B)K = K T B T S σ + (I + µ(a BK) T )S σ (A BK) + K T RK (612) Moreover, boh sides of (612) are equal o (I + µa T )S σ (A BK) Proof We use (69) o rewrie he lef-hand side of (612) as (I + µa T )S σ A (I + µa T )S σ B(R + µb T S σ B) 1 (R + µb T S σ B)K and we use (69) again o rewrie he righ-hand side of (612) as = (I + µa T )S σ (A BK), K T B T S σ + (I + µa T )S σ (A BK) µk T B T S σ (A BK) + K T RK = (I + µa T )S σ (A BK) K T B T S σ (I + µa) + K T (R + µb T S σ B)K = (I + µa T )S σ (A BK) Hence (612) holds

17 The Linear Quadraic Regulaor on Time Scales 165 Theorem 69 If R + µb T S σ B is inverible and S is symmeric, hen S solves (611) if and only if i solves S = Q + (A BK) T S σ + (I + µ(a BK)) T S σ (A BK) + K T RK (613) and if and only if i solves S = Q + A T S σ + (I + µa T )S σ (A BK) (614) Proof This saemen follows direcly from Lemma 68 When an opimal conrol is in erms of he curren sae, i is said o be a closed-loop conrol We now consider he form of an opimal conrol ha minimizes (33) Theorem 61 Le R + µb T S σ B be inverible Suppose ha x, u, and λ solve (37) such ha (62) holds Then u() = K()x() (615) Proof Using (37c), (62), (21), and (36a), we have u() = R 1 B T λ σ () = R 1 B T S σ ()x σ () = R 1 B T S σ () ( x() + µ()x () ) = R 1 B T S σ ()(I + µ()a)x() µ()r 1 B T S σ ()Bu() Now combining like erms and pre-muliplying by R we have which implies (R + µ()b T S σ ()B)u() = B T S σ ()(I + µ()a)x(), u() = (R + µ()b T S σ ()B) 1 B T S σ ()(I + µ()a)x() Hence (615) follows using (69) The inpu (615) is referred o as he closed-loop conrol law ha minimizes he cos funcional (33) Noe ha in order o deermine his inpu, we need only K and he soluion o he Riccai equaion, S Bu recall ha we obained S from he sweep condiion, no from he Riccai equaion The Riccai equaion iself is used o find an opimal cos Now he closed-loop plan is given by x () = (A BK())x(), (616) which can be used o find an opimal rajecory for any given x( ) A block diagram describing his conrol scheme appears in Figure 61 Finally, we deermine he opimal cos

18 166 Marin Bohner and Nick Winz u() x () B + A K() S = A T S σ + (I + µa T )S σ ( I + µbr 1 B T S σ) 1 ( A BR 1 B T S σ) + Q K() = (R + µ()b T S σ ()B) 1 B T S σ ()(I + µ()a) Figure 61: The free-final-sae LQR Theorem 611 Suppose ha S solves (613) If x and u saisfy (616) and (615), hen he cos funcional (33) can be rewrien as J = 1 2 xt ( )S( )x( ) Proof Firs noe ha we may use he produc rule, (21), and (616) o find (x T Sx) = (x T S) x + (x T S) σ x = (x ) T S σ x + x T S x + (x + µx ) T S σ x = x T [(A BK) T S σ + S + (I + µ(a BK)) T S σ (A BK)]x Using his and (615) in (33), we evaluae J = 1 2 xt ( )S( )x( ) = 1 2 xt ( )S( )x( ) = 1 2 xt ( )S( )x( ) f f [(x T Sx) + x T Qx + u T Ru](τ) τ [(x T Sx) + x T Qx + x T K T RKx](τ) τ {x T [S + (A BK) T S σ ]x}(τ) τ { x T [(I + µ(a BK)) T S σ (A BK) + Q + K T RK]x } (τ) τ

19 The Linear Quadraic Regulaor on Time Scales 167 Since S saisfies (613), he inegrand is zero This concludes he proof From Theorem 611, if he curren sae and S are known, we can deermine he opimal cos before we apply he opimal conrol or even calculae i 7 Examples Example 71 Le θ() represen he emperaure of some objec a ime T Assume ha θ m represens he emperaure of he surrounding medium held consan Le u() be rae of hea supply o he medium Then he rae of change in he objec s emperaure may be modelled by he dynamic equaion θ () = a()(θ() θ m ) + b()u() Suppose ha we wan o find a minimum inpu needed o hea he objec over he inerval [, f ] Firs, define he sae o be he difference beween he objec s emperaure and is surrounding environmen, ha is Then he sae equaion is given by Nex, define he associaed cos funcional x() = θ() θ m x () = a()x() + b()u() J = 1 2 u 2 (τ) τ Then from he general form of he Hamilonian, we have H(, x, u, λ) = ru2 2 + λ(a()x + b()u), (71) where r is a consan Finally, he sae and cosae equaions and saionary condiion are given by and x () = H λ (, x(), u(), λ σ ()) = a()x() + b()u(), λ () = H x (, x(), u(), λ σ ()) = a()λ σ (), = H u (, x(), u(), λ σ ()) = ru() + b()λ σ () (72) Rewriing (72), we find he opimal conrol o be u() = b()λσ () (73) r

20 168 Marin Bohner and Nick Winz Now plugging he opimal conrol ino he sae-cosae equaions, we have x () = a()x() b2 ()λ σ (), λ () = a()λ σ () (74) r Assuming ha a R and solving for λ(), we have λ() = e a ( f, )λ( f ), (75) where λ( f ) is ye o be deermined Plugging (75) ino (74) yields We use Theorem 217 o find x () = a()x() b2 () r e a( f, σ())λ( f ) x() = e a (, )x( ) 1 b 2 (τ)e a (, σ(τ))e a ( f, σ(τ)) τλ( f ) r = e a (, )x( ) e a(, f ) b 2 (τ)e 2 r a( f, σ(τ)) τλ( f ) = e a (, )x( ) e a(, f ) b 2 (τ)e 2 a ( f, σ(τ)) τλ( f ), (76) r where 2 a = 2a + µa 2 In order o find our sae and cosae, we mus find λ( f ) Now we consider wo differen conrol schemes ha can be used o deermine λ( f ) Case 1 (Fixed Final Sae): Assume ha he emperaure of he objec is originally he same emperaure as is surrounding medium, θ m = 7 F (ie, x( ) = ) Now we are looking for an opimal conrol such ha he final emperaure of he objec is θ( f ) = 1 F This means ha he value ha he final sae mus ake on is given by x( f ) = 3 F Since boh he iniial and final saes are fixed, he boundary condiions are given by η 1 ( ) = η 1 ( f ) = Noe ha by (76), he final sae can also be wrien as x( f ) = 3 = 1 b 2 (τ)e 2 a ( f, σ(τ)) τλ( f ) (77) r Solving (77) for he final cosae, we have λ( f ) = Now by (75), he final cosae is given by 3r b 2 (τ)e 2 a ( f, σ(τ)) τ λ() = 3re a ( f, ) b 2 (τ)e 2 a ( f, σ(τ)) τ

21 The Linear Quadraic Regulaor on Time Scales 169 Using (73), he opimal conrol is given by u() = Finally, he opimal rajecory is given by 3b()e a ( f, σ()) (78) b 2 (τ)e 2 a ( f, σ(τ)) τ x() = 3e a (, f ) b 2 (τ)e 2 a ( f, σ(τ)) τ f (79) b 2 (τ)e 2 a ( f, σ(τ)) τ Case 2 (Free Final Sae): Now suppose ha he final sae is no quie 3 F However, we would like he final sae o be as close o 3 F as possible Thus o make he difference x( f ) 3 small, we include i in he cos funcional J = 1 2 s(x( f) 3) u 2 (τ) τ, (71) where s is some posiive consan Here, we would like o find a minimum inpu ha minimizes boh (71) and x( f ) 3, laer which can be hough of as an error erm Noe ha since he inegrand remains unchanged, he Hamilonian is sill given by (71) Thus he equaions for he sae, cosae, and conrol are he same as before Nex, we need o deermine our boundary condiions As before, x( ) = On he oher hand, x( f ) is no fixed, which means ha η 1 ( f ) Therefore we require ha λ( f ) = s(x( f ) 3) (711) Plugging (711) ino (76), solving for x( f ) and using his again in (76), we find λ( f ) = Using his in (75), we have he opimal cosae λ() = Using his in (73), we have he opimal conrol u() = 3rs r + s b 2 (τ)e 2 a ( f, σ(τ)) τ 3rse a ( f, ) r + s b 2 (τ)e 2 a ( f, σ(τ)) τ 3sb() e a ( f, σ()) (712) r + s b 2 (τ)e 2 a ( f, σ(τ)) τ

22 17 Marin Bohner and Nick Winz By (76), he opimal sae is given by x() = 3se a (, f ) b 2 (τ)e 2 a ( f, σ(τ)) τ f (713) r + s b 2 (τ)e 2 a ( f, σ(τ)) τ In Example 71, we were unable o evaluae he inegral for a general ime scale due o he way he exponenial is defined In he nex example, we consider a specific sae coefficien ha allows for evaluaion of his inegral Example 72 Le T be a general ime scale and suppose here exiss a consan c such ha c b2 () for all T (2 a)() Then we may use Theorem 216(e) o evaluae he relevan inegral b 2 (τ)e 2 a ( f, σ(τ)) τ = c From (76), we find, using x( ) =, (2 a)(τ)e 2 a ( f, σ(τ)) τ = c [e 2 a ( f, )] (τ) τ = c[e 2 a ( f, ) e 2 a ( f, )] x() = e a (, )x( ) c r e a(, f )[e 2 a ( f, ) e 2 a ( f, )]λ( f ) = c r e a(, f )[e 2 a( f, ) e 2 a( f, )]λ( f ) = c r [e a( f, ) e a (, )e a ( f, )]λ( f ) Case 1 (Fixed Final Sae): From (78), he opimal conrol is given by u() = From (79), he opimal sae is given by 3(2 a)e a ( f, ) b()[e 2 a ( f, ) e 2 a ( f, )] (714) x() = 3[e a( f, ) e a (, )e a ( f, )] (715) [e 2 a ( f, ) 1] Case 2 (Free Final Sae): From (712), we have he opimal conrol u() = 3s(2 a)b()e a ( f, σ()) r(2 a)() + sb 2 ()[e 2 a ( f, ) 1] (716)

23 The Linear Quadraic Regulaor on Time Scales 171 From (713), he opimal sae is given by x() = 3sb2 ()[e a ( f, ) e a (, )e a ( f, )] r(2 a)() + sb 2 ()[e 2 a ( f, ) 1] (717) b 2 () Example 73 Le T = R wih c (2 a)() = b2 () Then he sae and cosae 2a() equaions and saionary condiion are given by and ẋ() = a()x() + b()u(), λ() = a()λ(), = ru() + b()λ() Case 1 (Fixed Final Sae): By (714), he opimal conrol is given by u() = while he opimal sae by (715) is given by 6a()e a( f ) b()[e 2a( f ) e 2a( f ) ], x() = 3[ea(f ) e a( ) e a( f ) ] [e 2a( f ) 1] Case 2 (Free Final Sae): From (716), we have he opimal conrol u() = wih he opimal sae by (717) given by 6sa()b()e a( f ) 2ra() + sb 2 ()[e 2a( f ) 1] x() = 6sa()b2 ()[e a(f ) e a( ) e a( f ) ] 2ra() + sb 2 ()[e 2a( f ) 1] b 2 () Example 74 Le T = Z wih c (2 a)() = b 2 () Then he sae and a()[2 + a()] cosae equaions and saionary condiion are given by x() = a()x() + b()u(), and λ() = a()λ( + 1), = ru() + b()λ( + 1)

24 172 Marin Bohner and Nick Winz Case 1 (Fixed Final Sae): By (714), he opimal conrol is given by where u() = 3(2 a)()e a ( f, ) b()[e 2 a ( f, ) e 2 a ( f, )], e m ( f, ) = From (715), he opimal sae is given by τ T [, f ) (1 + m(τ)) x() = 3[e a( f, ) e a (, )e a ( f, )] [e 2 a ( f, ) 1] Case 2 (Free Final Sae): By (716), we have he opimal conrol u() = wih he opimal sae by (717) given by 3s(2 a)()b()e a ( f, + 1) r(2 a)() + sb 2 ()[e 2 a ( f, ) 1] x() = 3sb2 ()[e a ( f, ) e a (, )e a ( f, )] r(2 a)() + sb 2 ()[e 2 a ( f, ) 1] Example 75 Le T = q N wih q > 1 and c b2 () (2 a)() = b 2 () a()[2 + (q 1)a()] Then he sae and cosae equaions and saionary condiion are given by and D q x() = a()x() + b()u(), D q λ() = a()λ(q), = ru() + b()λ(q) Case 1 (Fixed Final Sae): By (714), he opimal conrol is given by where u() = e m ( f, ) = 3(2 a)()e a ( f, ) b()[e 2 a ( f, ) e 2 a ( f, )], τ T [, f ) From (715), he opimal sae is given by (1 + (q 1)mτ) x() = 3[e a( f, ) e a (, )e a ( f, )] [e 2 a ( f, ) 1]

25 The Linear Quadraic Regulaor on Time Scales 173 Case 2 (Free Final Sae): By (716), we have he opimal conrol u() = From (717), he opimal sae is given by 3s(2 a)()b()e a ( f, q)() r(2 a)() + sb 2 ()[e 2 a ( f, ) 1] x() = 3sb2 ()[e a ( f, ) e a (, )e a ( f, )] r(2 a)() + sb 2 ()[e 2 a ( f, ) 1] Example 76 Here we rewrie he Riccai equaion (613) for various ime scales a If T = R, hen (613) becomes Ṡ() = Q + (A BK())T S() + S()(A BK()) + K T ()RK(), where K() = R 1 B T S() b If T = Z, hen (613) is given by S() S( + 1) = Q + (A BK()) T S( + 1) + K T ()RK() +(I + A BK()) T S( + 1)(A BK()), where K() = (R + B T S( + 1)B) 1 B T S( + 1)(I + A) c If T = hz, hen (613) urns ino S() S( + h) h = Q + (A BK()) T S( + h) + K T ()RK() +(I + h(a BK()) T )S( + h)(a BK()), where K() = (R + hb T S( + h)b) 1 B T S( + h)(i + ha) d If T = q N for q > 1, hen (613) is given by D q S() = Q + (A BK()) T S(q) + K T ()RK() +(I + (q 1)(A BK()) T )S(q)(A BK()), where K() = (R + (q 1)B T S(q)B) 1 B T S(q)(I + (q 1)A) Remark 77 Aside from he examples given, he LQR can also be exended o include applicaions in racking and disurbance/rejecion We sudy such applicaions in a forhcoming paper I should be noed ha he LQR mirrors he conrollabiliy properies we sudied in [4] In fuure work, we seek an opimal sae esimae ha minimizes an error covariance This mirrors he observabiliy properies we sudied in [4] Such an observer is called a Kalman filer In he discree and coninuous cases, he Riccai equaions for he LQR and he Kalman filer are mahemaically dual o each oher We show ha his dualiy is preserved in heir unificaion and exension

26 174 Marin Bohner and Nick Winz References [1] Marin Bohner Calculus of variaions on ime scales Dynam Sysems Appl, 13(3-4): , 24 [2] Marin Bohner and Allan Peerson Dynamic equaions on ime scales Birkhäuser Boson Inc, Boson, MA, 21 [3] Marin Bohner and Allan Peerson, ediors Advances in dynamic equaions on ime scales Birkhäuser Boson Inc, Boson, MA, 23 [4] Marin Bohner and Nick Winz Conrollabiliy and observabiliy of ime-invarian linear dynamic sysems Mah Bohem, 211 To appear [5] Arhur E Bryson, Jr and Yu Chi Ho Applied opimal conrol Hemisphere Publishing Corp Washingon, D C, 1975 Opimizaion, esimaion, and conrol, Revised prining [6] R E Kalman Conribuions o he heory of opimal conrol Bol Soc Ma Mexicana (2), 5:12 119, 196 [7] R E Kalman When is a linear conrol sysem opimal? Trans ASME Ser D J Basic Engineering, 86:81 9, 1964 [8] R E Kalman and R W Koepcke Opimal synhesis of linear sampling conrol sysems using generalized performance indexes Trans ASME Ser D J Basic Engineering, 8: , 1958 [9] Huiber Kwakernaak and Raphael Sivan Linear opimal conrol sysems Wiley- Inerscience [John Wiley & Sons], New York, 1972 [1] Frank L Lewis and Vassilis L Syrmos Opimal conrol Wiley, second ediion, 1995 [11] N Winz The Kalman Filer on Time Scales PhD hesis, Missouri Universiy of Science and Technology, Rolla, Missouri, USA, 29