34-5 LINEAR SYSTEMS Lecure : Riccai Equaions and Leas Squares Feedback Conrol 5.6.4 Sae Feedback via Riccai Equaions A recursive approach in generaing he marix-valued funcion W ( ) equaion for i for he purpose of realizing he opimal conrol law in real ime. is o find a differenial Thus he marix differenial equaion can be approximaed by a difference equaion which can be solved numerically. Theorem: (properies of he backward conrollabiliy Grammian) The backward conrollabiliy Grammian W ( has he following properies for : (a) W ( is symmeric (b) W ( is posiive semidefinie d W ( ) = AW ( ) ( ) + W ( ) A ( ) BB ( ) ( ) W ( ) = θn n d (c) (d) W ( saisfies he equaion W( ) = W( ) +Φ( ) W( ) Φ ( ) Similar o Theorem on properies of he forward conrollabiliy Grammian in L8. Noe: The forward and backward conrollabiliy Grammians are relaed as follows: W ( ) =Φ( ) M ( ) Φ ( ) M ( ) =Φ( ) W ( ) Φ ( ) (5.7) Proposiion: The inverse of he backward conrollabiliy Grammian W equaion: ( ) saisfies he marix differenial Riccai d W ( A () W ( W ( A () W ( B () B () W = + ( d W ( ) = (5.73) L- /9
34-5 LINEAR SYSTEMS Differeniae: d d W ( ) = W ( ) W ( W ( ) d d Remarks: (a) One way of generaing W ( ) in real ime is o solve recursively a difference equaion approximaing he differenial Riccai Equaion (5.73) subjec o he iniial condiion W (. (b) The opimal conrol for he ransfer o he origin ( x ) ( ) is he sae feedback u = B W x. op () () ( () 5.7 Leas Squares Sae Feedback Conrol The leas-squares feedback conrol problem is defined as follows. Given: A sysem x () = A () x () + Bu () () x ( ) = x A se Σ x ( ) o which x ( belongs A cos funcion I N() u() J ( x u ) u ( ) x ( ) d x ( ) Qx( ) [ ] N () L() + x() The leas squares problem is ha of finding a conrol law is minimized subjec o he consrains. u U ad such ha he cos J ( x u ) [ ] This problem was parly solved for he minimum-norm inpu bu here we add he sae in he cos funcion in order o minimize i as well. The leas-squares problem is classified ino: (i) (ii) Free-end-poin problem Fixed-end-poin problem. 5.7. Free-End-Poin Problem The free-end-poin problem is defined as follows. Given: A sysem x () = A () x () + Bu () () x ( ) = x L- /9
34-5 LINEAR SYSTEMS A cos funcion [ ] = + + J ( x u ) u ( ) u( ) x ( ) L( ) x( ) d x ( ) Qx( ) L () L = () Q= Q ( N () = θ in he general problem definiion) Find a conrol law u U ad such ha he cos J ( x u ) is minimized subjec o he consrains. [ ] Noe ha because L () and Q are no assumed o be nonnegaive i is no generally rue ha a minimum exiss for J ( x u ). However in many conrol applicaions hese condiions will hold. [ ] I urns ou ha he deerminaion of condiions under which a minimum exiss as well as he acual calculaion of he minimizing conrol can be made o depend upon he soluion of he following Riccai equaion: P () A = () P () P () A () + PBB () () () P () L () (5.74) Since his equaion is nonlinear i is no clear ha for a given iniial condiion P a soluion will exis. Moreover even if soluions do exis for some ime inervals hey may fail o exis over longer inervals because of he possibiliy of having a finie escape ime. Example: (finie escape ime) Suppose ha he Riccai equaion is scalar and has he form p p p p ( ) = an( ) p( π /) = + () = () + () = Thus p () does no exis over he inerval. Noaion: The soluion o he Riccai equaion (5.74) will be denoed as Π (; P (or P ()) meaning ha is value a ime is Π ( ; P ) = P and is value a ime is generaed from Π ( ; P ) = P. Lemma: Suppose P () = P () and P () on he inerval [ ]. Then for he sysem x () = Ax () () + Bu () () we have: B ( ) P( ) u () u x d x P x = PB () () P () A() P () P () A () x () + + () () () () (). L- 3/9
34-5 LINEAR SYSTEMS If x() is a differeniable rajecory and if P() is a differeniable marix-valued funcion of ime hen: d x () Px () () d x () Px () () d = = + + x() Px () () x() P () x () x() Px () () x() Px () () d { } A () x () + Bu () () P () x () + x () { A() P () P () A () + PBB () () () P () L ()} x () + x() () () () () () P Ax Bu B ( ) P( ) u () u () x () d () () () () () () () x () PB P + A P + P A { + } d Theorem: Suppose L () = L() Q= Q and here exiss over [ ] a soluion o he Riccai differenial equaion P () A = () P () P () A () + PBB () () () P () L () P ( = Q hen: (a) There exiss a conrol u() which minimizes J ( x u op[ ]) subjec o he consrains x () = A () x () + Bu () () x ( ) = x (b) The minimum value of he cos funcion is ( op[ ] ) = Π( ; J x u x Q x (c) The opimal feedback conrol law is uop () =B() Π(; Q ) x() (d) The opimal open-loop conrol law is uop[ ]() =B() Π(; Q ) Φ ( ) x where Φ( ) is he sae ransiion marix for x () = A () BB () () Π(; Q x (). To ease he noaion le's use P () insead of Π (; Q o denoe he soluion of he Riccai equaion. From he lemma we have he ideniy for x () P() x() which we use in he cos funcion: L- 4/9
34-5 LINEAR SYSTEMS J [ ] ( x u ) I u( ) u () x () d + x ( ) Qx( ) L ( ) x ( ) I u( ) u x d + x P x + x P x L ( ) x ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) I u( ) u () x () L ( ) d x ( ) B ( ) P( ) u () + u ( ) x ( ) d+ x ( ) P( ) x( ) PB () () P () A() P () P () A () x () + + () () u () I B P u () x () d () () () () () () () () x () + x ( ) P( ) x( ) PB P + A P + P A + L I B () P() u () u () x () d x ( ) P( ) x( ) + PB () () PBB () () () P () x () = u () + B() Px () () d+ x( ) P ( ) x ( ) From his las equaliy we conclude ha he inpu ha minimizes he cos funcion is u () = B () P() x() which proves (a) and (c) and for which he cos funcion reaches is op minimum op ( op[ ] ) ( ; J x u = x Π Q x proving (b). For (d) noice ha he opimal rajecory x is given by he closed-loop equaion: x op () = A() B() B () Π(; Q ) xop (). Thus if Φ ( ) is he sae ransiion marix for he closed-loop sysem we have x () ( ) x( ) () = () () Φ ( ) ( ) is he opimal open-loop op policy. =Φ and hence uop B P x Remarks: Suppose ha L () = which amouns o weighing only he conrol signal in he cos funcion. Then he Riccai equaion reduces o P () A = () P () P () A () + PBB () () () P (). L- 5/9
34-5 LINEAR SYSTEMS Hence P() saisfies he same equaion as he inverse of he backward conrollabiliy Grammian W ( ). Furhermore i can be shown ha provided he inverse exiss. Π ( Q ; = W ( +Φ( Q Φ ( Theorem: Assume Q = Q. Le W ( be he backward conrollabiliy Grammian of x () = A () x () + Bu () () x ( ) = x. If he marix W ( ) +Φ( ) Q Φ ( ) is inverible [ ] hen here exiss a conrol ha minimizes [ ] = + J ( x u ) u ( ) u( ) d x ( ) Qx( ). Follows from he previous remark. 5.7. Fixed-End-Poin Problem The fixed-end-poin problem is defined as follows. Given: A sysem x () = A () x () + Bu () () x ( ) = x x ( = x A cos funcion ( N () Find a conrol law J ( x u[ ] ) u ( ) u( ) + x ( ) L( ) x( ) d L () = L () Q= Q = θ in he general problem definiion) u U ad such ha he cos J ( x u ) is minimized subjec o he consrains. [ ] This is an exension of he conrollabiliy problem sudied earlier in which a weigh on he sae is added. However here we discuss he soluion from he opimal conrol poin of view. If he erminal sae is required o belong o a cerain se insead of a fixed vecor he mehodology can be modified o accoun for his requiremen. In general calculus of variaions is he subjec ha deals wih such opimizaion problems. From Secion 5.6 we found an expression for he minimum-norm conrol: [ ] u () = B() Φ ( ) W ( ) Φ( ) x x (5.75) op which leads o he following lemma. L- 6/9
34-5 LINEAR SYSTEMS Lemma: Le W ( be he backward conrollabiliy Grammian of x () = Ax () () + Bu () (). If uop () is any conrol of he form uop () =B() Φ ( ) ξ where vecor ξ is a soluion of [ ] W( ) ξ= Φ( ) x x (5.76) hen he conrol uop () drives he sae from ( x ) ( x ). If u () is any oher conrol wih he same propery i.e. ( x ) ( x ) hen op = op op L u u () u () d u u() d. (5.77) Moreover if W ( is nonsingular hen op L[ ] op op [ ] [ ] u = u () u () d = x Φ( ) x W ( ) x Φ( ) x Theorem:. (5.78) Suppose ha L () = L () and ha here exiss a symmeric marix P ( such ha he soluion Π (; P ( ) ) of he differenial Riccai equaion P () A = () P () P () A () + PBB () () () P () L () (5.79) holds. Then (a) A differeniable rajecory x() defined on he inerval [ ] and saisfying x () = A () x () + Bu () () x ( ) = x x ( ) = x minimizes he cos J ( x u[ ] ) u ( ) u( ) + x ( ) L( ) x( ) d (5.8) if and only if i minimizes he coss ( [ ]) ( ) ( ) = L[ ] J x u v vd v (5.8 subjec o he consrains x () = A () BB () () Π (; P ( ) ) x () + Bv () () x ( ) = x x ( ) = x (5.8) L- 7/9
34-5 LINEAR SYSTEMS (b) Along any sae rajecory saisfying he boundary condiions we have J x u J x u x P x x P x (5.83) ( [ ] ) ( [ ] ) = + Π( ; ( ( From he Lemma in 5.7. we have J [ ] ( x u ) I u( ) u () x () d L ( ) x ( ) I u( ) u () x () d L ( ) x ( ) B ( ) P( ) u () + u () x () d PB () () P () A() P () P () A () x () x () P() x() + + () () u () I B P u () x () d x () P() x() PB () () P () A() P () P () A () L () x () + + + I B () P() u () u () x () d PB () () PBB () () () P () x () x () P() x() = u () + B() Px () () d+ x P ( ) x x P ( ) x Leing v (): = u () + B () Px () () (which proves (b)) we obain he equivalen problem of minimizing he norm of v subjec o he consrain x () = A () x () + B () v () B() Px () () = A () BB () () P () x () + Bv ()() x ( ) = x x ( ) = x (5.84) The soluion of which is given by he above lemma i.e. v () =B() Φ ( ) W ( ) Φ( ) x x (or oher sae feedback forms obained) op [ ] where he sae ransiion marix and he backward conrollabiliy Grammian correspond o he sysem (5.84) wih he "new A marix" A () BB () () P (). Once v () is compued u () can be solved for. op op L- 8/9
34-5 LINEAR SYSTEMS Noe: The relaionship beween v () and u () is shown in he block diagram below. v () - + u () + B () s I + x () A () B () Π(; P( ) ) L- 9/9