448 Asan Journal o Control Vol. 7 No. 4 pp. 448-454 December 25 Bre Paper SENSIIVIY APPROACH O OPIMAL CONROL FOR AFFINE NONLINEAR DISCREE-IME SYSEMS Gong-You ang Nan Xe Peng Lu ABSRAC hs paper deals wth the optmal control problem or a class o ane nonlnear dscrete-tme systems. By ntroducng a senstvty parameter epng the system varables nto a Maclaurn seres around t we transorm the orgnal optmal control problem or ane nonlnear dscrete-tme systems nto the optmal control problem or a sequence o lnear dscrete- tme systems. he optmal control law conssts o an accurate lnear term a nonlnear compensatng term whch s an nnte sequence o adjont vectors. In the present approach teraton s requred only or the nonlnear compensaton seres. By nterceptng a nte sum o the seres we obtan a suboptmal control law that reduces the complety o the calculatons. A numercal smulaton shows that the algorthm can be easly mplemented has a ast convergence rate. KeyWords: Dscrete-tme systems nonlnear systems optmal control senstvty approach. I. INRODUCION Physcal systems are nherently nonlnear n nature. he optmal control or nonlnear systems s one o the most mportant subjects o research n the eld o control theory. Lee studed the suboptmal local eedback control or a class o constraned nonlnear dscrete-tme control problems n []. Hager [2] Nagurka [3] Stojano [4] ang [5] also gave some mportant results. For the quadratc perormance nde the optmal control problem oten leads to a Hamlton-Jacob-Bellman (HJB) equaton or a nonlnear two-pont boundary value (PBV) problem. For the general optmal control problem or nonlnear systems however an analytcal soluton does not est. Manuscrpt receved December 8 23; accepted January 6 25. he authors are wth College o Inormaton Scence Engneerng Ocean Unversty o Chna Qngdao 2667 PRChna. Nan Xe s also wth School o Computer Scence echnology Shong Unversty o echnology Zbo 25549 P.R Chna. (emal: gtang@ouc.edu.cn). hs work was supported by the Natonal Natural Scence Foundaton o Chna (674) the Natural Scence Foundaton o Shong Provnce (Y2G2). hs has nspred researchers to propose approaches to obtanng an appromate soluton to the HJB equaton or the nonlnear PBV problem as well as to obtanng an appromate optmal control or nonlnear systems. One o these approaches s power seres appromaton (PSA). hs approach nvolves usng a power seres [6-9] or so-called Adoman decomposton [] to separate the nonlnear terms appromate the soluton o the HJB equaton. In order to obtan the appromate optmal control law n a seres orm ths approach needs an teratve soluton o a seres o HJB equatons (nonlnear matr derental equatons). Another approach s successve Galerkn appromaton (SGA) where an teratve process s used to nd a sequence o appromatons approachng the soluton o the HJB equaton [-3] whch s done by solvng a sequence o generalzed HJB equatons. However beng an teratve method SGA s dependent on the teratve ntal value. I t s not well chosen the method may converge very slowly or even not converge at all. Recently ang [4] presented a successve appromaton approach to optmal control or nonlnear systems wth a quadratc perormance nde. he optmal control law can be obtaned by an adjont vector sequence. Other approaches have been presented n [5-7].
G.-Y. ang et al.: Senstvty Approach to Optmal Control or Ane Nonlnear Dscrete-me Systems 449 he man objectve o ths artcle s to develop a new PSA optmal control algorthm based on the senstvty approach or dscrete-tme nonlnear systems. By ntroducng a senstvty parameter we transorm the optmal control problem or ane nonlnear dscrete-tme systems nto an optmal control problem or lnear dscrete-tme systems. In the present approach the optmal control law conssts o an accurate lnear term a nonlnear compensatng seres. Iteraton s requred only or the nonlnear compensaton seres.e. the soluton o a sequence o adjont vector derence equatons. By nterceptng a nte summaton o the seres we obtan a suboptmal control law that can reduce the complety o the calculatons. hs paper s organzed as ollows. In secton 2 by ntroducng a senstvty parameter epng the system varables nto a Maclaurn seres wth respect to that parameter we obtan a suboptmal control law. In secton 3 a numercal eample s gven to llustrate the proposed approach. Several conclusons are drawn n the last secton. II. DESIGN OF SUBOPIMAL CONROL Consder the ollowng ane nonlnear dscrete-tme systems: ( k+ ) = g( ( k)) + Bu( k) k N () = () where R n s the state vector u R r s the control vector B s an n r constant matr s the ntal state vector N = { 2 }. Assume that g C (R n U R n ) s a polynomal vector uncton that g() =. he nonlnear uncton g can be rewrtten n the ollowng orm: g( k ( )) = Ak ( ) + ( k ( )) (2) where s a nonlnear polynomal vector uncton o A = g()/. hereore system () can be rewrtten as ( k+ ) = A( k) + ( ( k)) + Bu( k) k N () =. (3) he man objectve o ths paper s to nd an optmal control law u (k) whch mnmzes the quadratc perormance nde N J = ( N) Q ( N) + [ ( k) Q( k) + u ( k) Ru( k)] 2 2 k = (4) subject to () where R R r r s a postve-dente matr Q Q R n n are sem-postve-dente matrces. It s well known that the optmal control problem n (3) (4) oten leads to the ollowng soluton o a nonlnear PBV problem: λ ( k) = Q( k) + A λ ( k + ) + ( ( k)) λ ( k + ) ( k+ ) = Ak ( ) + Buk ( ) + ( k ( )) k N (5a) wth the boundary condtons () = λ ( N) = Q ( N) (5b) where = / N = { 2 N }. he optmal control law s uk ( ) = R B λ ( k+ ) k N. (6) Unortunately n general an analytcal soluton or ths nonlnear PBV problem does not est. hereore t s necessary to nd appromaton approaches to solvng the optmal control problem or nonlnear dscrete-tme systems. We propose a senstvty approach that can smply the PBV problem n (5) get the optmal control law. Construct the ollowng PBV problem n whch a senstvty parameter s ntroduced: λ( k ) = Q( k ) + A λ ( k+ ) + ( ( k )) λ ( k+ ) k ( + ) = Ak ( ) + Buk ( ) +( k ( )) k N (7a) wth boundary condtons ( ) = λ( N ) = Q ( N) (7b) the control law uk ( ) = R Bλ ( k+ ) k N (8) where. In the ollowng we assume that u(k ) (k ) λ(k ) are nntely derentable wth respect to around =. Epng them nto a Maclaurn seres we obtan uk ( ) = u ( k) ( k ) = ( k) =! =! λ( k ) = λ ( k) (9)! = where the superscrpt () denotes the th-order dervatve o the seres wth respect to when =. Obvously the PBV problem n (7) control law (8) are equvalent to the orgnal problem n (5) (6) when = respectvely. Note that the Maclaurn seres ((k )) can be eped as
45 Asan Journal o Control Vol. 7 No. 4 December 25 = ( ) ( k ( )) ( ( k )) = ()! that ( ( k )) λ ( k+ ) can be eped as ( ( k )) λ ( k+ ) where j ( j) ( j) C k k = = [( )!] ( ) ( ( )) λ ( + ) ( ) j! k C λ ( ) = =. j!( j)! = = Substtutng (9) () () nto (7) we obtan = ( k+ )! ( ) = A ( k) ( ( k)) Bu ( k) + +! λ ( k)! j ( j) ( = Q ( k) + C ( ) ( ( k)) λ ( k + ) = + A λ ( k+ )! u () ( k ) ( ).! R B k = λ +! k N = = () (2) (3) From (3) we can easly obtan ( ) ( k + ) = A ( k) + ( ( k)) + Bu ( k) j ( j) λ ( k) = Q ( k) + C ( ) ( ( k)) ( ( ) λ ( k + ) + A λ ( k + ) u ( k) = R B λ ( k+ ) k N (4a) wth boundary condtons () () () = λ ( N) = Q ( N) () = λ ( N) = I + (4b) where I + = { 2 }. When = we get the th-order PBV problem () () () ( k + ) = A ( k) + Bu ( k) () () () λ ( k) = Q ( k) + A λ ( k+ ) () () u ( k) = R B λ ( k+ ) k N (5a) wth boundary condtons () () () = λ ( N) = Q ( N). (5b) Lettng () () λ ( k) = P( k) ( k) k N (6) substtutng (6) nto (5) we obtan the ollowng th-order optmal control law: () () u ( k) = S ( k + ) B P( k + ) A ( k) k N. (7) Here S(k) = R + B P(K) B P(k) s the postve-dente soluton o the ollowng Rccat matr derence equaton: Pk ( ) = A I Pk ( ) BS ( k ) B + + Pk ( + ) A+ Q PN ( ) = Q. k N (8) When = 2 we get the th-order PBV problem ( ) ( k+ ) = A ( k) + Bu ( k) + ( ( k)) λ ( k) = Q ( k) + A λ ( k+ ) j ( j) ( + C ( ) ( ( k)) λ ( k + ) u ( k) = R B λ ( k + ) k N (9a) wth boundary condtons () = λ ( N) =. (9b) Note that () j ( j) ((k)) C ( ) ( ( k)) ( j λ ) ( k + ) are known unctons whch have been obtaned n the ( )th teraton then the PBV problem n (9) becomes a lnear nonhomogeneous one. In order to solve ths problem we let λ ( k) = P( k) ( k) + g ( k) k N. (2) Substtutng (2) nto (9) we can decouple the PBV problem (9) get the th costate equaton
G.-Y. ang et al.: Senstvty Approach to Optmal Control or Ane Nonlnear Dscrete-me Systems 45 g ( k) = A I P( k + ) BS ( k + ) B g k+ + P k + k ( ) ( ) ( ) ( ( )) + C k λ k + k N j ( j) ( ( ) ( ( )) ( ) g ( N) = I +. (2) By solvng (2) we can get the soluton o g (k). Substtutng (2) nto (9) we can also get ( k + ) = I + BR B P( k+ ) ( ) A ( k) BR B g ( k ) ( ( k)) + + () = I + (22) ( ) u ( k) = S ( k + ) B P( k + ) A ( k) + ( ( k)) S ( k + ) B g ( k + ) I +. (23) Lettng g (k) substtutng (7) (9) (23) nto (9) we can get uk ( ) = S ( k+ ) B =! ( ) =! ( Pk ( + ) A ( k) + g( k+ )) + P( k + ) ( ( k)) (24) Summarzng the above we obtan the ollowng theorem. heorem. Consder the problem o mnmzng the cost unctonal (4) subject to system (3); the control law * u ( k) = S ( k+ ) B Pk ( + )( Ak ( ) + ( k ( ))) + = g ( k + )! (25) s optmal where P(k) g (k) are solved by (8) (2) respectvely. Remark. From optmal control law (25) we know that an accurate result o g ( k + ) (!) s nearly mpossble = to get. By replacng wth M n (25) we can get a suboptmal control law as ollows: u ( k) = S ( k+ ) B M M g ( ) ( )( ( ) ( ( ))) k+ Pk+ Ak + k +. =! (26) In the ollowng we present an teratve procedure or ndng the suboptmal control law. Denote g ( k+ ) h ( k) = S ( k+ ) B = 2 M (27)! [ ] u ( k) = S ( k+ ) B P( k+ ) A( k) + ( ( k)). (28) From (26) we obtan u ( k) = u ( k) + h( k) = 2 M. (29) Remark 2. In (2) we can calculate () ((k)) as ollows: j n d ( ) () k j j! = d ( ) () =. (3) j n d ( ) n () k j! d j n d 2 ( ( )) k k j! d Remark 3. When N the quadratc perormance nde (4) can be descrbed as J = ( k) Q( k) u ( k) Ru( k) 2 + (3) k = (8) (2) respectvely become APA P APBS BPA+ Q= (32) ( ) g( k) = A I PBS B g( k + ) + P ( ( k)) + C k λ k + j ( j) ( ( ) ( ( )) ( ) g ( N) = I + (33) where ( k+ ) = I + BR B P ( ) A ( k) BR B g ( k ) ( ( k)) + +
452 Asan Journal o Control Vol. 7 No. 4 December 25 () = k N. (34) hereore we obtan the suboptmal control law M g ( k + ) um ( k) = S B P( A( k) + ( ( k))) +. =! (35) Smlarly to (27)-(29) the teratve procedure or ndng the suboptmal control law n (35) s as ollows. Denote g ( k + ) h ( k) = S B! = 2 M (36) [ ] u ( k) = S B P A( k) + ( ( k)). (37) From (35) we obtan 2 ( k) 2 ( k) ( ) =.4 2 3( k) 2( k) ( k) 2( k) 3( k). (39) + + 4 ( k) he quadratc cost unctonal s chosen as J = k + k + k + k + u k 2 2 2 2 2 2 ( ( ) 2( ) 3( ) 4( ) ( )). (4) 2 k = When we choose the suboptmal control law when = 2 3 4 the values o the quadratc cost uncton are J =.86 J = 8.583 J 2 = 8.42 J 3 = 7.738 J 4 = 7.4765 respectvely. Lettng =.5 we can get (J 4 J 3 ) /J 4 =.34 < then we can stop teratng. Smulaton results are presented n the ollowng. u ( k) = u ( k) + h ( k) = 2 M. (38) o nd the suboptmal control law (26) we propose the ollowng algorthm or the system n (3) wth the quadratc perormance nde n (4). Algorthm. Step. Work out the value o P rom epresson (8) gven α > ; Step 2. obtan the th-order control law u (k) rom (5) calculate the value o J lettng = ; Step 3. work out g (k) rom (2); Step 4. obtan the th-order control law u (k) rom (29) calculate the value o J ; Step 5. ( J J )/J < α then let M = obtan the output u M (k) stop. Else let = + turn to step 3. Fg.. he curves o (k) when = 2 4. Remark 3. When N we can easly calculate P g (k) u (k) J rom (32) (33) (38) (3) respectvely. Usng the above algorthm we can get the suboptmal control law (35). III. A NUMERICAL EXAMPLE Consder the 4th-order dscrete-tme nonlnear system descrbed by (3) where..2.5 A= B = () =...2.2.5 Fg. 2. he curves o 2 (k) when = 2 4.
G.-Y. ang et al.: Senstvty Approach to Optmal Control or Ane Nonlnear Dscrete-me Systems 453 the suboptmal control law manly result rom g M. he weaker the nonlnear term s the smaller the errors wll be whch brngs the suboptmal control trajectory closer to the theoretc optmal control law reduces the number o teratons M. IV. CONCLUSION Fg. 3. he curves o 3 (k) when = 2 4. he man contrbutons o ths paper are summarzed as ollows. By ntroducng a senstvty parameter epng the system varables nto a seres we tranorm the optmal control problem or ane nonlnear dscrete-tme systems nto an optmal control problem or lnear dscrete- tme systems. A suboptmal control law has been obtaned that reduces the complety o the calculatons. An algorthm or solvng the suboptmal control law has also been gven. REFERENCES Fg. 4. he curves o 4 (k) when = 2 4. Fg. 5. he curves o u(k) when = 2 4. From the above gures t s clear that the larger the number o teratons the better the control precson. When = 4 the derence between the values o the quadratc cost uncton based on the 4th-order control law on the 3rd-order control law s small enough. hs ndcates that the 4th suboptmal control law u 4 s very close to the optmal control law u. It can be seen that the errors due to. Lee H.W.J. K.L. eo V. Rehbock Suboptmal Local Feedback Control or a Class o Constraned Dscrete me Nonlnear Control Problems Comput. Math. Appl. Vol. 36 pp. 33-48 (998). 2. Hager W.W. Multpler Methods or Nonlnear Optmal Control SIAM J Numer. Anal. Vol. 27 pp. 6-8 (99). 3. Nagurka M.L. V. Yen Fourer-Based Optmal Control o Nonlnear Dynamc Systems rans. ASME J. Dyn. Syst. Meas. Contr. Vol. 2 pp. 7-26 (99). 4. Stojano V.S. Optmal Dampng Control Nonlnear Ellptc Systems SIAM J. Contr. Optm. Vol. 29 pp. 594-68 (99). 5. ang G.-Y. Z.-W. Luo Suboptmal Control o Lnear Systems wth State me-delay Proc. IEEE Con. Syst. Man Cybern. okyo pp. 4-9 (999). 6. Garrard W.L D.F. Enns S.A. Snell Nonlnear Feedback Control o Hghly Manoeuvrable Arcrat Int. J. Contr. Vol. 56 No. 4 pp. 799-82 (992). 7. Nshkawa Y. N. Sannomya H. Itakura A Method or Suboptmal Desgn o Nonlnear Feedback Systems Automatca Vol. 7 pp. 73-72 (97). 8. ang G.-Y. H.-P. Qu Y.-M. Gao Senstvty Approach o Suboptmal Control or a Class o Nonlnear Systems J. Ocean Unv. Qngdao Vol. 32 pp. 65-62 (22) (n Chnese). 9. ang G.-Y. X.-H. Zhao Y.-J. Lu Senstvty Approach o Optmal Control or Lnear Dscrete Large-Scale Systems wth State me-delay Proc. World Cong. Intell. Contr. Autom. Vol. Hangzhou Chna pp. 5-9 (24).. Chanane B. Optmal Control o Nonlnear Systems:
454 Asan Journal o Control Vol. 7 No. 4 December 25 A Recursve Approach Comput. Math. Appl. Vol. 35 pp. 29-33 (998).. Beard R.W. G.N. Sards J.. Wen Galerkn Appromaton o the Generalzed Hamlton-Jacob- Bellman Equaton Automatca Vol. 33 pp. 259-277 (997). 2. Ral W.B. W.M. mothy Successve Galerkn Appromaton Algorthms or Nonlnear Optmal Robust Control Int. J. Contr. Vol. 7 pp. 77-743 (998). 3. Sards G.N. C.S.G. Lee An Appromaton heory o Optmal Control or ranable Manpulators IEEE rans. Syst. Man Cybern. Vol. 9 pp. 52-59 (979). 4. ang G.-Y. Suboptmal Control or Nonlnear Systems: A Successve Appromaton Approach Syst. Contr. Lett. Vol. 54 No. 5 (25). 5. Banks S.P. Eact Boundary Controllablty Optmal Control or a Generalsed Korteweg de Vres Equaton Nonln. Anal. Vol. 47 No. 8 pp. 5537-5546 (2). 6. Goh C.J. On the Nonlnear Optmal Regulator Problem Automatca Vol. 29 pp. 75-756 (993). 7. Huang C.S. S. Wang K.L. eo Solvng Hamlton-Jacob-Bellman Equatons by a Moded Method o Characterstcs Nonln. Anal. Vol. 4 pp. 279-293 (2).