A Variation Evolving Method for Optimal Control

Transcription

1 A Variaion Evolving Mehod or Opimal Conrol Sheng ZHANG, En-Mi YONG, Wei-Qi QIAN, and Kai-Feng HE (7. Absrac: A new mehod or he opimal soluions is proposed. Originaing rom he coninuous-ime dynamics sabiliy heory in he conrol ield, he opimal soluion is anicipaed o be obained in an asympoically evolving way. By inroducing a virual dimension he variaion ime, a dynamic sysem ha describes he variaion moion is deduced rom he Opimal Conrol Problem (OCP, and he opimal soluion is is equilibrium poin. hrough his mehod, he inracable OCP is ransormed o he Iniial-value Problem (IVP and i may be solved wih maure Ordinary Dierenial Equaion (ODE numerical inegraion mehods. Especially, he deduced dynamic sysem is globally sable, so any iniial value will evolve o he exremal soluion ulimaely. Key words: Opimal conrol, dynamics sabiliy, variaion evolving, iniial-value problem. I. INRODUCION Opimal conrol heory aims o deermine he inpus o a dynamic sysem ha opimize a speciied perormance index while saisying consrains on he moion o he sysem. I is closely relaed o engineering and has been widely sudied []. Because o he complexiy, usually Opimal Conrol Problems (OCPs are solved wih numerical mehods. Various numerical mehods are developed and generally hey are divided ino wo classes, namely, he direc mehods and he indirec mehods []. he direc mehods discreize he conrol or/and sae variables o obain he Nonlinear Programming (NLP problem, or example, he widely-used direc shooing mehod [] and he classic collocaion mehod [3]. hese mehods are easy o apply, whereas he resuls obained are usually subopimal [4], and he opimal may be ininiely approached. he indirec mehods ransorm he OCP o a boundary-value problem (BVP hrough he opimaliy condiions. ypical mehods o his ype include he well-known indirec shooing mehod [] and he novel symplecic mehod [5]. Alhough be more precise, he indirec mehods oen suer rom he signiican numerical diiculy due o ill-condiioning o he Hamilonian dynamics, ha is, he sabiliy o cosaes dynamics is adverse o ha o he sae dynamics [6]. he recen developmen, represenaively he Pseudo-specral (PS mehod [7], blends he wo ypes o mehods, as i uniies he NLP and he BVP in a dualizaion view [8]. Such mehods inheri he advanages o boh ypes and blur heir dierence. he auhors are wih he Compuaional Aerodynamics Insiuion, China Aerodynamics Research and Developmen Cener, Mianyang, 6, China. ( zszhangshengzs@homail.com.

2 heories in he conrol ield oen enlighen sraegies or he opimal conrol compuaion, or example, he nonlinear variable ransormaion o reduce he variables [9]. In his paper, he dynamics sabiliy heory [], regarding he coninuous-ime sysem evolving, moivaes he new mehod he Variaion Evolving Mehod (VEM, and he OCPs are ransormed o he Iniial-value Problems (IVPs. A virual variaion ime dimension τ, disinguished rom he normal ime variable in he OCP, is inroduced o describe he evoluion o he opimal soluion. Generally, he BVP can be ransormed o he IVP only or special cases (see [] and [] or example, while here ypical OCPs may be ransormed o he IVPs wih respec o τ, rom a new angle o view. By guaraneeing he exremum as he equilibrium poin o he deduced dynamic sysem, he opimal soluion will be gradually approached. Fig. illusraes he idea o he VEM in solving he OCP. hrough variaion moion, he iniial guess o variable will evolve o he opimal soluion. Fig.. he illusraion o he variable evolving in he VEM. In he ollowing, irs he oundaional VEM is presened by applying o he unconsrained calculus-o-variaions problem. hen he opimal conrol or he dynamic sysem is solved. We presen some commens abou he work in Secion IV and Secion V concludes he paper a he end. II. HE FOUNDAIONAL VARIAION EVOLVING MEHOD In he paper, our work is buil upon he assumpion ha he soluion or he opimizaion problem exiss. We do no describe he exising condiions or he purpose o breviy. Relevan researches such as he Filippov-Cesari heorem are documened in [3]. From his premise, we demonsrae he oundaional VEM by solving he unconsrained calculus-o-variaions problem ha is deined as Problem : For he ollowing uncional depending on variable vecor y( ( n J = F y(, y (, d ( where is ime. he elemens o y belong o C [, ], which denoes he se o variables wih coninuous second-order derivaives (indicaed by he superscrip. he uncion F : and is irs-order and second-order parial n n

3 3 derivaives are coninuous wih respec o y, is ime derivaive d y = y and. and are he ixed iniial and erminal ime. d Find he exremum ŷ ha minimizes J, i.e. y ˆ = arg min( J ( hrough he variaion heory, he exremum or his uncional saisies he Euler-Lagrange equaion [4] d Fy ( Fy = (3 d and he boundary condiions F = (4 y ( F ( = (5 y where F y F = y and F F y = y are he shorhand noaions o parial derivaives. Insead o direcly solving he BVP deined by Eqs. (3-(5 o ge he soluion, i is enlighened by he saes evolving in coninuous-ime sable dynamic sysems ha any iniial guess y (, whose elemens belong o C [, ], will evolve o he exremum along he variaion dimension. Like he decrease o δ J a Lyapunov uncion, hrough inroducing he variaion ime, τ, i J decreases wih respec o τ, i.e., <, we may inally obain he opimal soluion. Diereniaing Eq. ( wih respec o τ produces δj δ y δ y d δ y = Fy Fy + F ( F d y y d (6 δ J where he superscrip denoes he ranspose operaor. By enorcing, we may se ha δ y d = K Fy ( Fy, (, (7 d δ y( KF ( (8 = y δ y( = KFy ( (9

4 4 where K = diag( k, k,..., k n is a posiive diagonal marix. o solve he variaion dynamic equaions, he iniial guess y ( = y (, [, ] may be arbirary variable proile wih elemens belonging o C [ τ =, ]. Equaions (7-(9 describe he variaion moion o y ( saring rom y (, and he moion is direced o he exremum. heorem : Solving he IVP deined by Eqs. (7-(9 rom he VEM, when τ +, y ( will saisy he opimaliy condiions o Problem. δ J Proo: Subsiuing Eqs. (7-(9 o Eq. (6, we have. he uncional J will decrease unil δ J =, which occurs when τ + due o he asympoical approach. When δ J =, his deermines he opimal condiions, namely, Eqs. (3-(5. Remark : I in Problem he boundary condiions o he variable vecor y ( are prescribed as y( = y and y( = y, hen we may solve he problem using Eq. (7 and wih he iniial guess y ( = y τ = and y ( = y. τ = δ y( = ( δ y( = ( Remark : Equaions (7-(9 render a asympoically converge o he exremum o Problem, while he limied ime convergence may be achieved by δ y d = K sign Fy ( Fy, (, ( d δ y( = K sign ( Fy ( (3 δ y( = K sign ( Fy ( (4 where sign( a sign( a sign( a = is he sign uncion.... sign( an

5 5 o veriy he mehod, we consider an example rom [4]. Example : Solve he scalar variable y, which minimizes he ollowing uncional wih prescribed boundary condiions y ( = and y( π =. π ( J = y ycos( d In solving his example wih he VEM, y ( was direcly discreized wih uniormly-spaced poins as y ( i =,,...,, and he iniial guess was given by y ( =. hus a dynamic sysem wih saes was obained hrough Eqs. (7-(9 wih one-dimensional marix K =.. he calculus-o-variaions problem was ransormed o an IVP. he Ordinary Dierenial Equaion (ODE inegraor ode45 in Malab was employed o solve he IVP and he inie dierence mehod was used o compue he irs-order and second-order derivaives a he discreizaion poins. he analyic soluion obained hrough solving he BVP is yˆ = cos( + ( / π Fig. compares he soluions, and shows ha he inegraion resuls approach he opimal soluion quickly. A τ = 6s, i is hard o disinguish he numerical soluion rom he analyic. Fig. 3 plos he proile o uncional value agains he variaion ime. I monoonously declines and approaches he minimum rapidly. i Variable y τ = s τ = s τ = s he analyic soluion Numerical soluions wih VEM τ = 3s τ = 4s -. τ = 6s ime (s Fig.. he evolving o numerical soluion o y o he analyic soluion. -.5 Minimum o J Funcional value o numerical soluion Funcional J Variaion ime τ (s Fig.3 he approach o he minimum o uncional.

6 6 III. OPIMAL CONROL PROBLEMS WIH DYNAMIC CONSRAIN In his secion, we consider he opimal conrol problem wih dynamic equaion consrain. Problem :Consider perormance index o Bolza orm ( J ϕ( x(, L x(, u (, d (5 = + subjec o he dynamic equaion x = ( x,u, (6 where is ime. n x is sae vecor and is elemens belong o C [, ]. u m is conrol vecor and is elemens belong o C [, ]. he uncion L : and is irs-order and second-order parial derivaives are coninuous n m m wih respec o x, u and. he uncion ϕ : and is irs-order and second-order parial derivaives are coninuous n m n wih respec o x and. he vecor uncion : and is irs-order and second-order parial derivaives are coninuous and Lipschiz in x, u and. he iniial ime is ixed and he erminal ime is ree. he iniial boundary condiions are prescribed as x( = x (7 and he erminal saes are ree. Find he exremum ( xu ˆ, ˆ ha minimizes J, i.e. ( xu ˆ, ˆ = argmin( J (8 I is well known ha using he adjoining mehod [3], his problem may be reormulaed as an unconsrained augmened uncional o be ( J = ϕ( x(, + L+ λ ( x d (9 where λ is he cosae vecor. From his augmened perormance index, he irs-order opimaliy condiion may be obained as wih ransversaliy condiions x H = ( λ λ + H = ( x H = u ( λ ( ϕ ( (3 = x

7 7 H ( + ϕ = (4 where H = L+ λ is he Hamilonian. Alhough he opimaliy condiions o Problem may be deduced rom Eq. (9, he exremum canno be compued rom i. his is because exremums in Problem are uned o saddle poins in he augmened uncional [5], and applying he VEM direcly will no produce he righ soluion. Pracically, we may consruc an equivalen unconsrained uncional problem ha has he same exremum as Problem, ha is Problem 3:Consider he ollowing unconsrained uncional { } ( ϕ ( x ( x ( λ ( λ λ λ x x u u J = H ( + + H H + + H + H + H H d (5 n where x, λ and heir elemens belong o C [, ], u m and is elemens belong o C [, ]. he Hamilonian H and is irs-order and second-order parial derivaives, wih respec o x, λ, u and, are coninuous. he iniial ime is ixed and he erminal ime is ree. he boundary condiions are prescribed as x( = x (6 λ ( ϕ ( (7 = x Find he exremum ( xˆ, λˆ, u ˆ ha minimizes J, i.e. ( xˆ, λˆ, u ˆ = argmin( J (8 I is readily o ind ha he exremum o Problem is also he exremum o Problem 3. Especially, Problem 3 is a convex uncional opimizaion problem, and is soluion is he one ha saisies Eqs. (-(. Replacing he uncion and variables in Eq. ( respecively wih ( x ( x ( λ ( λ λ λ x x u u F = H H + + H + H + H H (9 x y = λ u (3 According o he VEM and wih exra consideraion on he ree erminal ime, we may deduce he variaion dynamic evolving equaions as δ y (, = Kr, (3

8 8 δ y( = K ( λ + H x (3 ru ( x ( H + ϕ ( H x + ϕx + H λ δ y = K ( H + ϕ H u (33 δ ( ( ϕ ( ϕ x x λ λ u u x x λ λ k = H H + H H + H H + H H wih iniial value x ( = x τ = and λ ( ϕ ( τ, where = = x (34 rx x r = = H + + H rλ yyv My x λ (35 r u H H H xx x xu = yy x u Hux u Huu is he Hessian marix, ( H x + λ v = ( x is he opimaliy vecor, he marix M is Hu M = H x u xx x Hxu, d x x = and d d λ λ =. d δ y δ y( δ = + y is he derivaive o variaion in he erminal variable wih respec o τ. K = diag( k, k,..., k n + m is he posiive diagonal marix and k is a posiive consan. heorem : Solving he IVP deined by Eqs. (3-(34 rom he VEM, when τ +, we have J and ( x, λ, u will saisy he opimaliy condiions o Problem. δ J Proo: Diereniaing Eq. (5 wih respec o τ and subsiuing Eqs. (3-(34 in, we have. he convex uncional J will decrease unil J =, which occurs when τ +. Because he exremum o he unconsrained uncional deined in Problem 3 saisies he opimaliy condiions o Problem, when J reaches he minimum, i.e., J =, his deermines he opimaliy condiions o Problem. Remark 3: I in Problem he erminal ime is ixed, hen he equivalen unconsrained uncional may be {( x ( x ( λ ( λ } = d λ λ x x u u J H H H H H H (36

9 9 Using he VEM, he variaion dynamic evolving equaions deduced is similar as he ree case excep ha Eq. (34 is no applicable and Eq. (33 is reormulaed as δ y ( x H = K ru λ (37 wih he iniial value λ ( ϕ (. = τ = x Remark 4: I in Problem he erminal boundary condiion o he saes is prescribed as x( = x, hen using he VEM he variaion dynamic evolving equaions deduced is similar as he ree erminal saes case excep Eq. (33 is reormulaed as δ y = K ( H + ϕ H ( H λ + λ + (38 x ( H + ϕ H u wih he iniial value x ( x. = τ = Remark 5: I in Problem he erminal ime is ixed and he erminal boundary condiion o he saes is prescribed as x( = x, hen using he uncional (36, he variaion dynamic evolving equaions deduced by he VEM is similar as he ree erminal saes case excep Eq. (37 is reormulaed as δ y = K ( λ + H x (39 ru wih he iniial value x ( x. = τ = Remark 6: Analogous o he Lyapunov uncion in he nonlinear dynamic sysem, he uncional (5 isel is a reasonable index o measure he disance o he exremum. Wih he VEM, i is globally monoonously decreases and his ensures ha ( x, λ, u will approach he exremum, i.e., ( xˆ, λˆ, u ˆ, over ime. An example rom [6] is now considered. Example : Consider he ollowing dynamic sysem

10 x = Ax+ bu where x x = x, A =, b =. Find he soluion ha minimizes he perormance index J = u d wih he boundary condiions x ( =, x ( = where he iniial ime = and he erminal ime = are ixed. As he principle, he equivalen unconsrained uncional was consruced as Eq. (36 where H = u + + u λ Ax λ b, Hλ = Ax + b u, H = x A λ and Hu = u+λ b. We also discreized he ime horizon [, ] uniormly, wih 4 poins. he iniial guess was given by.5 x ( =.5, λ ( = and u ( =. hrough Eqs. (3, (3 and (39, wih parameers k i K and k equaling, we obained a large IVP wih 5 saes. Since we did no specially scale he problem, we employed a si ODE inegraion mehod, ode5s in Malab, or he numerical inegraion. he analyic soluion by solving he BVP is 3 xˆ = xˆ = ˆ λ = 3 ˆ λ = uˆ = Figs. 4 and 5 show he evolving process o x and λ soluions o he opimal respecively, and a τ = 3s, he numerical soluions are indisinguishable rom he analyic. Fig. 6 plos he numerical soluions o x, λ, and u a τ = 3s. hey are almos idenical wih he analyic and his shows he eeciveness o he VEM. τ = 3s Variable x.8 τ = 5s.6.4 τ = s τ =.3s τ =.5s τ = s. he analyic soluion Numerical soluions wih VEM.5.5 ime (s Fig. 4 he evolving o numerical soluion o x o he analyic soluion.

11 3.5 τ = 5s τ = 3s Variable λ τ = s τ = s he analyic soluion Numerical soluions wih VEM ime (s τ = 5s τ = s Fig. 5 he evolving o numerical soluion o λ o he analyic soluion. Variables x, λ, u he analyic soluion o x he analyic soluion o λ he analyic soluion o u Numerical soluions a τ =3s ime (s Fig. 5 he numerical soluions o x, λ, and u wih VEM a τ = 3s. Now we consider a nonlinear example wih ree erminal ime, he Brachisochrone problem [7], which describe he moion curve o he ases descending. Example 3: Consider he ollowing dynamic sysem x = ( x, u where x x = y, V Vsin( u = V cos( u, g = is he graviy consan. Find he soluion ha minimizes he perormance index g cos( u J = wih he boundary condiions x y V = x =, y = his example has ixed erminal posiion boundary condiions and ree erminal velociy V( variaion dynamic equaions are synhesized accordingly beween Eqs. (33 and (38. An iniial guess o. hus, he boundary sae τ = = s was used and

12 he ime horizon was again discreized uniormly, wih poins. he iniial guess was given by x ( =, λ ( =, and u ( =. We also used ode5s o solve he large IVP. Here we compued he opimal soluion wih GPOPS-II [8], a Radau PS mehod based OCP solver. Fig. 7 gives he saes curve in x y posiion plane, showing ha he numerical resul approaches he opimal soluion over ime. In Fig. 8 he erminal ime proile agains he variaion ime τ is ploed. he resul o increases rapidly a irs and hen gradually decreases o he minimum decline ime, and i almos keeps unchanged aer τ = 7s. A τ = 4s, we compue ha =.866s, very close o he opimal resul o.865s. τ = s -.5 τ = s τ = 5s Variable y τ = s τ = 6s τ = 4s - he opimal soluion Numerical soluions wih VEM Variable x Fig. 7 he evolving o numerical soluion in x y posiion plane o he opimal soluion..8 Minimum decline ime Evolving proile o erminal ime (s (s.5 τ (s Variaion ime τ (s Fig.8 he evolving proile o o he minimum decline ime. IV. FURHER COMMENS o aciliae he propagaion o he discovery and acknowledge he auhors work, i no improper, he equaions (3-(34 may be called he ZS equaion. Acually, wihou considering he boundary condiions a and, he ZS equaion may be presened as a Parial Dierenial Equaion (PDE by replacing he variaion operaion δ wih he parial dierenial operaor o be

13 3 λ x H x + x = H τ λ K yy x λ (4 + H x u H u Consider in his way, he ormer examples are acually solved by he well-known semi-discree mehod in he ield o PDE numerical calculaion [9]. For Eq. (4, recall Fig., he iniial boundary condiions o x (, τ, λ (, τ and u (, τ a τ = may be arbirary and heir value a τ = + is he exremal soluion o he OCP. Since he boundary value a τ =+ is he ruly demanded, eicien mehods may be developed or compuing his PDE. For he sae- and conrol-consrained OCPs, he sraegy developed above is no generally applicable, because he analogous equivalen uncional is no available when complex pah consrains are involved. Furher sudies along ha hread may require employmen o echniques such as he Karush Kuhn ucker (KK variables or he slack variables []. However, or a paricular class o OCP, he ime-opimal conrol problems wih conrol consrain, an eecive alernaive may be developed and his will appear in a orhcoming paper. Acually, any discree ieraion mehod may have is coninuous-ime evolving counerpar heoreically. For example, rom he same principle, he coninuous-ime dynamic equaion may also be derived or he soluions o parameer opimizaion problems. Since he opimizaion procedure is driven by he engine o ODE inegraion mehods, his may moivae us o develop eicien and easy-o-use opimizaion ramework. V. CONCLUSION In his paper, a novel mehod, originaing rom he sable dynamics evoluion, is proposed or he opimal conrol soluion. By using he variaion moion, he Variaion Evolving Mehod (VEM ransorms he Opimal Conrol Problem (OCP o he Iniial-value Problem (IVP, hus avoiding he complexiy o he wo-poin Boundary Value Problem (BVP. Especially, because he reormulaed uncional, which measures he disance o he opimal soluion, decreases globally, he soluion is ensured o approach he opimal in an asympoically way. he VEM also synhesizes he direc and indirec mehods, bu rom a new sandpoin. Compared wih he discree ieraion mehods, dauning ask o searching reasonable sep size and annoying oscillaion phenomenon around he exremum, as occurs in he discree gradien mehod, are eliminaed and he maure Ordinary Dierenial Equaion (ODE inegraion mehods may be employed o solve he OCPs.

14 4 REFERENCES [] H. J. Pesch and M. Plail, he maximum principle o opimal conrol: A hisory o ingenious ideas and missed opporuniies, Conrol & Cyberneics, vol. 38, no. 4, pp , 9. [] J.. Bes, Survey o numerical mehods or rajecory opimizaion, J. Guid. Conrol Dynam., vol., no., pp. 93-6, 998. [3] C. Hargraves and W. Paris, Direc rajecory opimizaion using nonlinear programming and collocaion, J. Guid. Conrol Dynam., vol., no. 4, pp , 987. [4] O. V. Sryk and R. Bulirsch, Direc and indirec mehods or rajecory opimizaion, Ann. Oper. Res., vol. 37, no., pp , 99. [5] H. J. Peng, Q. Gao, Z. G. Wu, and W. X. Zhong, Symplecic approaches or solving wo-poin boundary-value problems, J. Guid. Conrol Dynam., vol. 35, no., pp ,. [6] A. V. Rao, A survey o numerical mehods or opimal conrol, in Proc. AAS/AIAA Asrodynam. Specialis Con., Pisburgh, PA, 9, AAS Paper [7] D. Garg, M. A. Paerson, W. W. Hager, A. V. Rao, e al, A Uniied ramework or he numerical soluion o opimal conrol problems using pseudospecral mehods, Auomaica, vol. 46, no., pp ,. [8] I. M. Ross and F. Fahroo, A perspecive on mehods or rajecory opimizaion, in Proc. AIAA/AAS Asrodynam. Con., Monerey, CA,, AIAA Paper No [9] I. M. Ross and F. Fahroo, Pseudospecral mehods or opimal moion planning o dierenially la sysems, IEEE rans. Auom. Conrol, vol. 49, no. 8, pp. 4-43, 4. [] H. K. Khalil, Nonlinear Sysems. New Jersey, USA: Prenice Hall,, pp. -8. [] J. P. Chiou and. Y. Na, On he soluion o roesch s nonlinear wo-poin boundary value problem using an iniial value mehod, J. Compu. Phys., vol. 9, no. 3, pp. 3-36, 975. [] R. Fazio and S. Iacono, On he ranslaion groups and non-ieraive ransormaion mehods, in Applied and Indusrial Mahemaics in Ialy III, E. De Bernardis, R. Spligher, and V. Valeni, Ed. Singapore: World Scieniic,, pp [3] R. F. Harl, S. P. Sehi, and R. G. Vickson, A survey o he maximum principles or opimal conrol problems wih sae consrain, SIAM Rev., vol. 37, no., pp. 8-8, 995. [4] D. G. Wu, Variaion Mehod. Beijing, China: Higher Educaion Press, 987, pp [5] A. E. Bryson and Y. C. Ho, Applied Opimal Conrol: Opimizaion, Esimaion, and Conrol. Washingon, DC, USA: Hemisphere, 975, pp [6] X. S. Xie, Opimal Conrol heory and Applicaion. Beijing, China: singhua Univ. Press, 986, pp [7] H. J. Sussmann and J. C. Willems, he Brachisochrone problem and modern conrol heory, in Conemporary rends in Nonlinear Geomeric Conrol heory and is Applicaions, A. Anzaldo-Meneses, B. Bonnard, J. P. Gauhier, and F. Monroy-Perez, Ed. Singapore: World Scieniic,, pp [8] M. A. Paerson and A. V. Rao, GPOPS-II: AMALAB soware or solving muliple-phase opimal conrol problems using hp-adapive Gaussian quadraure collocaion mehods and sparse nonlinear programming, ACM rans. Mah. Soware, vol. 4, no., pp. -37, 4. [9] H. X. Zhang and M. Y. Shen. Compuaional Fluid Dynamics Fundamenals and Applicaions o Finie Dierence Mehods. Beijing, China: Naional Deense Indusry Press, 3, pp [] D. G. Hull, Opimal Conrol heory or Applicaions, New York, USA: Springer, 3, pp