On-line Adapive Opimal Timing Conrol of Swiched Sysems X.C. Ding, Y. Wardi and M. Egersed Absrac In his paper we consider he problem of opimizing over he swiching imes for a muli-modal dynamic sysem when he complee cos-o-go is no available. The insananeous cos is assumed o be unavailable before run-ime, bu can be measured in real ime. Since he cos-o-go for opimizaion is ime-varying, his problem falls under he caegory of online opimizaion of swiched sysems. The goal of he paper is o presen an ieraive process o updae he swiching imes for he sysem in such a way ha hey remain opimal wih respec o he changing cos-o-go funcion. I is shown ha he informaion required o updae he swiching imes is he rae of change of he insananeous cos, which is in he form of he parial derivaive of he insananeous cos funcion. I. INTRODUCTION Consider a swiched-mode hybrid dynamical sysem henceforh denoed as a swiched sysem described by he equaion ẋ = f 1 x, [0, τ 1 ] f 2 x, τ 1, τ 2 ]... f N1 x, τ N, T], where τ i } N i=1 is a monoone-nondecreasing finie sequence of swiching imes wih τ 1 0, τ N T, for a given final ime T. The sae x R n and he iniial condiion x0 = x 0 R n is given. The funcions f i : R n R n represen he various modes of he sysem and hence he sequence f i } is called he modal sequence. I is assumed ha hese funcions are coninuously differeniable so ha here exiss a unique, well-defined soluion o Equaion 1 wih he given iniial condiion x 0. A paricular mode may be repeaed in he modal sequence, and in his case, f i = f j for some i j. The sandard swich-ime opimizaion problem can be defined as follows. Le Lx, : R n [0, T] R be an insananeouscos cos defined over he sae rajecory, and le J be a corresponding performance funcional, defined via J = 0 1 Lx, d. 2 We view J as a funcion of he swiching imes, denoed collecively by τ := τ 1,..., τ N T superscrip T indicaing ranspose, and hence we label i as J τ. The problem of minimizing J τ over all swiching-ime sequences in some given feasible se has been exensively invesigaed in he pas few years. Following is inroducion in a general X.C. Ding, Y. Wardi and M. Egersed are wih he School of Elecrical and Compuer Engineering, Georgia Insiue of Technology, Alana, GA 30332, USA. Email: ding,ywardi,magnus}@ece.gaech.edu nonlinear hybrid-sysems seing in [4], opimaliy condiions, based on he maximum principle, were derived in [9], [10], [12]. For he general case of nonlinear sysems, several algorihms have been proposed and analyzed in [2], [6], [8], [10], [11], [13], [14], among ohers. A paricular kind of problems arises when he cos funcion Lx, is no known a-priori, bu raher he sysem learns abou i incremenally as ime advances. This is for insance he case when he sysem is racking a arge wih unknown dynamics. In his case, i is impracical o aemp o opimize he cos funcional J as defined in 2 and insead we consider he relaed problem of opimizing a suiable cos-ogo performance funcional. This problem requires an on-line approach ha corresponds o he evoluion of he cos o go. We propose an on-line algorihm ha adapively adjuss an esimaed value of he opimal swiching-ime vecor as new informaion abou he cos funcion becomes available. In oher words, i aemps o rack he soluion poin of he opimal cos-o-go problem as i learns abou changes in he cos funcion. This approach may require an off-line soluion of he problem a he iniial ime = 0, which is assumed o be compuable. A similar approach has been developed for a differen kind of real-ime problems, corresponding o siuaions where he cos funcion is known a priori, bu he sae variable x canno be measured and has o be esimaed via an observer [3], [5]. However, despie he similariy of he approach, he problem considered here is more general, and his is refleced in he algorihms and heir analysis, which are differen from hose in [3], [5]. We poin ou ha he variable parameer of he algorihm consiss of he swiching imes beween he modes, bu he sequence of he modes is assumed o be fixed. The moredifficul problem of opimizing J wih respec o he modal sequence has been addressed in [2] in an off-line seing, and is exension o he on-line seing will be invesigaed a a fuure ime based on he resuls derived in his paper. The res of he paper is organized as follows. Secion II carries ou an analysis for he case of a single modeswiching, and Secion III discusses an exension of he analysis o a swiched sysem wih muliple swichings. The analysis of he wo cases are idenical bu he muliswiching case involves more complicaed noaion. Therefore we presen a deailed analysis only for he case of a single swiching. Secion IV presens a numerical example of a double-ank sysem whose inpu flow rae o he firs ank is conrolled by a valve wih discree modes. The problem a hand is o conrol he swiching ime of he sysem so ha he average fluid level for he second ank racks a reference
value which is coninuously varying in real-ime. Finally, Secion V concludes he paper and discuss direcions for fuure research. II. THE SINGLE-SWITCH CASE Consider he sysem defined via Equaion 1 for he case where N = 1, namely here is a single swiching ime. We can simplify he noaion by deleing he subscrip from τ 1, and denoe he swiching ime by τ. In his case, according o 1, ẋ = f 1 x for x τ, and ẋ = f 2 x for > τ. Since we consider dynamic compuaions of he swiching ime, we will denoe i by τ o emphasize is dependence on. Furhermore, we sar by defining τ o be a minimum poin of he cos-o-go defined laer, and since his may vary coninuously wih, we firs describe τ} as a coninuous process; our algorihm will be based on i and of course will perform is compuaion a a discree se of imes. In ligh of his, a simple ranslaion of Equaion 1 wih N = 1 ha emphasizes he dependence of he swiching ime on may no be well defined, and we have o carefully define he sae rajecory x and he swiching ime τ simulaneously. To his end, i is convenien o firs define he sae rajecory of he dynamics associaed wih he cos o go. Given x R n, τ [0, T], and [0, T], define xs,, x, τ, s, o be he sae rajecory forward from, as a funcion of s, wih he iniial condiion of x a he ime s = ; ha is, xs,, x, τ is defined via he equaion x s = f1 x, if s τ f 2 x, if s > τ, wih he iniial condiion x,, x, τ = x. x should be hough of as he prediced or simulaed sae rajecory saring a ime s = and evolves unil s = T. Given [0, T], le Lx, s, be a projeced cos funcion of x R n and fuure imes s. I has he following meaning: A ime he sysem has learned abou he cos funcion of he sae variable x and he fuure imes s, and his funcion is Lx, s,. For example, if i is desired o rack a moving arge whose posiion and velociy are known a ime, Lx, s, may indicae he square-disance of he sae x from he goal s projeced posiion a fuure imes s based on is posiion and velociy a ime. Generally, he funcion Lx, s, forms he basis for he cos o go, and we assume ha i is coninuous, coninuously differeniable in x for every [0, T] and s [, T], and piecewise coninuously differeniable in and s for every x R n. Lasly, we assume ha x, s, o be differeniable in he reason for his assumpion o be needed should be clear laer in his secion. For every τ [0, T], [0, T], and x R n, we define he cos-o-go performance funcional, J, x, τ, by J, x, τ = L xs,, x, τ, s, ds. 3 We define τ as a local minimum poin of he cos o go from ime and he iniial sae x a ha ime, where x is he rue rajecory of he sysem. As menioned earlier, we have o formally define he processes τ} and x} simulaneously, and his is done in he following way: x saisfies he differenial equaion dx d = f1 x, if τ f 2 x, if > τ wih a given boundary condiion x0 = x 0 ; and as for τ, as long as < τ hen τ is defined as a funcion ha saisfies, x, τ = 0, 4 wih a given boundary condiion τ0 = τ 0 [0, T] such ha 0, x0, τ0 = 0. If τ, we define τ = 0. We poin ou ha Equaion 4 is based on he sandard necessary opimaliy condiion in nonlinear programming. The condiion τ = 0 when τ reflecs he fac ha once he swiching ime has occurred i become a par of he pas and canno be modified, in oher words, once τ, τξ has a consan value ξ. Throughou his paper, we assume ha he processes x} and τ} are well defined in he above manner and τ is coninuous in. Proving such a hypohesis from verifyable assumpions is beyond he scope of he general seing of his paper, bu would be done for specific classes of problems in forhcoming publicaions. Consider a ime-poin such ha < τ. Since, x, τ = 0, aking he oal derivaive wih respec o, i follows ha d, x, τ = 0 d as well. This oal derivaive has he following form, 2 J 2 J ẋ, x, τ, x, τ 2 J, x, τ τ = 0, 2 and hence, 2 J 2 J 1 τ =, x, τ 2 ẋ, x, τ. 5, x, τ 2 J By he assumpion ha he process τ} is well defined and coninuous, he second derivaive 2 J, x, τ is 2 posiive. We nex simplify he Righ-hand-side RHS of Equaion 5. Define he funcion φ ξ, ξ, by φ ξ = ξ, xξ, τ. Then dφ dξ ξ = 2 J 2 J ẋξ, ξ, xξ, τ ξ, xξ, τ 6 and a ξ =, dφ dξ = 2 J, x, τ 2 J, x, τ ẋ. 7
Plug his in 5 o obain, 2 J 1 dφ τ =, x, τ 2. 8 dξ Le us nex derive an expression for dφ dξ. Given [0, T] and x R n, denoe by Φ,x s, s 0 he sae ransiion marix associaed wih he marix-valued T funcion of s f2 xs,, x, τ. Consequenly s Φ,xs, s 0 = f 2 TΦ,x xs,, x, τ s, s 0, 9 wih Φ,x s 0, s 0 = I he ideniy marix in R n. Given [0, T and > 0 such ha T. Lemma 1: For all s and for all s 0, Φ, x,,x,τ s, s 0 = Φ,x s, s 0. 10 Proof: Fix [0, T and > 0 such ha T. s [, T] and s 0 [, T], equaion 9 is in force. Moreover, he same equaion holds wih and x,, x, τ in lieu of and x, respecively, meaning ha d ds Φ, x,,x,τs, s 0 = f2 xs, T, x,, x, τ, τ Φ, x,,x,τ s, s 0. 11 Now s, x s,, x, τ = s f1 x, if s τ f 2 x, if s > τ wih he boundary condiion x,, x, τ = x, and x f1 x, if s τ s,, x,, x, τ, τ = s f 2 x, if s τ wih he boundary condiion x,, x,, x, τ, τ = x,, x, τ. We see ha xs,, x, τ and xs,, x,, x, τ, τ saisfy he same differenial equaion in s s, and hese wo funcions have he same value a s = ; consequenly xs,, x, τ = xs,, x,, x, τ, τ 12 for all s [, T]. Therefore, by 9 and 11, Φ, x,,x,τ s, s 0 and Φ,x s, s 0 saisfy he same differenial equaion in s, wih he same value a s = s 0 i.e., I, and his implies 10. We now can characerize he righmos erm in he RHS of 8, and hence obain a compuable expression for τ. Proposiion 1: Suppose ha < τ. Then he derivaive erm dφ dξ τ Proof: Fix [0, T and x R n, and consider Jx,, τ as a funcion of τ >. Define he cosae ps R n, s [τ, T], via he following differenial equaion, ṗs = f2 xs, T, x, τ ps xs, T, x, τ, s, wih he boundary condiion pt = 0. By Equaion 9, ps = ξ a ξ = has he following form, Now suppose ha < τ. Then ξ [, ], dx dξ = f 1 xξ and x ξ ξ,, x, τ = f1 xξ,, x, τ dφ dξ = wih x,, x, τ = x, and hence, ξ [, ], 2 L xs,, x, τ, s, Φ,x τ, s T xξ,, x, τ = xξ. ds f 1 xτ,, x, τ f2 xτ, In paricular, for ξ =,, x, τ. 13 x,, x, τ = x. 20 s Φ,x s, ξ Tdξ, xξ,, x, τ, ξ, 14 and a s = τ, and replacing ξ by s in 14, we obain, pτ = Nex, by Reference [6], τ Φ,x τ, s Tds. xs,, x, τ, s, 15, x, τ = pτt f 1 xτ,, x, τ f2 xτ,, x, τ, 16 and hence, and by 15,, x, τ = xs, Tds, x, τ, s, Φ,x τ, s f 1 xτ,, x, τ f2 xτ,, x, τ. 17 τ Use 17 wih wo specific values:, x, τ, and, x, τ, o obain he following wo equaions, τ and, x, τ = Tds xs,, x, τ, s, Φ,x τ, s f 1 xτ,, x, τ f2 xτ,, x, τ, 18 τ, x, τ = xs,, x, τ, s, Tds Φ,x τ, s f 1 xτ,, x, τ f 2 xτ,, x, τ. 19
By Lemma 1 and Equaion 20, s and s 0, Φ,x s, s 0 = Φ,x s, s 0. Therefore, and by 19, τ, x, τ = xx,, x, τ, s, Tds Φ,x τ, s f 1 xτ,, x, τ f 2 xτ,, x, τ. 21 Moreover, s [, τ], s xs,, x, τ = f 1 xs,, x, τ and s xs,, x, τ = f 1 xs,, x, τ ; while he iniial condiions of he respecive erms in hese wo differeniable equaions are x,, x, τ = x by definiion, and x,, x, τ = x by Equaion 20. Consequenly, s [, τ], xs,, x, τ = xs,, x, τ. Using his in 21 we obain,, x, τ = xs,, x, τ, s, τ Tds Φ,x τ, s f 1 xτ,, x, τ f2 xτ,, x, τ. 22 By 22 and 18,, x, τ, x, τ = [ xs,, x, τ, s, τ xs, T ], x, τs, Φ,x τ, s ds f 1 xτ,, x, τ f2 xτ,, x, τ. 23 Dividing boh erms of Equaion 23 by, aking he limi 0, and recalling he definiion of φ ξ, Equaion 13 follows Finally, we obain an expression for τ. Proposiion 2: For every [0, T such ha < τ, we have ha 2 J 1 τ =, x, τ 2 T 2 L xs, Tds, x, τ, s, Φ,x τ, s τ f 1 xτ,, x, τ f2 xτ,, x, τ. 24 Proof: Follows direcly from Equaion 8 and Proposiion 1. We remark ha Equaion 24 is a differenial equaion in τ, and he algorihm ha we use amouns o is numerical soluion by he forward Euler mehod; his will be illusraed in Secion IV for an example of a nonlinear sysem. III. THE CASE OF MULTIPLE SWITCHING TIMES Consider he case where here are N swiching imes, τ i, i = 1,...,T, denoed collecively by he vecor τ := τ 1,...,τ N T R N. We assume ha 0 τ 1... τ N T, and we define τ 0 := 0 and τ N1 := T. For a fixed vecor τ, he sae equaion is given via 1, bu in he even ha τ is a funcion of ime and hence denoed by τ, we define i ogeher wih he sae rajecory as i was done for he single-swiching case. Given a swiching vecor τ, x R n, [0, T], and s [, T], define he projeced sae rajecory xs,, x, τ by he following differenial equaion, x s = f i x s [τ i 1, τ i ] [, T], i = 1,...,N 1, wih he iniial condiion x,, x, τ = x. Nex, given [0, T] and x R n, le Lx, s, be a cos funcion defined for s, and define he cos-o-go performance funcional, J, x, τ, by J, x, τ = L xs,, x, τ, s, ds. 25 Finally, Define x and τ = τ 1,..., τ N T simulaneously in he following way: x saisfies he differenial equaion dx d = f i x, [τ i 1, τ i, i = 1, 2,.., N 1, 26 wih a given boundary condiion x0 = x 0 ; and as for τ, if < τ i hen i, x, τ = 0 27 wih a given iniial condiion τ i 0 = τ 0, and if τ i hen τ i = 0. Furhermore, we assume ha x and τ are coninuous funcions of. This definiion reflecs he fac ha swiching imes ha occurred prior o ime remain fixed and canno be changed in he fuure. As a maer of fac, he number of fuure swiching imes may decline as ime advances, and herefore he dimension of he opimizaion problem, namely he number of fuure swiching imes, may go down. In he forhcoming discussion we assume ha < τ 1 and we consider variaions in N fuure swiching imes. The main resul of his secion concerns an expression for τ ha is similar o Equaion 24 for he single-swiching case. Is proof is almos idenical o ha of Proposiion 2, and hence i is no presened here. Given [0, T] such ha < τ 1, and given x R n and i = 1,...,N, denoe by Φ i,,x s, s 0 he sae
ransiion marix associaed wih he marix-valued funcion T of s fi xs,, x, τ. Furhermore, define he vecor p i R n in a backwards-recursive manner as follows: For i = N 1,...,1, q i T = Φ i,,x τ i, τ i1 q i1 T τi1 τ i Φ i,,x τ i1, s 2 L xs, T, x, τ, s, ds, 28 wih he boundary condiion x 1 x 2 τ N q N = Φ i,,x τ N1, s 2 L xs, T, x, τ, s, ds. 29 Nex, define p i R by p i = q i1 T T Φ i,,x τ i, τ i1 τi1 Furhermore, le p R N be he vecor whose ih coordinae is p i. We now presen he main resul of his secion. Proposiion 3: If < τ 1, hen τ 2 J =, x, τ 1 p. 31 2 I should be noed ha again he compuaion of he vecor p only requires inegraion forward once. IV. NUMERICAL EXAMPLE Consider he double-ank sysem shown in Figure 1, where i is desired o conrol he fluid level in he second lower ank, so as o have i rack a given reference signal, r, by he inpu valve o he firs upper ank. The reference signal is no known in advance, bu is pas values a any ime are known a ha ime. The valve can be eiher open or closed, corresponding o he respecive modes of inpu flow o he firs ank a he rae of eiher a given u > 0, or 0. We denoe he fluid level of he firs and second anks by x 1 x 2, respecively, and we consider x := x 1, x 2 T o be he sae of he sysem. By Torricelli s Principle, he dynamics of his swiched sysem can be described by he following equaions: or Fig. 1. [ ẋ = f 1 x := [ ẋ = f 2 x := The double ank process α 1 x1 α 1 x1 α 2 x2 α 1 x1 u α 1 x1 α 2 x2 ], 32 ], 33 2 L where f 1 corresponds o he mode mode 1 when he valve xs,, x, τ, s, τ i is closed, and f 2 corresponds o he mode mode 2 when Φ i,,x τ i1, s he valve is open. In his example we assume ha iniially T ds he valve is open Mode 2, and i can undergo a single f i xτi,, x, τ f i1 xτi,, x, τ mode-ransiion by becoming closed. Following he noaion of Secion II, τ denoes he opimal cos-o-go a ime,. 30 which is he opimal fuure swiching ime. Since he objecive is o rack he reference signal r by he fluid level in he second ank, we define he cos funcion L, xs,, x, τ, s by L, xs,, x, τ, s = 1 2 x 2s,, x, τ r 2, and correspondingly, he coso-go funcional is J, x, τ = 1 x 2 s,, x, τ r 2 ds. 2 34 I can be seen ha 2 L = [0, ṙ], and herefore, Equaion 24 has he form 2 J 1 τ =, x, τ [0, ṙ] 2 T Tds Φ,x τ, s f 1 xτ,, x, τ τ f 2 xτ,, x, τ. 35 We poin ou ha he RHS of 35 requires he compuaion or approximaion of he sae ransiion marix Φ,x τ, s for every s [τ, T], and his may require numerical inegraion. In his example we se α 1 = α 2 = 1, u = 2, and T is se o 10. Furhermore, o ensure numerical accuracy, we use a relaively small sep-size = 0.005 o updae he
dj/dτ 0.5 0.4 0.3 0.2 0.1 0 0.1 0.2 0.3 0.4 Gradien as funcion of ime 0.5 0 1 2 3 4 5 6 7 8 9 10 Fig. 2. Plo of he gradien dj, x, τ as a funcion of ime. The dτ verical doed line corresponds o he ime when mode swich occured. Afer he mode swich, he gradien is se o 0. τ 7 6 5 4 3 2 1 Swiching ime evoluion 0 0 1 2 3 4 5 6 Fig. 3. Plo of he swiching ime evoluion τ as funcion of ime. The diagonal line is he line τ = and mode swich akes place if τ his his line. The swiching ime does no change afer his poin since he mode swich has already occured. swiching ime and he sysem. The reference signal is r = 1 9 32 1.7. Using he mehod oulined in his paper, he resulan dj gradien dτ, x, τ for he cos-o-go 34 is shown in he Figure 2. The gradien is close o 0 for all ime which is he desired resul ha corroboraes he heoreical developmens. The small deviaions in he begining are due o numerical approximaions. Saring a opimal iniial condiion τ0 = 1.86 corresponds o r0 = 0.7, he evoluion of he opimal swiching ime using 35 is shown in Figure 3. The diagonal line represens τ = and mode swich akes place when τ his his line. A his poin, he sysem swichs from mode 2 o mode 1 and he sysem is no longer conrolled. In his paper we address he problem of adaping he opimal swiching imes of a swiched sysem in an online environmen where he insananeous cos funcion is assumed o be coninuously varying. We proposed an approach o obain he rajecory of he opimal swiching imes wih respec o he on-line cos-o-go funcion. We found ha o mainain his opimal swiching ime evoluion, on-line informaion in he form of rae of change of he insananeous cos is required. Fuure work includes exension of his framework o he problem when he insananeous cos funcion undergoes impulsive changes insead of coninuous updaes. REFERENCES [1] H. Axelsson. Opimal Conrol of Swiched Auonomous Sysems: Theory, Algorihms, and Roboic Applicaions. Ph.D Thesis, School of Elecrical and Compuer Engineering, Georgia Insiue of Technology, May 2005. [2] H. Axelsson, Y. Wardi, M. Egersed, and E. Verries. A Gradien Descen Approach o Opimal Mode Scheduling in Hybrid Dynamical Sysems. Journal of Opimizaion Theory and Applicaions, Vol. 136, No. 2, pp. 167-186, February 2008. [3] S. Azuma, M. Egersed, and Y. Wardi. Oupu-Based Opimal Timing Conrol of Swiched Sysems. Proc. Hybrid Sysems: Compuaion and Conrol, Springer-Verlag, pp. 64-78, Sana Barbara, CA, March 2006. [4] M.S. Branicky, V.S. Borkar, and S.K. Mier. A Unified Framework for Hybrid Conrol: Model and Opimal Conrol Theory. IEEE Trans. on Auomaic Conrol, Vol. 43, pp. 31-45, 1998. [5] X.C. Ding, Y. Wardi, and M. Egersed. On-Line Opimizaion of Swiched-Mode Dynamical Sysems. IEEE Transacions on Auomaic Conrol, o appear. [6] M. Egersed, S. Azuma, and Y. Wardi. Opimal Timing Conrol of Swiched Linear Sysems Based on Parial Informaion. Nonlinear Analysis: Theory, Mehods & Applicaions, 2006. [7] M. Egersed, Y. Wardi, and H. Axelsson. Transiion-Time Opimizaion for Swiched-Mode Dynamical Sysems. IEEE Trans. on Auomaic Conrol, Vol. AC-51, pp. 110-115, 2006. [8] S. Hedlund and A. Ranzer. Opimal Conrol of Hybrid Sysems. IEEE Conf. on Decision and Conrol, pp. 3972-3977, 1999. [9] B. Piccoli. Hybrid sysems and opimal conrol. Proc. 37h IEEE Conference on Decision and Conrol, Tampa, Florida, 1998. [10] M.S. Shaikh and P. Caines. On Trajecory Opimizaion for Hybrid Sysems: Theory and Algorihms for Fixed Schedules. IEEE Conf. on Decision and Conrol, pp. 1997 1998, 2002. [11] M.S. Shaikh and P. Caines. On he Opimal Conrol of Hybrid Sysems: Opimizaion of Trajecories, Swiching Times and Locaion Schedules. In 6h Inernaional Workshop on Hybrid Sysems: Compuaion and Conrol, 2003. [12] H.J. Sussmann. Se-Valued Differenials and he Hybrid Maximum Principle. IEEE Conf. on Decision and Conrol, pp. 558 563, 2000. [13] X. Xu and P. Ansaklis. Opimal Conrol of Swiched Auonomous Sysems. IEEE Conf. on Decision and Conrol, pp. 4401 4406, 2002. [14] X. Xu and P.J. Ansaklis. Opimal Conrol of Swiched Sysems via Nonlinear Opimizaion Based on Direc Differeniaions of Value Funcions. In. J. of Conrol, Vol. 75, pp. 1406-1426, 2002. V. CONCLUSION