Sliding Mode Extremum Seeking Control for Linear Quadratic Dynamic Game

Transcription

1 Sliding Mode Exremum Seeking Conrol for Linear Quadraic Dynamic Game Yaodong Pan and Ümi Özgüner ITS Research Group, AIST Tsukuba Eas Namiki --, Tsukuba-shi,Ibaraki-ken 5-856, Japan Deparmen of Elecrical Engineering, The Ohio Sae Universiy 5 Neil Avenue, Columbus, OH umi@ee.eng.ohio-sae.edu Absrac The exremum seeking conrol has been proposed o find a se poin and/or rack a ime-varying se poin so ha a performance index of he sysem reaches is exremum poin. In his paper, he exremum seeking conrol wih sliding mode is exended o solve he Nash equilibrium soluion for an n-person linear quadraic dynamic game. For each player, a sliding mode exremum seeking conroller is designed o le he player s linear quadraic performance index rack a decreasing signal so ha he Nash equilibrium poin is reached. Keyword: Noncooperaive Dynamic Game, Linear Quadraic Performance Index, Nash Equilibrium Soluion, Exremum Seeking, Sliding Model. I. INTRODUCTION Exremum seeking conrol approaches have been sudied since 5 s and have been successfully implemened in he conrol of a axial-flow compressor, he opimizaion of he operaion of biological reacors, and he opimizaion of spark igniion auomoive engines. Wih he exremum seeking conrol, a sepoin and/or a ime-variable sepoin is racked o minimize or maximize a performance index of he sysem As one of he exremum seeking conrol approaches, he sliding mode exremum seeking conroller shown in Figure ensures (9) ha he sysem converges o a predesigned sliding mode in a finie ime, eners a viciniy of he exremum poin on he sliding mode, and says here wih oscillaion. Alhough i is a radeoff problem o deermine he conrol accuracy and convergence speed by choosing conroller parameers, i is possible o obain boh high conrol accuracy and fas convergence speed afer Fig.. Exremum Seeking Conrol Using Sliding Mode inroducing ime-variable parameers. Recenly, i is found ha he sliding mode exremum seeking conrol can also be implemened o solve he Nash Soluion. For an n-person noncooperaive dynamic game, each player defines a performance index and adjuss some of he conrol parameers o minimize his own performance index o find a Nash equilibrium soluion. In, i is assumed ha he performance index for each player is a funcion on he sae, he inpu and he oupu variables of he sysem, which may be in an unknown form bu is measurable or can be calculaed from he measurable variables. In his paper, we will consider he n-person noncooperaive linear quadraic dynamic game, where he conrol inpus o be adjused by each player are he feedback conrol gains K i (i =,,..., n) and he performance index (i =,,..., n) is he inegraion of a linear quadraic funcion on he sysem sae and inpu. In his case, he calculaion of he performance indices needs he fuure informaion of he sysem sae and inpu. The performance

2 indices can no be calculaed on-line and hus can no be direcly used in he exremum seeking conroller. As he firs sep o design an exremum seeking conroller for he n-person noncooperaive linear quadraic dynamic game, he linear quadraic index is ransferred o an equivalen performance index, which can be calculaed on-line based on he sysem parameers and he feedback gain. The arrangemen of he paper is as follows. Secion describes he problem formulaion and he ransformaion of he linear quadraic index; Secion proposes he sliding mode exremum seeking approach o find he Nash equilibrium soluion for a linear quadraic dynamic game; and Secion gives simulaion resuls. II. PROBLEM FORMULATION Consider an n-person noncooperaive linear quadraic dynamic game described by a linear dynamic sysem d n d x = Ax + B i u i () i= wih a linear quadraic performance index for i-h player = (x T Q i x + u T R i u)d, (i N) () where A and B i (i N) are known consan marices of appropriae dimensions, (A, B i ) (i N) are conrollable pairs, N is he index se of player defined as N = {,,, n}, x R m is he sysem sae variable, u i R (i N) is he conrol variable for each player, is he sysem iniial ime, Q i R m m and R i R (i N) are semi-posiive and posiive definie symmeric marices, respecively, and (Q i, A) (i N) are observable pairs. To simply he noaion and he discussion, he -player case is considered from now on. Exension o he n-player case is sraighforward. The opimal conrol of he above sysem is a game problem. The Nash Soluion wih he Nash feedback sraegy is given by u i = R i B T i P i x, (i =, ) which ensures for any sysem iniial sae x( ) (x( ) R m ) ha J u =u,u =u J u U,u =u J u =u,u =u u =u,u U where U and U are ses of possible conrol inpu for player and, respecively, P and P are he posiive definie soluions of he following coupled algebraic Riccai equaions: P A + A T P P B R BT P P B R BT P P B R BT P = Q P A + A T P P B R BT P P B R BT P P B R BT P = Q. Insead of solving he above Riccai equaions, in his paper he opimal feedback gains K i (K i = R BT P, i =, ) as he Nash soluion o minimize he performance index (i =, ) are calculaed by he sliding mode exremum seeking conrol approach. as Denoe he linear feedback conrol inpu for each player u i = K i x, (i =, ). () Then he closed-loop sysem of () wih he conrol inpu () is deermined by d x = Āx, () d and he linear quadraic performance index given in () can be rewrien as where = x T Q i xd, (i =, ), (5) Ā = A B K B K Q i = ĀT (Q i + Ki T R i K i )Ā. (i =, ) As (A, B i ) and (Q i, A) (i =, ) are conrollable and observable pairs, respecively, here exis wo parameer ses K and K so ha for any feedback gain K i (K i K i, i =, ), he Lyapunov funcions M i Ā + ĀT M i = Q i (i =, ) (6) have posiive definie symmeric soluions M i (i =, ). I is clear ha M i (i =, ) is a funcion on K and K and hus may be denoed as M i = M i (K, K ). (i =, ) (7) Then he linear quadraic performance index () can be rewrien as = x T M i x = x T ( )M i x( ) = r(x( )x T ( )M i ) K i K i, (i =, ) where r(.) is he race operaion.

3 According o he linear quadraic opimal conrol heory, he opimal soluion of he above performance index is independen o he iniial ime and he sysem iniial sae x( ), i.e. he opimal feedback gain Ki (i =, ) is unique no maer he sysem iniial sae x( ) is. Denoe he column vecor of he ideniy marix I m as e j (j =,,..., m), i.e. I m = diag{,,..., } = e e e m. III. EXTREMUM SEEKING WITH SLIDING MODE To design an exremum seeking conroller wih sliding mode for he i-h player (i =, ), a swiching funcion is defined as s i = g i () where he reference signal g i R is deermined by ġ i = ρ i, () Then he opimal feedback gain Ki (i =, ) can be where ρ i (i =, ) are posiive consans. described as Le he variable srucure conrol law be {K, K } = Arg min )x T ( )M i ) sgn(sin(πs i /α i )) K i K i,i=, = Arg min sgn(sin(πs i /α i )) je T j M i ), (j =,,..., m) v i = k i, (i =, ) K i K i,i=,... = Arg min m sgn(sin( m πs i /α i )) r(e j e T j M i ) () K i K i,i=, j= = Arg min r( m e j e T j M i ) K i K i,i=, j= = Arg min r(i mm i ) K i K i,i=, where α i and k i (i =, ) are posiive consans. = Arg min r(m i). (8) Assumpion : The parial derivaive of he performance K i K i,i=, index (i =, ) saisfies Thus he conrol objecive o solve he Nash equilibrium soluion is urned o minimize he performance index (K, K ) >> (K, K ), j i(i, j =, ) K i K j (K, K ) = r(m i ) = r(m i (K, K )) (i =, ) (9) by adjusing he feedback gain K i by each player (i =, ) independenly. I follows from he linear quadraic opimal conrol heory ha a unique Nash equilibrium soluion (K, K ) exiss such ha J (K, K ) (K, K ), K K J (K, K ) (K, K ), K K. During searching he Nash soluion, he feedback gain K i (i =, ) is adjused on-line by he exremum seeking conroller. Therefore K i (i =, ) is a funcion on ime. The performance index given in (9) hus can be described as a funcion on ime, oo, i.e. = (K, K ) = r(m i (K, K )) (i =, ) () To simplify he noaion, he same symbol is used o denoe he performance indices in (), (9), and (). and he feedback gain K i (i =, ) saisfy K i = v i, (i =, ) () which means ha each player can adjus his performance index mos effecively Assumpion : The Nash equilibrium poin (K, K ) is in he viciniy of he iniial -uple of K i () (i =, ). Thus he parial derivaive of he performance index on K i is bounded by a posiive consan γ i, i.e., (K, K ) γ i. (i =, ) (5) K i Theorem : Consider he dynamic noncooperaive game described by he sae equaion in () wih he linear quadraic performance index (), he exremum seeking conroller wih sliding mode for he i-h player (i =, ) designed by Equaions (), (), (), and () ensures ha he performance index (i =, ) are minimized o ge he Nash equilibrium soluion J (K, K ) and J (K, K ) if posiive consans ρ i, k i, and α i (i =, ) as he conroller parameers are chosen suiable. Proof: The compleed proof of his heorem needs much space. In his paper, only he seps of he proof are described as follows, which are similar o he resuls in and.

4 Based on he above assumpions, he derivaive of he swiching funcion s i is given by d d s i = j= From which, i can be shown ha K j (K, K ) K j ġ i K i (K, K ) K i ġ i. (6) For each player here exiss a viciniy of he minimum poin, which is deermined by ρi k i his viciniy, a sliding mode (i =, ). Ouside, s i, g i Performance Index, Swiching Funcion s i and Reference Signal g i (Player i) s i g i s i = lα i or s i = lα i will happen for some number l deermined by he iniial condiion of he sysem. On he sliding mode, he sysem converges o he viciniy. Afer enering he viciniy, i is possible ha eiher he sysem says inside he viciniy or go hrough he viciniy. In he laer case, anoher sliding mode will happen and he sysem will ener he viciniy again on he sliding mode. In boh cases, i.e. he case saying inside he viciniy and he case moving ou of he viciniy, he performance index J oscillaes and decreases in each oscillaion period as shown in Figures, and. 5 6 Fig.. Swiching Funcion wih Sliding Mode Exremum Seeking Conroller Performance Index (Player i) Convergence wih Sliding Mode Exremum Seeking Conrol (Player i) Ki Ki Fig.. Performance Index wih Sliding Mode Exremum Seeking Conroller, Ki, Ki 5 has been shown in ha fas convergen speed and high conrol accuracy may be obained if he conroller parameers ρ i, k i, and α i are chosen suiably and adjused online. IV. EXAMPLES 5 6 Fig.. Convergence wih Sliding Mode Exremum Seeking Conroller Finally he sysem oscillaes around he Nash equilibrium poin. In his way, he Nash soluion can be found by he proposed exremum seeking conroller wih sliding mode. And i Consider a wo-person noncooperaive linear quadraic dynamic game described by a second-order linear sysem..7. ẋ = x + u + u...9 The parameer marices of he performance index () are respecively given by Q =, R =.

5 Q =, R =. The proposed exremum seeking conrol algorihm is implemened o he above sysem wih sampling inerval as. T =. second and oher conroller parameers as k i =.5, ρ i =.5. (i =, ) The simulaion resuls wih α = α =.5 are given in Figures 5 and 6, which shows ha he sysem reaches a sliding mode in a finie ime, converges o he viciniy of he Nash equilibrium poin and hen oscillaes while he performance index keeps decreasing in each oscillaion period unil he Nash equilibrium poin is reached. s, s Swiching Funcions s s Fig. 5. Conrol Swiching Funcions wih Sliding Mode Exremum Seeking The ampliude of he oscillaion can be reduced by increasing he posiive consan ρ i (i =, ) as shown in Figure 7 wih ρ = ρ =. bu in his case, he convergen speed is slow. In his paper, a kind of on-line adjusing mehod is proposed. The parameer ρ is deermined by a ime-variable funcion as { ρ i =. >. (i =, ) The simulaion resuls in Figure 8 wih he above imevariable parameer ρ i (i =, ) show ha he Nash soluion is obained quickly wih higher conrol accuracy. To confirm he resuls obained by he proposed sliding mode exremum seeking conrol approach, he opimal feedback gains for he Nash soluion are obained as K =.6.65 K = (7) (8) by he ɛ-coupling approach, which are he same o hose resuls shown in Figures 6, 7, and 8. V. CONCLUSION The sliding mode exremum seeking conrol approach is successfully exended o he Nash equilibrium soluion for an n-person noncooperaive linear quadraic dynamic game. Wih he designed exremum seeking conroller for each player in he game, he sysem reaches a sliding mode, eners a viciniy of he Nash equilibrium poin, and says here wih oscillaing behavior. The simulaion resuls show he effeciveness. REFERENCES H. Tsien, Engineering Cyberneics. PA, USA: McGraw-HILL Book Company, Inc., 95. S. Y. Hsin-Hsiung Wang and M. Krsic, Experimenal applicaion of exremum seeking on an axial-flow compressor, in IEEE Transacions on Conrol Sysems Technology, Vol., No., pp. 8,. M. K. Hsin-Hsiung Wang and G. Basin, Opimizing bioreacors by exremum seeking, in Inernaional Journal of Adapive Conrol and Signal Processing,, pp , 999. P. G. Scoson and P. E. Wellsead, Self-uning opimizaion of spark igniion auomoive engines, IEEE Conrol Sysem Magazine, vol. -, pp. 9, B. Blackman, Exremum-seeking regulaors, in An Exposiion of Adapive Conrol, (New York), pp. 6 5, The Macmillan Company, K. Asrom and B. Wienmark, Adapive Conrol, nd ed. MA: Addison-Wesley, M. Krsic and H. Wang, Sabiliy of exremum seeking feedback for general nonlinear dyanmic sysems, Auomaica, vol. 6, pp ,. 8 M. Krsic, Performance improvemen and limiaions in exremum seeking conrol, Sysem & Conrol Leers, vol. 9, pp. 6,. 9 S. Drakunov and Ümi Özgüner, Opimizaion of nonlinear sysem oupu via sliding mode approach, in IEEE Inernaional Workshop on Variable Srucure and Lyapunov Conrol of Uncerain Dynamical Sysem, (UK), pp. 6 6, 99. S. Drakunov, Ümi Özgüner, P. Dix, and B. Ashrafi, ABS conrol using opimum search via sliding modes, IEEE Trans. on Conrol Sysems Technology, vol. -, pp , 995. Y. Pan, Ümi Özgüner, and T. Acarman, Sabiliy and performance improvemen of exremum seeking conrol wih sliding mode, o be published in In. J. Conrol,. Y. Pan, T. Acarman, and Ümi Özgüner, Nash soluion by exremum seeking conrol approach, in he h Conference on Decision and Conrol, (Las Vegas, Nevada), pp. 9,. T. Basar, Dynamic Noncooperaive Game Theory. Philadephia: SIAM, 999. Ümi Özgüner and W. Perkins, A series soluion o he nash sraegy for large scale inerconneced sysem, Auomaica, vol., pp. 5, 977.

6 .6. Exremum Seeking wih Sliding Mode (Player ) J K K.5 Exremum Seeking wih Sliding Mode (Player ) K K..5 J, K, K.8.6, K, K Fig. 6. Nash Soluion by Exremum Seeking Conrol(α = α =.5).6. Exremum Seeking wih Sliding Mode (Player ) J K K.5 Exremum Seeking wih Sliding Mode (Player ) K K. J, K, K.8.6, K, K Fig. 7. Nash Soluion by Exremum Seeking Conrol(α = α =.).6. Exremum Seeking wih Sliding Mode (Player ) J K K.5 Exremum Seeking wih Sliding Mode (Player ) K K. J, K, K.8.6, K, K Fig. 8. Nash Soluion by Exremum Seeking Conrol(α = α =.5.)