he inea Quadatic Dynamic Game fo Discete-ime Descipto Systems Hua Xu 3 and Hioai Muaidani 33 3 Gaduate School of Systems Management he Univesity of suuba, 3-9- Otsua Bunyo-u, oyo -0, Japan xuhua@gssm.otsua.tsuuba.ac.jp 33 Faculty of Infomation Sciences Hioshima City Univesity, 3--, Oua-Higashi saminami-u, Hioshima, 73-39 Japan bstact he linea quadatic eo-sum dynamic game fo discete time descipto systems is consideed. method, which involves solving a linea quadatic eo-sum dynamic game fo a educed-ode discete time state space system, is developed to nd the linea feedbac saddle-point solutions of the poblem. Checable conditions, which ae descibed in tems of two dual algebaic Riccati equations and a Hamiltonian matix, ae given such that the linea quadatic eo-sum dynamic game fo the educed-ode discete time state space system is available. Sucient conditions fo the existence of the solutions ae obtained. In contast with the dynamic game in state space systems, the dynamic game in descipto systems admits uncountably many linea feedbac saddle-point solutions. ll these solutions have the same existence conditions and achieve the same value of the dynamic game. Intoduction In this pape, we conside the linea quadatic eosum dynamic game fo discete time descipto systems Ex+ x + B u + B u,wheee is in geneal a singula matix and the system stuctue is in geneal noncausal(ref. ). Fo the descipto system descibed above, if the system is causal, it can be tansfomed to a egula full-ode o educed-ode state space system depending on whethe the matix E is invetible, o not. It is well nown that, unde cetain conditions, the linea quadatic eo-sum dynamic game fo state space systems admits a unique linea feedbac saddlepoint solution (Ref. ). In papes (Refs. 3,), two methods based on the \completion of squae" technique and the dynamic pogamming technique have been povided to solve the linea quadatic eo-sum dieential game fo continuous time descipto systems. Howeve, it seems that those methods ae not suitable fo the same poblem of discete time descipto systems. In fact, we have not found any appopiate (genealied) discete time Riccati equation which allow us to apply the dynamic pogamming technique to solve the poblem. In this pape, we povide a dieent method, which involves solving a linea quadatic eo-sum dynamic game fo a educed-ode discete time state space system, to solve the linea quadatic eo-sum dynamic game fo discete time descipto systems. Checable conditions depending on the solvability oftwo dual algebaic Riccati equations and the invetible condition of a Hamiltonian matix ae given such that the linea quadatic eo-sum dynamic game fo the educedode discete time state space system is available. Suf- cient conditions fo the existence of the solutions ae given in tems of the conditions of the linea quadatic eo-sum dynamic game fo the educed-ode discete time state space system. Simila to the countepat esults of continuous time descipto systems, we show that the dynamic game in discete time descipto systems admits uncountably many linea feedbac saddlepoint solutions. ll these solutions have the same existence conditions and achieve the same value of the dynamic game. heefoe, the nonunique featue of the linea feedbac saddle-point solutions in this pape is dieent fom the so-called infomational nonuniqueness in state space systems, whee playes have access to closed-loop state infomation (with memoy) and diffeent saddle-point equilibia do not necessaily equie the same existence conditions (Ref. ). Poblem Fomulation Conside the linea discete time descipto system Ex+ x + B u + B u ()
fo K : f0 :::N 0 g whee x is the n- dimensional descipto vecto, u is the m-dimensional contol vecto of Playe, u is the l-dimensional contol vecto of Playe. he matix E is a squae matix of an n. he pencil (se0) is assumed to be egula (i.e., j(se 0 )j 6 0). It is assumed that the initial state x0 is nown to both playes. he cost function is given by NX fx Qx +u u 0u u g J NE x Q N Ex N + 0 () which is to be minimied by Playe and to be imied by Playe, whee Q N 0andQ 0. he supescipt denotes the tanspose of the matix. he infomation stuctue to be used in the pape is the closed-loop no-memoy infomation on x, unde which the stategy spaces fo Playe and Playe at each stage K ae denoted by 0 and 0 espectively. 0 and 0 ae composed of linea feedbac stategies of x, denoted by and, such that the esulting closed-loop system is causal(refs. 5,6). hei openloop ealiations ae u and u espectively. et us intoduce the notation i 0 i : f i 0i Kg i : (3) hen, we have the notion of a saddle-point equilibium. Denition. Fo the dynamic game posed above, a pai of stategies ( 3 3 ) 0 0 is in a saddle-point equilibium if J( 3 ) J( 3 3 ) J( 3 ) () fo all ( ) 0 0. We now see the linea feedbac saddle-point solution to the poblem fomulated above. t this point it is impotant to ecall that the dynamic pogamming technique is a useful tool to nd the linea feedbac saddlepoint solution fo the poblem fomulated in state space systems. Howeve, the dynamic pogamming technique is not valid (at least, at pesent time) fo the poblem fomulated in this pape. he eason is that we have not yet found any appopiate discete time Riccati equation to descibe the cost-to-go function. It has been indicated that an obvious modication to the usual Riccati equation of state space system may esult in no solution in descipto system (Refs. 5,6). heefoe, an altenative method is needed to solve the poblem of this pape. In the next section, we constuct the educed-ode linea quadatic dynamic game fo state space systems whose solution plays an impotant ole in the solution of the dynamic game fo the descipto system. 3 Reduced-Ode Dynamic Game in State Space Systems Suppose that the positive semidenite weighting matix Q is factoed into Q C C and y Cx is the output of the system (). hen, a basic assumption is made thoughout the pape. ssumption. he descipto system () is causally contollable and econstuctible. Moeove, thee exist nonsingula matices M and H such that MEH I 0 0 0 an E: (5) Based on the tansfomation (5), we dene MH B i MB i i B i (6a) Q H QH Q C 3 C Q Q C C (6b) M 0 Q N M QN QN Q Q : (6c) N N Obviously, ssumption holds tue if and only if the ows of the matices [ B B ]and[ C ]ae independent espectively(refs. 5,6). (7a) et us futhe dene some othe matices(ref. 7). 0S Q 0S Q 0S 3 0Q 0 0S 0Q 0 whee (7b) S B B 0 B B (8a) S B B 0 B B (8b) S B B 0 B B : (8c) ssumption. he matix is invetible. ssumption 3. he following two dual algebaic Riccati equations admit eal symmetic solutions X and Y espectively, X + X 0 XS X + Q 0 Y + Y 0 YQ Y + S 0: (9a) (9b)
Rema. Dieent fom Ref. 7, it is woth to note that exists only if ssumption is satised. Howeve, ssumption is not sucient fo to be invetible. Moeove, the existence of a eal symmetic solution X to (9a) is equivalent to the existence of a eal symmetic solution Y to (9b) if Q > 0, which means that the system () is causally obsevable, a moe esticted condition than that in ssumption. Since exists, we can calculate Q 0S 0 3 (0) diectly by some numeical methods. Howeve, fo the pupose of theoetical analysis, we need the explicit expessions of S Q. emma. Suppose that ssumptions -3 ae satised. hen, thee exist matices B, Rm B such Rl that S B B 0 B. Moeove, B Q is a positive semidenite matix. Poof. See ppendix fo the poof. We ae now in the position to constuct the linea quadatic dynamic game fo the educed-ode state space system. Because of emma, we can fomulate the standad linea quadatic dynamic game fo the educed-ode state space systems as follows. Reduced-Ode Dynamic Game. Find the linea feedbac saddle-point solution of the cost function J N Q N N + N X 0 f subject to the system equation Q +u u 0u u g () + + B u + B u () whee [ ] H x, hence 0 is also nown by both playes. heoem. he linea quadatic eo-sum dynamic game descibed by (),() admits a unique linea feedbac saddle-point solution if, and only if, I + B P +B > 0 ( K) (3a) B P +B > 0 ( K) (3b) in which case the unique equilibium stategies ae given by f 3 3 ()0B ( K) (a) 3 ()B ( K) (b) and the coesponding unique state tajectoy Kg satises the dieence equation 3 + Z 3 (5) whee P + (I + S P + ) (6) Z (I + S P + ) : (7) P + satises the discete-time Riccati equation P Q + P +(I + S P + ) P N Q N : (8) Poof. See Basa and Olsde(Ref. ) fo the poof. Since the unique linea saddle-point solution exists, the uppe value and the lowe value of the dynamic game must be equal to the value of the dynamic game. Hence, we aive at the following conclusions. Coollay. Suppose that the unique linea saddlepoint solution exists. hen, the following elations hold. (i) wo discete-time Riccati equations P Q + B B +( 0 B B ) P + (I0 B B P +) ( 0 B B ) P N Q N (9a) P Q + B B +( + B B ) P + (I+ B B P +) ( + B B ) P N Q N (9b) (ii) have the same solution P as that of (8). (I + B B P +) ( + B B ) ( B B (I + S P + ) (0a) P +) ( 0 B B ) (I + S P + ) : (0b) Poof. o pove the elations given above, we only need to x u 3 ()(o, u 3 ( )) of (),() and solve the coesponding imiing (minimiing) poblem obtained fom (),(). he details ae omitted hee. Dynamic Game fo Descipto Systems In this section, using the esults obtained in Section 3, we will solve the dynamic game fo descipto systems. Befoe doing that, we st dene the following notations. Z Moeove, we have 0 3 I : () 0 XZ N 0 N : ()
he eade is efeed to ppendix B fo the explicit expessions of () and the deivations of (). heoem. Fo the linea quadatic eo-sum dynamic game of the discete time descipto systems, suppose that ssumptions -3 ae satised. hen, (i) he dynamic game admits a linea feedbac saddlepoint solution if, and only if conditions (3a),(3b) of heoem ae satised. (ii) Unde conditions (3a), (3b), thee exist uncountably many linea feedbac saddle-point solutions, with the family of the equilibium stategies given by 3 (x )0B M 0 0 F Z F H x ( K) (3a) 3 (x )B M 0 0 F Z F H x ( K) (3b) whee F, F ae abitay two(n0)(n0) matices maing 0 B F + B B F invetible. B Poof. Fist, let F F X in (3). hen, 0 BB X + BB X ^ is invetible. Substituting 3(x )into (),() and maing tansfomations of (5),(6) yield and + + + B u (a) 0 + + B u (b) N X J N Q N N+ f 0 0 u u Q Q Q Q g (5) whee the eade is efeed to ppendix C fo the expessions of the elated tems. We now solve the imiing poblem of Playe descibed by (),(5). Intoducing the codescipto vectos,, the necessay conditions fo (5) to be imied ae (Refs. 6,8) + + + B u (6a) 0 + + B u (6b) + + + + Q + Q (6c) 0 + + + + Q + Q (6d) u B + + B +: (6e) Substituting (6e) into (6a) and (6b) yields + BB Q + B + B Q 0 BB 0Q 0 B + B 0Q 0 + Dene B ^ B Q B ^ B Q ^ 3 BB 0Q 0 B ^ B 0Q 0 Since X I ^ B B ^ exists, we can calculate ^Q 0Q 0 B B 0 0 ^ ^S ^ + (7a) + : (7b) (8a) (8b) 0X I ^ 0 ^ ^ (8c) : (8d) (9) ^ 3 (30) using (8),(9) diectly. Futhemoe, we can pove heefoe, we obtain + 0 B B ^ 0 B B (3a) ^S B B (3b) ^Q Q + B B : (3c) B B Q + B B ( 0 B B ) + (3) fom (7). et ^P, we have the discete-time Riccati equation ^P Q + ( B B B B +( 0 B B ) ^P+ ^P + ) ( 0 B B ) ^PN Q N : (33) Using the solution ^P of (33), we aive at the following equa tions + ^Z (3)
whee ^Z ( B B ^ ^P+ ( B B + ^ (35) ^P + ) ( 0 B B ) (36) ^P + ) ( 0 B B ): (37) aing into account the facts of (9),(0), we eadily have ^P P ^Z Z ^ : (38) Futhemoe, substituting + into (7b) yields which ae obtained fom the fomula ^Z ^ 0 ^ ^ 3 We can also pove ^Z (39) + ^ (0) I^ : () ^ 0 X ^Z 0 XZ () ^Z Z : (3) heefoe, the imiing poblem of Playe admits a solution 3 (x )B M 0 H x 0 XZ X : ( K) () Symmetically, substituting 3(x ) of () into (),(), we can obtain a minimiing poblem of Playe. Solving the minimiing poblem in a simila way asabove gives the esult that 3(x )0B M 0 H x 0 XZ X (5) constitutes a minimiing solution to the poblem. heefoe, we conclude that the dynamic game admits a linea feedbac saddle-point solution which is given by (),(5). he coesponding unique state tajectoy is H[3 3 ] Kg, whee 3 fx 3 dieence equation (3)(o (5)) and 3 equation (39). satises the satises the In the following, we will show that the linea feedbac saddle-point solutions ae not unique. o do this, it suf- ces to show that the stategies (3) and the stategies (),(5) lead to the same state tajectoy. Substituting (3) into () and maing some tansfomations give the closed-loop system whee + c + c (6a) 0 c + c (6b) c 0 B B 0 B B ( 0 F Z ) +B B + B B ( 0 F Z ) (7a) c 0 B B F + B B F (7b) c 0 B B 0 B B ( 0 F Z ) +B B + B B ( 0 F Z ) (7c) c 0 B B F + B B F : (7d) Since 0 BB F + B B F is invetible, (6) admits a unique solution, fo example, (^ ^ ). On the othe hand, it is easy to veify that ( 3 3)aealso the solutions of (6) because of 3 Z 3. Hence, ^ 3 and ^ 3, which means that (3) is a linea feedbac saddle-point solution of the poblem. heeby, we have nished the poof of the theoem. 5 Conclusions In this pape, we have investigated the linea quadatic eo-sum dynamic game fo discete time descipto systems. his poblem is solved though the solution of the educed-ode linea quadatic eo-sum dynamic game fo standad discete time state space system. Checable conditions ae given such that such a educed-ode eo-sum dynamic game is available. hese conditions ae descibed in tem of the solvability of two dual algebaic Riccati equations and the invetible condition of the Hamiltonian matix. Futhemoe, the sucient conditions fo the existence of the linea feedbac saddle-point solutions ae obtained which ae the same as the conditions of the linea quadatic eosum dynamic game fo the educed-ode discete time state space system. Simila to the countepat esults of continuous time descipto systems, we show that the dynamic game in discete time descipto systems admits uncountably many linea feedbac saddle-point solutions. ll these solutions have the same existence conditions and achieve the same value of the dynamic game. heefoe, the nonunique featue of the linea feedbac saddle-point solutions in this pape is dieent fom the so-called infomational nonuniqueness in state space systems, whee playes have access to closed-loop state infomation (with memoy) and dieent saddlepoint equilibia do not necessaily equie the same existence conditions (Ref. ). ppendix. Poof of emma. Unde ssumptions,3, we have X I 0S 0Q 0 ^ 0 ^S 0 ^ 0 0 ^0 0X I : (8)
whee ^ 0 S X. Using (8) in the calculation of (0 ) yields + N + N S N + S N S S + N S + N S N + S N Q Q 0 N 0 N S N 0 N whee Hence, whee (9a) (9b) (9c) N 0 ^ ^ N ^Q ^ (50a) ^ 0 S X ^Q Q + X: S B B (50b) 0 B B (5) B B + N B B B + N B : (5) On the othe hand, 0S 0Q 0 ~ 0 0 0 ~ Q ~ 0 0 ~ Y 0 I I Y : (53) 0 I whee ~ 0 Q Y. Using (53) in the calculation of (0) gives + M + M Q + M Q M (5a) S S 0 M 0 M 0 M Q M (5b) Q Q + Q M + M Q + M Q M (5c) whee M 0 ~ ~ M ~ S ~ (55a) ~ 0 Q Y S ~ S + Y: (55b) heefoe, Q (C + C M ) (C + C M ) 0: (56) ppendix B. Deivations of Z, in Section. Using (8) to () yields Z (^ S + ^ S N) 0 ^ 0 ^ S N (57) X( ^ S + ^ S N ) 0X( Hence, ^ + ^ S N )+N 0 N : (58) 0 XZ N 0 N : (59) 0 B [B + B ( 0 XZ )] 0 B B X 0 B [B + B ( 0 XZ )] + 0 B B X: (60) Q Q Q Q 0 Z X B B BB Q Q Q Q 0 X B B B B 0 0 XZ : (6) X Refeences. Dai,., Singula Contol Systems, ectue Notes in Contol and Infomation Sciences, Edited by M.homa and.wyne, Spinge-Velag, Belin, 989.. Basa,., and Olsde, G. J., Dynamic Noncoopeative Game heoy, cademic Pess, New Yo, New Yo, 98. 3. Xu, H., and Miuami, K., inea-quadatic Zeo-Sum Dieential Games fo Genealied State S pace Systems, IEEE ansactions on utomatic Contol, Vol. 39, pp.3-7, 99. Xu, H., and Miuami, K., On the Isaacs Equation of Dieential Games fo Descipto Systems, Jounal of Optimiation heoy and pplications, Vol. 83, pp.05-9, 99. 5. Bende, D. J., and aub,. J., he inea- Quadatic Optimal Regulato fo Descipto Systems, IEEE ansactions on utomatic Contol, Vol. 3, pp.67-688, 987. 6. Bende, D. J., and aub,. J., he inea- Quadatic Optimal Regulato fo Descipto Systems: Discete-ime Case, utomatica, Vol. 3, pp.7-85, 987. 7. Wang, Y. Y., Shi, S. J. and Zhang, Z. J., Descipto-System ppoach to Singula Petubation of inea Regulatos, IEEEansactions on utomatic Contol, Vol. 33, pp.370-373, 988. 8. Mantas, G. P., and Kielis, N. J., inea Quadatic Optimal Contol fo Discete Descipto Systems, Jounal of Optimiation heoy and pplications, Vol. 6, pp.-5, 989. ppendix C. Deivations of the elated tems in (),(5)