Linear Quadratic Stochastic Differential Games under Asymmetric Value of Information

Transcription

1 Preprint of the 2th World Congre The International Federation of Automatic Control Touloue, France, July 9-4, 27 Linear Quadratic Stochatic Differential Game under Aymmetric Value of Information Dipankar Maity and John S. Bara Department of Electrical & Computer Engineering, Intitute for Sytem Reearch Univerity of Maryland, USA ( {dmaity, Abtract: Thi paper conider a variant of two-player linear quadratic tochatic differential game. In thi framework, none of the player ha acce to the tate obervation for all the time, which retrict the poibility of continuou feedback trategie. However, they can oberve the tate intermittently at dicrete time intance by paying ome finite cot. Having on demand cotly meaurement enure that open-loop trategy i not the only trategy for thi game. The individual cot function for each player explicitly incorporate the value of information and the aymmetry that come along with different cot of tate obervation for different player. We tudy the tructural propertie of the Nah equilibrium for thi particular cla of problem when the cot of obervation i finite and poitive. We how that the game problem implifie into two decoupled game problem: one for deciding the control trategie, and the other for deciding the obervation acquiition time. The tudy alo reveal that under two extreme cae -cot of obervation being or - the trategie coincide with feedback and open-loop trategie repectively. Keyword: game theory, linear-quadratic game, tochatic game, non-cooperative game, event-baed game. INTRODUCTION Game theory ha been an active topic for reearch in control for it wide applicability in tochatic control, robut control; and it ha been tudied extenively by the community a can be found in Baar and Older (995), Jame and Bara (996), Engwerda (25), Başar and Bernhard (28), Fleming and Hernández-Hernández (2) and many other. A differential tochatic game encompae many apect of a control problem uch a optimality, tochaticity and filtering, and etimation; hence the reult can reveal everal propertie related to thoe. Linear-quadratic-Gauian i a ubcla of uch differential game problem that attain a cloed form analytical olution for the Nah trategy a reported in Cruz Jr. and Chen (97), Jacobon (973), Weeren et al. (999). The olution of linear-quadratic differential game are generally contructed by certain Riccati equation; for detail, ee Jacobon (973), Weeren et al. (999), and the reference therein. Studie on the neceary and ufficient condition for a trategy to be a Nah trategy for a linear-quadratic game can be found in the work of Foley and Schmitendorf (97), and Bernhard (979). Baar (976) tudied the uniquene property of a Nah trategy. The work of Weeren et al. (999) tudie the aymptotic behavior of the Nah trategy over an infinite horizon. Reearch partially upported by ARO grant W9NF and W9NF , and by National Science Foundation (NSF) grant CNS Unlike the well perceived fact about linear control law being optimal for a linear-quadratic-gauian problem, Baar (974) provided a counterexample howing that the optimality i achieved by ome nonlinear Nah trategy for a linear-quadratic game problem. A non-cooperative game i inherently a joint problem on the control and the deciion making proce and hence the olution relie on the knowledge of the behavior of the opponent. In the vat majority of the pat work in the community, the tudied problem either aume that the tate information i available to the player for all time or only the initial tate information i available. The former ituation reult in a f eedback-type Nah trategy wherea the latter exhibit an openloop Nah trategy. To the bet of our knowledge, the problem of having multiple dicrete tate meaurement for thi problem cla remained unaddreed. In thi cenario, the player have le information about the tate of the ytem ince the meaurement are available at only certain dicrete time intance, not for all time; however, they have more information than merely having the knowledge of the initial tate. In thi work we addre thi linear-quadratic game problem under dicrete meaurement where the player are given the freedom to elect their time intance to acquire the meaurement of the tate. Moreover, it i impoed that each uch query about the tate information require ome finite cot. Thi new framework introduce certain change in the well known behavior of the Nah trategy ince the feedback trategy i not plauible, and the open loop i not necearily optimal. Given the fact that each obervation Copyright by the International Federation of Automatic Control (IFAC) 9287

2 Preprint of the 2th IFAC World Congre Touloue, France, July 9-4, 27 require finite cot, the player mut decide optimal time intance for oberving the tate. Therefore, the problem include deigning a ampling policy to meaure the tate and yntheizing a controller uch that they contitute a Nah equilibrium for the game. In thi work, we aume that the ampling i done intantaneouly and there i no delay or noie in communicating the ampled value to the controller. We conider aynchronou witching i.e. player can chooe their witching policy irrepective of the policy choen by their opponent. We alo aume that whenever a player receive a ample, the opponent i notified about that but the opponent doe not get the value of the ample. In thi tudy, we how that given a witching policy, there alway exit a Nah trategy for controller ynthei and the controller i a dynamic controller that reet it value in an optimal way every time the witch i cloed. The problem i decompoed into two decoupled ubproblem for deigning the witching policy and deigning the controller. The tudied game i aymmetric ince the parameter aociated with the player are different (e.g. cot per ample i different for different player) and that eentially lead to different trategy for them. 2. NOTATION x(t): tate of the game, C i : controller of player-i, S i : witching policy of player-i, a 2 B = a Ba for matrice a and b of proper dimenion, T i (t): et of ampling time until t for player-i, X i (t): et of ampled tate value for player-i until t, I i (t): total information available to playeri that include X i (t), T (t) and T 2 (t). For any matrix M, Φ M (t, ) denote the aociated tate tranition matrix. 3. PROBLEM FORMULATION Let u conider the following tochatic linear differential game dynamic: dx = (Ax + B u + B 2 u 2 + GdW t () where x R n, u R m, u 2 R m2 and W t i a p dimenional Wiener proce noie, independent of the initial tate x(), acting on the ytem. The aociated quadratic cot i: J(u, u 2 ) = E (x Lx + u R u u 2R 2 u 2 (2) where L, R i. All the matrice A, B i, L, R i, G are time varying unle or otherwie mentioned in the paper. The objective of player- (or player-2) i to minimize (or maximize) the cot functional (2) with the knowledge of x(t) at ome finite number of dicrete time intance. Let u conider the chematic preented in Figure where player-i ha to deign it controller C i and the optimal witching policy S i. The witch S i cloe only for a time intance and open immediately o that the controller get the tate value only at a ingle time intance. We aume there i no delay in the witching action or in the noie-le channel o that the controller C i get the tate information preciely at the witching time intance. Prior work on linear-quadratic differential game either conider the witche S i are cloed for all t or open for Fig.. Schematic of the game: C i repreent the controller (dynamic) of player-i and S i implement a witch that ample the tate x at ome optimal intance. ZOH i a zero order hold circuit. The witche S i are initially cloed and they open at t = + o that each player ha the knowledge of x. all t >. We tudy the characteritic of the game and the aociated Nah trategy() for thi pecial et up of the game. Maity and Bara (26a) tudie the problem when the witche S and S 2 operate ynchronouly i.e. they ample the tate at the ame time intance. In thi work we aume that the tate x i fully obervable, however the tudy eaily extend under partial obervation framework along the line of Maity et al. (27). Moreover, the player are given the freedom to elect their witching intance for ampling the tate by incurring a finite cot. Let T i (t) = {τ, i τ2, i, τn i i(t) } be the et of elected time intance for cloing the witch S i of player-i till time t, where τk i < τ k+ i < < τ n i < t. Let T (t) = T i(t) (t) T 2 (t) denote the et of all intance for oberving the tate. The tate information available to player-i at time t i denoted a X i (t) = {x(τ) τ T i (t)}. The total information available to player-i at any time intance t i I i (t) = X i (t) T (t). However a mentioned, the information acquiition i not free and in order to contruct I i (t), player-i need to pay λ i (> ) for each ample of the tate and c ij (> ) for each element in T j (player-j i the opponent of player-i). Thu, it hould be noted that the cot function J(u, u 2 ) i implicitly a function of the information et I and I 2 ince the trategie u and u 2 are I and I 2 meaurable function repectively (J(u, u 2 ) J(u, u 2, I, I 2 )). Therefore, player- (P) need to minimize: J (u, u 2, I, I 2 ) =E ( x 2 L + u 2 R u 2 2 R 2 and player-2 (P2) hould maximize: + λ N + c 2 N 2, (3) J 2 (u, u 2, I, I 2 ) =E ( x 2 L + u 2 R u 2 2 R 2 λ 2 N 2 c 2 N. (4) where N i = n i (T ) i the number of ample in T i (T ). In their repective cot function, appropriate term have been added to account for the cot of ampling (or could be thought a the cot of communication over the channel). It hould be noted right away that the new cot function do 9288

3 Preprint of the 2th IFAC World Congre Touloue, France, July 9-4, 27 not allow infinite number of witching and hence the Zeno behavior i not poible for an optimal witching policy. The game i tudied under aymmetric information tructure and remark are made when the game i ymmetric (i.e. c 2 = c 2, λ = λ 2, B = B 2, R = R 2 ). 4. NASH STRATEGY In thi ection we aim to tudy the exitence of Nah trategy for the propoed game framework. The Nah trategy include deigning a Nah witching policy (S i ) and deigning a Nah controller (C i ) for both the player. We eek for Nah equilibrium olution (u, u 2, I, I2 ) uch that J = J (u, u 2, I, I2 ) = min J (u, u u,i 2, I, I2 ) (5) J2 = J 2 (u, u 2, I, I2 ) = max J 2 (u u 2,I 2, u 2, I, I 2 ) (6) Note that for player-i, chooing an Ii eentially mean to chooe a Ti. It can be eaily hown that J = min min J(u, u I u 2, I, I2 ) + λ N + c 2 N2 (7) J2 = max max J(u I 2 u 2, u 2, I, I 2 ) λ 2 N 2 + c 2 N (8) Equation (7) and (8) decouple the problem that i the minimization (or maximization) i performed in two tage rather than in a ingle tage. Therefore, a a firt tep toward the proof, the Nah trategie (u, u 2) (u (I, I 2 ), u 2(I, I 2 )) are found for the cot function J(u, u 2, I, I 2 ) for a given (I, I 2 ). Let u denote J # (I, I 2 ) to be the value of J(u, u 2, I, I 2 ) at the Nah equilibrium (u, u 2). That i, for a fixed (I, I 2 ), and for any u, u 2, J(u, u 2, I, I 2 ) J # (I, I 2 ) J(u, u 2, I, I 2 ). (9) Theorem 2 characterize the Nah trategy (u, u 2). Uing completion of quare, it can be hown that (cf. Maity and Bara (26b)): J(u, u 2, I, I 2 ) = E x() 2 T P () + tr(p GG () T + E ( u + R B P x 2 R u 2 R2 B 2P x 2 R 2 where P (t) atifie the Riccati equation: P + A P + P A + L + P (B 2 R2 B 2 B R B )P = P (T ) = () Aumption. In order to enure the exitence and welldefinedne of the olution of the Riccati equation (), we aume that B 2 R2 B 2 B R B The admiible trategy u i (t) ha to be I i (t) meaurable. Lemma. For a given witching I i, the optimal control trategy for player-i i of the form, u i (t) = ( ) i R i B ip ˆx i (t) (2) for ome I i (t) meaurable optimal ˆx i (t). A proof thi lemma can be found in (Maity et al., 27, Propoition 3.2). Therefore, the goal i to find the Nah equilibrium of T J (ˆx, ˆx 2 ) = E ( ˆx x 2 Q ˆx 2 x 2 Q 2 for a given (T, T 2 ) (or equivalently (I, I 2 )), where Q i = P B i R i B i P. Theorem 2. J ha a unique addle point at (ˆx, ˆx 2) uch that: J (ˆx, ˆx 2 ) J (ˆx, ˆx 2) J (ˆx, ˆx 2) (3) for all ˆx, ˆx 2. The optimal ˆx and ˆx 2 atify the following differential equation: ˆx = (A P Q + P Q 2 )ˆx (4) ˆx (τ ) = x(τ ) ˆx 2(τ 2 ) = x(τ 2 ) for all τ T and τ 2 T 2. ˆx 2 = (A P Q + P Q 2 )ˆx 2 (5) Proof: The proof of thi theorem i preented in the Appendix A. Therefore, Theorem 2 enure that, for a fixed witching pair (T, T 2 ), there exit a Nah controller pair (C, C2). Uing the reult from Theorem 2, we can write (the are removed from ˆx i for brevity), dˆx dx =(A + P Q 2 )(ˆx x + P Q 2 (x ˆx 2 GdW t (6) dˆx 2 dx =(A P Q )(ˆx 2 x P Q (x ˆx GdW t with the reetting condition ˆx (τ ) = x(τ ) and ˆx 2 (τ 2 ) = z = z 2 x(τ 2 ) for all τ T and τ 2 T 2. Denoting z = ˆx x, we can write ˆx 2 x A + P dz = Q 2 P Q 2 GdWt P Q A P zdt Q GdW t with z i (τ) = for all τ T i, i =, 2. Σ Σ Let u denote Σ(t) = 2 = Ez(t)z Σ 2 Σ (t). Therefore, 22 Σ = ĀΣ + ΣĀ + ḠḠ (7) G A + P where Ḡ =, and G Ā = Q 2 P Q 2 P Q A P. At Q τ T, Σ (τ) = Σ 2 (τ) = Σ 2 (τ) = and at τ T 2, Σ 22 (τ) = Σ 2 (τ) = Σ 2 (τ) =. The olution of (7) i given by, Σ(t) = Φ Ā (t, )Σ()Φ Ā (t, ) + Φ Ā (t, r)ḡḡ Φ Ā (t, r)dr where σ (, t, σ T (t). Φ (t, ) Φ One can how that, Φ Ā (t, ) = 2 (t, ) ha the Φ 3 (t, ) Φ 4 (t, ) following expreion (we leave out the detail here). Φ (t, ) = Φ A (t, ) + where à = A P Q + P Q 2. Φ 2 (t, ) = Φ A (t, σ)p (σ)q 2 (σ)φã(σ, )dσ Φ A (t, σ)p (σ)q 2 (σ)φã(σ, )dσ, 9289

4 Preprint of the 2th IFAC World Congre Touloue, France, July 9-4, 27 and, Φ 3 (t, ) = Φ 4 (t, ) = Φ A (t, ) Φ A (t, σ)p (σ)q (σ)φã(σ, )dσ I Since Σ = I O Σ O Σ (t) = Φ A (t, σ)p (σ)q (σ)φã(σ, )dσ. O and Σ = O I Σ we get, I Φ A (t, σ)g(σ)g(σ) Φ A(σ, )dσ+ (8) Φ (t, )Σ ()Φ (t, ) + Φ (t, )Σ 2 ()Φ 2(t, ) + Φ 2 (t, )Σ 2 ()Φ (t, ) + Φ 2 (t, )Σ 22 ()Φ 2(t, ) Similar expreion can be found for Σ 22 a well. Therefore, the game problem at thi tage i repreented with the objective J (ˆx, ˆx 2) = J # (T, T 2 ) = where Q = tr(q Σ Q 2 Σ 22 tr( QΣ (9) = Q. Σ atifie the dynamic: Q 2 Σ = ĀΣ + ΣĀ + ḠḠ (2) At τ T i, Σ ii (τ) = Σ ij (τ) = Σ ji (τ) = for all i, j =, 2. The dynamic game at thi point ha a linear dynamic (2) with a linear cot criterion (9), however, the action of the player are witching action i.e. the action partially reet the value of Σ. Clearly, when player- trategically elect a witching intance τ, it reet Σ to zero and conequently the cot i reduced, however thi reduction in the cot come with an additional witching cot of λ. The objective of player- i to minimize J # (T, T 2 ) + λ n (ince the other term, c 2 n 2, depend olely on the opponent action) and player-2 aim to maximize J # (T, T 2 ) λ 2 n 2, where n i i the cardinality of T i (T ). It hould be noted at thi point that the game i totally characterized by Σ(t) which can uniquely be determined whenever the witching intance are known. Thi game ubproblem i decoupled from the game ubproblem eeking the Nah control trategie (Theorem 2). Due to pace contraint, olving for the Nah witching trategy i beyond the cope of thi paper. Intead, we will aume a olution to thi witching game problem exit and we will provide ome characterization of the olution in the ret of the paper. If (T, T2 ) i an equilibrium trategy for (9) with optimal number of witching being (n, n 2), then we have J # (T, T 2 ) λ 2 (n 2 n 2) J # (T, T2 ) (2) J # (T, T2 ) + λ (n n ) for all T and T 2 with cardinality being n and n 2 repectively. For all (T, T 2 ) uch that n i = n i, we obtain: J # (T, T 2 ) J # (T, T2 ) J # (T, T2 ). (22) It can be hown that for any witching trategy played by Player-2, if player- elect no-witching trategy (i.e. doe not attempt to reet Σ(t)), denoted by T, then the following inequality hold: J # (T, T 2 ) tr(q Σ λ n (23) for all (T, T 2 ) with cardinalitie n and n 2 repectively, and Σ (t) = Φ A(t, )GG Φ A (, )d. Similarly for all (T, T 2 ), it can be hown that J # (T, T 2 ) Combining (23) and (24), tr(q Σ J # (T, T 2 ) tr(q 2 Σ + λ 2 n 2. (24) tr(q Σ (25) for all (T, T 2 ). Propoition 3. If n i i the number of witching of player-i at equilibrium, then n i tr ( ) (Q + Q 2 )Σ dt. λ i Proof: From (23), J # (T, T2 ) tr(q Σ λ n. Uing (25), we obtain tr(q Σ tr(q Σ λ n. Hence, n tr ( ) (Q + Q 2 )Σ dt. λ i Similarly we can proceed for n 2. Propoition 3 provide an upper bound on the number of equilibrium witching and it i inverely proportional to the cot of witching λ i, a expected. Alo notice that a λ i, upper bound on n i, reembling the continuou-cloed-loop trategy. Alo, when λ i, n i reembling the open-loop trategy. Let (T, T2 ) and (T3, T4 ) two ditinct equilibrium trategie with number of witching equal to (n, n 2) and (n 3, n 4) repectively. Therefore, J # (T, T2 ) J # (T3, T2 ) + (n 3 n )λ J # (T3, T4 ) + (n 3 n )λ (n 4 n 2)λ 2 J # (T, T4 ) (n 4 n 2)λ 2 J # (T, T2 ) Therefore, all the inequalitie in the above equation hould be equalitie, and that reult into J # (T, T2 ) + n λ n 2λ 2 = J # (T3, T4 ) + n 3λ n 4λ 2. If there exit two equilibria (T, T2 ) and (T3, T4 ) uch that n = n 3 and n 2 = n 4, then J # (T, T2 ) = J # (T3, T4 ). Moreover, the cot incurred by both the player at thee two different equilibria remain the ame for both the player. We conclude thi ection by citing ome reult for a ymmetric game. 4. Symmetric Game In thi ection, we extend the reult for a game where λ = λ 2 = λ and B R B = B 2 R2 B 2. In thi cae, Ã = A, and Q = Q 2 = Q. One can verify that J # (T, T ) = for all witching trategy T (with n number of witching). In thi cae, the cot of player- i nλ and the ame for player-2 i nλ. 929

5 Preprint of the 2th IFAC World Congre Touloue, France, July 9-4, 27 Let (T, T2 ) be a equilibrium witching trategy for under thi ymmetric ituation. Let n i be the number of element in Ti. Therefore, J # (T, T2 ) + n λ J # (T2, T2 ) + n 2λ = n 2λ (26) and J # (T, T2 ) n 2λ J # (T, T ) n λ = n λ (27) Combining the above two inequalitie, J # (T, T2 ) = (n 2 n )λ (28) Under thi ituation, the cot incurred by player- i n 2λ ( ) and by player-2 i n λ ( ). Remark 4. If the player cooperate, then no-witching for all t > produce the bet cot for both the player. In thi ituation J # = and the cot incurred by both the player are. 5. CONCLUSION In thi work we have conidered a variant of two-player linear quadratic game where we retricted the poibility of feedback trategie by putting a finite cot for acceing the tate. Thi work tudie the tructure of the Nah controller of the player under thi aymmetric game etup. We how that the game problem can be olved by independently olving two impler game problem. The cotly witching behavior make thi problem challenging and intereting. The reult how that for a high enough cot (λ i > tr( ) (Q + Q 2 )Σ dt) player-i opt for openloop trategy (i.e. S i i alway open). Thu, the reult how the value of information of a ample obtained by performing the witching. It i a trade-off between the reduction in cot (J) by ampling the tate, and the cot incurred to obtain the tate information. REFERENCES Baar, T. (976). On the uniquene of the Nah olution in linear-quadratic differential game. International Journal of Game Theory, 5(2-3), Baar, T. (974). A counterexample in linear-quadratic game: Exitence of nonlinear Nah olution. Journal of Optimization Theory and Application, 4(4), Başar, T. and Bernhard, P. (28). H-infinity optimal control and related minimax deign problem: a dynamic game approach. Springer Science & Buine Media. Baar, T. and Older, G.J. (995). Dynamic noncooperative game theory, volume 2. SIAM. Bernhard, P. (979). Linear-quadratic, two-peron, zeroum differential game: neceary and ufficient condition. Journal of Optimization Theory and Application, 27(), Cruz Jr., J. and Chen, C. (97). Serie Nah olution of two-peron, nonzero-um, linear-quadratic differential game. Journal of Optimization Theory and Application, 7(4), Engwerda, J. (25). LQ dynamic optimization and differential game. John Wiley & Son. Fleming, W.H. and Hernández-Hernández, D. (2). On the value of tochatic differential game. Commun. Stoch. Anal, 5(2), Foley, M. and Schmitendorf, W. (97). On a cla of nonzero-um, linear-quadratic differential game. Journal of Optimization Theory and Application, 7(5), Jacobon, D.H. (973). Optimal tochatic linear ytem with exponential performance criteria and their relation to determinitic differential game. Automatic Control, IEEE Tranaction on, 8(2), Jame, M.R. and Bara, J. (996). Partially oberved differential game, infinite-dimenional hamilton-jacobiiaac equation, and nonlinear h control. SIAM Journal on Control and Optimization, 34(4), Maity, D. and Bara, J.S. (26a). Optimal trategie for tochatic linear quadratic differential game with cotly information. In Deciion and Control (CDC), 26 IEEE 55th Conference on, IEEE. Maity, D. and Bara, J.S. (26b). Strategie for twoplayer differential game with cotly information. In Dicrete Event Sytem (WODES), 26 3th International Workhop on, IEEE. Maity, D., Raghavan, A., and Bara, J.S. (27). Stochatic differential linear-quadratic game with intermittent aymmetric obervation. In American Control Conference (ACC), 27, (to appear). IEEE. Weeren, A., Schumacher, J., and Engwerda, J. (999). Aymptotic analyi of linear feedback Nah equilibria in nonzero-um linear-quadratic differential game. Journal of Optimization Theory and Application, (3), Appendix A. PROOF OF THEOREM 2: Player- want to minimize J (ˆx, ˆx 2 ) = E ˆx x 2 Q ˆx 2 x 2 Q 2 I dt (A.) wherea, player-2 want to maximize (with light abue of notation) J (ˆx, ˆx 2 ) = Let u denote the olution of () a: E ˆx x 2 Q ˆx 2 x 2 Q 2 I 2 dt (A.2) x(t) =Φ A (t, t )x(t ) + K t,t ˆx (t) + K t,t 2 ˆx 2 (t) + K t,t 3 W (t) (A.3) for t t. K t,t i for i =, 2, 3 are linear operator defined a follow: K t,t i f(t) = ( ) i Φ A (t, )P ()Q i ()f()d, t for i =, 2 and (A.4) K t,t 3 W (t) = Φ A (t, )G()dW (). (A.5) t We eek the Nah equilibrium (ˆx, ˆx 2) of (A.). We proceed uing calculu of variation technique by tudying the firt and econd order Gateaux differential of the cot functional J and finally we look for a addle point of J. Let u calculate the Gateaux differential of the functional J : 929

6 Preprint of the 2th IFAC World Congre Touloue, France, July 9-4, 27 δj ˆx, ˆx 2 (h, h 2 ) = lim ɛ J (ˆx + ɛh, ˆx 2 + ah 2 ) J (ˆx, ˆx 2 ) ɛ (A.6) where the notation δj ˆx, ˆx 2 (h, h 2 ) mean the Gateaux differential of J evaluated at the point (ˆx, ˆx 2 ) in the direction (h, h 2 ). Note that J ˆx, ˆx 2 (, ) i a linear functional parameterized by ˆx and ˆx 2. Let u denote x h,h2 (t) to be the perturbed olution of the following dynamic: x h,h2 (t) = Φ A (t, )x() + K t, ˆx + h (t) + K t, 2 ˆx 2 + h 2 (t) + K t, 3 W (t), (A.7) Therefore, 2 δj ˆx, ˆx 2 (h, h 2 ) = E ( ˆx x, h K t, h K t, 2 h 2 Q h 2 K t, h K t, 2 h 2, ˆx 2 x Q2 ) I dt (A.8) where a, b C = a Cb and a, b, C are matrice (or vector) of compatible dimenion. In order for (ˆx, ˆx 2 ) to be a Nah Equilibrium (addle point of J ), the neceary condition i δj ˆx, ˆx 2 (h, h 2 ) = for all (h, h 2 ). Let u conider h 2 (t) = ΦÃ2 (t, )P Q h d (A.9) where ΦÃ2 i the tate tranition matrix correponding to the drift matrix A + P Q 2 i.e. ΦÃ2 (t, ) = (A(t) + P (t)q 2 (t))φã2 (t, ) and ΦÃ2 (, ) = I n n Thi choice of (h, h 2 ) implie, K t, 2 h 2(t) = = = = h 2 (t) Φ A (t, )P Q 2 ΦÃ2 (, σ)p Q h dσd Φ A (t, )P Q 2 ΦÃ2 (, σ)d P Q h dσ σ d d (Φ A(t, )ΦÃ2 (, σ))d P Q h dσ σ = h 2 (t) K t, h (t). Φ A (t, )( P Q h )d Subtituting thee (h, h 2 ), we obtain 2 δj ˆx, ˆx 2 (h, h 2 ) = ) E ( ˆx x, h h 2 Q I dt = (A.) Equation (A.) hold true for all choice of h. Thu the neceary condition become, Ex(t) ˆx (t) I (t) = (A.) or Ex(t) I (t) = ˆx (t) for all t. Similarly, h (t) = ΦÃ (t, )P Q 2 h 2 d (A.2) implie K t, h (t) + K t, 2 h 2(t) = h (t). With thi pair of (h, h 2 ), 2 δj ˆx, ˆx 2 (h, h 2 ) = ) E ( ˆx2 x, h h 2 Q2 I dt (A.3) Therefore, another neceary condition i: Ex(t) ˆx 2 (t) I (t) = or Eˆx 2 (t) I (t) = ˆx (t). (A.4) Similarly, one can how uing (A.2) that the following relation hold: ˆx 2 (t) =Ex(t) I 2 (t) (A.5) ˆx 2 (t) =Eˆx (t) I 2 (t) (A.6) From (A.) and (A.4), and uing the fact EW t I (t) = for all t, we obtain t ˆx = (A P Q + P Q )ˆx (A.7) ˆx (τ ) = x(τ ) for all τ T. Similarly by conidering (A.2), one can how that t, ˆx 2 = (A P Q + P Q )ˆx 2 (A.8) ˆx 2 (τ 2 ) = x(τ 2 ) for all τ 2 T 2. Therefore, equation (A.7) and (A.8) are neceary condition for ˆx and ˆx 2 to be a Nah Equilibrium. To prove that ˆx and ˆx 2 atifying (A.7),(A.8) are a addle point pair (hence Nah Equilibrium) for J, we need to evaluate the econd order Gateaux differential of J. We do not preent the detail of thi derivation due to pace limitation, but one can check that: 2 δ2 J ˆx, ˆx 2 (h, h 2 ) = D D 2 (A.9) where for i =, 2, D i = h i K t, h K t, 2 h 2 2 Q i. (A.2) We need to prove that δ 2 J ˆx, ˆx 2 i indefinite i.e. depending on the direction (h, h 2 ), δ 2 J can be poitive a well a negative. Let u conider a (h, h 2 ) pair uch that h 2 identically for all t and h (t) = ΦÃ (t, )P ()Q 2 ()h 2 ()d (A.2) Therefore for ome h 2 (ay h 2 = contant) we have D 2 > and D = implying that δ 2 J ˆx, ˆx 2 (h, h 2 ) <. Alo in a imilar fahion, by chooing h 2 (t) = ΦÃ2 (t, )P ()Q ()h ()d one can how that δ 2 J ˆx, ˆx 2 (h, h 2 ) >. (A.22) Thi prove that the pair (ˆx, ˆx 2 ) atifying (A.7)-(A.8) i a addle point of J. The uniquene of the addle point i due to the uniquene property of the olution of a linear differential equation. 9292