Math-Net.Ru All Russian mathematical portal

Transcription

1 Math-Net.Ru All Russian mathematical portal Anna V. Tur, A time-consistent solution formula for linear-quadratic discrete-time dynamic games with nontransferable payoffs, Contributions to Game Theory and Management, 015, Volume 8, Use of the all-russian mathematical portal Math-Net.Ru implies that you have read and agreed to these terms of use Download details: IP: November 4, 018, 06:46:46

2 Contributions to Game Theory and Management, VIII, A Time-consistent Solution Formula for Linear-Quadratic Discrete-time Dynamic Games with Nontransferable Payoffs Anna V. Tur St.Petersburg State University, Faculty of Applied Mathematics and Control Processes, Universitetskii pr. 35, St.Petersburg, , Russia a.tur@spbu.ru Abstract The solution formula for the payoff distribution procedure of a bargaining problem in a cooperative differential game with nontransferable payoffs that lead to a time consistent outcome is proposed by Leon A. Petrosyan and D.W.K. Yeung 014. In this paper we study this formula for linear-quadratic discrete-time dynamic games with nontransferable payoffs. Pareto-optimal solution is studied as optimality principle. The time consistency and irrational behavior proof condition of this solution are investigated. As an example the government debt stabilization game is considered. Keywords: linear-quadratic games, discrete-time games, games with nontransferable payoffs, Pareto-optimal solution, time consistency, PDP, irrational behavior proof condition. 1. Introduction Consider N-person discrete-time dynamic game Γk 0,x 0 which is described by the state equation n xk +1 = Akxk+ B i ku i k, k 0 k K <, xk 0 = x 0, 1 where x is m-dimensional state of system, u i is a r-dimensional control variable of player i, xk 0 = x 0 is the arbitrarily chosen initial state of the system, Ak,B i k are matrices of appropriate dimensions. The quadratic cost function of player i N is J i k 0,x 0,u = x T kp i kxk+u T i kr i ku i k + +x T KP i KxK, i = 1,...,n, P i k = P T i k, R ik = R T i k, P i k positive semidefinite matrices, R i k positive definite matrices, i N. This work was supported by the Saint-Petersburg State University under grant No

3 90 Anna V. Tur Suppose that payoffs are nontransferable. We will assume that the players use feedback strategies u i k,x = M i kx to minimize their costs. Suppose that players agree to use a Pareto-optimal solution as optimality principle and use vector of weights α = α 1,...,α n : n α i = 1, 0 < α i < 1 on their costs to obtain a Pareto-optimal outcome. According to Engwerda, 005 find optimal cooperative strategies of players solving n min α i J i k 0,x 0,u. 3 Let Then J α k 0,x 0,u = u 1,...,u n u α 1,...,u α n = arg min J α k 0,x 0,u = P α k = u 1,...,u n n α i J i k 0,x 0,u, 4 n α i J i k 0,x 0,u, n α i P i k, α 1 R 1 k O... O R α k = O α R k... O O O... α n R n k x T kp α kxk+ukr α kuk+ x T KP α KxK. 5 Finding of Pareto-optimal solution is reduced to linear-quadratic optimal control problem 1-5 with one control variable uk. According to Başar and Olsder, 1999 there exists the unique control in class of admissible {u α i k = Mα i kx, i = 1,...,n}, minimizing Jα k 0,x 0,u, where M α i k i-th block ofmatrix Mα k,{m α k, Θ α k} solution ofsystem ofmatrix equations Ak+BkM α k T Θ α k +1Ak+BkM α k Θ α k+ +P α k+m α k T R α km α k = 0, M α k = R α k+b T kθ α k +1Bk 1 B T kθ α k +1Ak, k = 1,...,K 1, Θ α K = P α K, 6

4 A Time-consistent Solution Formula for Linear-Quadratic Games 91 where Θ α k is symmetric. Here Bk = B 1 k,...,b n k. The cooperative state trajectory x α k one can find by substituting the cooperative strategies {u α i k} in 1 and solving the system: And payoffs of players are: xk +1 = Akxk+Bku α k. 7 Ji α k 0,x 0,u α = x α k T P i kx α k+u α i k T R i ku α i k + +x α K T P i Kx α K. 8. Time-consistency Suppose that there exists such α, that inequalities J α i k 0,x 0,u α V i k 0,x 0, i = 1,...,n. 9 requiring for individual rationality in the cooperative game are satisfied at initial time. Here V i k 0,x 0 is Nash outcome of player i in game Γk 0,x 0. But if there exists k > k 0 such that for some i: J α i k,xα k,u α > V i k,x α k, then time-inconsistency of the individual rationality condition is appear. To overcome the time inconsistency problem in the game with nontransferable payoffs the notion of Payoff Distribution Procedure PDP was introduced by L.A. Petrosyan The solution formula for the payoff distribution procedure in a cooperative differential game with nontransferable payoffs that leads to a time consistent outcome is proposed by D.W.K. Yeung and Leon A. Petrosyan 014. In this paper the PDP and time-consistency of Pareto-optimal solution are detailed for linear-quadratic discrete-time dynamic games. Definition 1. Vector βk = β 1 k,...,β n k, k 0 k K 1 is a PDP Petrosyan, 1997 if x α k T P i kx α k+u α i k T R i ku α i k = β i k, i = 1,...,n. Definition. Pareto-optimal solution is called time-consistent Petrosyan, 1993; 1997 if there exists a PDP such that the condition of individual rationality is satisfied k=l β i k+x α K T P i Kx α K V i l,x α l, 10 l,k 0 l K, i = 1,...,n, 11 where V i l,x α l is Nash outcome of player i in subgame Γl,x α l.

5 9 Anna V. Tur According to Başar and Olsder, 1999 if Nash equlibrium {u NE i = Mi NE kx, i = 1,...,n} exists in game Γl,x α l, then it can be found by solving the system of matrix equations Ak + n B i kmi NE k T Θ i k +1Ak+ Θ i k+p i k+mi NE k T R i kmi NE k = 0, Mi NE k = R i k+bi T kθ ik +1B i k 1 Bi T kθ ik +1 Ak+ B j kmj NE k, k = k 0,...,K 1, j i Θ i K = P i K, i = 1,...,n. n B i kmi NE k And J i k,x α k,u NE = x α k T Θ i kx α k, i = 1,...,n. In Yeung and Petrosyan, 014 the formula for PDP, which guarantees a timeconsistency in differential game with nontransferable payoffs, is considered. The following theorem gives an analog of this formula. Theorem 1. Let inequalities J α i k 0,x 0,u α V i k 0,x 0, i = 1,...,n, are satisfied for some Pareto-optimal solution. Then PDP βk computed by formula β i k = Jα i k 0,x 0,u α V i k 0,x 0 V i k +1,x α k +1+V i k,x α k 1 i = 1,...,n, k = k 0,...,K 1 13 guarantees time-consistency of this Pareto-optimal solution along the cooperative trajectory x α k. Proof. Show that βk is a PDP: β i k = Ji αk 0,x 0,u α V i k 0,x 0 V i k +1,x α k +1+V i k,x α k = = Ji α k 0,x 0,u α V i k 0,x 0 V i K,x α K+V i k 0,x 0 = = x α k T P i kx α k+u α i k T R i ku α i k + +x α K T P i Kx α K x α K T P i Kx α K = = x α k T P i kx α k+u αi kt R i ku αi k. 14

6 A Time-consistent Solution Formula for Linear-Quadratic Games 93 So βk satisfies definition 1. Now show that the condition of individual rationality is satisfied. Using 13 we obtain k=l k=l β i k = Ji αk 0,x 0,u α V i k 0,x 0 V i k +1,x α k +1+V i k,x α k = K l J α i k 0,x 0,u α V i k 0,x 0 V i K,x α K+V i l,x α l. 15 We can see that in 15 K l J α i k 0,x 0,u α V i k 0,x 0 0. Taking into account the system 1 and the fact, that matrices P i K are positive semidefinite defined, we have so V i K,x α K = x α K T P i Kx α K 0, K l J α i k 0,x 0,u α V i k 0,x 0 V i K,x α K+V i l,x α l It means that k=l β i k+x α K T P i Kx α K V i l,x α l,.1. Irrational Behavior Proof Condition = V i l,x α l. l,k 0 l K, i = 1,...,n. The condition under which even if irrational behaviors appear later in the game the concerned player would still be performing better under the cooperative scheme was considered in Yeung, 006. The irrational behavior proof condition for differential games with nontransferable payoffs is proposed in Belitskaia, 01. In this paper this condition is concretized for linear-quadratic discrete-time dynamic games with nontransferable payoffs. Definition 3. Pareto-optimal solution J α 1 k 0,x 0,u α,..., J α nk 0,x 0,u α satisfies the irrational behavior proof condition Yeung, 006 in the game Γk 0,x 0, if the following inequalities hold l β i k+v i l+1,x α l+1 V i k 0,x 0, i = 1,...,n 16

7 94 Anna V. Tur for k 0 l K 1, where βk = β 1 k,...,β n k is time-consistent PDP of J α 1 k 0,x 0,u α,..., J α n k 0,x 0,u α. So if for all i = 1,...,n the following inequalities holds β i k+v i k +1,x α k +1 V i k,x α k 0, k k 0, 17 then the Pareto-optimal solution satisfies the irrational behavior proof condition. Theorem. If in linear-quadratic discrete-time dynamic games with nontransferable payoffs for some Pareto-optimal solutions and its PDP the following inequalities hold β i k+x α k Ak+BkM T α k T Θ i k +1Ak+BkM α k Θ i k x α k 0, k 0 k K 1, 18 where M α k solution of the system 6, Θ i k solution of the system 1, x α k cooperative optimal trajectory, then the irrational behavior proof condition for this Pareto-optimal solutions is satisfied. Proof. The proof of the theorem follows directly from the condition 17, system 1 and the state equation 1. Proposition 1. Let inequalities J α i k 0,x 0,u α V i k 0,x 0, i = 1,...,n, are satisfied for some Pareto-optimal solution and PDP βk of this solution is calculated using formula 13, then the irrational behavior proof condition for this Pareto-optimal solution is satisfied. Proof. If then β i k = Jα i k 0,x 0,u α V i k 0,x 0 V i k +1,x α k +1+V i k,x α k, β i k+v i k +1,x α k +1 V i k,x α k = Jα i k 0,x 0,u α V i k 0,x 0, where Jα i k0,x0,uα V ik 0,x 0 K k 0 0 for all i = 1,...,n. So β i k+v i k +1,x α k +1 V i k,x α k 0 and according to the theorem it guarantees that irrational behavior proof condition for this Pareto-optimal solution is satisfied.

8 A Time-consistent Solution Formula for Linear-Quadratic Games Application to a Monetary and Fiscal Regulation Dynamic Game As an example consider the government debt stabilization game van Aarle, Bovenberg and Raith, Assume that government debt accumulation, dk, is the sum of interest payments on government debt, rdk, and primary fiscal deficits, fk, minus the seignorage i.e. the issue of base money mk. So, dk +1 = r+1dk+ft mt, d0 = d 0, here r > 0 the interest rate. The objective of the fiscal authority is: J 1 = k=0 k 1 fk f +ηmk m +λdk d. The objective of monetary authorities is: J = k=0 k 1 mk m +γdk d. FollowingRincon-Zapatero et. al., 000 we consider the finite-horizon game, where players wish to minimize the deviations from fixed target f, m, d. We suppose that the two institutions wish a balanced budget, that is to say, rd+f m = 0. In this case the game can be formulated as follows: k 1 x 1 k = dk d, 1 x k = d 1 u 1 k = u k = Then our system can be rewritten as r +1 A = 0 J i = 1 1 k+1 k fk f, k 1 mk m. xk +1 = Axk x T kp i xk+, B i u i k,, B 1 = 1 0 1,B = 1 u T j kr ij u j k, i = 1,...,n, j=1 0 1,

9 96 Anna V. Tur P 1 = λ 0,P 0 0 = γ 0, R = 1, R 1 = η, R 1 = 0, R = 1. Following Basar and Olsder, 1999 to find the Nash equilibrium we solve the system A+ B i Mi NE k T Θ i k +1A+ Θ i k+p i +Mj NE k T R ij Mj NE k+ +Mi NE k T R ii Mi NE k = 0, Mi NE k = R ii +Bi T Θ i k +1B i 1 Bi T Θ i k +1 A+B j Mj NE k, j i. Θ i K = 0, i = 1, B i Mi NE k 1 1 For λ = 1,η =, = 1 4,r = 0,3,γ =,α = ,K = 9,k 0 = 0,d 0 = 100, d 99 = 1,x 0 = tables 1 and give us the decision of this system. 1/4 Table 1: Θ 1k k k = 0 k = 1 k = k = 3 k = 4 k = 5 k = 6 k = 7 k = 8 Θ 1k

10 A Time-consistent Solution Formula for Linear-Quadratic Games 97 Table : Θ k k k = 0 k = 1 k = k = 3 k = 4 k = 5 k = 6 k = 7 k = 8 Θ k ]] We solve the following system to find the Pareto solution A+BM α k T Θ α k +1A+BM α k Θ α k+ +P α +M α k T R α M α k = 0, M α k = R α +B T Θ α k +1B 1 B T Θ α k +1A, k = 1,...,K 1, Θ α K = 0. Tables 3 and 4 give Mi α k decision of system. We can find the corresponding optimal trajectory x α k in table 5. k Table 3: M α 1 k M α 1 k k = k = k = k = k = k = k = k =

11 98 Anna V. Tur k Table 4: M α k M α k k = k = k = k = k = k = k = k = k Table 5: x α k T x α k T k = ,5 k = k = k = k = k = k = k = k = k = It can be shown that J α 1 k 0,x 0,u α = , J α k 0,x 0,u α = We can see that for the chosen value of parameter α at the moment k = 0 conditions 9 are satisfied J α 1 k 0,x 0,u α V 1 k 0,x 0 = , J α k 0,x 0,u α V k 0,x 0 = , J α i k 0,x 0,u α V i k 0,x 0, i = 1,, But J α 1 5,x α 5,u α > V 1 5,x α 5, i = 1,, It means, that time-inconsistency of the individual rationality condition is appear. To avoid this problem, use PDP, calculated by formula 13:

12 A Time-consistent Solution Formula for Linear-Quadratic Games 99 k β 1k β k k = k = k = k = k = k = k = k = k = References Aarle, B. van, Bovenberg, L. and Raith, M Monetary and fiscal policy interaction and government debt stabilization. Journal of Economics, 6, Başar T. and Olsder G. J Dynamic Noncooperative Game Theory, nd edition. Classics in Applied Mathematics, SIAM, Philadelphia. Belitskaia, A. V. The D.W.K. Yeung Condition for Cooperative Differential Games with Nontransferable Payoffs. Graduate School of Management, Contributions to game theory and management, 5, Engwerda, J. C LQ Dynamic Optimization and Differential Games. Chichester: John Wiley Sons, 497 p. Petrosyan L. A Differential Games of Pursuit. World Scientific, Singapore. Petrosyan L. A Agreeable Solutions in Differential Games. International Journal of Mathematics, Game Theory and Algebra, 7, Petrosyan, L. A. and N. N. Danilov 198. Cooperative differential games and their applications. Izd. Tomskogo University, Tomsk. Rincon-Zapatero, J. P., Martin-Herran, G., Martinez, J Identification of Efficient Subgame-Perfect Nash Equilibria in a Class of Differential Games. Journal of Optimization Theory and Applications, 1041, Yeung, D. W. K An Irrational-Behavior-Proof Condition in Cooperative Differential Games. IGTR, 81, Yeung, D. W. K. and L. A. Petrosyan 014. A Time-consistent Solution Formula for Bargaining Problem in Differential Games. Int. Game Theory Rev., 164,