Coalitional solutions in differential games

Size: px

Start display at page:

Download "Coalitional solutions in differential games"

Aleesha Reynolds
6 years ago
Views:

1 2012c12 $ Ê Æ Æ 116ò 14Ï Dec., 2012 Operations Research Transactions Vol.16 No.4 Coalitional solutions in differential games Leon A. Petrosyan 1 Abstract The two-stage level coalitional solution for n-person differential game played over the time interval [, is proposed. The paper emerges from the idea that it is natural not to assume that coalitions on the first level can form a grand coalition. At first level the Nash equilibrium in the game played by coalitions is computed. Secondly the value of each coalition is allocated according to the Shapley value. The results are illustrated by an example of n-person differential emission reduction model. Keywords differential game, Hamilton Jacobi Bellman equation, coalitional partition, Shapley value, Nash equilibrium, PMS-value Chinese Library Classification O Mathematics Subject Classification 91A23, 49N90 éü é *Ûƒ 1 Á JÑžm«m[, þn< éüüãé. 31 ãøu/ Œé b g, =uù gž. 31 ã±é Š Û <éü O ŽÙBŸþï ƒ éz é ÂÃUShapleyŠ? 1. n< ~ü. ~f²þã J. ' c éü Hamilton Jacobi Bellman é ShapleyŠ Nashþ ï PMS-Š ã aò O225 êæ aò A23, 49N90 0 Problem statement Consider n-person non zero-sum differential game with objective for i I = {1,..., n} and state dynamics g i [xs, u 1 s,, u n s]e s t0 ds 0.1 ẋs = f[xs, u 1 s,, u n s], x = x Since s does not appear in g i [xs, u 1 s,, u n s], the objective function and the state dynamics, the game 0.1, 0.2 is an autonomous problem. Consider the game Γx with ÂvFÏ: 2012c918F 1. St. Petersburg University, Universitetsky pr. 35, St. Petersburg, , Russia ÏÕŠö Corresponding author, spbuoasis7@peterlink.ru

2 4Ï Coalitional solutions in differential games 87 player set I = {1,, n} min u i for i I = {1,, n} subject to t g i [xs, u 1 s,, u n s]e s t0 ds ẋs = f[xs, u 1 s,, u n s], xt = x. The infinite-horizon autonomous problem Γx is independent of the choice t and dependent only upon the state at the starting time, that is x. Let S 1,, S m be the partition of the player set I such that S i Sj =, m S i = I, S i = n i, m n i = n. Suppose that each player i from I is playing in the interests of coalition S k to which he belongs, trying to minimize the sum of payoffs of its members, i. e. min u i,i S k i S k t g i [xs, u 1 s,, u n s]e s t0 ds, 0.3 i I, k {1,, m}. Define u Sk = {u i, i S k } as strategy of coalition S k. It is supposed that coalitions S k, S k I are playing noncooperatively as individual players with objectives 0.3 and state dynamics 0.2, and players inside the coalition S k as it was mentioned above play cooperatively. 1 Solution of the problem By assumption, each player i S k is playing in the interests of coalition S k. Without loss of generality it can be assumed that S k are acting as players. Then a two-stage solution is proposed. On the first stage the Nash equilibrium for the game with player set M = {S 1,, S m } is computed. The subgame-perfect Nash equilibrium can be calculated with the help of Hamilton Jacobi Bellman equation [1]. The total cost of coalition S k is allocated among players according to Shapley value of the corresponding differential cooperative subgame Γx, S k. The subgame Γx, S k is defined as follows: let S k be the set of players involved in the game Γx, S k, Γx, S k is a differential cooperative game. In the cooperative game Γx, S k, S k M computation of characteristic function is not standard. When the values of characteristic function are computed for the subcoalition K S k, S k M, it is necessary to define the behavior of players outside S k, i. e. players i I \ K. There are two different cases: i S k \ K and i I \ S k. In the first case the left out players stick to their Nash strategies played in the noncooperative game ΓS k between players from S k, and in the second case players stick to their Nash strategies played in the noncooperative game Γx between players from I S k. As result the proposed two-stage solution forms PMS-vector [2]. This implies that PMS-vector is defined by the following way:

3 88 Leon A. Petrosyan 16ò Definition 1.1 Vector PMSx = [PMS 1 x,, PMS n x] is a PMS-vector in the game Γx, if PMS i x = Sh i S k, x, i S k, S k M, where n k k!k 1! Sh i S k, x = [vk, x vk \ {i}, x], 1.1 n k! i K,K S k and n k = S k, k = K, S 1,, S m is the partition of the set I, vk, x is characteristic function defined in the cooperative game Γx, S k, S k M. Time-consistency of the PMS-vector can be investigated similar to [3]. 2 Example: emission cost reduction game The dynamics of the model is proposed by [4]. Let I be the set of countries involved in the game of emission reduction: I = {1, 2,, n}. The game starts at the instant of time from initial state x 0. Emission of player i i = 1, 2,, n at time t, t [,, is denoted u i t. Let xt denote the stock of accumulated pollution by time t. The evolution of this stock is governed by the following differential equation ẋt = u i t δxt, x = x 0, 2.1 where δ denotes the natural rate of pollution absorption, δ > 0. Let denote u i t = u i and xt = x. Let C i u i be the emission reduction cost incurred by country i when limiting its emission to level u i : C i u i t = 2 u it ū i 2, 0 u i t ū i, > 0. D i xt denotes its damage cost D i x = xt, > 0. Both functions are continuously differentiable and convex. The objective function of player i is defined as K i x 0,, u = e t t0 C i u i t + D i xtdt, subject to the equation dynamics 2.1, where u = u 1,..., u n and δ is the common social discount rate. Coalitional model of emission reduction game was considered in the paper [10], in what follows we present the results of this paper. Let S 1, S 2,..., S m be the partition of the set I such that m m S i Sj =, S i = I, S i = n i, n i = n. Suppose that each player i from I is playing in interests of coalition S k, to which he belongs, trying to minimize the sum of payoffs of its members, i.e. min K i u, x 0, = min e t t0 {C i u i t + D i xt}dt, u i,i S k u i,i S k i S K i S K

4 4Ï Coalitional solutions in differential games 89 subject to the equation dynamics 2.1. On the first step we compute the Nash equilibrium in the game played by coalitions supposing coalitions S k are acting as players. The optimization problem for the coalition S k is W S k, x 0, = min u i,i S K i S k K i u, x 0, = min u i,i S K e t t0 i S k C i u i + D i xdt, subject to pollution accumulation dynamics 2.1. Denote by W S k, x, t = W Sk the Bellman function of this problem. To obtain the Nash equilibrium, assuming differentiability of the value function, the system of Hamilton-Jacobi-Bellman equations must be satisfied [ W Sk = min u i,i S k C i u i + D i x + W S k i S k u i t δxt ]. 2.2 Differentiating the right hand side of formulas 2.2 with respect to u i and equating to zero leads to u N i = 1 W Sk, i S k. 2.3 Substituting u N i W Sk in 2.2 we get = n k WSk 2 + x + W Sk 1 2 It can be shown in the usual way that the linear functions satisfies the equation 2.2 see [3]. Now note that m j=1 W Sj δx. 2.4 W Sk = A Sk x + B Sk, k = 1, 2,..., m, 2.5 W Sk = A S k. 2.6 Substituting 2.5 and 2.6 into the formula 2.4 we get the coefficients A Sk and B Sk as follows: A Sk =, B S k = A n m S k n i A S i + n k 2 A S k. 2.7 If we combine 2.3 and 2.7 we get u N i = 1,i k, 2.8 for i I, if i S k. As a result we obtain the total cost of coalition S k, k = 1,..., m, in the following form: W Sk = A S k x + n m + n k 2 A S k. 2.9

5 90 Leon A. Petrosyan 16ò Substituting 2.7 for A SK and A Sj in 2.9, we get n m W Sk = x + l S j + n k Substituting the Nash equilibrium strategy 2.8 and solving equation of dynamics 2.1 we obtain coalitional trajectory x N t = x 0 1 π [ m j n ] i j S i e δt t0 + 1 [ π m j ] n i j S i δ δ Compute now the characteristic function. Recall that the total cost of coalition S k is allocated among players according to the Shapley value. Similarly we have to find the characteristic function for the game Γx 0, S k and the Shapley value. Computation of the characteristic function of this game isn t standard. When the characteristic function is computed for the coalition K S k, the left-out players from S k \ K stick to their Nash strategies players from the set I \ S k stick to strategies defined by 2.8. Compute now the Nash equilibrium in the game Γx 0, S k. We can easily do it. Each player seeks to minimize a stream of discounted sum of emission reduction cost and damage cost. We have following system of optimization problems: W {i}, x 0, = min u i K i u, x 0, = min u i e t t0 {C i u i + D i x}dt, i S k, 2.12 subject to equation dynamics 2.1. The value function W i = W {i}, x, t of system of dynamic programming problem 2.12 must satisfy the following system of Hamilton-Jacobi- Bellman equations: [ W i = min u i C i u i + D i x + W i u i δxt ], i S k Differentiating the right hand side 2.13 with respect to u i and equating to zero leads to the following Nash emission strategy: u n i = 1 W i, i S k Recall that u n i = un i, i S j, i / S k, where u N i is given by the formula 2.8. Substituting 2.14 and 2.8 in 2.13 we obtain W i = 1 2 Wi 2 + πi x + W i Taking into account 2.5 we get m A i x + B i = 1 n 2 A2 i + x + A i 1 m W j δxt, i S k. 1 A S k δx.

6 4Ï Coalitional solutions in differential games 91 It follows easily that By assumption, A i =, B i = A i n 1 m j= A i W i = A i x + B i, i S k. So we have W i = x + n 1 m j=1 l S j l S i Compute now the outcomes for all remaining possible coalitions in the game Γx 0, S k. The characteristic function for the intermediate coalition L S k is computed by the solution of the following optimization problem: W L, x 0, = min u i,i L K iu, x 0, = e t t0 i L,L S k C i u i + D i xdx, 2.17 subject to the equation dynamics 2.1. Let W L = W L, x, t be the Bellman function of the problem The solution of the problem 2.17 is equivalent to the solution of the following Hamilton-Jacobi-Bellman equation: [ W L = min C i u i + D i x + W L u i δx ] u i,i L t i L Suppose the players from I\S k stick to 2.8 and the players from S k \L stick to Differentiating the right hand side of expression 2.18 subject to u i, i L, we get the strategies u L i i L. Substituting u L i, un i, un i W L = l 2 WL m u L i = 1 in the formula 2.18 leads to 2 + x + W L l j L 1 where l = L. Combining this with 2.5 and 2.6, we get: W L, 2.19 W L A i δxt, L S k, A L x + B L = l 2 A2 L + m x + A L j L 1 A i l A L δxt, L S k.

7 92 Leon A. Petrosyan 16ò It can be easily checked that W L = A L x + B L, L S k, where j L A L =, B L = A L n m 1 A i l A L. As a result we get j L W L = x + n m l S j 1 l S i l l L The characteristic function of the game Γx 0, S k is given by the following formula: 0, K = ; W V K, x 0 = i x 0, K = {i}; W Sk x 0, K = S k ; W L x 0, K = L. Where W i x is given by 2.16, W L x is given by 2.20, W Sk x is given by 2.9. There is a problem which is connected with the approach used for the computation of the characteristic function. The characteristic function isn t superadditive in general. Thus superadditivity of the characteristic function 2.21 in this special case is proved by N. Kozlovskaya in [5-10]. To be definite, denote A K = i K Hence V K, x is given by formula 0, K = ; A i n x + 1 m A i, K = {i}; j=1 V K, x = A Sk n m x + + n k 2 A S k, K = S k ; A L n m x + 1 A i l A L, K = L. Compute now the Shapley Value of the game Γx 0, S k. The value Shapley is computed by formula n k k!k 1! [ ] Sh i S K, x 0 = V K, x 0 V K\{i}, x 0. K i,k S k n k! Consider the difference V K, x 0 V K\{i}, x 0 = A K A K\{i} x + B K B K\{i},

8 4Ï Coalitional solutions in differential games 93 we obtain A K A K\{i} = After some calculations we get m B K B K\{i} = ū j j I + 1 S k \K\{i} A K\{i} m = ū j j I 2 πi K\{i} k. n j A K S k \K n j Transition from summing sets to summing powers leads to = = Analogously we obtain = K i n k!k 1! n! As a result we get K i n k k=1 n k n k k=1 n k k!k 1! n k! n k k!k 1! n k! 1 ū i x = B K B K\{i} Sh i S k, x 0 = A i x + A i k 1A 2 K\{i} ka2 K πi 2 2k S k \K A K A K\{i} x n k 1! k 1!n k k! x x n 3 m n A2 + 2n k Sk 3 A ia Sk n k 6 A2 i πi A 2 j K\{i} π 2 j Construct now the PMS-vector. We obtained the Shapley Value 2.22 for any game Γx 0, S k, where S 1, S 2,, S m is a coalitional partition of the set of players I. Taking into account definition 1.1, we obtain the formula for PMS-vector PMSx 0 = PMS 1 x 0, PMS 2 x 0,, PMS n x 0,.

9 94 Leon A. Petrosyan 16ò where PMS i x 0 = Sh i S k, x 0 if i S k, see 2.22: PMS i x 0 = xn t + j I l S k n k πi n k 3 ū j m l S k πj 2, i S k, l S j where coalitional trajectory x N t is given by the formula 2.11 see [11]. References [1] Albizur M, Zarzuelo J. On coalitional semivalues [J]. Games and Economic Behaviour, 2004, 2, [2] Bloch F. Sequantal formation of coalitions with fixed payoff division [J]. Games and Economic Behaviour, 1966, 14, [3] Dockner E J, Jorgensen S N, van Long, et al. Differential Games in Economics and Management Science [M]. Cambridge: Cambridge University Press, 2000, [4] Filar J A, Gaertner P S. A regional allocation of wourld CO 2 emission reductions [J]. Mathematics and Computers in Simulations, 1997, 43, [5] Kozlovskaya N, Petrosyan L, Zenkevich N. Coalitional Solution of a Gema-Theoretic Emission Reduction Model [J]. International Game Theory Review, 2010, 123: [6] Owen G. Values of games with a priory unions [M]// Mathematical Economy and Game Theory Berlin, 1997, [7] Petrosyan L. Differential Games of Pursuit [M]. World Sci. Pbl, [8] Petrosyan L, Zaccour G. Time-consistent Shapley value allocation of pollution cost reduction [J]. Journal of Economic Dynamics and Control, 2003, 27, [9] Petrosyan L, Mamkina S. Dynamic games with coalitional structures [J]. International Game Theory Review, 2006, 82: [10] Yeung D, Petrosyan L. Subgame Consistent Economic Optimization [M]. Birkhauser, 2012, 392. [11] Kaitala V, Pohjola M. Sustainable international agreements on green house warming: a game theory study [J]. Annals of the International Society of Dynamic Games, 1995, 2,

Political Economy of Institutions and Development: Problem Set 1. Due Date: Thursday, February 23, in class.

Political Economy of Institutions and Development: 14.773 Problem Set 1 Due Date: Thursday, February 23, in class. Answer Questions 1-3. handed in. The other two questions are for practice and are not