Hyperbolicity of Systems Describing Value Functions in Differential Games which Model Duopoly Problems. Joanna Zwierzchowska

Decision Making in Manufacturing and Services Vol. 9 05 No. pp. 89 00 Hyperbolicity of Systems Describing Value Functions in Differential Games which Model Duopoly Problems Joanna Zwierzchowska Abstract. Based on the Bressan and Shen approach Bressan and Shen, 004; Shen, 009, we present an extension of the class of non-zero sum differential games for which value functions are described by a weakly hyperbolic Hamilton Jacobi system. The considered value functions are determined by a Pareto optimality condition for instantaneous gain functions, for which we compare two methods of the unique choice Pareto optimal strategies. We present the procedure of applying this approach for duopoly. Keywords: duopoly models, semi-cooperative feedback strategies, Pareto optimality, hyperbolic partial differential equations Mathematics Subject Classification: 9A, 49N70, 49N90, 5L65 Revised: February, 05. INTRODUCTION Dynamic models describe situations in which two or more players make their decisions about their own behavior under the same circumstances. In this paper, we shall consider games with a finite duration of time. We shall be interested in solving theoretical maximizing problems that can be applied to finding better strategies in models of duopoly. Our effort is focused on finding a better solution than the Nash equilibrium. On the one hand, we want the solution to provide greater payoffs for both players, but also we want to obtain a well-posed system of PDEs describing value functions. We assume that the evolution of state is described by the following differential equation: ẋ = fx + φxu + ψxu with initial data: xτ = y R m Nicolaus Copernicus University in Torun, Faculty of Mathematics and Computer Sciences, Torun, Poland, e-mail: joanna.zwierzchowska@mat.umk.pl DOI: http://dx.doi.org/0.7494/dmms.05.9..89 89

90 J. Zwierzchowska where f : R m R m, φ, ψ : R m M m n R and u i are feedback strategies they depend on time t and state xt, i =,. The goal of the i-th player is to maximize his payoff function; i.e., where and J i τ, y, u, u = g i xt T τ h i xt, u i tdt a terminal payoff g i : R m R is a non negative and smooth function 4 a running cost h i : R m R n R is a smooth function such that h i x, is strictly convex for every x R m 5 We consider the instantaneous gain functions: Y x, p, p, u, u = p F x, u, u h x, u Y x, p, p, u, u = p F x, u, u h x, u 6 where F x, u, u is the right side of the dynamic and the dot denotes the scalar product in, we have F x, u, u = fx+φxu +ψxu. Fixing x, p, p R m and s > 0, we can find Pareto optimal choices Ui P x, p, p, s for the static game Y i x, p, p,,, i =,, in the following way: if u P, u P is the maximum of the combined payoff Y s = sy +Y, then the strategies u P, u P give Pareto optimal payoffs in game Y, Y. As a result, strategies Ui P depend on s. We choose a smooth function sx, p, p and define feedback strategies Ui sx, p, p = Ui P x, p, p, sx, p, p for the problem. Such strategies are called semi-cooperative Bressan and Shen, 004. If functions: Vi s τ, y = J i τ, y, U s, U s are smooth enough, then they satisfy the following system: { V,t + H x, x V, x V = 0 with the terminal data: V,t + H x, x V, x V = 0 V T, x = g x and V T, x = g x 8 where the Hamiltonian functions are given by: H i x, p, p = Y i x, p, p, U s x, p, p, U s x, p, p and they depend on s. Functions V s i are usually called the value functions. We say that u P, up is a pair of Pareto optimal choices for the game, which is given by payoff functions Y i u, u i =,, if there exists no pair u, u such that Y u, u > Y u P, up and Y u, u > Y u P, up. This means that no pair of admissible strategies exists that improve both payoffs simultaneously. 7

Hyperbolicity of Systems Describing Value Functions... 9 In Section, we shall prove that system 7 is weakly hyperbolic and is hyperbolic except for some curves on the p, p -plane see Theorem.. If the system is hyperbolic, then it is well-posed and this fact is crucial for numerical solutions for more details, see Serre, 000. This result is a generalization of Theorem from Bressan and Shen, 004. In that paper, it is shown that, if we consider the dynamic: ẋ = fx + u + u, 9 then system 7 is weakly hyperbolic and is hyperbolic except some curves on the p, p -plane. In Section, we compare two methods of stating a Pareto optimum for functionals Y, Y. In both cases, the referential point is a Nash equilibrium payoff Y N, Y N, and we require Pareto optimal outcomes to be greater than those for the Nash equilibrium: Y P i Y N i for i =, Obviously, the above criterion does not determine Pareto optimal strategies uniquely. Bressan and Shen 004 receive the uniqueness of Pareto optimal choices by using the following condition: Y P Y N = Y P Y N The second criterion of choosing Pareto optimal strategies is based on the concept of the Nash solution to the bargaining problem see Nash, 950. The pair Ỹ P, Ỹ P is such a solution if: Ỹ P Y N Ỹ P Y N Y P Y N Y P Y N for every Y P, Y P. The above condition can be reformulated using function s: sx, p, p = arg max {Y P s Y N Y P s Y N } s>0 The main advantage of the second approach is that, considering the dynamics such as the Lanchester duopoly model used in Chintagunta and Vilcassim, 99 and Wang and Wu, 00 in which the dynamic is given by the equation: ẋ = u x u x 0 and the duopoly model from Bressan and Shen, 004 given by the formula: ẋ = x xu u it is possible to compute function s analytically, as we shall study in Section, and determine the system 7 effectively. In view of Theorem., the obtained systems are hyperbolic except for some curves on the p, p -plane. A natural consequence of the above result should be solving numerically received systems and using them to construct semi-cooperative strategies for empirical examples of a duopoly. Unfortunately, we have no ready algorithms for such problems at the moment. Although, the situation seems not to be hopeless. Hamilton Jacobi systems

9 J. Zwierzchowska can be transformed into systems of conservation laws, and for such problems, there exist numerical algorithms. For now, a numerical solvability will not be the subject of this paper. The Nash equilibrium is the most common approach to the problem of maximizing payoff for the dynamic given by and. As Bressan and Shen 004 show, in general, such an approach leads to unstable systems of partial differential equations. Therefore, we shall use a Pareto optimality condition. Moreover, our result is not only a theoretical generalization of the Bressan and Shen dynamic, because 9 is not sufficient for empirical research. Let us notice that duopoly models 0 and are not of the 9 form, but they are of the form.. PARETO OPTIMAL CHOICES THE MAIN RESULT First, we recall the basic definitions and facts concerning the hyperbolicity of linear and nonlinear systems of PDEs. One can find details in Bressan and Shen, 004; Serre, 000. We consider a linear system on R m with constant coefficients: V t + m A α V xα = 0 α= where t is time, x R m, V : R R m R k. Let us notice that k corresponds to the number of players in a game and m is the dimension of the state space. We define the linear combination: m Aξ = ξ α A α where ξ R m. Definition.. System is hyperbolic if there exists a constant C such that where: α= sup exp iaξ C ξ IR m exp iaξ = iaξ n Definition.. System is weakly hyperbolic, if for every ξ R m, the matrix Aξ has k real eigenvalues λ ξ,..., λ k ξ. n=0 In Bressan and Shen, 004, it is shown that the initial value problem for system is well-posed in L R m if and only if the system is hyperbolic. We have the following necessary condition of hyperbolicity. Lemma.. If system is hyperbolic, then it is weakly hyperbolic. n!

Hyperbolicity of Systems Describing Value Functions... 9 The next result refers to the one-dimensional case, when system takes the form: V t + AV x = 0 Lemma.. System is hyperbolic if and only if the matrix A admits a basis of real eigenvectors. It is also easy to see that the following statement is true. Remark. Let A M R. The matrix A has two real eigenvalues if and only if A A + 4A A 0. Moreover, if A A + 4A A > 0, then the eigenvectors span the space R. In view of Lemma., it is reasonable to check the weak hyperbolicity in the first place. We mainly receive nonlinear systems, so it is necessary to understand what the hyperbolicity means in this case. Consider the system of Hamilton Jacobi equations: V i t + H i x, V x,..., V k x = 0 i =,..., k 4 The linearization of 4 takes the following form: V i t + [ ] Hi x, p, p,..., p k V j = 0 p j,α jα x α i =,..., k 5 where x, p, p,..., p k R +km and p i = V i x. If we denote: then equations 5 are of the form. A α ij := H i p jα x, p, p,..., p k 6 Definition.. The nonlinear system 4 is hyperbolic weakly hyperbolic on the domain Ω R +km, if for every x, p, p,..., p k Ω its linearisation 5 is hyperbolic weakly hyperbolic. Due to the fact that we are interested in solving empirical problems in a duopoly and applying numerical methods, we need to know that our systems have a unique solution, and this solution s behavior changes continuously with the initial conditions. For this reason, hyperbolicity is crucial. Our aim is to study the hyperbolicity of a system of Hamilton Jacobi equations describing value functions generated by a Pareto optimality condition for instantaneous gain functions. The evolution of the state is described by with the initial data given by. The goal of the i-th player i =, is to maximize his payoff function, where g i and h i satisfy the assumptions 4, 5. We shall consider instantaneous gain functions: Y x, p, p, u, u = p fx + φxu + ψxu h x, u Y x, p, p, u, u = p fx + φxu + ψxu h x, u

94 J. Zwierzchowska Fixing x, p, p R m and s > 0, we can find Pareto optimal choices Ui P x, p, p, s for static game Y i x, p, p,,, i =,, in the following way: if u P, u P is the maximum of function Y s = sy + Y, then strategies u P, u P give Pareto optimal payoffs in game Y, Y. This is the reason why strategies Ui P depend on s. We choose a smooth function sx, p, p and define the semi-cooperative feedback strategies: U s i x, p, p = U P i x, p, p, sx, p, p i =, 7 We define the Hamiltonian functions as follows: H x, p, p = Y x, p, p, U s x, p, p, U s x, p, p If value functions: H x, p, p = Y x, p, p, U s x, p, p, U s x, p, p V s i τ, y = J i τ, y, U s, U s i =, are smooth, then they satisfy the system of Hamilton Jacobi equations: { V,t + H x, x V, x V = 0 V,t + H x, x V, x V = 0 8 Theorem.. Consider problem 5. As gradients p, p of the value functions range in open region Ω R m, assume that the players adopt Pareto optimal strategies of form 7 for some smooth function s = sx, p, p. Then, system 8 is weakly hyperbolic on domain Ω. Moreover, if we consider one-dimension case m =, system 8 is hyperbolic except for some curves on the p, p -plane. The method of proof is similar to the proof of Theorem in Bressan and Shen, 004. Proof. We define functions k i : R m R n i =, as follows: and: k ξ = k ξ, v, x = arg max ω R n{ξfx + φxω + ψxv h x, ω} k ξ = k ξ, v, x = arg max ω R n{ξfx + φxv + ψxω h x, ω} where x R m and v R n. Since h, h are smooth functions that satisfy 5, one can observe that h u x, k ξ = ξφx and h u x, k ξ = ξψx 9 We seek Pareto optimal choices by maximizing function Y s = sy + Y. In view of 9, we can formulate Pareto optimal strategies using functions k and k : u P x, p, p, s = k p + p s and up x, p, p, s = k sp + p 0

Hyperbolicity of Systems Describing Value Functions... 95 The necessary condition for a local maximum implies that: Denoting: s Y u + Y u = s Y u + Y u = 0 Y P i = Y i x, p, p, u P x, p, p, s, u P x, p, p, s i =, and recalling, we obtain the following equality: Y P = s Y P Now, we compute the linearization of system 8. From 0, we get: Y P = p fx + φxk p + p + ψxk sp + p h x, k p + p s s Y P = p fx + φxk p + p + ψxk sp + p h x, k sp + p s To clarify further computations, let us temporarily assume that m = n = and that s = const. The linearization takes the following form: [ f + φk + ψk + p φk + sψk h k p s φk + ψk s h k ] p φk + sψk sh k f + φk + ψk + p s φk + ψk h k where: h = h u and h = h u Let a := ψp k s φp k. We can write matrix as follows: where: [ ] f + φk + ψk A := + sa a s = f + φk a f + φk + ψk sa + ψk I + A I = [ ] 0 0 and A = [ ] sa a s a sa In view of Remark, it is obvious that A is weakly hyperbolic. Now let s = sx, p, p. In this situation, the linearization matrix is the following: A = f + φk + ψk I + A + A where: A = Y P Y P p p Y P Y P p p

96 J. Zwierzchowska Using and denoting c := Y P p, d := Y P A = [ ] c d sc sd p, we obtain: From Remark, it is easy to verify that matrix A has two real eigenvalues: λ = f + φk + ψk and λ = f + φk + ψk + c sd; thus, the system is weakly hyperbolic. Furthermore, the system is hyperbolic when c sd and p i 0 for i =,. Now let m, n N. We need to verify if matrix: Aξ = is weakly hyperbolic where, as in 6, A α = m ξ α A α α= [ ] Hi x, p, p p jα i,j= Repeating the reasoning for α s coordinate of p and p, we receive: A α = f α + φk α + ψk α I + A α + A α where: [ ] [ ] A saα a α = α, A cα d sa α sa α = α α sc α sd α and a α = ψdk p α s φdk p α, c α = Y P that matrix Aξ has the following form: Aξ = p α, d α = Y P m ξ α A α = ξ f + ξ φk + ξ ψk I + A ξ + A ξ α= p α. This means where: A ξ = [ ] [ ] ξ sa ξ a ξ s, A ξ c ξ d ξ = a ξ sa ξ sc ξ sd Matrix Aξ has the two real eigenvalues: λ ξ = ξ f + φk + ψk and λ ξ = ξ f + φk + ψk + c sd. Remark. If s is constant, then our problem becomes a cooperative game, and there is no guarantee that Pareto optimal payoffs dominate Nash payoffs. Such dominance is crucial for our considerations, because we want to improve outcomes in a reasonable way.

Hyperbolicity of Systems Describing Value Functions... 97. THE UNIQUENESS OF THE PARETO OPTIMAL CHOICES The choice of Pareto optimal strategies is a very important issue. Since Pareto optimal outcomes are not unique, we present two meaningfully different criteria. In this section, we compare the Bressan and Shen criterion 004 with the criterion proposed by us, which is based on the Nash solution to the bargaining problem. Finally, we determine Pareto optimal solutions for two duopoly models. Bressan and Shen formulate the choice of s basing on the fairness conditions: and: Yi P s > Yi N for i =, 4 Y P s Y N = Y P s Y N 5 Condition 4 is necessary to receive better outcomes than the Nash equilibrium ones, and it is essential to convince players to use a Pareto optimal approach. Unfortunately, conditions 4, 5 are not easy to apply in the examples. Accordingly, we suggest using the Nash solution to the bargaining problem. Firstly, the choice should not make the payoffs worse: Yi P s Yi N for i =, 6 Pair Ỹ P, Ỹ P is the Nash solution to the bargaining problem if: Ỹ P Y N Ỹ P Y N Y P Y N Y P Y N for every Y P, Y P In the examples, we use the following reformulated form: sx, p, p = arg max {Y P s Y N Y P s Y N } 7 s>0 If the intersection of the image of function Y = Y, Y and set {y, y : y i Y N i, i =, } is convex, then conditions 6, 7 provide the unique s. We compare these two approaches for two dynamics, the Lanchester duopoly model: and the duopoly model from Bressan and Shen, 004: ẋ = u x u x 8 ẋ = x xu u 9 In both cases, state x [0, ] characterizes the market share. We shall use the following payoff function: T [ J i τ, y, u, u = x i T + x i t ] τ u i t dt 0 where x t = xt is the market share of the first company at time t [τ, T ], while x t = xt is the market share of the second. Both methods require comparing new values with Nash equilibrium payoffs Y N, Y N the instantaneous gain functions

98 J. Zwierzchowska for Nash equilibrium strategies u N, u N. The strategies can be found from the following conditions: Y Y = 0 and = 0 u u Example Let us consider the Lanchester duopoly model, which is given by 8, with the payoff function 0. The instantaneous gain functions take form: Y i x, p, p, u, u = p i u x u x + x i u i i =, where x = x and x = x. Using, we obtain that u N = p x. The Nash payoffs are the following: u N = p x and Y N = Y x, p, p, u N, u N = p x + p p x + x Y N = Y x, p, p, u N, u N = p x + p p x + x Now, we find the set of Pareto optimal choices. We maximize function Y s = sy + Y : Y s x, p, p, u, u = sp u x u x+sx s u +p u x u x+ x u. Using necessary condition: Y s u = 0 and Y s u = 0 we receive u P = p + s p x and u P = sp + p x. The respective payoffs are: Y P = p p + s p x + sp + p x + x p + s p x Y P = p p + s p x + sp + p x + x sp + p x Firstly, we shall use fairness condition 4 and 5. Condition 5 allows us to present function s as one of the solutions of the following equation: p x s 4 + p x s p x s p x = 0 Unfortunately, condition does not provide solutions that could be presented in one simple, analytically computed formula. On the other hand, applying conditions 6 and 7 we obtain that the seeking function s is given by the formula: p x sx, p, p = p x 4

Hyperbolicity of Systems Describing Value Functions... 99 The Hamilton functions for 4 are the following: H x, p, p = x p p x p x x + p p + p x p x + x H x, p, p = + + p x x p p x x + p p + p p x + x x The matrix of the linearization of 8, in this case, takes the form: p φ xp p x φ x p 4 p x p + 4p p φx p p4 φ x x + p + p 4 p x φx x p p p φ x x x + p + 4φxp p x p + φx p p4 where φx = x x. p = xv,p = xv Example Let us consider the second duopoly model, which is given in 9. The payoff functions are given in 0; thus, the instantaneous gain functions take the following form: Y i x, p, p, u, u = p i x xu u + x i u i i =, where x = x and x = x. Using, we obtain that u N = p x x with the Nash payoffs: u N = p x x and Y N = x x p + p p + x, Y N = x x p + p p + x Now, we need to find the set of Pareto optimal choices. To do that, we shall maximize function Y s = sy + Y, s > 0 is fixed: Y s x, p, p, u, u = sp x xu u +sx s u +p x xu u + x u Using necessary condition, we get u P = p + s p x x and u P = sp +p x x, and the Pareto optimal payoffs are: [ ] Y P = x x + s p + p p p s + x

00 J. Zwierzchowska [ Y P = x x + p + p p s p ] + x s Applying 5, we get a very similar polynomial as in Example : p s 4 + p s p s p = 0 On the other hand, from conditions 6 and 7, we find that seeking function s is given by the following formula: sx, p, p = p p 5 The Hamilton functions for 5 are the following: H x, p, p = x x p + p p + H x, p, p = x x p + p p + The linearization of the system takes the following form: [ ] V φx p + p + p p φx p + p 4 + p V t φx p + p p4 φx p + p + p p where φx = x x. p p + x p p + x p = xv,p = xv [ ] [ ] V 0 = V 0 x Criteria 6 and 7 provide an effective analytical formula describing Hamilotnian functions H, H in system 8 in both duopoly models. Our next aim is to solve the received systems numerically and to compare the obtained solutions with empirical data. REFERENCES Basar, T., Olsder, G.J., 999. Dynamic Noncooperative Game Theory. SIAM. Bressan, A., Shen, W., 004. Semi-cooperative strategies for differential games. International Journal of Game Theory,, pp. 56 59. Chintagunta, P.K., Vilcassim, N.J., 99. An empirical investigation of advertising strategies in a dynamic duopoly. Management Science, 8, pp. 0 44. Nash, J., 950. The bargaining problem. Econometrica, 8, pp. 55 6. Nash, J., 95. Non-cooperative games. Ann Math, 54, pp. 86 95. Serre, D., 000. Systems of Conservation Laws I, II. Cambridge University Press. Shen, W., 009. Non-Cooperative and Semi-Cooperative Differential Games. Advances in Dynamic Games and Their Applications, Birkhauser Basel, pp. 85 06. Wang, Q., Wu, Z., 00. A duopolistic model of dynamic competitive advertising. European Journal of Operational Research, 8, pp. 6.