Lecture 9. Dynamic games
. Introduction Definition. A dynamic game is a game Γ =< N, x, {U i } n i=, {H i } n i= >, where N = {, 2,..., n} denotes the set of players, x (t) = f (x, u,..., u n, t), x(0) = x 0, x = (x,..., x m ), 0 t T, indicates a controlled system in the space R m, U,..., U n are the strategy sets of players,..., n, respectively, and a function H i (u,..., u n ) specifies the payoff of player i N.
. Introduction A controlled system is considered on a time interval [0, T ] (finite or infinite). Player strategies represent some functions u i = u i (t), i =,..., n. Depending on selected strategies, each player receives the payoff H i (u,..., u n ) = T 0 g i (x(t), u (t),..., u n (t), t)dt+g i (x(t ), T ), i =,..., n Actually, it consists of the integral component and the terminal component; g i and G i, i =,..., n are given functions. There exist cooperative and noncooperative dynamic games. Solutions of noncooperative games are comprehended in the sense of Nash equilibria.
. Introduction Definition 2. A Nash equilibrium in the game Γ is a set of strategies (u,..., u n) such that H i (u i, u i ) H i (u ) for arbitrary strategies u i, i =,..., n.
2. Discrete-time Dynamic Games (Fish Wars) Imagine a certain dynamic system governed by the system of difference equations x t+ = (x t ) α, t = 0,,..., where 0 < α. For instance, this is the evolution of some fish population. The initial state x 0 of the system is given. The system admits the stationary state x =. If x 0 >, the population diminishes approaching infinitely the limit state x =. In the case of x 0 <, the population spreads out with the same asymptotic line.
2. Discrete-time Dynamic Games (Fish Wars) Suppose that two countries (players) perform fishing; they aim at maximizing the income (the amount of fish) on some time interval. The utility function of each player depends on the amount of fish u caught by this player and takes the form log(u). The discounting coefficients δ (player ) and δ 2 (player 2) are given, 0 < δ i <, i =, 2. Find a Nash equilibrium in this game.
First, we study this problem on a finite interval. Take the one-shot model. Assume that players decide to catch the amounts u and u 2 at the initial instant (here u + u 2 x 0 ). At the next instant t =, the population of fish has the size x = (x 0 u u 2 ) α. The game finishes and, by the agreement, players divide the remaining amount of fish equally. The payoff of player makes up H (u, u 2 ) = log u + δ log ( 2 (x u u 2 ) α) = = log u + αδ log(x u u 2 ) δ log 2, x = x 0. Similarly, player 2 obtains the payoff H 2 (u, u 2 ) = log u 2 + αδ 2 log(x u u 2 ) δ 2 log 2, x = x 0.
The functions H (u, u 2 ) and H 2 (u, u 2 ) are convex, and a Nash equilibrium exists. For its evaluation, solve the system of equations H / u = 0, H 2 / u 2 = 0, or αδ = 0, u x u u 2 And so, the equilibrium is defined by u = u 2 αδ 2 ( + αδ )( + αδ 2 ) x, u 2 = αδ 2 x u u 2 = 0. The size of the population after fishing constitutes x u u 2 = αδ ( + αδ )( + αδ 2 ) x. α 2 δ δ 2 ( + αδ )( + αδ 2 ) x. The players payoffs in the equilibrium become H (u, u 2) = (+αδ ) log x +a, H 2 (u, u 2) = (+αδ 2 ) log x +a 2,
where the constants a, a 2 follow from the expressions ( αδ j (α 2 δ δ 2 ) αδ ) i a i = log [( + αδ )( + αδ 2 ) ] +αδ δ i i log 2, i, j =, 2, i j. Now, suppose that the game includes two shots, i.e., players can perform fishing twice. We have determined the optimal behavior and payoffs of both players at the last shot. Hence, the equilibrium in the two-shot model results from maximization of the new payoff functions H 2 (u, u 2 ) = log u + αδ ( + αδ ) log(x u u 2 ) + δ a, x = x 0, H 2 2 (u, u 2 ) = log u + αδ 2 ( + αδ 2 ) log(x u u 2 ) + δ 2 a 2, x = x 0. The Nash equilibrium appears from the system of equations αδ ( + αδ ) = 0, u x u u 2 αδ 2( + αδ 2 ) = 0. u 2 x u u 2
Again, it possesses the linear form: u 2 = αδ 2 ( + αδ 2 ) ( + αδ )( + αδ 2 ) x, u2 2 = αδ ( + αδ ( + αδ )( + αδ 2 ) x. We continue such construction procedure to arrive at the following conclusion. In the n-shot fish war game, the optimal strategies of players are defined by u n = αδ 2 n n (αδ ) j (αδ 2 ) j x, u n (αδ 2 ) j 2 n = j= αδ n n (αδ ) j (αδ ) j x. n (αδ 2 ) j j= ()
After shot n, the population of fish has the size x u n u n 2 = α 2 n δ δ 2 (αδ ) j n (αδ 2 ) j n (αδ ) j n (αδ 2 ) j j= As n, the expressions (), (2) admit the limits u = lim n un = Therefore, αδ 2 ( αδ )x ( αδ )( αδ 2 ), u 2 = x. (2) αδ ( αδ 2 )x ( αδ )( αδ 2 ). where x u u 2 = lim n x un u n 2 = kx, k = α 2 δ δ 2 x ( αδ )( αδ 2 ).
Suppose that at each shot players adhere to the strategies u, u 2. Starting from the initial state x 0, the system evolves according to the law x t+ = (x t u (x t ) u 2(x t )) α = k α x α t = k α (kx α t ) α = k α+α2 x α2 t =... = k t j= α j x αt 0, t = 0,, 2,... Under large t, the system approaches the stationary state x = ( ) α α αδ +. (3) αδ 2 In the case of δ = δ 2 = δ, the stationary state has the form ( ) α x = αδ α 2 αδ.
We focus on the special linear case. Here the population of fish demonstrates the following dynamics: x t+ = r(x t u u 2 ), r >. Apply the same line of reasoning as before to get the optimal strategies: u n = (δ 2 ) j δ 2 n x, u n n (δ ) j (δ 2 ) j 2 n = j= (δ ) j δ n x. n n (δ ) j (δ 2 ) j j= As n, we obtain the limit strategies u = δ 2 ( δ )x ( δ )( δ 2 ), u 2 = δ ( δ 2 )x ( δ )( δ 2 ).
As far as x u u2 x = δ + δ 2, the optimal strategies of the players lead to the population dynamics ( ) t r x t = δ + δ 2 x r t = δ + x 0, t = 0,,... δ 2 Obviously, the population dynamics in the equilibrium essentially depends on the coefficient r/( δ + δ 2 ). The latter being smaller than, the population degenerates; if this coefficient exceeds, the population grows infinitely. And finally, under strict equality to, the population possesses the stable size. In the case of identical discounting coefficients (δ = δ 2 = δ), further development or extinction of the population depends on the sign of δ(r + ) 2.