Extensive Form Games with Perfect Information

Extensive Form Games with Perfect Information Pei-yu Lo 1 Introduction Recap: BoS. Look for all Nash equilibria. show how to nd pure strategy Nash equilibria. Show how to nd mixed strategy Nash equilibria. Emphasize that if a mixed strategy s i is a best response of i to i, all pure strategies in the support of s i is also a best response to i. In our previous description of a game, i.e. Normal Form Game, a game is a list N; (S i ) i2n ; (u i ) i2n. It is assumed that the game itself is common knowledge. The description does not contain information on timing of movements. But what players know may change as the game proceeds. How will this extra information about timing and what the players know a ect our prediction? Example 1 BoS where Alice moves rst. Now when Bob moves, Bob knows what Alice has chosen. Is this good for Bob or bad? What is Bob s strategy space? Because Bob knows where Alice is when Bob chooses, his choice can depend on Alice s choice. So S B = f(o; O) ; (O; B) ; (B; O) ; (B; B)g. Alice does not know what Bob has chosen when Alice chooses, so S A = fo; Bg. In this game, the payo matrix is Bob (O; O) (O; B) (B; O) (B; B) Alice O (2; 2) (2; 1) (0; 0) (0; 0) B (0; 0) (1; 2) (0; 0) (1; 2) (O; (O; O)), (O; (O; B)) ; (B; (B; B)) are all the pure strategy Nash equilibria. But is it credible that Bob will choose Bar after seeing that Alice has chosen Opera? Given that Alice chooses O, both (O; B) and (B; B) are best responses for Bob. This is why (B; (B; B)) is a NE. This is because given that Alice chooses B, what Bob would do in the situation when Alice chooses O does not a ect Bob s payo. But what 1

Bob would do in that situation is crucial for Alice to decide what to do. We would like to incorporate the information that if Alice chooses O, Bob knows that before he decides what to do. We use a tree to incorporate the information on timing on what players know when they make a move. Example 2 An entrant chooses rst whether to enter the market. After learning the entrant s decision, the incumbant decides whether to ght (F) or accomodate (A) the entrant. If the entrant stays out (Out), then entrant s vnm utility is 1, incumbant s payo is 5. If entrant enters (I) and incumbant ghts (F), then both get 0. If entrant enters and incumbant accomodates, then both get 2. I s strategy set is ff; Ag, and C s strategy set is fe; NgThe payo matrix is Competitor E N Incumbent F 1; 1 2; 0 A 1; 2 2; 0 Both (F; N) and (A; E) are NE of this game. But is it credible that Incumbent will ght once the competitor enters? Game tree. extensive form representation. Strategy of each player. Normal form representation. 2 Extesive Form Representation, Formally An extensive form game incorporate timing information into the description. perfect information: when a player makes a move, she knows all the previous moves made by all players. (Draw game tree for the de nition) De nition 3 it consists of is an Extensive form game with perfect information if 1. N =players 2. H set of histories such that (a) ; 2 H: beginning of the game, initial node (b) if a 1 ; :::; a K 2 H, then a 1 ; :::; a k 2 H for all k K (c) a 1 ; :::a k ; ::: 2 H if a 1 ; :::; a k 2 H for all k (d) Z =the set of terminal histories 3. P : HnZ! N 2

4. u i over Z De nition 4 = N; H; P; (u i ) i2n is nite if H is nite. De nition 5 The action set at history h: A (h) = fa : (h; a) 2 Hg : So when player i moves at h where i = P (h), i chooses among A (h). If P (h) is not necessarily a singleton for every nonterminal history h, (i.e. more than 1 player may move at a history), then we need to add to the description A i (h) for all h 2 HnZ and i 2 P (h). Example 6 BoS where girl moves rst. Game tree. extensive form representation. Strategy of each player. Normal form representation. De nition 7 A strategy of player i 2 N in = N; H; P; (u i ) i2n is a function s i that assigns an action in A i (h) to every h where i makes a move. De nition 8 Outcome function O (s): the terminal history that will happen if i uses strategy s i for all i 2 N. De nition 9 Normal form representation of = N; H; P; (u i ) i2n is N; (S i ) i2n ; (~u i ) i ~u i (s) = u i (O (s)). if it is a NE in the normal form repre- De nition 10 s is a NE of sentation of. 3 Solution Concept 3.1 Backward Induction We want a solution concept that can eliminate the NE which contain threats that are not credible. As the game proceeds, players learn about actions that have been taken. They can then reevaluate their plan. A strategy by player i is credible if the player will still want to stick with it once she learns more about actions that have been taken. In nite extensive form games, we do so by solving the game backwards. Sequential BoS. entrant game three period centipede game. 3

De nition 11 Backward induction solution of an extensive form game with perfect information where only one player moves at a time is a strategy pro le s such that for every i, at every history h 2 H where i moves at h, s i (h) maximizes player i s payo among all her available actions a 2 A i (h) given all players follow s at every history following h. Example 12 (Stackelberg Game.) Cournot competition where rm 1 chooses rst and rm 2 observes rm 1 s output level before it chooses its own. Payo of rm i is i (q i ; q i ) = q i (a q 1 q 2 c). Game tree. A pure strategy of rm 1 is q 1 2 [0; 1). A pure strategy of rm 2 is a function s 2 (q 1 ). Given q 1, expect rm 2 to choose s 2 (q 1 ) that maximizes its payo. So s 2 (q 1 ) solves the FOC 0 = a q 1 2q 2 c. So s 2 (q 1 ) = a c q 1 2. If rm 1 expects rm 2 to do so after any q 1, then a rational rm 1 will choose q 1 to maximize q 1 (a q 1 s 2 (q 1 ) c) = q 1 a q 1 a c q 1 2 c. So rm 1 will choose q1 = a c 2. This strategy pro le is (q1; s 2 (q 1 )) the unique backward solution. And the outcome (q1; s 2 (q1)) = a c; a c 2 4, the history that takes place given this strategy pro le is the backward solution outcome. Example 13 long centipede game. Consider nite extensive form games. Can we possibly have a problem with existence? Proposition 14 (Zerneki s Theorem) Every nite extensive form game with perfect information has a backward induction solution, which is also a pure strategy Nash equilibrium. If no player has the same payo at two di erent terminal nodes, then the backward induction solution is unique. The idea is that we predict that when a player makes a move, she will choose a move that is optimal among her possible moves given what she has learned about the past history, what she expects others will do after each of her possible moves, and she expects others to choose optimally given their choices. 4

3.2 Subgame Perfect Nash Equilibrium What about extensive form games with perfect information with simultaneous moves, i.e. when P : HnZ! 2 N n;? Example 15 Entrant game with Cournot Competition. The competitor decides rst whether to enter. If the competitor enters, he pays an entry cost $1. After it enters, both rms choose quantity simultaneously. Market price is a q 1 q 2 where a > 3. Marginal cost is 0. Competitor s payo is 0 if it does not enter, and is equal to its pro t from the market minus the entry cost of 1 if it enters. The incumbent s payo is simply its pro t from the market. Whether entrant will enter depends on what the entrant expects will happen once the entrant enters. After entrant enters, the situation is like a static cournot competition game. One may then predict that a NE will happen in the cournot competition stage. In a cournot game, there is a unique NE where each player produces qi = a. i s payo under 3 a 2. the cournot game is 3 Expecting this, entrant prefers to enter. good prediction? In a perfect information game, as the game proceeds, actions that have been taken, i.e. the past history, become common knowledge to all the players. The strategic situation going forward is also a "game", which is part of the original game. De nition 16 The subgame of = (N; H; P; (u i )) following history h is (h) = (N; Hj h ; P j h ; (u i j h )) where Hj h is the set of subsequences following h, i.e. Hj h = fh 0 : (h; h 0 ) 2 Hg, P j h (h 0 ) = P (h; h 0 ) and u i j h (h 0 ) = u i (h; h 0 ). Find subgames in the examples we have talked about. De nition 17 s is a SPNE in subgame of. = (N; H; P; (u i )) if it is a NE in every In nite games, one can use backward induction method to nd all the SPNE. Example 18 Entry game with cournot competition. Example 19 Commitments. Alice and Bob Inc. are duopolists in a amrket. They choose quantity simultaneously. D (Q) = 14 Q. Marginal costs are both equal to 2. Before they make production decisions, Alice can choose to install a new technology that reduces MC A to 0. The 5

new technology involves a xed cost of $10. Alice s technology choice is observable to Bob before they choose quantity simultaneously. Write down the game: s A = (a; q A ) 2 S A = fi; Ng [0; 1) s B : fi; Ng! [0; 1): u A ((a; q A ) ; s B ) = (14 q A s B (a) 2 (1 a)) q A 10a u B ((a; q A ) ; s B ) = (14 q A s B (a) 2) (q B ). Consider ^s A = (N; 4) ^s B (1) = 4 = ^s B (0). Show that (^s A ; ^s B ) is a NE. In this NE, ^u A = ^u B = 16. Solution 20 u A = (14 q A q B ) q A 10 u B = (12 q A q B ) q B FOC for A FOC for B r A (q B ) = 14 2 q B r B (q A ) = 12 q A. 2 16 (qa; qb) = 3 ; 10. 3 u A = 256 9 u B = 40 9 < 16. 10 = 166 9 > 16 Remark 21 value of commitment comes from a ecting opponents actions. 6

4 How to nd SPNE For long extensive games, the set of strategies is huge. one-shot deviation principle for nite games and continuous in nite period games h = a 0 ; a 1 :::; a k ; :::. Then de ne h k = a 0 ; a 1 :::; a k De nition 22 " if h k = ^h k. is continuous at in nity if 8" > 0, 9k s.t. u (h) u ^h < De nition 23 Given s i, ^s i is a pro table one-shot deviation at h 2 H for i from s i if ^s i (h 0 ) = s i (h 0 ) for all histories h 0 following h and u i ^s i ; s ijh > u i s i ; s ijh. Theorem 24 Let be an extensive form game that is nite or continuous at in nity. Then s is a SPNE of i there is no pro table one-shot deviation from s for i at any h 2 H for any i 2 I. Proof. If s is a SPNE, then at any history h 2 H, s j h is a NE in the subgame after h, so no player has any pro table deviation at h, so of course no pro table one-shot deviation. To show the other direction, suppose to the contrary that there is no pro table one-shot deviation for any i from s but s is not SPNE. Because s is not a SPNE, 9 a player i and a history ^h such that player i has a pro table deviation in the subgame starting at ^h. Let s rst consider nite games. ^s i 6= s i and ^h such that u i ^s i ; s ij^h > u i s i ; s ij^h. Let s rst consider nite games. Because the set of histories^h nite, we can nd a pro table deviation ^s i to s i in the subgame that deviates from s i at fewest histories. Because all histories are nite, we can nd a history h such that ^s i (h ) 6= s i (h ) but ^s i (h) = s i (h) for all histories h that follow h. Therefore, in the subgame starting from h, ^s i j h is a one-shot deviation from s i j h. By hypothesis, there is no pro table one-shot deviation. So u i ^s i ; s ijh u i s i ; s ijh. De ne ~s i (h) to be equal to ^s i (h) for all history h 6= h and equal to s i (h) at h = h. Then ~s i must does at least as well as ^s i against s i in the subgame after ^h. So ~s i is a pro table deviation to s i which deviate at fewer histories than ^s i, contradiction to the construction of ^s i. So our hypothesis must be false. Now let s consider in nity games continuous at in nity. Because di erences in histories long in the future matter verylittle (continuity at in nity), 9K such that u i s 0 i ; s ij^h > u i s i ; s ij^h and s 0 i ^h; h = 7

^s i ^h; h for any jhj K and s 0 i ^h; h = s i ^h; h for jhj > K. That is, if i can do better by deviating in in nitely many periods, then i can do better by deviating at nitely many periods. We can then use the same argument for nite games. Corollary 25 In an extensive form game with perfect information without simultaneous moves, a backward induction solution is a SPNE. Remark 26 If the game is not continuous at in nity, this principle may not hold. Example. At every history, choose (C; D). u (C; :::C; D) = 0 but u (C; C; ::::C; :::) = 1. Unique SPNE is s where s (h) = D at all h 2 Hn fc; ::::C; ::g. Theorem 27 (Kuhn s Theorem) Every nite extensive form game with perfect information (and maybe simultaneous moves) has a SPNE. Proof. Induction on length of the subgame using backward induction idea. Game is nite, so there is a last period K + 1. At h K, there is only one subgame which is j h K. It is a nite game. There exists a NE (maybe in mixed strategies). If P h K = i, then there exists an action that i can choose that maximizes i s payo among all u i h K ; a where a 2 A h K. De ne s i h K 2 arg max a2a(h K ) u i h K ; a if P h K = fig and s h K to be a NE in the subgame at h K. De ne R h K to be the outcome h K ; s h K. Suppose we have de ned s i (h) and outcome R (h) for all histories that go on to period k + 1 or higher. If only player i moves at h k, i.e. if P h k = fig, de ne s i h k 2 arg max a2a(hk ) u i R h k ; a. De ne R h k = R h k ; s i h k. If there are more than one player that move at h k,.the game N; A i h k ; ~u i (:) where ~u i (a) = u i R h k ; a is a nite strategic form game and by Nash theorem has a NE a. De ne s i h k = a i. By induction, we have de ned a strategy pro le s. By de nition, at every history h, if every player follows s in the subgame after h, the outcome will be R (h). Using one-shot-deviation principle, this is a SPNE because at every history h, if i 2 P (h) chooses a 0 i 6= s i (h), then i get u i (R (h; a 0 i; )) = ~u i (a 0 i). But a = (s i (h)) i2p (h) is a NE in the game N; A i h k ; ~u i (:). Therefore, u i h; a 0 i; a i ; R (h) = ~ui a 0 i; a i ~u i s i (h) ; s i (h) = u i (R (h; s (h))). So there is no pro table one-shot deviation from s i given s i. 8

5 Discussion of the solution concept of SPNE Example 28 Chain store paradox. A chain store (CS) is the incumbent in city k = 1; ::; K. There is one potential entrant in each city. In period k entrant in city k (player k) and CS plays the entrant game. CS and every entrant knows the outcome in every city. Unique SPNE. Example 29 Centipede game. Unique SPNE. strategies are not really a plan of action because some choices early on precludes i from making a move later on but a strategy still speci es what i will do there. interpretations: i s strategy is a belief that others hold about i. In a SPNE, every player other than i holds the same belief and the correct belief about i. rationality, so i chooses a best response against i s belief about what others will do. SPNE: every player chooses a best response against the correct belief about others actions at every history. So every player believes that everyone is rational at every history, even at histories that are inconsistent with backward induction. 5.1 Example Two countries. One rm in each country. Firm in country i chooses h i for home market, e i for export. Price in country i is P i = a h i e j. Government i imposes tari on imports. Firms have constant marginal costs c. Assume that rms want to maximize pro ts. Government want to maximize total welfare: home consumer surplus + home rm s pro ts + tari revenue. Firm i s pro t is Government i s payo is h i (a h i e j c) + e i (a h j e i c). 1 2 (h i + e j ) 2 + h i (a h i e j c) + e i (a h j e i c) + t i (h i + e j ) 5.2 Repeated games D C D 1; 1 5; 0 C 0; 5 4; 4 Example 30 nitely repeated prisoner s dilemma: unique SPNE is (s 1; s 2) where s i (h) = D at all non-terminal history h. 9

D C R D 1,1 5,0 0,0 C 0,5 4,4 0,0 R 0,0 0,0 3,3 Table 1: Table Caption Example 31 Finitely repeated Cournot game. unique SPNE is to produce cournot output at every non-terminal history. Proposition 32 If G has a unique NE, then there exists a unique SPNE where in G (T ) where s i (h) = at every non-terminal history. Proof. backward induction. Example 33 if there are multiple NE in G, a outcome that is non a NE outcome in G may be sustained in a SPNE in the rst few periods of G (T ). Two pure NE (D; D; ), (R; R). In G (2), de ne (s 1; s 2): play C in period 1 play R in period 2 if h = ((C; C)). play D in period 2 otherwise. Use (D; D) to punish bad rst-period behavior. Check the following subgame after (C; C) subgame after h 6= ;; (C; C) subgame at ;. use one-shot deviation principle: 10