Extensive Form Games with Perfect Information Levent Koçkesen 1 Extensive Form Games The strategies in strategic form games are speci ed so that each player chooses an action (or a mixture of actions) once and for all. In games that involve a sequential character this speci cation may not be suitable as players may nd it pro table to reassess their strategies as the events in the game unfold. Extensive form provides an explicit description of a strategic interaction by specifying who moves when, to do what, and with what information, and hence provides a richer environment to study interesting question such as commitment, repeated interaction, reputation building etc. However, as we shall see, a given game can be represented in both a strategic form and an extensive form, and the choice depends on the questions that we ask of that situation rather than on a formula. In this section we will study extensive form games with perfect information. These games are such that moves are taken in a sequence and every player observes every event that has taken place until that player has to take an action. 1.1 Preliminaries De nition 1. An extensive game with perfect information is a tuple (N;H;P;(u i ) i2n ) where ² N is a nite set of players, ² H is a set of sequences (histories) such that The empty sequence (history) ;2H: If (a k ) k=1;:::;k 2 H and L<Kthen (a k ) k=1;:::;l 2 H: If a k satis es k=1;:::: (ak ) k=1;:::;k 2 H for every integer K; then a k 2 H: k=1;:::: Each member of H is a history (a sequence of actions) Ahistory(a k ) k=1;:::;k 2 H is terminal if there exists no a K+1 such that (a k ) k=1;:::;k+1 2 H The set of all terminal histories denoted by Z 1
² A player function P : HnZ! N ² A payo function for each i 2 N u i : Z! R If H is nite, the game is nite. If the length of every history is nite, then the game has a nite horizon. De ne the length of a history h =(a k ) k=1;:::;k as the number of elements in the sequence (a k ) k=1;:::;k ; not counting the empty history, and denote it by jhj: Set j;j =0: Let h 2 H be a history of length k; and h 0 be a another sequence with length l; then we denote by (h; h 0 ) the history of length k + l consisting h followed by h 0 : We will usually write h 0 for (;;h 0 ) : Although we did not make any reference to action sets of players, it is implicit in the de nition. After any nonterminal history, player P (h) choosesanactionfromtheset A(h) =fa :(h; a) 2 Hg : We let H i be the set of histories h such that P (h) =i: Example 1. Sequential Battle of the Sexes. ² N = f1; 2g ² H = f;;b;s;(b;b) ; (B;S) ; (S; B) ; (S; S)g Z = f(b;b) ; (B;S) ; (S; B) ; (S; S)g ² P (;) =1;P(B) =P (S) =2 ² u 1 ((B;B)) = 2; u 1 ((B;S)) = 0; u 1 ((S; B)) = 0; u 1 ((S; S)) = 1 u 2 ((B;B)) = 1; u 2 ((B;S)) = 0; u 2 ((S; B)) = 0; u 2 ((S; S)) = 1: A (;) =fb;sg ;A(B) =A (S) =fb;sg : H 1 = f;g ;H 2 = fb;sg De nition 2. A strategy of player i is a function that maps every h 2 H i to an action in A (h) ; i.e., s i : H i! [ A (h) h2h i with s i (h) 2 A (h) for all h 2 H i : We denote the set of all strategies by player i by S i : 2
1b H B HHHHH S r r B @ S B @ S r r 2; 1 0; 0 0; 0 1; 2 Figure 1: Sequential Battle of the Sexes IntheBoSgame,wehave s 1 : f;g! fb;sg s 2 : fb;sg!fb;sg : For example, a strategy for player 1 could be s 1 (;) =B or s 1 (;) =S: Astrategyforplayer 2couldbes 2 (B) =B; s 2 (S) =B (which we may denote by BB) or s 2 (B) =B; s 2 (S) =S, etc. Therefore, S 1 = fb;sg ; and S 2 = fbb;bs;sb;ssg : This de nition requires us to specify the actions taken by a player after every history it is her turn to move, even after those histories which will not be reached if the strategy is followed. Consider the extensive form game in Figure 2. A strategy for player 1 has to specify an action after every history he is called upon to move. So, he has four strategies: LL 0 ;LR 0 ; RL 0 ;RR 0 : Notice that in strategies LL 0 and LR 0 we specify an action after the history (R; r) even though player 1 s move itself precludes reaching that history. 1b L @ R 2r 3; 3 l @ r 1r 10; 0 L 0 @ R 0 r 1; 10 2,1 Figure 2: Another Extensive Form Game Astrategypro leiss =(s i ) i2n ; and an outcome O (s) is the terminal history that is reached when s if followed. For example in the sequential BoS game O(B;BS) =(B;B); O(S; BS) =(S; B); O(B;BB) =(B;B): We will denote the set of all strategy pro les by S: 3
1.2 Nash Equilibrium De nition 3. A Nash equilibrium of the extensive form game with perfect information =(N;H;P;(u i ) i2n ) is a strategy pro le s such that for every player i 2 N we have u i (O (s )) u i O s i ;s i for every strategy s i of player i Nash equilibrium of an extensive form game can also be de ned as the Nash equilibrium of its strategic form. (Note: Sometimes we will write u i (s) for u i (O (s)) if doing so does not cause confusion.) De nition 4. The strategic form of the extensive form game with perfect information = (N;H;P;(u i ) i2n ) is the strategic form game G =(N;(S i ) ; (v i )) where for each i 2 N; ² S i is the set of strategies of player i ² v i is de ned by v i (s) =u i (O (s)) for all s 2 i2n S i Example 2. Sequential BoS BB BS SB SS B 2; 1 2; 1 0; 0 0; 0 S 0; 0 1; 2 0; 0 1; 2 Example 3. The following is the strategic form of the extensive form game in Figure 2. l r LL 0 3; 3 3; 3 LR 0 3; 3 3; 3 RL 0 10; 0 1; 10 RR 0 10; 0 2; 1 Notice that the strategies LL 0 and LR 0 by player 1 give the same payo to each player irrespective of what strategy player 2 uses. In that sense they can be regarded equivalent. Exercise 1. Let be an extensive form game and suppose the strategy s 0 i of player i in diers from s i only in the action it prescribes after histories that are precluded by strategy s i : Show that if s i;s i is a Nash equilibrium of ; then so is s i ;si 0 : The above exercise leads us to de ne the reduced strategic form of an extensive form game. 4
De nition 5. Let be an extensive form game with perfect information and let G be its strategic form. We say s i and s 0 i are equivalent if for each s i 2 S i we have v j (s i ;s i )=v j (s i ;s 0 i ) for all j 2 N: The reduced strategic form of is obtained by for each i 2 N; eliminating all but one member of each set of equivalent strategies in S i : ThereducedstrategicformofthegameinFigure2isgivenby l r L 3; 3 3; 3 RL 0 10; 0 1; 10 RR 0 10; 0 2; 1 There are two Nash equilibria of the above game: (L; r) and (RL 0 ;l): However, notice that in the second equilibrium, player 1 plays L 0 after history (R; r): This is not plausible as player 1 would choose action R 0 if the history (R; r) were ever to occur. The only reason why RL 0 is a best response to l is because the history (R; r) is precluded by player 2 s strategy. Also, the only reason why l is a best response to RL 0 ; is because player 1 threatens player 2 with playing L 0 if she plays l: Yet, this threat is not credible and the following equilibrium concept eliminates this type of Nash equilibria. 1.3 Subgame Perfect Equilibrium We rst de ne a subgame. De nition 6. The subgame of =(N;H;P;(u i )) that follows history h is the extensive game (h) =(N;Hj h ;Pj h ; (u i j h )) ; where Hj h, P j h ; and (u i j h ) are restrictions of H; P; and u i to histories after h: In other words, Hj h is the set of sequences of actions h 0 such that (h; h 0 ) 2 H; Pj h is de ned by P j h (h 0 )=P(h; h 0 ) for each nonterminal history h 0 2 Hj h ; and u i j h is de ned by u i j h (h 0 )=u i (h; h 0 ) for all terminal histories h 0 in Hj h : Also denote the restriction of a strategy s i in game to (h) by s i j h ; i.e., s i j h (h 0 )= s i (h; h 0 ) for all h 0 2 Hj h : When it does not cause a confusion we will denote u i j h (sj h ) by u i (sjh) : Example 4. The game in Figure 2 has three nonterminal histories, H=Z = f;;r;(r; r)g ; and hence three subgames. (;) = ; trivially. The other two subgames, (R) and (R; r) ; are dierent from ; and they are sometimes called proper subgames to distinguish them from (;) :Hj R = fl; r; (r; L 0 ) ; (r; R 0 )g ;Pj R (;) =2;Pj R (r) =1: If s 1 = LL 0 and s 2 = r; then s 1 j R = L 0 and s 2 j R = r; and u 1 j R (s 1 j R ;s 2 j R )=1. 5
De nition 7. A subgame perfect equilibrium (SPE) of =(N;H;P;(u i )) is a strategy pro le s such that for all i 2 N and every h 2 HnZ for which P (h) =i we have u i j h s i j h ;s i j h ui j h s i j h ;s i for all si in (h) : Equivalently, a SPE of is a strategy pro le s such that s j h is a Nash equilibrium of every subgame (h) : Example 5. Sequential BoS. Is s 1 = B; s 2 = SB a subgame perfect equilibrium? We have to check if and u 2 j B (S) u 2 j B (B) u 2 j S (B) u 2 j S (S) u 1 (B;SB) u 1 (S; SB) : Clearly not. So, (B;SB) is not a SPE. Check to see that the only SPE is (B;BS) : Example 6. In the game in Figure 2, the unique SPE is (LR 0 ;r) : Example 7. Entry Deterrence Game. The unique SPE is (in; C) : Eb out @ in Ir 5; 1 F @ C r 0; 0 2,2 Figure 3: Entry Deterrence Game The following result is extremely useful in verifying if a strategy pro le is a subgame perfect equilibrium. Proposition 1. (One-Deviation Property). Let be a nite horizon extensive form game with perfect information. The strategy pro le s is a SPE of if and only if for every player i 2 N and for every history h 2 HnZ for which P (h) =i we have u i j h s i j h ;s i j h ui j h s i j h ;s i for every strategy s i history in (h): that diers from s i j h onlyintheactionitprescribesaftertheinitial 6
Proof. Ifs is a SPE, then it clearly satis es the condition. To prove the other direction, suppose, for a contradiction that, the one-deviation property holds but s is not a SPE. Then there exists a player i; ahistoryh 0 ; and a strategy s i in the subgame (h 0 ) such that s i (h) 6= s i j h 0 (h) for some h 2 Hj h 0,andplayings i is strictly better for player i: Notice that since the game is of nite horizon the number of histories h for which s i (h) 6= s i j h 0 (h) is nite. Among all such s i,lets min i be the strategy which diers from s i j h of histories. Also let h max be the longest history of (h 0 ) for which s min i initial history of (h max ) is the only history in (h max ) at which s min i 0 at the least number (h) 6= s i j h 0 (h) : The diers from s i j h0: It max does must be the case that s min i j h max is a pro table deviation in (h max ). Otherwise, s i j h as well as s min i j h max: But, then we could nd a strategy ~s i in (h 0 ) which is identical with s min i until h max (and hence when combined with s ij h 0, it reaches h max if and only if s min i does too) and after that history it is equal to s i j h max: Therefore, ~s i does as well as s min i in (h 0 ) (~s i does as well as s i in the subgame (h max ), therefore, whether ~s i reaches h max ornotitalsodoes as well as s min i in (h 0 )); and hence it does strictly better than s i j h 0 in (h 0 ) and it diers from s i j h 0 at fewer histories than does smin i : This contradicts the hypothesis that s min i diered from s i j h 0 at the least number of histories, and hence establishes that s min i j h max is a pro table deviation in (h max ) which diers from s i j h max only at the initial history of (hmax ) : This, of course, contradicts the hypothesis that s satis es the one-deviation property. [The proof uses the fact that the game is of nite horizon. However, it is possible to show that if the payos of an in nite horizon game satis es a certain regularity condition (continuity at in nity: see Fudenberg and Tirole, 1991, p. 110), then the one-deviation property holds for in nite horizon games as well.] The following proposition is due to Kuhn and its proof uses backward induction, whichis also an algorithm to calculate SPE of nite extensive form games with perfect information. Proposition 2. (Kuhn s Theorem) Every nite extensive form game with perfect information has a subgame perfect equilibrium.. Proof. Let l ( ) = K 1: Also let H k = fh 2 H : jhj = kg : Since the game has a nite horizon k is nite. Since A (h) is nite for all h; players who can move at h 2 H K 1 has an optimal action. Choose one for each player, and eliminate actions that are not chosen. Do thesameforh 2 K k; k =2; 3;::: ;K: This, by induction, de nes a unique history h (and hence a unique strategy pro le s ) such that each player chooses optimally at every initial history of every subgame given that every player follows s after that initial history. By the one-deviation property s is a subgame perfect equilibrium. Notice that in the proof we used the facts that the game is nite and has a nite horizon. We used the fact that it has nite horizon in the induction, and we used the fact that A (h) is nite for every h to assure the existence of an optimal action (see Harris, 1985, Econometrica for an existence proof for in nite action games). Also, notice that if no player is indierent 7
between any two outcomes, then a nite extensive form game has a unique subgame perfect equilibrium. 1.3.1 Counter-theoreticals Since the strategies of players have to specify an action even after histories that are precluded by their own actions it is di cult to interpret strategies as plan of actions. A more appropriate interpretation is that strategies are beliefs held by the other players regarding a particular player s plan of action. However, one then faces the problem of counter-theoreticals in subgame perfect equilibrium concept (as well as some other equilibrium notions that we will discuss). Any satisfactory equilibrium concept has to specify what players believe and how they behave at out-of-equilibrium situations. To illustrate consider the game in Figure 2 again. The unique subgame perfect equilibrium of that game is given by (LR 0 ;r): In other words, player 1 is supposed to play L at the beginning. (A rational player 1 who knows player 2 is rational, and knows that player 2 knows that player 1 is rational, would play L). Now, what a rational player 2 is supposed to believe about what player 1 will do if history (R; r) is reached? Either player 1 is not rational, or he does not think that player 2 is rational, or does not think that player 2 knows that player 1 is rational. In sum, after such an history rationality of both players cannot be common knowledge. Furthermore, it is possible that rationality is common knowledge at the beginning of the game, yet the players do not play according to the backward induction solution. (see P. Reny, 1992, Journal of Economic Perspectives, for more on this issue.) The other alternative is simply that player 1 is rational, and so on, but he simply made a mistake. As we will see later on, most equilibrium concepts based upon sequential rationality take this last position ( simple mistake theories ). Principles of backward induction, and of sequential rationality at out-of-equilibrium histories in general, have the problem of requiring players to assume rationality of other players, even after they violated that assumption. We will see another illustration of what might be problematic about the simple mistake theories when we look at the centipede game below. One other possibility is that player 2 thinks that player 1 is just malicious, and if he is given the chance he will choose L 0 : Then it would be better for player 2 to choose l rather that r that the SPE prescribes. This would, of course, destroy the subgame perfect equilibrium. 1.3.2 Centipede Game Two players are involved in a process which they alternately get a chance to stop. Player 1 moves rst and the game ends after T periods by player 2 s last move (T even). If a player stops the process at period t; t 2; the payo of the player who chose to stop is t and the 8
payo of the other player is t 2: If player 2 stops in the rst period the payo pro le is (1; 0) : If player 2 continues in period T; the payo pro le is (T;T 1) : Asixperiodcentipedegame isgiveninfigure4. b1 C r2 C r1 C r2 C r1 C r2 C S S S S S S r r r r r r 1; 0 0; 2 3; 1 2; 4 5; 3 4; 6 r 6; 5 Figure 4: A six-period centipede game. In the unique subgame perfect equilibrium of this game, each player plays S at every period. Therefore, the game ends with player 1 stopping at the beginning. Interestingly enough, this is also the only Nash equilibrium outcome. To see this, rst note that there can be no Nash equilibrium outcome that ends with player 2 choosing C in the last period. He could change his strategy only in the last component and increase his payo. Similarly, if the outcome is such that it ends with player i choosing S in period t 2; then player j would be better of by playing S at period t 1: Therefore, in all Nash equilibrium outcomes player 1 chooses S in the rst period. For that to be a Nash equilibrium strategy, it has to be that player 2 chooses S in the second period. So, any Nash equilibrium has that player 1 and 2 choose to stop in the rst two periods. Whatever they do afterwards is irrelevant. The subgame perfect equilibrium concept clearly has the problems that are discussed in the previous section. However, we now see that Nash equilibrium leads us to the same outcome. This is at odds with experimental evidence in which, at least in the early rounds, players choose to continue. (see McKelvey and Palfrey, 1992, Econometrica). 9