6 The Principle of Optimality

Transcription

1 6 The Principle of Optimality De nition A T shot deviation from a strategy s i is a strategy bs i such that there exists T such that bs i (h t ) = s i (h t ) for all h t 2 H with t T De nition 2 A one-shot deviation from a strategy s i is a strategy bs i such that there exists some unique t and e h t 2 H t such that bs i eht 6= s i eht In other words, s i (h t ) = bs i (h t ) for every h t 6= e h t Proposition Suppose that there exists some M < such that M u i (s) M for all s 2 S Then, a necessary and su cient condition for s = (s ; ; s n ) to be subgame perfect in the in nitely repeated game G (; ) is that there exists no pro table one-shot deviation after any history h t 2 H For proving this. Necessity is trivial. 2. We rst use continuity to show that, if there is a pro table deviation, then there is a pro table nite period deviation.. The second step proceeds inductively to locate a pro table one-shot deviation for every nite period deviation. Proof. Suppose that s is not subgame perfect. We want to show that there is a one-shot deviation. 9

2 By de nition of subgame perfection there is some h and i 2 I such that s i jh is not a best reply against s i jh Thus, there exists bs i such that the continuation payo from sticking to s i is below that of deviating to bs i We write this as U i (s i ; s i jh ) < U i (bs i ; s i jh ) De ne " = U i (bs i ; s i jh ) U i (s i ; s i jh ) > 0 and let T be such that T M < " Now, consider a T shot deviation s i where < bs i (h y ) s i (h t ) = s i (h t ) if t T if t > T Now, let b h T be the history implied by playing continuation strategies (bs i ; s i ) up to time T after history h Then, we may write = = U i (s i ; s i jh ) U i (s i ; s i jh ) X t u i a t (s i ; s i jh ) U i (s i ; s i jh ) t= X t u i t= a t (s i ; s i jh ) {z } t=t + t u i(a t (sj b h T)) t u i a t (bs i ; s i jh ) X t u i = P T t= t u i (a t (bs i ;s i jh ))+ P + X t=t + = U i (bs i ; s i jh ) U i (s i ; s i jh ) + = " + X t=t + t u i a t sj b h T {z } T M> " Hence, there is a nite shot deviation. Now, t=t + X t=t + X t=t + t u i U i (s i ; s i jh ) a t (bs i ; s i jh ) t u i a t sj b h T X t=t + t u i a t (bs i ; s i jh ) " 2 T M < " {z } T M< " 40 a t (bs i ; s i jh )

3 Let s i be a T -shot deviation from s i. Without loss of generality, let the deviation start at t = For each t = ; ; T let h t be the history induced by play of (s i ; s i ) Now, consider the one-shot deviation s T i where < s s T i (h t ) if h t = h T i (h t ) = s i (h t ) if h t 6= h T There are now two possibilities. Either s T i is a pro table one show deviation from s i ; 2. or playing is a pro table T s i < s i (h t ) if t T s i (h t ) if t > T -shot deviation. Proceeding inductively it follows that there must be a pro table one-shot deviation. 6. Supporting Ine cient Play in In nitely Repeated Games The focus in many applications of repeated games is on supporting good/e cient outcomes. However, the same ideas can also be used to support bad/ine cient outcomes. Consider the stage game L C R T 5; 5 4; 4 0; 0 M 4; 4 ; 0; 0 B 0; 0 0; 0 2; 2 4

4 Clearly (T,L) is the unique stage game Nash equilibrium as well as the unique Pareto e cient outcome. Consider the in nite repetition and let s (h 0 ) = < M if (a t 2 ; a t ) = (MC,MC) or a t 2 6= MC and a t M and s (h t ) = B otherwise = BR s 2 (h 0 ) = < C if (a t 2 ; a t ) = (MC,MC) or a t 2 6= MC and a t C and s 2 (h t ) = R otherwise = BR If the players follow this strategy the outcome is (MC,MC,...) giving each player a payo of = + + X t A one shot deviation to T (for player ) or L (for player 2) would give payo t=2 4 + X t t=2 Hence (this is the best on the equilibrium path deviation), the speci ed strategy is a Nash equilibrium if = + + () X t 4 + t=2 X t t=2 4 To check for subgame perfection, we now only need to rule out pro table deviations from the punishment phase. Using the Principle of optimality, we only need to check one-shot deviations, that is whether X t t=2 () 2 5 X t t=2 Hence, the strategy pro le is subgame perfect. 42

5 7 Folk Theorems 7. A Simple Folk Theorem Now consider an arbitrary stage game G = (n; A; u) Let NE (G) denote the set of Nash equilibria to the stage game. Now De nition A strategy pro le s is called a Nash reversion pro le if there exists some stage game Nash equilibrium a 2 NE (G) and a sequence fa t g 2 A such that < a t s i if h t = (a ; ; a t ) i (h t ) = a i if h t 6= (a ; ; a t ) Lemma A Nash reversion strategy that calls for playing path fa t g with Nash reversion threat a is subgame perfect if and only if U i s jh t = a ; ; a t X = u i (a ) u i for every a i 2 A i ; t and i =t a i ; a t i + u i (a ) Proof. Obvious as playing a static Nash in every period is an equilibrium of subgames following a deviation and the condition stated in the Lemma says that deviating from the sequence is worse than triggering Nash reversion, implying that following the sequence is a Nash equilibrium in every subgame on the equilibrium path as well. It is more or less immediate that Proposition 2 Let a 2 A be a stage game action pro le such that u i (a) > u i (a ) for all i 2 I Then, there exists some < such that playing a 2 A in every period is supportable as a subgame perfect equilibrium of the in nite repetition of G Proof. By Lemma we have that we can support playing a 2 A in every period as a subgame perfect equilibrium by a threat of Nash reversion if and only if u i (a) u i a i ; a t i + u i (a ), u i (a) u i (a ) ( ) u i a i ; a t i 4

6 Since ( ) u i a i ; a t i! 0 as! and ui (a) u i (a ) > 0 it follows that there is < such that u i (a) u i (a ) ( ) u i a i ; a t i whenever 7.2 More Sophisticated Folk Theorems Nash reversion can be used to support also non-stationary paths of play, which can be used to convexify the set of supportable stage game payo s De nition 4 The set of feasible (average) payo s in an in nitely repeated game is F = CONV fx 2 R n jthere exists a 2 A s.t. u (a) = xg Proposition For any v 2 F such that there exists some stage game Nash equilibrium a such that v i > u i (a ) there exists some < such that there is a subgame perfect equilibrium of the in nite repetition of G where the payo is v i for each player i 2 I IDEA. If x 2 F there exists some a ; a 2 ; ; a k and some 2 k such that x i = X j a j i 2. Can approximate this by strategies where a j is played approximately j percent of the time. If is close enough to the exact timing isn t important (but technical work is needed to show this).. Use same idea as in the simple Folk theorem to use Nash reversion to support the play. Next, we note that it is not necessary to use Nash reversion as punishments. More severe punishments do exist. De nition 5 The minmax value in stage game is v i = min max u i (a i ; a i ) a i a i 44

7 De nition 6 The set of individually rational payo s is I = fx 2 R n jv i v i for every i 2 Ig Proposition 4 For every v 2 F \ I there exists some < such that there is a subgame perfect equilibrium of the in nite repetition of G where the payo is v i for each player i 2 I play. Proof somewhat di cult, but idea similar to construction of equilibrium with ine cient Remark Issues. Minmax doesn t always exist. 2. Cheat with boundaries...must restrict to interior or rely on public randomizations. 7. Renegotiation Proofness Subgame perfection without bite in many repeated game contexts. One idea suppose that layers can renegotiate in every period. Equilibria such as trigger strategy equilibria are then somewhat suspect as players may try to strike a deal upon reaching punishment phase. "Let bygones be bygones and return to the jolly equilibrium path"...which would potentially destabilize the equilibrium. This idea is formalized as renegotiation proofness, which may be viewed as importing some cooperative ideas into non-cooperative game theory. Consider the nite case rst. Let G T be a T fold repetition of stage game G De nition 7 Let the set of Nash equilibria of G T that are not Pareto dominated be called the e cient equilibria of G T ; denoted P G T That is, P G T = s 2 NE G T jthere exists no s 0 2 NE G T s.t. u i (s 0 ) > u i (s) for all i 2 I 45

8 De nition s is renegotiation proof if s jh t is an e cient equilibrium of the continuation game after any history h t That is s jh T 2 P (G) for every history leading to the nal stage, s jh T 2 P (G 2 ) for every history leading to the penultimate stage and s jh t 2 P G T t+ for every history of length t Example Consider the case with G given by the following mix of a coordination game and a prisoners dilemma D A D B C D A ; 0; 0 4 ; 0 4 D B 0; 0 2; 2 4 ; 0 4 C 0; 4 4 0; 4 4 ; We have two stage game Nash equilibria. (D A ; D A ) and (D B ; D B ) First, consider G and let4 s i = C < s 2 i (h 2 ) = s i (h ) = D B D A if a = CC if a 6= CC Since any history leads to repeated play of a stage game Nash in period 2 and we have that play is Nash in all subgames starting at times 2 and. Hence we need to check whether there is a pro table one shot deviation in the rst period, that is if ( + )

9 which clearly holds for Next, let T > and consider the following strategies 2 s i = C < C if t T and h s t t = (CC; ; CC) i = D A if t T and h t 6= (CC; ; CC) < D s T i (h 2 ) = s T B if h T = (CC; ; CC) i (h ) = D A if h T 6= (CC; ; CC) It is easy to check that these strategies are subgame perfect given that However, this 2 is not renegotiation proof. In the last period we see that (D B ; D B ) is the only e cient equilibrium in the stage game. 2. It follows that (D B ; D B ) in the rst period and (D B ; D B ) in the second period is the only e cient equilibrium in G 2. By induction, there is a unique renegotiation proof subgame perfect equilibrium which is to play (D B ; D B ) after any history of play. Example 2 Consider the case with G given by the following mix of a coordination game and a battle of the sexes F O C F ; 0; 0 5; 0 O 0; 0 ; 5; 0 C 0; 5 0; 5 4; 4 Again, we have two stage game Nash equilibria. (F; F ) and (O; O) First, consider G and 47

10 let s i = C < F s 2 i (h 2 ) = O < O s i (h ) = F if a = C or a 2 6= C if a 6= C and a 2 = C if a 2 = C or a 6= C if a 2 6= C and a = C The outcome path is CC; F F; OO With payo s for player for player 2 It is clear that the play in the subgames starting at time 2 and is subgame perfect. Need to check for one shot deviation at time. If player 2 doesn t have a deviation, then there is no deviation for player either (since get rewarded earlier). The pro le is immune towards deviations from 2 if which is true for every q ; In this case renegotiation proofness doesn t have any bite at all as there is no Pareto dominated equilibrium in stage game. Proposition 5 In a nitely repeated game, there is either a unique renegotiation proof equilibrium, or every renegotiation proof equilibrium is near Pareto e cient if is near unity. 7.4 Renegotiation in in nitely repeated games There are variants. Here is one 4

11 De nition 9 Consider a two player game. A strategy pro le s in G () is internally consistent if for all h; h 0 ui (sjh) > ui (sjh 0 ) ) uj (sjh) uj (sjh 0 ) for i; j = ; 2 IDEA Rule out punishments where both players are made worse o for the remainder of the game. Such a punishment would be vulnerable to a deal -let s go back to the equilibrium path! De nition 0 s is negotiation proof if it is internally consistent and subgame perfect Example D C D ; 4; 0 C 0; 4 ; Grim-Trigger is not negotiation proof as 0 A A Need strategies that. Punishes the deviator 2. Rewards the other player Consider s i = C < C if a t s t j = C or if a t 2 ; a t 2 f(cd; DC) ; (DC; CD) (CC; DD)g i = D otherwise 49

12 Outcome path (CC) in every period. Nash if = ( ) ( + ) = ( + ) 4 Deviation from punishment not pro table if 0 + ( ) = (obvious that the player that is rewarded has no pro table deviation). Continuation payo s are A in cooperative phase" (which includes after DD) A after deviation by player 2 only A after deviation by player only These are the only continuation payo s to consider, and we see that so the strategy is negotiation proof. 4 + > > 0 + ; 50