Reputations. Larry Samuelson. Yale University. February 13, 2013

Transcription

1 Reputations Larry Samuelson Yale University February 13, 2013

2 I. Introduction I.1 An Example: The Chain Store Game Consider the chain-store game: Out In Acquiesce 5, 0 2, 2 F ight 5,0 1, 1 If played once, this game has a unique subgame-perfect equilibrium, (Acquiesce, In).

3 What if the game is played against a sequence of (finitely) many entrants? One s intuition is that player 1 will fight early entry, in order to deter later entrants. However, the (finitely repeated) game has a unique subgame perfect equilibrium, in which (Acquiesce, In) is played in every period. This is a simple backward-induction argument. This is known as the chain store paradox. Similarly, it is intuitive that players might cooperate in the early rounds of a finitely-repeated prisoners dilemma, but here the only Nash equilibrium outcome is that both players always defect.

4 Let s add some incomplete information: Now suppose that with probability 1 ˆµ 0, player 1 is the normal type. With probability ˆµ 0 (small), player 1 is a commitment type who always fights entry. Player 1 s type is 1 s private information. We can interpret the commitment type as having different payoffs, that make it optimal to fight, or as a type who is simply committed to fighting.

5 Some things that are no longer equilibrium outcomes: It is not an equilibrium outcome for the normal type to acquiesce in every period. It is also not an equilibrium outcome for the normal type to fight in every period. For example, the normal type will acquiesce in the last period.

6 The sequential equilibrium has the following properties (Kreps, Milgrom, Roberts and Wilson (1982)): There is an initial phase in which (F ight,out) is played, and 2 s beliefs remain unchanged at the prior. There is a terminal phase in which 2 mixes between In and Out, and 1 mixes between Acquiesce and F ight. If (Acquiesce, In) occurs, then 1 is revealed to be normal, and then (Acquiesce, In) is played in every subsequent period. If Out is played, 2 s beliefs remain unchanged. If (F ight,in) is played, 2 revises upward the posterior probability that 1 is the commitment type.

7 In the final period, player 2 mixes while the normal player 1 plays Acquiesce. As the number of repetitions grows: The length of the terminal phase remains fixed. This is determined by backward-induction calculations that do not depend on the length of the game. The initial phase grows long, consuming virtually all of the game. We thus see (F ight,out) most of the time. (Conlon (2003))

8 I.2 The Reputation Literature The idea of reputation has been modeled many different ways in the literature. Examples include: Models of expert advice, such as Morris (2001). Models of career concerns, such as Holmström (1982,1999). Models of bargaining, such as Abreu and Gul (2000).. Models based on repeated games.

9 I.3 Reputations in Repeated Games The idea of reputation is applied to repeated games in two ways: As an interpretation of equilibria in repeated games of complete information. By incorporating incomplete information.

10 I.4 The Objective Reputation ideas were originally applied to finitely-repeated games, such as the chain-store game or the prisoners dilemma, as a way of expanding the set of equilibrium outcomes to avoid seemingly counterintuitive implications. More recently, reputation ideas have been applied to infinitely-repeated games, as a way of putting bounds (referred to as reputation effects) on equilibrium payoffs. The techniques are much the same. This is commonly interpreted as a robustness exercise, but this interpretation must be adopted with care.

11 II. Reputations in Repeated Games of Perfect Monitoring II.1 The Model Players include: The normal player 1, a long-run, optimizing player. One or more commitment types of player 1. Player 2, a short-run player.

12 II.2 Stackelberg Types and Actions Define: v 1 = max a 1 A 1 â 1 arg max a 1 A 1 v 1 = sup α 1 A 1 min u 1(a 1,α 2 ) α 2 BR(a 1 ) min u 1(a 1,α 2 ) α 2 BR(a 1 ) min α 2 BR(α 1 ) u 1(α 1,α 2 ).

13 For example, consider the product-choice game: h l H 2,3 0,2 L 3,0 1,1

14 II.3 The Basic Reputation Result Proposition 1 Let A 2 be finite and let µ 0 attach positive probability to both the normal type and the pure Stackelberg type. Then there exists a (parameter dependent) k such that in every Nash equilibrium, player 1 s payoff is at least δ k v1 + (1 δk )minu a 1 (a). Fudenberg and Levine (1989). We ll first interpret this result, then comment on Stackelerg types, and then sketch the proof.

15 II.4 Interpretation The implication is that a patient player 1 gets arbitrarily close to the Stackelberg payoff. A chance of being the Stackelberg player is just as good as being that player. It is familiar that results in repeated games require patience. However, in the folk theorem, patience is important in strengthening incentives, while here is is important in reducing the cost of reputation building. Despite this interpretation of the role of patience, this result and its proof says nothing about equilibrium behavior. A 2 can be infinite, at the cost of a more cumbersome argument.

16 We can also accommodate multiple short-run players. We simple replace best response with Nash equilibrium in the induced short-run player game in the Stackelberg definitions.

17 II.5 Should We Expect Stackelberg Types? This result is sometimes criticized for requiring the fortuitous appearance of (only) the Stackelberg type. The commitment type was considered intuitive in the chain store game, but this may not always be the case. The result still holds if the commitment type is not the Stackelberg type, but it may give a lower payoff bound. Whatever commitment type is there, player 1 can do as well as being known to be that commitment type. There may be many commitment types, such as countably many. In this case, player 1 gets to choose his favorite type.

18 II.6 Proof: Overview There are three parts to the proof: A characterization of player 2 s beliefs on a useful set of states, in the form of a belief lemma. A characterization of player 2 s behavior, given these beliefs. The combination of these two elements to get the proof.

19 II.7 Proof: Beliefs Some notation: Ω Set of states Ω Ω States in which a 1 is chosen in every period States in Ω whose histories all have positive probability ˆµ 0 Prior probability of Stackelberg type Probability player 2 attaches to a 1 in period t η z Number of periods q t < z. q t

20 The belief lemma: Lemma 1 P { η z > } ln ˆµ0 lnz Ω = 0.

21 The argument: On Ω, P ( Stackelberg type h t) = P ( Stackelberg type h t 1 q t 1 This follows immediately from Bayes rule if only 1 s actions are observed. A straightforward technical argument confirms that observing player 2 s behavior as well does not confound the inference. Then we note that ) Either q t 1 is large or the probability attached to the Stackelberg type increases. A high probability of the Stackelberg type implies that q t 1 is large.

22 More formally, for ω Ω, P(ˆξ h t (ω)) = P(ˆξ h t 1 (ω)) q t 1. Using this result, we can write 0 < ˆµ 0 = q 0 P(ˆξ h 1 ) = q 0 q 1 P(ˆξ h 2 ) = and since P (ˆξ h t ) 1 for all t,. t 1 q τ τ=0 t 1 τ=0 q τ ˆµ 0. P(ˆξ h t ),

23 Taking limits, or But, and so, or P { ω Ω : τ=0 qτ (ω) ˆµ 0} = P(Ω ) = P(Ω ) τ=0 q τ = P { τ=0 qτ ˆµ 0 Ω } = 1. q τ {τ:q τ z} {τ:q τ >z} q τ < P { z nz ˆµ 0 Ω } = 1, P { n z lnz < ln ˆµ 0 Ω } = P { n z > {τ:q τ z} } ln ˆµ0 Ω lnz q τ z nz, = 0.

24 II.8 Proof: Best Responses The best response lemma: Lemma 2 There exists z (0,1) such that α 1 (â 1 ) > z BR(α 1 ) BR(â 1 ). This is obvious when A 2 is finite, and otherwise requires a continuity argument.

25 II.9 Proof: The Reputation Result Fix z > z. Let k = ln ˆµ0 lnz. Let the normal player 1 play â 1 in each period. Then in all but k periods, 2 plays a best response to â 1. Then 1 s payoff is at least δ k v1 + (1 δk )minu a 1 (a).

26 III. Reputations in Repeated Games of Imperfect Monitoring We are interested in imperfect monitoring for two reasons: Monitoring may often be imperfect. We can then consider mixed commitment types in perfect monitoring games.

27 The arguments are much the same, but require two changes: We must revisit the Stackelberg payoff notion. We need either a new belief lemma or a new proof. We will examine both possibilities.

28 III.1 The Model Player i now has a finite action set A i. Player 1 s actions are not observed. Instead, there is a finite signal set of public signals Y i. The distribution ρ(y a 1 ) gives the probability of signal y given action a 1 from player 1. The obvious interpretation (as in Fudenberg and Levine (1992)) is that y is a noisy public signal and player 2 cannot observe the actions of previous player 2 s. It requires only some additional notation to cover the case in which signals also depend on player 2 s actions. This allows us to include cases in which player 2 can observe previous player-2 actions, and to include perfect monitoring with mixed commitment types. Only minor modifications are required if we assume the signals are privately observed by player 2.

29 III.2 A Proof Based on a New Belief Lemma Stackelberg Payoffs: Let ˆα 1 be the commitment action. The bound on player 1 s payoff will be something like inf α 2 BR(ˆα 1 ) u 1(ˆα 1,α 2 ). However, we need to be more precise about BR(ˆα 1 ). Let BR ε (ˆα 1 ) = {α 2 α 1 s.t. α 2 BR(α 1 ), ρ( α 1,α 2 ) ρ( ˆα 1,α 2 ) < ε}.

30 The Reputation Result: Proposition 2 For any ε > 0, there exists k such that in any Nash equilibrium, player 1 s payoff is at least (1 ε)δ k inf α 2 B ε (ˆα 1 ) u 1(ˆα 1,α 2 ) + (1 (1 ε)δ k ) min a A u 1(a). Fudenberg and Levine (1992).

31 An Example: Consider the product choice game. h l H 2,3 0,2 L 3,0 1,1 Suppose h and l are public, but H and L give rise to public signals y and y with probabilities (p,1 p) and (q,1 q) (with p > q).

32 If we set ˆα 1 = 1 2 H L, then for all ε, BR ε(ˆα 1 ) = A 2, and we have a lower bound on player 1 s payoff of 1 2. If we shade the commitment type just slightly toward H, then BR ε (α 1 ) = {h}, and we get a bound close to 3 2. This exceeds the public-monitoring complete information payoff.

33 Sorin s (1999) Belief Lemma: Let Ω be a Borel space with filtration F t. Let ˆP and P be measures on Ω, and let P be their (nontrivial) mixture µˆp + (1 µ) P. Then for Lemma 3 For any ε and (1 z) > 0, there exists k such that where ˆP { {t : d t (P, ˆP) (1 z)} > k } ε, d t (P, ˆP) = sup A F t+1 P(A F t ) ˆP(A F t ).

34 III.3 An Alternative Argument (Gossner (2011)) Entropy: The relative entropy or Kullback-Leibler distance between probability distributions p and q over the finite set X is (logs are base e) d(p q) := x X p(x)log p(x) q(x). (For us, the the support of q will always contain the support of p, and 0log0 = 0, so this is well defined.) d(p q) is the relative entropy of q with respect to p. d(p q) 0; d(p q) = 0 iff p = q. (Cover and Thomas (2006)).

35 To interpret this, notice that the probability of n draws from X, identically and independently distributed according to p, is x p(x) n x. An observer who believes the data is distributed according to q assigns to the same sample probability x q(x) n x. The log likelihood ratio of the sample is x p(x) n x log x q(x) n = L(x 1,...,x n ) = x x n x log p(x) q(x). As n grows large, p-almost surely L(x 1,...,x n )/n d(p q).

36 Let p and q be two distributions over a finite product set X Y, with marginals p X and q X on X, and conditional probabilities p Y ( x) and q Y ( x) on Y given x X. The chain rule for relative entropy is d(p q) = d(p X q X ) + x p X(x)d(p Y ( x) q Y ( x)) = d(p X q X ) + E px d(p Y ( x) q Y ( x)). The error in predicting the pair xy can thus be decomposed into the error predicting x, and conditional on x, the error in predicting y.

37 Lemma 4 Let X be a finite set of outcomes. Suppose q = µp + (1 µ) p for some ε > 0 and p, p (X). Then, d(p q) log µ. Proof Since q(x)/p(x) µ, we have d(p q) = x q(x) p(x)log p(x) p(x)log µ = log µ. x

38 Fix a (possibly mixed) action ˆα 1 (A 1 ) and suppose the commitment type ˆξ = ξ(ˆα1 ) has positive prior probability. We show that since player 2 s beliefs assigns positive probability to the event that player 1 is always playing ˆα 1, then when player 1 does so, player 2 s one-step prediction error must disappear asymptotically. For any period t, denote the marginal of the unconditional distribution P on H t 2, the space of t period histories of public signals, by Pt 2. Similarly, the marginal on H t 2 of P (the distribution conditional on ˆξ) is denoted P t 2. We have: d( P t 2 Pt 2 ) log µ0 (ˆξ). (1)

39 Consider the one-step-ahead prediction errors, d(ˆp p(h τ 2 )). The chain rule implies d( P t 2 Pt 2 ) = t 1 τ=0 EˆP d(ˆp p(hτ 2 )), that is, the prediction error over t periods is the total of the t expected one-step ahead prediction errors. Since (1) holds for all t, and then τ=0 EˆP d(ˆp p(hτ 2 )) log µ0 (ˆξ). d δ 2 ( P P) := (1 δ) t=0 δt E Pd(ˆp p(h t 2 )) (1 δ)log µ0 (ˆξ), so the discounted sum of prediction errors is small.

40 An action α 2 (A 2 ) is an ε-entropy confirming best response to ˆα 1 (A 1 ) if there exists α 1 (A 1 ) such that 1. α 2 is a best response to α 1 ; and 2. d(ρ( ˆα 1 ) ρ( α 1 )) ε. The set of ε-entropy confirming best responses to ˆα 1 is denoted B d ε(ˆα 1 ).

41 Define v ˆα1 (ε) := min α 2 B d ε (ˆα 1) u 1 (ˆα 1,α 2 ), and denote by w ˆα1 the largest convex function below v ˆα1. Proposition 3 Fix a potentially mixed action ˆα 1 (A 1 ) and suppose µ 0 (ˆξ) > 0. Then, the set of normal player 1 s Nash equilibrium payoffs of the game Γ(µ,δ) is bounded below by w ˆα1 (ˆε), where ˆε := (1 δ)log µ 0 (ˆξ).

42 Proof Since in any Nash equilibrium (σ 1,σ 2 ), player 1 has the option of playing ˆα 1 in every period, we have U 1 (σ,ξ 0 ) = (1 δ) t=0 δt E P [u 1 (σ 1 (ht 1 ),σ 2 (ht 2 )) ξ 0] (1 δ) t=0 δt E Pu 1 (ˆα 1,σ 2 (ht 2 )) (1 δ) t=0 δt E Pv ˆα1 (d(ˆp p(h t 2 ))) (1 δ) t=0 δt E Pw ˆα1 (d(ˆp p(h t 2 ))) (and applying Jensen s inequality) w ˆα1 ( (1 δ) t=0 δt E Pd(ˆp p(h t 2 ))) = w ˆα1 ( d δ 2 ( P P) ) w ˆα1 ( (1 δ)log µ 0 (ˆξ) ).

43 Corollary 1 Fix a potentially mixed action ˆα 1 (A 1 ) and suppose µ 0 (ˆξ) > 0. Then, for all η > 0, there exists a δ < 1 such that, for all δ ( δ,1), the normal player 1 s payoff in any Nash equilibrium of the game Γ(µ,δ) is bounded below by v ˆα1 (0) η. Proof Since the distribution of signals is independent of player 2 s actions, if a mixture is an ε-entropy confirming best response to ˆα 1, then so is every action in its support. This implies that B0 d(ˆα 1) = Bε(ˆα d 1 ) for ε sufficiently small, and so v ˆα1 (0) = v ˆα1 (ε) for ε sufficiently small. Hence, v ˆα1 (0) = w ˆα1 (0) = lim ε 0 w ˆα1 (ε). The Corollary is now immediate from Proposition 3.

44 u 1 v ˆα1 v H w ˆα1 w H 0 ε log 2 relative entropy Figure 1: The functions v H, w H, v ˆα1, and w ˆα1 for the perfect monitoring version of the product-choice game. The relative entropy of 1 2 H L with respect to H is log For higher relative entropies, l is an ε- entropy confirming best response to H. The relative entropy of 1 2 H L with respect to ˆα 1 is ε For higher relative entropies, l is an ε- entropy confirming best response to ˆα 1.

45 IV. Temporary Reputations IV.1 The Model Imperfect monitoring poses no obstacles to building a reputation, but it does make it difficult to maintain a reputation. Suppose we have: Full support public or private monitoring. Player 1 is either normal or commitment type. Player 2 has unique best response to commitment action, that does not give Nash equilibrium of the stage game. For every player-1 pure action, the signal distributions induced by player 2 s pure actions are linearly independent.

46 IV.2 The Temporary Reputations Result P(Commitment type F t 2 ) 0 P almost surely. If in addition to public imperfect monitoring of player 1 s actions, the actions of player 2 are public, then not only does player 2 learn the type of player 1, but the continuation play of any Nash equilibrium converges to a Nash equilibrium of the complete information game Cripps, Mailath and Samuelson (2004,2007).

47 IV.3 The Intuition Suppose not. Player 2 s beliefs are a martingale, and so must converge to an interior belief. 2 s beliefs can converge to something in the interior only if play converges to ˆα 1. Then 2 will play a best response to ˆα 1. Then the normal player 1 will deviate from ˆα 1, a contradicting 2 s beliefs. This is not a limiting result in δ, but the limits must be interpreted carefully.

48 1 δ δ Corollary 2 bound applies δ 0 µ µ 0 µ Figure 2: Illustration of reputation effects under imperfect monitoring. Suppose ˆξ is a Stackelberg type. For fixed η > 0, δ is the lower bound for prior probability µ(ˆξ) = µ 0. For players 1 s discount factor δ > δ, the reputation result applies in period 0. The arrows indicate possible belief updates of player 2 about the likelihood of player 1 being the Stackelberg type ˆξ. When player 1 is normal, almost surely the vertical axis is reached.

49 More precisely, assume there are only two types of player 1, the normal type ξ 0 and the simple action type, ˆξ. Also: Assumption 1 (Full Support) ρ(y ˆα 1,a 2 ) > 0 for all a 2 A 2 and y Y. Assumption 2 (Identification of 1) For all a 2 A 2, the A 1 columns in the matrix [ρ(y a 1 a 2 )] y Y,a1 A 1 are linearly independent. Assumption 3 (Identification of 2) For all a 1 A 1, the A 2 columns in the matrix [ρ(y a 1 a 2 )] y Y,a2 A 2 are linearly independent. Assume the actions of the short-lived player are not observed by the player 1, or subsequent short-lived players.

50 Let µ t (h t 2 ) := P(ˆξ h t 2 ). Proposition 4 Suppose Assumptions 1 3 are satisfied. Suppose player 2 has a unique myopic best response â 2 to ˆα 1 and that (ˆα 1,â 2 ) is not a Nash equilibrium of the stage game. If (σ 1,σ 2 ) is a Nash equilibrium of the incomplete information game, then µ t 0, P-a.s.

51 By Assumption 1, Bayes rule determines µ t after all histories. At any Nash equilibrium of this game, the belief µ t is a bounded martingale and so converges P-almost surely (and hence P- and P-almost surely) to a random variable µ. The idea of the proof is: 1. En route to a contradiction, assume that there is a positive P-probability event on which µ is strictly positive. 2. On this event, player 2 believes that both types of player 1 are eventually choosing the same distribution over actions ˆα 1 (because otherwise player 2 could distinguish them). 3. Consequently, on a positive P-probability set of histories, eventually, player 2 will always play a best response to ˆα 1.

52 4. Assumption 3 then implies that there is a positive P-probability set of histories on which player 1 infers that player 2 is for many periods best responding to ˆα 1, irrespective of the observed signals. 5. This yields the contradiction, since player 1 has a strict incentive to play differently than ˆα 1.

53 V. Discussion