Strategies and Interactive Beliefs in Dynamic Games

Transcription

1 Strategies and Interactive Beliefs in Dynamic Games Pierpaolo Battigalli 1 Alfredo Di Tillio 2 Dov Samet 3 1 Department of Decision Sciences Bocconi University 2 Department of Economics Bocconi University 3 Faculty of Management Tel Aviv University Games and Decisions Pisa, 10 July 2013

2 Introduction Topic: formal analysis of strategic thinking in dynamic games, backward and forward induction reasoning. Tools: epistemic (type) structures where states determine truth value of conditional statements. Distinctive feature: we use only epistemic conditionals of the form: "if i learned h he would believe E with probability p"; state=(actual actions,epistemic conditionals). Motivation: strategies cannot be (irreversibly) chosen, they are just beliefs on own contingent choices obtained from planning (objective vs subjective strategies). Focus: generic perfect information games.

3 Background literature Hierarchies of probabilitstic beliefs: Mertens & Zamir (IJGT 1985), Brandenburger & Dekel (JET 1993), Heifetz & Samet (JET 1998) Rationalizability in games: Bernheim (Ecma 1984), Pearce (Ecma 1984) Dynamic interactive epistemology: Ben Porath (REStud 1997), Battigalli & Siniscalchi (JET 1999,2002)

4 Road map 1 Preview on conditionals and Backward Induction (BI): semi-formal analysis of an example 2 Setup: perfect information (PI) games and epistemic structures 3 Strategies as epistemic constructs: some results 4 Conclusions

5 Preview: two kinds of type structures Variants of Battigalli & Siniscalchi JET99: types are implicit representations of hierarchies of conditional beliefs. But, what are rst-order beliefs about? H=histories, Z=paths (terminal nodes), S i,s i,s=strategies "Traditional" S-based structures: (T i ; i ) i2i, states in = S T, (s i ; t i )=state of i (s i is "objective"), H i =conditioning events, H i = fs i (h) T i : h 2 Hg i : T i! H i (S i T i ) [(S i T i )] H i Our Z-based structures: (T i ; i ) i2i, states in = Z T, t i =epist.state of i (beliefs about others+plan), H = fz(h) T i : h 2 Hg i : T i! H (Z T i ) [(Z T i )] H

6 Preview on conditionals and BI: an example Stackelberg Minigame outputs: low (left) high (right) high (up) 3,1 0,0 low (down) 2,2 1,3 tree: (3,1) (0,0) L - % R Bob " U Ann # D Bob l. & r (2,2) (1,3)

7 Preview on conditionals and BI: an example Stackelberg Minigame Ann believes "Bob would go Right given Up" =belief about a behavioral conditional Given Up, Ann would believe "Bob goes Right" =conditional belief about behavior In traditional analysis: behavioral conditionals and beliefs about conditionals and opponents beliefs In our analysis: actual actions (paths) and conditional beliefs about actions and beliefs

8 Preview on conditionals and BI: traditional analysis States! specify strategies s i = i (!) objectively Bob would go Right given Up false at! i Bob (!) 2 fl:r; L:lg Rat Bob [L:r] = f! : i (!) = L:rg ) B Ann (Rat Bob ) B Ann (L:r) (uncond. belief) States also specify conditional beliefs By built in independence: B Ann (L:r) B Ann (LjU)\B Ann (rjd) = =B Ann (util = 3jU)\B Ann (util = 1jD) ) Rat Ann \ Rat Bob \ B Ann (Rat Bob ) [U; L:r] [U; L] (3,1) (0,0) L - % R Bob " U Ann # D Bob l. & r (2,2) (1,3) ) BI strategies and path obtain

9 Preview on conditionals and BI: our analysis strategy is an epistemic concept: i s cond. beliefs ) (beliefs about others and) contingent plan of i Rat. Planning: RP Bob =B Bob (LjU)\B Bob (rjd) Material Consistency=path cons. with plan Rationality=Material cons.+rat.plan. R Bob [U; L] [ [D; r] But no event [L:r]. B Ann (L:r) not expressible! B Ann (R Bob ) B Ann ([U; L] [ [D; r]) 6 6B Ann (LjU)\B Ann (rjd) e.g. if B Ann (D) (3,1) (0,0) L - % R Bob " U Ann # D Bob l. & r (2,2) (1,3)

10 Preview on conditionals and BI: an example Our analysis Recall: R Bob [U; L] [ [D; r], but B Ann (R Bob ) 6B Ann (LjU)\B Ann (rjd) e.g. if B Ann (D) (Ann plans D) Since there is no event [L:r], B Ann (L:r) not expressible If B Ann (D) (Ann plans D) the following is possible B Ann ([U; L] [ [D; r]) \ :B Ann (LjU) 9! 2 [D; r]\r Ann \R Bob \B Ann (R Bob ) (3,1) (0,0) L - % R Bob " U Ann # D Bob l. & r (2,2) (1,3) ) Imperfect Nash equilibrium may obtain

11 Preview on conditionals and BI: an example How to recover elementary BI, route 1: Strong Belief in rationality Ann strongly believes E if she believes E given each C with C \ E 6= ; [U] \ R Bob 6= ;, [D] \ R Bob 6= ; hence SB Ann (R Bob ) B Ann (R Bob ju)\ B Ann (R Bob jd) but R Bob [U; L] [ [D; r] hence SB Ann (R Bob ) B Ann (LjU) \ B Ann (rjd) (BI cond. beliefs) thus R Ann \ R Bob \ SB Ann (R Bob ) [U; L] (BI path)

12 Preview on conditionals and BI: an example How to recover elementary BI, route 2: Own-Action Independence Suppose (i) Ann s conditional beliefs about Bob s beliefs (hence his plan) are independent of her actions (Ind), (ii) Ann strongly believes that Bob is materially consistent: Ind \ B Ann (RP Bob ) \ SB Ann (MC Bob ) B Ann (LjU) \ B Ann (rjd) Thus R Ann \ R Bob \ Ind \ B Ann (RP Bob ) \ SB Ann (MC Bob ) [U; L] (Note: such independence and belief in consistency are implicit in traditional analysis)

13 Route 1 (SB, with higher-level epistemic assumptions) leads to the BI-path in generic PI games Route 2 (OAI) leads to the BI-path in generic two-stage PI games, not in longer games such as the Centipede (even with higher-level epistemic assumptions)

14 "Centipede" example: strong belief analysis Forward induction reasoning yields the BI outcome C c C 0 Ann! Bob! Ann! (0,3) # D # d # D 0 (1,0) (0,2) (3,0) R Ann :[C; c; C 0 ] R Bob \ SB Bob (R Ann ) [D] [ [C; d] R Ann \ SB Ann (R Bob \ SB Bob (R Ann )) [D]

15 "Centipede" example: initial common belief analysis Not enough to get the BI outcome C c C 0 Ann! Bob! Ann! (0,3) # D # d # D 0 (1,0) (0,2) (3,0) R Ann :[C; c; C 0 ] But, without SB Bob (R Ann ), R Bob has no behavioral implication! Thus R Ann \R Bob \Ind\B Ann (RP Bob )\SB Ann (MC Bob ) :[C; c; C 0 ] ) initial common belief in R \ Ind \ SB(MC) only buys :[C; c; C 0 ] (We can show this by example and as corollary of a general theorem)

16 Setup Perfect Information (PI) games i 2 I, players h 2 H, histories/nodes (H i, owned by i), H nite z 2 Z H, terminal histories/paths; Z(h) = fz : h zg a 2 A(h), actions at h 2 HnZ u i : Z! R, utility/payo s.t. no relevant ties

17 Setup Epistemic structures for PI games: general! 2 = X T = X Q i2i T i, states of the world x 2 X, external (non-epistemic) states, nite (e.g. X = S or X = Z) (implicitly understood path function : X! Z) for Y X, [Y] = Y T external events i : T i! B i, i (t i ) epistemic state of i at! (B i = Beliefs i to be speci ed)

18 Setup Conditional Probability Systems Conditioning events (hypotheses) correspond to histories: [h] = f(x; t) 2 X T i : (x) 2 Z(h)g Probability measures concentrated on conditioning events C = [h], D = [h 0 ]... related by chain rule: E C D ) i (EjD) = i (EjC) i (CjD) (ch.r) Write i;h () = i (j[h]); i = ( i;h ()) h2h 2[(X T)] H Conditional probability systems (CPS s) on (X T; H): H (X T i ) = i : i;h ([h]) = 1 and (ch.r) holds

19 Setup We focus on epistemic type structures where external states are complete sequences of actions: X = Z = Z Y i2i T i, i : T i! H (Z T i ), (i 2 I) meaning: no knowledge about z at the outset information about moves cannot directly disclose anything about types/beliefs

20 Type structures = Z Y i T i, i : T i! H (Z T i ), Technical assumptions (in a sense, w.l.o.g.): for each i 2 I T i compact metrizable, hence H (Z T i ) compact metrizable (B&S JET99) i continuous De nition A type structure is complete if i is onto for every i 2 I. Main example: the canonical structure of hierarchies of beliefs.

21 Canonical structure: types as hierarchies of beliefs First-order beliefs: T 1 i = H (Z) Second-order beliefs: n T 2 i = ( 1 i ; 2 i ) 2 T1 i H Z T 1 i : 8h 2 H; marg Z 2 i;h () =... recursive def. of T n i T n 1 i H (Z T n 1 i )... T i = projective limit of (T n i ) n2n, set of in nite hierachies of beliefs satisfying common full belief in coherence homeomorphism i : T i! H (Z T i ) by a generalization of Kolmogorov s extension theorem

22 Setup Belief operators Standard (monotonic) belief operators: [for E Z T, let E ti = f(z; t i ) : (z; t i ; t i ) 2 Eg section at t i ] conditional belief B i (Ejh) = f(z; t i ; t i ) : i (t i )(E ti j[h]) = 1g, initial belief B i (E) = B i (Ejh 0 ) (h 0 =empty hist., root) Nonmonotonic belief operator strong belief SB i (E) = \ h:e\[h]6=; B i (Ejh)

23 Setup Dynamic programming Given i 2 H (Z T i ) derive i (ajh)= i ([h; a]j[h]) for all h 2 HnZ, a 2 A(h). Consider only the conditional prob. of i s opponents actions: i (ajh), h 2 H i, a 2 A(h) ) subjective decision tree i( i ) for i i is consistent with dynamic programming on 8h 2 H i, i arg max a2a(h) V i((h; a); i )jh with V i ((h; a); i )= X z2z(h;a) i ( i ) i (OSD) = 1, u i (z) i (zjh; a)

24 Strategies as beliefs Rational planning The plan of i at (z; t i ; t i ) (plan of t i ) is derived from i (t i ) De nition Pl. i plans rationally at (z; t i ; t i ) if 8h 2 H i, i (t i ) arg max V i((h; a); i )jh = 1: a2a(h) Event: RP i RP i is just a property of i s beliefs/types: i expects to take locally maximizing actions conditional on each h 2 H i. Interpretation: i has beliefs about others and computes his plan (beliefs about himself) by dynamic programming ("folding back") on the corresponding subjective decision tree.

25 Strategies as beliefs Material Consistency and Material Rationality Connect beliefs to behavior: De nition Pl. i is materially consistent at (z; t i ; t i ) if he does not violate his plan on path z 8h 2 H i ; 8a 2 A(h), (h; a) z ) i (t i )(ajh) > 0: Event: MC i De nition Pl. i is materially rational at (z; t i ; t i ) if he plans rationally and does not violate his plan at (z; t i ; t i ).Event: R i = MC i \ RP i.

26 Strategies as beliefs Common belief in R and Nash equilibrium Recall: results apply to nite PI games with NRT, in such games the BI strategy is unique and (mixed) Nash equilibria yield a unique path with prob. 1. Similar to traditional analysis: initial common belief in material rationality does not yield BI or Nash paths in games of depth d > 2 (e.g. Centipede).

27 Unlike traditional analysis: in games of depth 2, correct belief in rationality does not yield BI, only a Nash path (cf. initial example). Proposition In a game of depth 2, 8z 2 Z, z is a Nash path if and only if (z; t) 2 \ i R i \ B i (R i ) for some t, in some type structure (T i ; i ) i2i for.

28 Strategies as beliefs Common Strong Belief in Material Rationality One way to obtain elementary BI (games of depth 2) is to assume that the rst mover strongly believes in the material rationality of the co-player. We can go further and replicate "traditional" results on common strong belief in rationality and Nash eq. (Battigalli-Friedenberg, 2010), or EFR and BI in complete structures (Battigalli-Siniscalchi, 2002).

29 Strategies as beliefs Common Strong Belief in Material Rationality R 0 i =R i, R k+1 i =R k i \ SB i (R k i ) CSBR= \ R k i, correct Common Strong Belief in R k;i : S! Z strategy-path function Proposition (i) In any type structure proj Z CSBR fnash-pathsg (ii) In a complete (or otherwise "su ciently rich") type structure proj Z CSBR = (EFR) = fbi-pathg:

30 Strategies as beliefs Own Action Independence Reason for non-bi result: Ann may initially believe in R Bob but she need not believe in R Bob conditional on taking an unplanned action. This cannot happen if Ann s beliefs about Bob s type conditional on her own actions do not depend on the conditioning action, and she strongly (hence always) believes in MC Bob. Own-action independence=i s conditional beliefs about t i do not depend on i s actions (event Ind i )

31 Proposition In every "rich" (e.g., complete) type structure for a game of depth 2 T proj Z R i \ Ind i \ B i (R i ) \ SB i (MC i ) = fbi-pathg. i What about longer games (e.g. Centipede)?

32 Strategies as beliefs Independence RInd 0 i =R i \ Ind i \ SB i (MC i ), RInd k+1 i =RInd k i \ B i (RInd k i ) CBRInd= \ i;k RInd k i All the paths consistent with the "Dekel-Fudenberg procedure", (S 1 W), are also consistent with CBRInd: Proposition There are type structures (including the canonical one) such that 8s 2 S 1 W, 9(z; t) 2 CBRInd such that z = (s). Corollary There are non-bi paths consistent with CBRInd in Centipede (of depth d > 2).

33 Conclusions Many results of the traditional analysis with behavioral conditionals in the state of the world make a lot of sense. They are built on often implicit assumptions of consistency of plans with actual behavior, strong belief in consistency (perceived intentionality) and self/opponents independence. We rule out behavioral conditionals, but allow for epistemic ones. This forces an interpretation of strategies as epistemic constructs and imposes discipline: in this setup strategies cannot be chosen, they can only be planned. Consistency and own-action independence have to be assumed explicitly. This seems tting for a formal analysis of strategic reasoning.

34 Conclusions Imperfect information (with perfect recall) h i 2 H i information sets (personal histories) Conditions/hypotheses for i: [h i ] and [h i ; a i ] (a i 2 A i (h i )) Value of action a i at h i : E i (t i )[u i jh i ; a i ] Potential con ict between our analysis and standard decision theory. Own-action independence and strong belief in material consistency allow to reconcile our analysis with traditional theory.

35 References Battigalli P. and G. Bonanno (1999): "Recent results on belief, knowledge and the epistemic foundations of game theory", Research in Economics, 53, Battigalli P. and A. Friedenberg (2010): Forward Induction Reasoning Revisited, Theoretical Economics, 7, Battigalli P. and M. Siniscalchi (1999): Hierarchies of conditional beliefs and interactive epistemology in dynamic games", Journal of Economic Theory, 88, Battigalli P. and M. Siniscalchi (2002): Strong Belief and Forward Induction Reasoning", Journal of Economic Theory, 106, Ben Porath, E. (1997): Rationality, Nash Equilibrium, and Backward Induction in Perfect Information Games, Review of Economic Studies, 64,

36 References Bernheim D. (1984): "Rationalizable Strategic Behavior", Econometrica, 52, Brandenburger A. (2007): The Power of Paradox," International Journal of Game Theory, 35, Brandenburger A. and E. Dekel (1993): " Hierarchies of beliefs and common knowledge", Journal of Economic Theory, 59, Dekel E., and D. Fudenberg (1990): Rational Behavior with Payo Uncertainty, Journal of Economic Theory, 52, Di Tillio A., J. Halpern and D. Samet (2010): Conditional Belief Types", mimeo.

37 References Heifetz, A. and D. Samet (1998): Topology-free typology of beliefs, Journal of Economic Theory, 82, Mertens J.-F. and S. Zamir (1985): "Formulation of Bayesian analysis for games with incomplete information", International Journal of Game Theory, 14, Pearce, D. (1984): Rationalizable Strategic Behavior and the Problem of Perfection, Econometrica, 52, Samet D. (1996) Hypothetical Knowledge and Games with Perfect Information", Games and Economic Behavior, 17,