A Folk Theorem For Stochastic Games With Finite Horizon Chantal Marlats January 2010 Chantal Marlats () A Folk Theorem For Stochastic Games With Finite Horizon January 2010 1 / 14
Introduction: A story 2 players are playing a contribution game. They know that their relationship will end at T : no contribution. Contribute Don 0 t contribute Contribute 4, 4 0, 5 Don 0 t contribute 5, 0 1, 1 Chantal Marlats () A Folk Theorem For Stochastic Games With Finite Horizon January 2010 2 / 14
Introduction: A story 2 players are playing a contribution game. They know that their relationship will end at T : no contribution. Contribute Don 0 t contribute Contribute 4, 4 0, 5 Don 0 t contribute 5, 0 1, 1 Does not capture some aspects Investment may be irreversible Non standard preferences: the history of the play matters. Chantal Marlats () A Folk Theorem For Stochastic Games With Finite Horizon January 2010 2 / 14
Introduction: A story 2 players are playing a contribution game. They know that their relationship will end at T : no contribution. Contribute Don 0 t contribute Contribute 4, 4 0, 5 Don 0 t contribute 5, 0 1, 1 Does not capture some aspects Investment may be irreversible Non standard preferences: the history of the play matters. These games are stochastic games. What about cooperation when these games have nite horizon? Such a question can be comprehended thought a Folk Theorem for stochastic games with nite horizon. Chantal Marlats () A Folk Theorem For Stochastic Games With Finite Horizon January 2010 2 / 14
Folk Theorem roughly says "everything is an equilibrium". Chantal Marlats () A Folk Theorem For Stochastic Games With Finite Horizon January 2010 3 / 14
Folk Theorem roughly says "everything is an equilibrium". implication : E cient outcomes are equilibrium in repeated interactions whereas it was not the case in a one shot game. Chantal Marlats () A Folk Theorem For Stochastic Games With Finite Horizon January 2010 3 / 14
Folk Theorem roughly says "everything is an equilibrium". implication : E cient outcomes are equilibrium in repeated interactions whereas it was not the case in a one shot game. Finite horizon: players know when the relationship will end. Chantal Marlats () A Folk Theorem For Stochastic Games With Finite Horizon January 2010 3 / 14
Folk Theorem roughly says "everything is an equilibrium". implication : E cient outcomes are equilibrium in repeated interactions whereas it was not the case in a one shot game. Finite horizon: players know when the relationship will end. implication: For some classes of games the Folk theorem holds if horizon is in nite but not when it nite. Example: prisoner s dilemma. Chantal Marlats () A Folk Theorem For Stochastic Games With Finite Horizon January 2010 3 / 14
Folk Theorem roughly says "everything is an equilibrium". implication : E cient outcomes are equilibrium in repeated interactions whereas it was not the case in a one shot game. Finite horizon: players know when the relationship will end. implication: For some classes of games the Folk theorem holds if horizon is in nite but not when it nite. Example: prisoner s dilemma. Stochastic Games: payo s depend on the state variable. Action pro le and the current in uence the probability distribution over states. Two motives of deviations: to increase the current gain and to modify the state realization. Chantal Marlats () A Folk Theorem For Stochastic Games With Finite Horizon January 2010 3 / 14
Folk Theorem roughly says "everything is an equilibrium". implication : E cient outcomes are equilibrium in repeated interactions whereas it was not the case in a one shot game. Finite horizon: players know when the relationship will end. implication: For some classes of games the Folk theorem holds if horizon is in nite but not when it nite. Example: prisoner s dilemma. Stochastic Games: payo s depend on the state variable. Action pro le and the current in uence the probability distribution over states. Two motives of deviations: to increase the current gain and to modify the state realization. implication: the usual strategies of proof are not directly applicable. Chantal Marlats () A Folk Theorem For Stochastic Games With Finite Horizon January 2010 3 / 14
The main questions of the paper Under which conditions any rational payo s can be approximated a SPE in endogeonously changing environnement when players know exactly when the relationship ends? How the answer of such a question enlightenes strategic behavior in nitely repeated non-stochastic games? Chantal Marlats () A Folk Theorem For Stochastic Games With Finite Horizon January 2010 4 / 14
The main messages of the paper Any rational payo can be approximated if the horizon is su ciently large and if the following assumptions hold: state invariance and dimensionality of the convex hull of the set of feasible payo s. Any non stochastic game can be perturbed toward a stochastic game such that any rational payo of the original game can be approximated by a SPE In other word: If the strategic situation is slightly perturbed, then the player s behavior is the same irrespective whether the horizon is nite or in nite. Chantal Marlats () A Folk Theorem For Stochastic Games With Finite Horizon January 2010 5 / 14
The main messages of the paper Any rational payo can be approximated if the horizon is su ciently large and if the following assumptions hold: state invariance and dimensionality of the convex hull of the set of feasible payo s. richness assumption of the set of SPE in some nite truncations of the game. Any non stochastic game can be perturbed toward a stochastic game such that any rational payo of the original game can be approximated by a SPE In other word: If the strategic situation is slightly perturbed, then the player s behavior is the same irrespective whether the horizon is nite or in nite. Chantal Marlats () A Folk Theorem For Stochastic Games With Finite Horizon January 2010 5 / 14
A leading example: State s L R U 4, 4 0, 5 D 5, 0 1, 1 s 0 :1 State s 0 L R U 4, 4 s:1 0, 0 s:1 D 0, 0 s:1 1, 1 s:1 Chantal Marlats () A Folk Theorem For Stochastic Games With Finite Horizon January 2010 6 / 14
A leading example: State s L R U 4, 4 0, 5 D 5, 0 1, 1 s 0 :1 State s 0 L R U 4, 4 s:1 0, 0 s:1 D 0, 0 s:1 1, 1 s:1 Claim: (4, 4) can be approximated by a SPE strategy. Consider σ (T = F + L): On the normal path: The rst F periods: U, L in each state The last L periods D, R in s and U, L in s. On the punishment path: Play D, R in each state. Chantal Marlats () A Folk Theorem For Stochastic Games With Finite Horizon January 2010 7 / 14
A leading example State s L R U 4, 4 0, 5 D 5, 0 1, 1 s 0 :1 State s 0 L R U 4, 4 s:1 0, 0 s:1 D 0, 0 s:1 1, 1 s:1 Punishment strategy is credible No deviation in order to increase the current gain: the strategy plays a Nash in each state. No deviation in order to modify the transition function: payo s are state invariant. Chantal Marlats () A Folk Theorem For Stochastic Games With Finite Horizon January 2010 8 / 14
A leading example State s L R U 4, 4 0, 5 D 5, 0 1, 1 s 0 :1 State s 0 L R U 4, 4 s:1 0, 0 s:1 D 0, 0 s:1 1, 1 s:1 No deviation from the normal path strategy. A deviation at F gives 5+1+1+...+1 1+L No deviation gives 4+1+4+1+4+..+1 1+L So for L su ciently large a deviation is no pro table. Chantal Marlats () A Folk Theorem For Stochastic Games With Finite Horizon January 2010 9 / 14
A leading example State s L R U 4, 4 0, 5 D 5, 0 1, 1 s 0 :1 State s 0 L R U 4, 4 s:1 0, 0 s:1 D 0, 0 s:1 1, 1 s:1 L is xed independently from F So as F is large the average payo s of σ are close to (4, 4). Conclusion: there is SPE strategy whose payo s are arbritrarily close to (4, 4). Chantal Marlats () A Folk Theorem For Stochastic Games With Finite Horizon January 2010 10 / 14
Example of a perturbed State s L R U 4, 4 0, 5 D 5, 0 1, 1 s 0 :η State s 0 L R U 4, 4 s:1 0, 0 s:1 D 0, 0 s:1 1, 1 s:1 If η = 0. Then (4, 4) can not be approximated by any SPE in the stochastic game starting in s. If η > 0 then (4, 4) can be approximated by a SPE. Chantal Marlats () A Folk Theorem For Stochastic Games With Finite Horizon January 2010 11 / 14
Why does it works? Possibility of credibly repeated SPE strategies at the end of the game (the last L periods). In non stochastic games the strategy that consists in repeating a SPE strategy is a SPE (conjunction property). =) Folk Theorem in nitely repeated non stochastic games. In stochastic games the strategy that consists in repeating a SPE strategy is not necessarily a SPE. =) we can t apply the proof strategy used for non-stochastic games. Chantal Marlats () A Folk Theorem For Stochastic Games With Finite Horizon January 2010 12 / 14
Intuition of the proof We identify conditions under which it is possible to have a property similar the conjunction property. We make an assumption under which it is possible to construct strategies give distinct payo s are "conjunctionable" Under some other standard assumptions we give a proof of the Folk Theorem. Chantal Marlats () A Folk Theorem For Stochastic Games With Finite Horizon January 2010 13 / 14
An important corollary We show that any non stochastic game can be perturbed toward a stochastic game such that the Folk Theorem holds. For instance: if a prisoner s dilemma is slightly but appropriately perturbed then cooperation is an outcome So if the economic situation is slightly perturbed the behavior is the same irrespective whether the horizon is nite or in nite. Chantal Marlats () A Folk Theorem For Stochastic Games With Finite Horizon January 2010 14 / 14