Learning to Coordinate Very preliminary - Comments welcome Edouard Schaal 1 Mathieu Taschereau-Dumouchel 2 1 New York University 2 Wharton School University of Pennsylvania 1/30
Introduction We want to understand how agents learn to coordinate in a dynamic environment In the global game approach to coordination, information determines how agents coordinate In most models, information comes from various exogenous signals In reality, agents learn from endogenous sources (prices, aggregates, social interactions,...) Informativeness of endogenous sources depends on agents decisions We find that the interaction of coordination and learning generates interesting dynamics The mechanism dampens the impact of small shocks......but amplifies and propagates large shocks 2/30
Overview of the Mechanism Dynamic coordination game Payoff of action depends on actions of others and on unobserved fundamental θ Agents use private and public information about θ Observables (output,...) aggregate individual decisions These observables are non-linear aggregators of private information When public information is very good or very bad, agents rely less on their private information The observables becomes less informative Learning is impeded and the economy can deviate from fundamental for a long time 3/30
Roadmap Stylized game-theoretic framework Characterize equilibria and derive conditions for uniqueness Explore relationship between decisions and information Study the planner s problem Provide numerical examples and simulations along the way 4/30
Abbreviated Literature Review Learning from endogenous variables Angeletos and Werning (2004); Hellwig, Mukherji and Tsyvinksi (2005): static, linear-gaussian framework (constant informativeness) Angeletos, Hellwig and Pavan (2007): dynamic environment, non-linear learning, fixed fundamental, stylized cannot be generalized Chamley (1999): stylized model with cycles, learning from actions of others, public signal is fully revealing upon regime change and uninformative otherwise 5/30
Model Infinite horizon model in discrete time Mass 1 of risk-neutral agents indexed by i [0,1] Agents live for one period and are then replaced by new entrant Each agent has a project that can either be undertaken or not 6/30
Model Realizing the project pays π it = (1 β)θ t +βm t c where: θ t is the fundamental of the economy Two-state Markov process θt {θ l,θ h }, θ h > θ l with P(θ t = θ j θ t 1 = θ i) = P ij and P ii > 1 2 m t is the mass of undertaken projects plus some noise β determines the degree of complementarity in the agents payoff c > 0 is a fixed cost of undertaking the project 7/30
Information Agents do not observe θ directly but have access to several sources of information 1 A private signal v it Drawn from cdf Gθ for θ {θ l,θ h } with support v [a,b] Gθ are continuously differentiable with pdf g θ Monotone likelihood ratio property: gh (v)/g l (v) is increasing 2 An exogenous public signal z t drawn from cdf F z θ and pdf f z θ 3 An endogenous public signal m t Agents observe the mass of projects realized with some additive noise ν t m t(θ, ˆv) = mass of projects realized+ν t νt iid cdf F ν with associated pdf f ν Assume without loss of generality that F ν has mean 0 8/30
Timing Agents start with the knowledge of past public signals z t and m t 1 θ t is realized 2 Private signals v it are observed 3 Decisions are made 4 Public signals m t and z t are observed 9/30
Information Information sets: At time t, the public information is F t = { m t 1,z t 1} Agent i s information is F it = {v it } F t Beliefs: Beliefs of agent i about the state of the world p it = P (θ = θ h F it ) Beliefs of an outside observer without private information p t = P (θ = θ h F t ) 10/30
Agent s Problem Agents i realizes the project if its expected value is positive E [(1 β)θ t +βm t c F it ] > 0 For now, restrict attention to monotone strategy equilibria: There is a threshold ˆv t such that Agent iundertakes his project v it ˆv t Later, we show that all equilibria have this form With this threshold strategy, the endogenous public signal is m t = 1 G θ (ˆv t ) }{{} signal + ν t }{{} noise 11/30
Dynamics of Information For a given signal s t, beliefs are updated using the likelihood ratio LR it = P (s t θ h,f it ) P (s t θ l,f it ) Using Bayes rule, we have the following updating rule P (θ h p it,s t ) = 1 1+ 1 pit p it LR 1 := L(p it,lr it ) it 12/30
Dynamics of Information At the beginning of every period, the individual beliefs are given by ( p it (p t,v it ) = L p t, g ) h(v it ) g l (v it ) By the end of the period, public beliefs p t are updated according to pt end = L (p t, f h z (z ) t) P (m t θ h,f t ) fl z (z t ) P (m t θ l,f t ) Moving to the next period, p t+1 = pt end P hh + ( 1 pt end ) Plh Full expression for dynamic of p 13/30
Dynamics of Information Lemma 1 The distribution of individual beliefs is entirely described by (θ, p): ˆ P (p i p θ,p) = 1I 1 p dg θ (v i ). 1+ 1 p p g l (v i) g h (v i) Conditional on θ agents know that all signals come from G θ From G θ and p they can construct the distribution of beliefs Rich structure of higher-order beliefs in the background 14/30
Monotone Strategy Equilibrium Definition A monotone strategy equilibrium is a threshold function ˆv(p) and an endogenous public signal m such that 1 Agent i realizes his project if and only if his v i is higher than ˆv(p) 2 The public signal m is defined by m = 1 G θ (ˆv (p))+ν 3 Public and private beliefs are consistent with Bayesian learning Given the payoff function π(v i ;ˆv,p) = E [(1 β)θ +β(1 G θ (ˆv)) c p,v i ] the threshold function ˆv(p) satisfies π(ˆv(p); ˆv(p),p) = 0 for every p. 15/30
Equilibrium Characterization: Complete Information Lemma 2 (Complete info) If β c (1 β)θ 0, the economy admits multiple equilibria under complete information. In particular, there is an equilibrium in which all projects are undertaken and one equilibrium in which no projects are undertaken. 16/30
Equilibrium Characterization: Incomplete Information Assumption 1 The likelihood ratio g h g l is differentiable and there exists ρ > 0 such that ( gh g l ) ρ. Proposition 1 (Incomplete info) Under assumption 1, 1 If 2 If β 1 β θ h θ l, all equilibria are monotone, β 1 β Uniqueness requires: ρp hl P lh max{ g h, g l } 3, there exists a unique equilibrium. 1 an upper bound on β; Role of β 2 enough beliefs dispersion. Role of dispersion 17/30
Endogenous vs Exogenous Information Sample path with only exogenous information: 1.0 0.8 0.6 0.4 0.2 0.0 1 Gθ(ˆv) θ 0 200 400 600 800 1000 Sample path with only endogenous information: 1.0 0.8 0.6 0.4 0.2 0.0 1 Gθ(ˆv) θ 0 200 400 600 800 1000 From now on, focus on endogenous public signal only: Var(z t ) 18/30
Endogenous Information Lemma 3 If F ν N(0,σ 2 ν), then the mutual information between θ and m is where = G l (ˆv) G h (ˆv) 0. I(θ;m) = p(1 p) 2 2σν 2 +O ( 3) Version of the Lemma with general F ν : General Lemma The informativeness of the public signal depends on: 1 The current beliefs p 2 The amount of noise σ ν added to the signal 3 The difference between G l (ˆv) and G h (ˆv) Point 3 is the source of endogenous information. Definition of mutual information 19/30
Signal vs. Noise Example 1: Normal case with different means µ h > µ l gθ 4 3 2 1 0 g l Distributions 0 0.2 0.4 0.6 0.8 1 g h v i Gl Gh 1 0 Signal distance = G l (ˆv) G h (ˆv) 0 0.2 0.4 0.6 0.8 1 ˆv Result: more information when ˆv = µ h+µ l 2, i.e., 0 m 1. Alt. signals 20/30
Inference from Endogenous Signal m t = 1 G θ (ˆv t ) }{{} signal + ν t }{{} noise Example 1: Normal case with different means µ h > µ l 4 gθ 3 2 1 0 g l g h 0 0.2 0.4 0.6 0.8 1 ˆv 1 1 Gθ 0 1 G l (ˆv)±σ ν 1 G h (ˆv)±σ ν 0 0.2 0.4 0.6 0.8 1 21/30
Signal vs. Noise Example 2: Information contained in m under the equilibrium ˆv 0.7 0.6 Mutual information 0.5 0.4 0.3 0.2 0.1 0.0-0.1 0.0 0.2 0.4 0.6 0.8 1.0 Current beliefs p Result: in the extremes of the state-space, the endogenous signal reveals no information Parameters 22/30
Coordination Traps Proposition 2 (Coordination traps) Under the conditions of proposition 1, 1 If (1 β)θ l c (1 β)θ h, there exists p [0,1], such that for all p p, ˆv (p) = b, i.e., nobody undertakes the project; 2 If (1 β)θ l +β c (1 β)θ h +β, there exists p [0,1], such that for all p p, ˆv (p) = a, i.e., everyone undertakes the project; 3 For p p and p p, m contains no information about θ. Furthermore, the regions with no and full activity widen with the degree of complementarity β: p (β) < 0 and p (β) > 0. We refer to the set [0,p] [ p,1] has the no-learning zone. Details Agents disregard their private information and all act together m is independent of the true state of the world 23/30
Signal vs. Noise: Role of β Example 2: Information contained in m under the equilibrium ˆv 0.7 0.6 β = 0 β = 0.3 Mutual information 0.5 0.4 0.3 0.2 0.1 0.0-0.1 0.0 0.2 0.4 0.6 0.8 1.0 Current beliefs p Result: the complementarity lowers informativeness and widens the no-learning zones Parameters Details for p > 1 2, higher β implies more projects realized (ˆv a) for p < 1 2, higher β implies fewer projects realized (ˆv b) 24/30
Complementarity and the Persistence of Recession To summarize: Higher complementarity reduces informativeness of public signals in the extremes of the state space In the no-learning zone, agents get no information from public signal As a result, an economy with high complementarity might resist well to brief shocks; magnify the duration of booms/recessions after a lengthier shock. 25/30
Persistence of Recession The economy with high complementarity resists well to brief shocks... 1.0 0.8 0.6 0.4 0.2 0.0 θ m with β =0.0 m with β =0.3 0 10 20 30 40 50...but recovers slowly after lengthy shocks. 1.0 0.8 0.6 0.4 0.2 0.0 θ m with β =0.0 m with β =0.3 0 10 20 30 40 50 26/30
Bubble-like Behavior The complementarity makes the response to ν shocks highly non-linear. 2 σ f positive shock to ν: 1.0 0.8 0.6 0.4 0.2 0.0 1 Gθ(ˆv) ν 0 10 20 30 40 50 3 σ f positive shock to ν: 1.0 0.8 0.6 0.4 0.2 0.0 1 Gθ(ˆv) ν 0 10 20 30 40 50 4 σ f positive shock to ν: 1.0 0.8 0.6 0.4 0.2 0.0 1 Gθ(ˆv) ν 0 10 20 30 40 50 27/30
Efficiency Agents don t internalize the impact of their decision on m. There are two externalities: 1 Complementarity: a higher m increases the payoff of others 2 Information: m influences the amount of information revealed We adopt the formulation of Angeletos and Pavan (2007): Planner cannot aggregate the information dispersed across agents He maximizes the ex-ante welfare of agents according to their own individual beliefs ˆ b V(p) = maxe θ,ν E θ,ν [π it (θ,ˆv) F it ] +γv(p ) F t ˆv ˆv }{{} Agent i s expected payoff subject to the same law of motion for the public beliefs: p (p,ˆv). 28/30
Dynamics in the Efficient Allocation Response to shock in the efficient allocation vs equilibrium 1.0 0.8 0.6 0.4 0.2 0.0 θ Efficient m Equilibrium m 0 10 20 30 40 50 Planner s decision compared to equilibrium: Complementarity Information externality p low more agents act more agents act p high more agents act less agents act The planner responds to recessions more than to booms. 29/30
Conclusion Summary We have built a model in which the interaction of coordination motives and endogenous information generates persistent episodes of expansions and contractions. Optimal government intervention reduces the length of recessions while keeping the expansions mostly unchanged. Extensions Large government spending multiplier? Generalized payoff function and endogenous public signal Intensive margin and unbounded distributions Long-lived agents with dynamic decision Applications Unemployment fluctuations, investment dynamics, currency attacks, bank runs, asset pricing, etc. 30/30
Dynamic of Information The public beliefs evolve according to p = P hhpfh z (z)f (m 1+G h(ˆv))+p lh (1 p)fl z (z)f (m 1+G l (ˆv)) pfh z (z)f (m 1+G h(ˆv))+(1 p)f l z (z)f (m 1+G l (ˆv)) Details 30/30
General Statement of Mutual Information Lemma Lemma 4 The mutual information between θ and m is where = G l (ˆv) G h (ˆv) 0 and I(θ;m) = p(1 p) 2 Γ+O ( 3) Γ = ˆ [ d2 f ν dν 2 + 1 2f ν If F ν N(0,σ 2 ν ), then Γ = (2σ2 ν ) 1. ( ) ] df ν 2 dν. dν Return 30/30
Mutual Information Definition 1 The mutual information between θ and m is I(θ;m) = H(θ) H(θ m) = ˆ ( ) P(θ,m) P(θ,m)log dm P(θ)P(m) where H denotes the entropy. θ {θ L,θ H } m Return 30/30
Numerical Example Description Value Low fundamental value θ L = 0 High fundamental value θ H = 1 Persistence of fundamental q = 0.99 Cost of investment c = 0.5 Time discount γ = 0.5 Private signal in state H G H N(0.8,0.4) truncated on [0,1] Private signal in state L G L N(0.2,0.4) truncated on [0,1] Noise in public signal F N(0,0.1) Return 30/30
Signal vs. Noise Example 1.1: Truncated normals case with different variances σ h < σ l : gθ 4 3 2 1 0 Distributions 0 0.2 0.4 0.6 0.8 1 g l g h v i Gl Gh 1 0 Signal distance 0 0.2 0.4 0.6 0.8 1 ˆv Result: informativeness of signal depends on underlying distributions Return 30/30
Uniqueness: Intuition Recall the payoff function: π(v i ;ˆv,p) = (1 β)e i [θ] + βe i [1 G θ (ˆv)] c }{{}}{{} Fundamental Complementarity we re looking for π(ˆv;ˆv,p) = (1 β)e[θ ˆv]+βE[1 G θ (ˆv) ˆv] c Example: normal case with different means µ h > µ l gθ 4 3 2 1 0 g l g h 0 0 0.2 0.4 0.6 0.8 1 v i pi Ei[1 Gθ(vi)] Fundamental 1 0 0.2 0.4 0.6 0.8 1 vi Complementarity 1 0 0 0.2 0.4 0.6 0.8 1 vi 30/30
Role of complementarity β 0.2 π(ˆv;ˆv,p) 0 β = 0 β = 0.05 β = 0.1 β = 0.2-0.2 0 0.2 0.4 0.6 0.8 1 ˆv Result: Uniqueness requires upper bound on complementarity Return 30/30
Role of belief dispersion 0.2 low dispersion π(ˆv;ˆv,p) 0 high dispersion -0.2 0 0.2 0.4 0.6 0.8 1 ˆv Result: Uniqueness requires enough belief dispersion Return Distributions g h, g l sufficiently dispersed Fundamental sufficiently volatile (P hl and P lh high enough) 30/30
Coordination Traps 0.2 m = 1 π(ˆv; ˆv,p) 0 higher p p p m = 0-0.2 0 0.2 0.4 0.6 0.8 1 Result: endogenous channel uninformative for extreme values of p for p < p, no project realized: ˆv = b, θ l and θ h are indistinguishable 1 G h (b) = 1 G l (b) = 0 for p > p, all projects realized: ˆv = a, θ l and θ h are indistinguishable 1 G h (a) = 1 G l (a) = 1 ˆv Return 30/30
Signal vs. Noise: Role of β 0.2 β =.02 β =.1 p > 1 2 π(ˆv;ˆv,p) 0 p < 1 2-0.2 0 0.2 0.4 0.6 0.8 1 ˆv Result: high complementarity induces convergence in strategies for p > 1 2, higher β implies more projects realized (ˆv a) for p < 1 2, higher β implies fewer projects realized (ˆv b) Return 30/30