Distributional stability Dai Zusai 1 Distributional stability and equilibrium selection Evolutionary dynamics under payoff heterogeneity (II) Dai Zusai Tohoku University (Economics) August 2017
Outline Distributional stability Dai Zusai 2 1 Outline Motivation This paper s contribution 2 Model 3 Stationarity and stability of equilibrium composition 4 Nonaggregable dynamic properties of aggregate equilibria 5 Distributional stability
Outline Motivation Distributional stability Dai Zusai 3 Evolutionary dynamics This series of my papers: effects of heterogeneity on aggregate social dynamics in the context of evolutionary dynamics. Evolutionary dynamic: social dynamic constructed from a pre-programmed agent s strategy revision process 1 Each agent randomly receives a revision opportunity, which follows a Poisson process. 2 Upon revision opportunity, an agent updates the information about the social state (payoff vector and/or strategy distribution) and revises the agent s own strategy according to revision protocol. 3 The dynamic of strategy distribution the population dynamic is constructed by gathering these individual revision processes over all the agents. e.g. Replicator (Taylor & Jonker): Imitation of another randomly sampled agent s strategy, with a payoff-based switching rate. e.g. Smith: Switching to a randomly chosen strategy, with a payoff-based switching rate. e.g. Standard BRD (Gilboa & Matsui, Hofbauer): Optimization with a constant switching rate. e.g. Tempered BRD (Zusai, IJGT, forthcoming): Optimization with a payoff-based switching rate. Deterministic dynamic in large population Assume a continuum of homogeneous agents. (A group of these agents is assigned to one player in a conventional game. Their strategy distribution corresponds to a mixed strategy of a player.) The population dynamic is defined by a deterministic differential equation/inclusion, which is constructed through tracking expected (mean) transition of a representative individual revision process. This is justified by appealing to the law of large number. (Rigorously proven by Benaim, Hofbauer, Sorin; Roth and Sandholm.)
Outline Motivation Distributional stability Dai Zusai 4 Heterogeneity in evolutionary dynamics Heterogeneity Different roles in game: extending to a multi-population dynamic, in which the social dynamic consists of a system of differential equations; each equation determines the dynamic of each different population. Different (more subtle, possibly continuous difference) tastes in preference: not much studied in the preceding literature on evolutionary dynamics. Different revision protocols: seeking for asymptotic symmetry (Sawa & Zusai) or unifying general proof (Zusai Gains ) The first two are not exclusive in construction. But heterogeneity in preference leads to technical difficulty of dealing with continuous type space, though it is normally used in economics. (e.g. discrete choice model like logit) Dimensionality Strategy composition: the distribution of strategies in each type, or the joint distribution of type and strategy. Aggregate strategy: the distribution of strategies in the whole society, i.e., the marginal distribution. The preceding literature on heterogeneous evolutionary dynamics (Blonski 1999, GEB; Ely & Sandholm, 2005, JET) relies on aggregability. In an aggregable dynamic like the standard BRD, the transition of the aggregate strategy is always totally determined by its current state, free from the strategy composition. But evolutionary dynamics are generally not aggregable.
Outline This paper s contribution Distributional stability Dai Zusai 5 This series of paper I work on two papers (split from one long paper) on heterogeneous evolutionary dynamics. I Nonaggregable evolutionary dynamics under payoff heterogeneity Considers a general class of evolutionary dynamics, and a game with an arbitrary finite number of strategies. Provides a regularity condition to well-define a diff l equation on a measure space to describe transition of the strategy composition in heterogeneous evolutionary dynamics; Verifies that stationarity of Nash equilibria and equilibrium stability in potential games are extended from a homogeneous dynamic to a heterogeneous dynamic of strategy composition, given the revision protocol of an individual agent; In particular, verifies that local stability of each equilibrium composition in a potential game under a heterogeneous dynamic can be checked just from local stability under the aggregable standard BRD. II Distributional stability and equilibrium selection (today s main dish, still working) Focuses on a binary game; Shows mid/long-run implications of nonaggregability, such as unstationarity of aggregate equilibrium; strategy composition may escape from an aggregate eqm, which is locally stable in the aggregable standard BRD. Proposes to utilize this difference to select the most robustly stable equilibrium under heterogeneity.
Model Distributional stability Dai Zusai 6 1 Outline 2 Model Population game Dynamics Aggregability 3 Stationarity and stability of equilibrium composition 4 Nonaggregable dynamic properties of aggregate equilibria 5 Distributional stability
Model Population game Distributional stability Dai Zusai 7 Population game with heterogeneous payoffs We consider a binary-choice large population game: Unit mass of continuously many agents ω Ω Each agent chooses either action In or Out. (I players: participants) Strategy profile a : Ω {O, I}: each agent s choice Strategy composition x : Θ [0, 1]: participation rate of each type Aggregate strategy x [0, 1]: the total mass of participants in the whole society Payoff heterogeneity: For agent ω, Payoff from In is F( x), depending on the aggregate state alone but common to all agents; Payoff from Out is θ(ω) Θ, varying with agents but constant. (Call θ(ω) agent ω s type.) Idiosyncratic payoff heterogeneity θ is assumed to be persistent over time. Notation about the type distribution P probability measure on type space Θ; p its density. E the aggregater (operator for expected value): x = Ex := Θ x(θ)dp(θ). N.B. Distance between strategy compositions is defined by variational norm, E x x. The paper: different symbols, distinction between the joint distribution of types and strategies (str composition) and the conditional distribution of strategies on types ( Bayesian strategy )
Model Population game Distributional stability Dai Zusai 8 Equilibrium composition and aggregate equilibrium Each type s best response: dependent on the aggregate state alone: I if F( x) θ; O if F( x) θ So type θ = F( x) is the threshold between I and O. BR composition and aggregate BR are determined from aggregate strategy { alone: 1 if θ < F( x), Strategy composition in best response is x BR (θ; x) := 0 if θ > F(y). Aggregate strategy in best response becomes Ex BR ( ; x) = P(θ < F( x)). Two kinds of equilibria: Strategy composition x is an equilibrium composition, if x = x BR ( ; Ex) Aggregate strategy x is an aggregate equilibrium, if x = P(θ < F( x)). x is eqm comp Eqm comp requires complete sorting by types. x := Ex is agg eqm.
Model Dynamics Distributional stability Dai Zusai 9 Exact optimization dynamics In general, an evolutionary dynamic normally assumes that an agent switches only to a better action. In a binary game, better must be the best. Thus it is not restrictive to limit attention to the following class of dynamics: Definition Exact optimization dynamics Given payoff vector π (for the agent s type), an agent who receives a revision opportunity switches from current action i to another action j = i with probability { 0 if j / argmax R ij (π(θ)) = a A π a,. Q ij (π(θ)) if {j} = argmax a A π a Here, Q ij : R A R + is assumed to be a Lipschitz continuous function. Then, the population dynamic is constructed as ẋ(θ) = v F (x(θ)) := (1 x(θ))r OI (F(Ex), θ) x(θ)r IO (F(Ex), θ) { (1 x(θ))qoi (F(Ex), θ) if I is optimal for θ, i.e., F(Ex) > θ = x(θ)q IO (F(Ex), θ) if O is optimal for θ, i.e., F(Ex) < θ.
Model Dynamics Distributional stability Dai Zusai 10 Aggregability of standard BRD (Ely & Sandholm 2005 GEB) The canonical example of exact optimization dynamics is the (standard) BRD. Definition Standard best response dynamic (Gilboa & Matsui; Hofbauer) 1 Each agent receives a revision opportunity with constant probability per unit of time (constant revision rate). 2 Upon the revision opportunity, the agent switches to the best response to the current state with sure, i.e., Q ij 1. Under the standard BRD, strategy changes towards BR from the current strategy at a given constant speed. ẋ(θ) = x BR (θ; Ex) x(θ). Ely & Sandholm rigorously verify that the aggregate str x = E Θ x under the heterogeneous standard BRD follows the homogeneized smooth BRD d x = P(θ < F( x)) x, dt in which the idiosyncratic payoff heterogeneity is only transient (independently drawn from the same P at each revision opportunity ) and the agent then switches to the ex-post optimal strategy with a constant rate. e.g. P: double exponential dist Homogenized smooth BRD: logit dynamic Standard BRD: Strategy composition does not affect the dynamic of aggregate strategy. x : aggregate eqm x stays there, whatever x is.
Model Dynamics Distributional stability Dai Zusai 11 Transition of strategy composition: standard BRD p(θ) : density of all the type-θ agents x(θ)p(θ) : density of type-θ participants Here let θ be the threshold type.
Model Dynamics Distributional stability Dai Zusai 12 Transition of strategy composition: standard BRD (case 2) p(θ) : density of all the type-θ agents x(θ)p(θ) : density of type-θ participants Here let θ be the threshold type.
Model Dynamics Distributional stability Dai Zusai 13 BRD with payoff-sensitive revision rate (tempered BRD) Constant revision rate sounds strong especially under payoff heterogeneity. More incentive to switch the strategy more likely to switch incentive of revision = the payoff from best response the current payoff (payoff deficit of the current strategy) Definition Tempered BRD (from stochastic status-quo bias) (Zusai) 1 Each agent receives a revision opportunity with constant probability per unit of time (constant revision rate). 2 Upon the revision opportunity, each agent compares the payoff deficit with a stochastic status-quo bias q, whose cdf is Q, and the best response to the current state only if the former is greater than the latter. Basic properties such as Nash stationarity and stability are proven. Payoff heterogeneity causes different revision rates through payoff-dependent revision rates. Assume Q(0) = 0 and Q (q) > 0 for all q.
Model Dynamics Distributional stability Dai Zusai 14 Transition of strategy composition: tempered BRD p(θ) : density of all the type-θ agents x(θ)p(θ) : density of type-θ participants Here let θ be the threshold type.
Model Dynamics Distributional stability Dai Zusai 15 Transition of strategy composition: tempered BRD (case 2) p(θ) : density of all the type-θ agents x(θ)p(θ) : density of type-θ participants Here let θ be the threshold type.
Model Aggregability Distributional stability Dai Zusai 16 Aggregability of heterogeneous dynamics Heterogeneous evolutionary dynamic d dt x = vf (x) is aggregable if there is an aggregate dynamic v F : [0, 1] [0, 1] such that [ x t = Ex t and d ] dt x t = v F [x t ] = d dt x t = v F ( x t ). Ely & Sandholm (2005) This condition is equivalent to interchangeabibility of aggregation and dynamic, i.e., Ev F [x] = v F (Ex). The standard BRD is aggregable: reduced to the homogenized smooth BRD. But generally an evolutionary dynamic is not aggregable. Theorem Generic nonaggregability (Zusai) Consider an exact optimization dynamic in a binary game. Assume continuous type distribution. Then, the dynamic is aggregable if and only if the correlation between the switching rates to the optimal action and the action distribution over different types is always zero.
Eqm composition Distributional stability Dai Zusai 17 1 Outline 2 Model 3 Stationarity and stability of equilibrium composition 4 Nonaggregable dynamic properties of aggregate equilibria 5 Distributional stability
Eqm composition Distributional stability Dai Zusai 18 Eqm stationarity and stability as general desirata Common general requirement for well-behaved dynamics Stationarity of an eqm (no profitable deviation no switch) Stability of an eqm in some good cases more specifically, asymptotic stability of the eqm set in a potential game. Potential game: the payoff function (mapping from a strategy distribution to a payoff vector) exhibits a function (potential function) that maps from a strategy distribution to a real number and whose derivative always coincides with the payoff vector. A binary game is a potential game. A potential maximizer is a Nash eqm, though not vice versa. Fundamental properties of revision protocols to dynamic properties of population dynamics In a homogeneous setting, it is known that BR stationarity: an agent does not switch action if its current action is optimal Stationarity of each Nash equilibrium Positive correlation: expected change in an agent s payoff by switch is positive if its current action is not optimal Global convergence to the Nash eqm set in a potential game, and local asymptotic stability of a potential-maximizing Nash eqm. Admissibility: satisfying both the BR stationarity and PC.
Eqm composition Distributional stability Dai Zusai 19 Stationarity and stability of eqm composition Theorem Stationarity and stability of eqm composition (Zusai, I) i) In a heterogeneous setting, Best response stationarity Stationarity of each eqm composition Positive correlation Global convergence to the set of eqm composition in a potential game, and local asymptotic stability of a potential-maximizing eqm composition. ii) Let x be an eqm composition and x = Ex be a corresponding aggregate eqm. Then, the followings are equivalent. x is asymptotically stable in an admissible heterogeneous evolutionary dynamic; x is asymptotically stable in the homogenized smooth BRD. These are proven in a general setting with finitely many actions and a general class of evolutionary dynamics. However, a basin of attraction to x is defined on the space of strategy composition. That is, even if the aggregate strategy is close to x (e.g. in the basin of attraction to x under the homogenized smooth BRD), the strategy composition may not converge to x unless the dynamic is aggregable.
Nonaggregability Distributional stability Dai Zusai 20 1 Outline 2 Model 3 Stationarity and stability of equilibrium composition 4 Nonaggregable dynamic properties of aggregate equilibria Stationarity Escape from a stable aggregate eqm Example 5 Distributional stability
Nonaggregability Stationarity Distributional stability Dai Zusai 21 Nonstationarity of an aggregate equilibrium Aggregable dynamics: an aggregate equilibrium is stationary. Once x reaches exactly an aggregate eqm, it never moves regardless of underlying composition. Nonaggregable dynamic: stationarity depends on the underlying composition. For an exact optimization dynamic, define the following terms: Sources of inflow Ω OI 0 and of outflow Ω0 IO Ω OI 0 := {ω Ω : agent ω is currently taking O but I is optimal at time 0 for its type θ(ω)} Ω IO 0 := {ω Ω : agent ω is currently taking I but O is optimal at time 0 for its type θ(ω)} Switching rate distributions in the source of inflows Q OI 0 and in that of outflows Q 0 IO ( ) ( )] Q OI 0 [(1 (q) := E x 0 (θ))1 I is optimal at time 0 for type θ 1 Q OI 0 q, Q IO 0 (q) := E [x a,0 (θ)1 ( ) ( )] O is optimal at time 0 for type θ 1 Q0 IO q Here Q ij 0 = Q ij(f( x 0 ), θ) is type-θ s switching rate from i to j at time 0, provided that j is optimal.
Nonaggregability Stationarity Distributional stability Dai Zusai 22 Balancing condition for stationarity of an aggregate equilibrium Theorem Stationarity of aggregate equilibrium in exact optimization dynamics Consider an exact optimization dynamic. Let x A be an aggregate equilibrium. The trajectory of the aggregate strategy { x t } stays at x for time interval [0, T] (with any T > 0), if and only if the underlying Bayesian strategy x 0 at time 0 satisfies the detailed balancing condition: Q OI 0 (q) Q IO 0 (q) for all q R, a A. cf. Aggregate eqm Q OI 0 (+ ) = P(ΩOI 0 ) = P(ΩIO 0 ) = Q 0 IO(+ ).
Nonaggregability Escape Distributional stability Dai Zusai 23 Stochastic dominance and finite-time escape What if the detailed balancing condition is not satisfied? In general, the transition of aggregate strategy x is determined from the switching rate distributions: d + dt x a,t = + q Q OI t (dq) Assumption Positive externality of common payoff F( x) increases with x [0, 1]. Assumption Monotonicity of switching rate function Q ij (π) never decreases whenever payoff deficit π i increases. q Q t IO (dq). Theorem Escape from aggregate equilibrium with no return in finite time Consider a monotone exact opt n dynamic in a binary game with positive externality. Let x 0 = x ; assume that x is an aggregate equilibrium. If Q 0 IO dominates Q OI 0 at time 0 in the sense of second-order stochastic domination, then x t is always smaller than x at any t (0, ). Distribution Q 0 IO on R dominates Q OI 0 in the sense of second-order stochastic domination if q IO Q 0 (q)dq q OI Q 0 (q)dq for any q R with > over some interval. This is weaker than first-order stochastic dom ce, i.e., Q0 IO( q) Q OI 0 ( q) for any q R with > over some interval.
Nonaggregability Escape Distributional stability Dai Zusai 24 Robust critical mass x is a robust critical mass to decrease x, if d dt x = E d dt x(θ) < 0 when x = x regardless of the underlying str composition. Say that the assumptions for the last theorem are satisfied. x t < x t > 0. Further, suppose that x < x and x is a robust critical mass to decrease x I. If the escaping path x t hits x in a finite time, then it never returns to x even asymptotically. Theorem Long-run escape from an aggregate equilibrium Consider a monotone exact opt n dynamic in a binary game with positive externality. Assume that Q IO (π) > 0 whenever π O > π I. Define x t [0, 1] by x t := x t If there exists t R + such that 0 0 t e qτ Q OI 0 (dq)dτ + e qt Q 0 IO(dq)dτ for each t R +. 0 0 x τ < x for any τ (0, t] and x t < x, then the aggregate mass of action-i players x first gets smaller than x, reaches the robust critical mass x in a finite time, and stays smaller than x forever since then.
Nonaggregability Escape Distributional stability Dai Zusai 25 How to find a robust critical mass? x is a robust critical mass to decrease x, if E d dt x(θ) < 0 for any str composition x with Ex(θ O) = x. This reduces to two (jointly sufficient) conditions: a) E dt d x(θ) < 0 at the perfectly sorted composition such that x(θ) = 0 for any θ O > P 1 ( x ) and x I (θ O ) = 1 for any θ < P 1 ( x ). b) This composition yields the greatest net increase d x I /dt of action-i players among all str compositions such that E Θ x(θ O ) = x 1 ẋ I (θ O ) x I (θ o ) 1 ẋ I (θ O ) x I (θ o ) 0 θ O x BR I (θ o ) F I 0 ( x I ) P Θ 1 ( x I ) θ O 0 θo x BR I (θ o ) F I 0 ( x I ) P Θ 1 ( x I ) θ O Perfectly sorted composition Unsorted composition
Nonaggregability Escape Distributional stability Dai Zusai 26 Sufficient condition for a robust critical mass Theorem Sufficient condition for a robust critical mass Consider a monotone exact opt n dynamic in a binary game with positive externality. Assume that Q IO (π) > 0 whenever π O > π I. x (0, 1] is a robust critical mass to decrease x, if a) P 1 ( x ) > F( x ), and b) Q IO (F( x ), P 1 ( x I )) sup θ Θ (,F( x )) Q OI (F( x ), θ). Note that condition a) d x I /dt < 0 under the homogenized smooth BRD. 1 ẋ I (θ O ) x I (θ o ) 1 ẋ I (θ O ) x I (θ o ) 0 θ O x BR I (θ o ) F I 0 ( x I ) P Θ 1 ( x I ) θ O 0 θo x BR I (θ o ) F I 0 ( x I ) P Θ 1 ( x I ) θ O Perfectly sorted composition Unsorted composition
Nonaggregability Example Distributional stability Dai Zusai 27 Example: standard BRD Specify the common payoff function and the type distribution as payoff fn: F( x) = (49 x 1)/20 c.d.f. of type dist: P(θ) = θ + 1 1 with support Θ = [0, 3] There are three aggregate eq ia: x = 0, x = 0.2, x = 0.25. Under the homogenized smooth BRD, Stable: x = 0 and x = 0.25 Unstable: x = 0.2. 0 x I 0.2 0.25 x I x I = P Θ (F I 0 ( x I )) x I Homogenized smooth BRD
Nonaggregability Example Distributional stability Dai Zusai 28 Example: tempered BRD Under the heterogeneous tbrd with Q( π) = π 3, x = 0.10 satisfies the sufficient conditions for a robust critical mass to decrease x I. As an extreme case, start from a reversely sorted composition: { = 1 for all θ>33/16, x(θ) = 0 for all θ<33/16. This composition aggregates to Ex(θ) = 0.25, a stable aggregate eqm under the heterogeneous standard BRD. But this composition satisfies the condition for long-run escape. The aggregate strategy escapes from x = 0.25 and then converges to x = 0. Standard BRD Tempered BRD
Distributional stability Distributional stability Dai Zusai 29 1 Outline 2 Model 3 Stationarity and stability of equilibrium composition 4 Nonaggregable dynamic properties of aggregate equilibria 5 Distributional stability Distributional stability Equilibrium selection
Distributional stability Distributional stability Distributional stability Dai Zusai 30 Distributional stability Definition Distributional stability Aggregate equilibrium x is distributionally stable, if x t converges to x whenever the initial aggregate strategy Ex 0 is sufficiently close to x, regardless of the underlying strategy composition. Note that distributional stability = local stability. Theorem Distributional stability by robust critical masses Let x < x, x be a robust critical mass to increase x, and x be a robust critical mass to decrease x. If aggregate equilibrium x lies in interval ( x, x ), then x is distributionally stable. In the last example, x = 0 is the only distributionally stable aggregate eqm under the heterogeneous tbrd with Q( π) = π 3.
Distributional stability Equilibrium selection Distributional stability Dai Zusai 31 Equilibrium selection by distributional stability We expect the transition of aggregate strategy to become more dependent on the strategy composition, as payoff sensitivity of switching rate function increases. Any tractable parameterization of payoff sensitivity? The tbrd with a bounded support of Q, i.e., Q stops increasing at π < +, i.e., Q( π) = Q( π ) for any π π. Then, x is a robust critical mass to decrease x I, if P 1 ( x ) F( x ) π i.e., P 1 ( x ) F( x ) + π. x is a robust critical mass to decrease x I, if F( x ) P 1 ( x ) π i.e., P 1 ( x ) F( x ) π. Let s call an agg eqm the most robust if it remains to be the only distributionally stable eqm when π increases.
Distributional stability Equilibrium selection Distributional stability Dai Zusai 32 Relation with risk dominance Coordination game: F( x) = a(1 x) + b x with a, b > 0. a < 0 is the payoff from I when nobody takes I b > 0 is the payoff from I when everybody takes I Without heterogeneity θ 0, there are three eq ia: x = 0, 1, a/(a + b). x = 1 is risk dominant, if I is optimal when x = 1/2. x = 0 is risk dominant, if O is optimal when x = 1/2. Assume that P(0) = 1/2 and P is point-symmetric around θ = 0. Each eqm is shifted when heterogeneity is introduced. Then, under tbrd with a bounded support of Q, [The most robust agg eqm] [(Shifted from) risk dominant eqm]
Distributional stability Equilibrium selection Distributional stability Dai Zusai 33 Discussion Summary Generic nonaggregability: the transition of aggregate strategy depends on the strategy composition. Stationarity and local stability of equilibrium composition cannot be reduced to aggregate equilibrium. That is, the dynamic on the type-strategy joint distribution is essential. However, we have a nice positive result to link local stability of eqm composition under nonaggregable heterogeneous dynamics with that of aggregate eqm under the homogenized dynamic So, we propose to use nonaggregability as a tool to further select the most robust aggregate eqm. Stochastic interpretation of distributional stability Distributional shock Switching costs in conventional models of stochastic evolution Technical merits of selection by distributional stability Deterministic dynamics Assessed in finite time