Population Games and Evolutionary Dynamics

Transcription

1 Population Games and Evolutionary Dynamics William H. Sandholm The MIT Press Cambridge, Massachusetts London, England

2 in Brief Series Foreword Preface xvii xix 1 Introduction 1 1 Population Games 2 Population Games 21 3 Potential Games, Stable Games, and Supermodular Games. 53 II Deterministic Evolutionary Dynamics 4 Revision Protocols and Evolutionary Dynamics Deterministic Dynamics: Families and Properties Best Response and Projection Dynamics 177 III Convergence and Nonconvergence of Deterministic Dynamics 7 Global Convergence of Evolutionary Dynamics Local Stability under Evolutionary Dynamics Nonconvergence of Evolutionary Dynamics 319 IV Stochastic Evolutionary Models 10 Stochastic Evolution and Deterministic Approximation Stationary Distributions and Infinite-Horizon Behavior Limiting Stationary Distributions and Stochastic Stability 451 References 541 Notation Index 565 Index 575

3 Series Foreword Preface xvii xix * 1 Introduction Population Games Modeling Interactions in Large Populations Definitions and Classes of Population Games Evolutionary Dynamics Knowledge, Rationality, and Large Games \ Foundations for Evolutionary Dynamics \ Deterministic Evolutionary Dynamics Orders of Limits for Stochastic Evolutionary Models Stationary Distributions and Stochastic Stability Remarks on History, Motivation, and Interpretation 13 Notes 15 1 Population Games 2 Population Games Population Games Populations, Strategies, and States Payoffs Best Responses and Nash Equilibria Prelude to Evolutionary Dynamics Examples Matching in Normal Form Games Congestion Games Two Simple Externality Models The Geometry of Population Games and Nash Equilibria Drawing Two-Strategy Games Displacement Vectors and Tangent Spaces 31

4 viii Contents Orthogonal Projections Drawing Three-Strategy Games Tangent Cones and Normal Cones Normal Cones and Nash Equilibria 41 2.A Appendix: Affine Spaces, Tangent Spaces, and Projections 44 2.A.1 Affine Spaces 44 2.A.2 Affine Hulls of Convex Sets 45 2.A.3 Orthogonal Projections 46 2.A.4 The Moreau Decomposition Theorem 50 Notes 51 Potential Games, Stable Games, and Supermodular Games Full Potential Games Full Population Games * Definition and Characterization Examples Nash Equilibria of Full Potential Games The Geometry of Nash Equilibrium in Full Potential Games Efficiency in Homogeneous Full Potential Games Inefficiency Bounds for Congestion Games Potential Games Motivating Examples Definition, Characterizations, and Examples Potential Games and Full Potential Games Passive Games and Constant Games Stable Games ' Definition Examples Invasion Global Neutral Stability and Global Evolutionary Stability Nash Equilibrium and Global Neutral Stability in Stable Games Supermodular Games Definition Examples Best Response Monotonicity in Supermodular Games Nash Equilibria of Supermodular Games A Appendix: Multivariate Calculus A.1 Univariate Calculus A.2 The Derivative as a Linear Map A.3 Differentiation as a Linear Operation A.4 The Product Rule and the Chain Rule 103

5 ix 3.A.5 Homogeneity and Euler's Theorem A.6 Higher-Order Derivatives A.7 The Whitney Extension Theorem A.8 Vector Integration and the Fundamental Theorem of Calculus A.9 Potential Functions and Integrability B Appendix: Affine Calculus B.2 Linear Forms and the Riesz Representation Theorem B.2 Dual Characterizations of Multiples of Linear Forms B.3 Derivatives of Functions on Affine Spaces B.4 Affine Integrability 113 Notes 115 II Deterministic Evolutionary Dynamics 4 Revision Protocols and Evolutionary Dynamics The Basic, Stochastic Evolutionary Model inertia and Myopia Revision Protocols Mean Dynamics Derivation, Target Protocols and Target Dynamics \ Examples Imitative Protocols and Dynamics Direct Protocols and Dynamics Deterministic Evolutionary Dynamics A Appendix: Ordinary Differential Equations A.2 Basic Definitions A.2 Existence, Uniqueness, and Continuity of Solutions A.3 Ordinary Differential Equations on Compact Convex Sets 135 Notes Deterministic Dynamics: Families and Properties Information Requirements for Revision Protocols Incentives and Aggregate Behavior Families of Evolutionary Dynamics Imitative Dynamics 153 5A.1 Definition Examples Biological Derivations of the Replicator Dynamic Extinction and Invariance Monotone Percentage Growth Rates and Positive Correlation Rest Points and Restricted Equilibria 164

6 5.5 Excess Payoff Dynamics Definition and Interpretation Incentives and Aggregate Behavior Pairwise Comparison Dynamics Definition Incentives and Aggregate Behavior Desiderata Revisited Multiple Revision Protocols and Hybrid Dynamics 173 Notes ' Best Response and Projection Dynamics The Best Response Dynamic Definition and Examples Construction and Properties of Solution Trajectories * Incentive Properties Perturbed Best Response Dynamics Revision Protocols and Mean Dynamics Perturbed Optimization: A Representation Theorem Logit Choice and the Logit Dynamic Perturbed Incentive Properties via Virtual Payoffs The Projection Dynamic Definition Solution Trajectories Incentive Properties Revision Protocols and Connections with the Replicator Dynamic A Appendix: Differential Inclusions A.I Basic Theory A.2 Differential Equations Defined by Projections B Appendix: The Legendre Transform B.2 Legendre Transforms of Functions on Open Intervals, B.2 Legendre Transforms of Functions on Multidimensional Domains C Appendix: Perturbed Optimization C.2 Proof of the Representation Theorem C.2 Additional Results 215 Notes 216 HI Convergence and Nonconvergence of Deterministic Dynamics 7 Global Convergence of Evolutionary Dynamics Potential Games Potential Functions as Lyapunov Functions Gradient Systems for Potential Games 228

7 7.2 Stable Games The Projection and Replicator Dynamics in Strictly Stable Games Integrable Target Dynamics Impartial Pairwise Comparison Dynamics, Summary Supermodular Games The Best Response Dynamic in Two-Player Normal Form Games Stochastically Perturbed Best Response Dynamics 253 7A Dominance Solvable Games v Dominated and Iteratively Dominated Strategies 258 7A.2 The Best Response Dynamic Imitative Dynamics A Appendix: Limit and Stability Notions for Deterministic Dynamics A.I co-limits and Notions of Recurrence A.2 Stability of Sets of States B Appendix: Stability Analysis via Lyapunov Functions B.2 Lyapunov Stable Sets B.2 w-limits and Attracting Sets B.3 Asymptotically Stable and Globally Asymptotically Stable Sets C Appendix: Cooperative Differential Equations \ 266 Notes 268. Local Stability under Evolutionary Dynamics Non-Nash Rest Points of Imitative Dynamics Local Stability in Potential Games Evolutionarily Stable States Single-Population Games Multipopulation Games Regular Taylor ESS Local Stability via Lyapunov Functions The Replicator and Projection Dynamics Target and Pairwise Comparison Dynamics: Interior ESS Target and Pairwise Comparison Dynamics: Boundary ESS Linearization of Imitative Dynamics The Replicator Dynamic General Imitative Dynamics Linearization of Perturbed Best Response Dynamics Deterministically Perturbed Best Response Dynamics The Logit Dynamic A Appendix: Matrix Analysis A.I Rank and Invertibility 299

8 xii Contents 8.A.2 Eigenvectors and Eigenvalues A.3 Similarity, (Block) Diagonalization, and the Spectral Theorem A.4 Symmetric Matrices A.5 The Real Jordan Canonical Form A.6 The Spectral Norm and Singular Values A.7 Hines's Lemma B Appendix: Linear Differential Equations B.2 Examples B.2 Solutions ' B.3 Stability and Hyperbolicity C Appendix: Linearization of Nonlinear Differential Equations 313 Notes Nonconvergence of Evolutionary Dynamics Conservative Properties of Evolutionary Dynamics ^Constants of Motion in Null Stable Games Preservation of Volume Games with Nonconvergent Evolutionary Dynamics Circulant Games Continuation of Attractors for Parameterized Games Mismatching Pennies The Hypnodisk Game Chaotic Evolutionary Dynamics Survival of Dominated Strategies A General Survival Theorem Examples and Discussion A Appendix: Three Classical Theorems on Nonconvergent Dynamics A.I Liouville's Theorem A.2 The Poincare-Bendixson and Bendixson-Dulac Theorems B Appendix: Attractors and Continuation B.2 Attractors and Repellors B.2 Continuation of Attractors 361 Notes 362 IV Stochastic Evolutionary Models 10 Stochastic Evolution and Deterministic Approximation The Stochastic Evolutionary Process Finite-Horizon Deterministic Approximation Kurtz's Theorem Deterministic Approximation of the Stochastic Evolutionary Process 372

9 10.3 Extensions Discrete-Time Models Finite-Population Adjustments A Appendix: The Exponential and Poisson Distributions A.1 Basic Properties A.2 The Poisson Limit Theorem B Appendix: Probability Models and Their Interpretation 382 2O.B.2 Countable Probability Models B.2 Uncountable Probability Models and Measure Theory B.3 Distributional Properties and Sample Path Properties C Appendix: Countable State Markov Chains and Processes C.1 Countable State Markov Chains C.2 Countable State Markov Processes: Definition and Construction C.3 ^Countable State Markov Processes: Transition Probabilities 393 Notes Stationary Distributions and Infinite-Horizon Behavior Irreducible Evolutionary Processes Full Support Revision Protocols Stationary Distributions and Infinite-Horizon Behavior Reversibility \ Stationary Distributions for Two-Strategy Games Birth and Death Processes The Stationary Distribution of the Evolutionary Process Examples Waiting Times and Infinite-Horizon Prediction Examples Discussion Model Adjustments for Finite Populations Finite-Population Games ClevertPayoff Evaluation Committed Agents and Imitative Protocols Potential Games and Exponential Protocols Finite-Population Potential Games Exponential Revision Protocols Reversibility and Stationary Distributions A Appendix: Long-Run Behavior of Markov Chains and Processes A2 Communication, Recurrence, and Irreducibility A.2 Periodicity A.3 Hitting Times and Hitting Probabilities AA The Perron-Frobenius Theorem 440

10 11.A.5 Stationary Distributions for Markov Chains, A.6 Reversible Markov Chains A.7 Stationary Distributions and Reversibility for Markov Processes A.8 Convergence in Distribution A.9 Ergodicity 448 Notes Limiting Stationary Distributions and Stochastic Stability Definitions of Stochastic Stability Small Noise Limits ' "' Large Population Limits Double Limits Double Limits: A Counterexample Exponential Protocols and Potential Games * Direct Exponential Protocols: The Small Noise Limit j. Direct Exponential Protocols: The Large Population Limit Direct Exponential Protocols: Double Limits Imitative Exponential Protocols with Committed Agents Noisy Best Response Protocols in Two-Strategy Games Noisy Best Response Protocols and Their Cost Functions The Small Noise Limit The Large Population Limit Double Limits Stochastic Stability: Examples Risk Dominance, Stochastic Dominance, and Stochastic Stability Imitative Protocols in Two-Strategy Games Imitative Protocols with Mutations Imitative Protocols with Committed Agents Imitative Protocols, Mean Dynamics, and Stochastic Stability Small Noise Limits Noisy Best Response Protocols and Cost Functions Limiting Stationary Distributions via Trees Two-Strategy Games and Risk Dominance The Radius-Coradius Theorem Half-Dominance Large Population Limits Convergence to Recurrent States of the Mean Dynamic Convergence to Stable Rest Points of the Mean Dynamic A Appendix: Trees, Escape Paths, and Stochastic Stability A.2 The Markov Chain Tree Theorem A.2 Limiting Stationary Distributions via Trees 521

11 22.A3 Limiting Stationary Distributions via Trees on Recurrent Classes A.4 Radius-Coradius Theorems A.5 Lenient Transition Costs and Weak Stochastic Stability B Appendix: Stochastic Approximation Theory B.2 Convergence to the Birkhoff Center B.2 Sufficient Conditions for Convergence to Stable Rest Points 533 Notes 537 References v. 541 Notation Index 565 Index 575