Markov Chain. Edited by. Andrew Gelman. Xiao-Li Meng. CRC Press. Taylor & Francis Croup. Boca Raton London New York. an informa business

Transcription

1 Chapman & Hall/CRC Handbooks of Modern Statistical Methods Handbook of Markov Chain Monte Carlo Edited by Steve Brooks Andrew Gelman Galin L. Jones Xiao-Li Meng CRC Press Taylor & Francis Croup Boca Raton London New York CRC Press Is an Imprint of the Taylor fit Francis Group, an informa business A CHAPMAN & HALL BOOK

2 Contents Preface X1* Editors xxi Contributors xxiii Part I Foundations, Methodology, and Algorithms 1. Introduction to Markov Chain Monte Carlo 3 Charles J. Geyer 1.1 History Markov Chains Computer Programs and Markov Chains Stationarity Reversibility Functionals The Theory of Ordinary Monte Carlo The Theory of MCMC Multivariate Theory The Autocovariance Function AR(1) Example A Digression on Toy Problems Supporting Technical Report The Example Variance Estimation Nonoverlapping Batch Means Initial Sequence Methods Initial Sequence Methods and Batch Means The Practice of MCMC Black Box MCMC Pseudo-Convergence One Long Run versus Many Short Runs Burn-In Diagnostics Elementary Theory of MCMC The Metropolis-Hastings Update The Metropolis-Hastings Theorem The Metropolis Update The Gibbs Update Variable-at-a-Time Metropolis-Hastings Gibbs Is a Special Case of Metropolis-Hastings Combining Updates Composition Palindromic Composition State-Independent Mixing Subsampling Gibbs and Metropolis Revisited 28 v

3 vi Contents 1.13 A Metropolis Example Checkpointing Designing MCMC Code Validating and Debugging MCMC Code The Metropolis-Hastings-Green Algorithm State-Dependent Mixing Radon-Nikodym Derivatives Measure-Theoretic Metropolis-Hastings Metropolis-Hastings-Green Elementary Update The MHG Theorem MHG with Jacobians and Augmented State Space The MHGJ Theorem 46 Acknowledgments 47 References A Short History of MCMC: Subjective Recollections from Incomplete Data 49 Christian Robert and George Casella 2.1 Introduction Before the Revolution The Metropolis et al. (1953) Paper The Hastings (1970) Paper Seeds of the Revolution Besag and the Fundamental (Missing) Theorem EM and Its Simulated Versions as Precursors Gibbs and Beyond The Revolution Advances in MCMC Theory Advances in MCMC Applications After the Revolution A Brief Glimpse at Particle Systems Perfect Sampling Reversible Jump and Variable Dimensions Regeneration and the Central Limit Theorem Conclusion 60 Acknowledgments 61 References Reversible Jump MCMC 67 Yanan Fan and Scott A. Sisson 3.1 Introduction From Metropolis-Hastings to Reversible Jump Application Areas Implementation Mapping Functions and Proposal Distributions Marginalization and Augmentation Centering and Order Methods Multi-Step Proposals Generic Samplers 78

4 Contents vn 3.3 Post Simulation Label Switching Convergence Assessment Estimating Bayes Factors Related Multi-Model Sampling Methods Jump Diffusion Product Space Formulations Point Process Formulations Multi-Model Optimization Population MCMC Multi-Model Sequential Monte Carlo Discussion and Future Directions 86 Acknowledgments 87 References Optimal Proposal Distributions and Adaptive Jeffrey S. Rosenthal MCMC Introduction The Metropolis-Hastings Algorithm Optimal Scaling Adaptive Comparing Markov Chains Optimal Scaling of Random-Walk Metropolis 95 MCMC Basic Principles Optimal Acceptance Rate as d -» oo Inhomogeneous Target Distributions Metropolis-Adjusted Langevin Algorithm Numerical Examples Off-Diagonal Covariance Inhomogeneous Covariance Frequently Asked Questions Adaptive MCMC Ergodicity of Adaptive MCMC Adaptive Metropolis Adaptive Metropolis-within-Gibbs State-Dependent Proposal Scalings Limit Theorems Frequently Asked Questions Conclusion 109 References MCMC Using Hamiltonian Dynamics 113 Radford M. Neal 5.1 Introduction Hamiltonian Dynamics Hamilton's Equations Equations of Motion Potential and Kinetic Energy A One-Dimensional Example 116

5 viii Contents Properties of Hamiltonian Dynamics Reversibility Conservation of the Hamiltonian Volume Preservation Symplecticness Discretizing Hamilton's Equations The Leapfrog Method Euler's Method A Modification of Euler's Method The Leapfrog Method Local and Global Error of Discretization Methods MCMC from Hamiltonian Dynamics Probability and the Hamiltonian: Canonical Distributions The Hamiltonian Monte Carlo Algorithm The Two Steps of the HMC Algorithm Proof That HMC Leaves the Canonical Distribution Invariant Ergodicity of HMC Illustrations of HMC and Its Benefits Trajectories for a Two-Dimensional Problem Sampling from a Two-Dimensional Distribution The Benefit of Avoiding Random Walks 130 from a 100-Dimensional Distribution Sampling 5.4 HMC in Practice and Theory Effect of Linear Transformations Tuning HMC Preliminary Runs and Trace Plots What Stepsize? What Trajectory Length? Using Multiple Stepsizes Combining HMC with Other MCMC Updates Scaling with Dimensionality Creating Distributions of Increasing Dimensionality by Replication Scaling of HMC and Random-Walk Metropolis Optimal Acceptance Rates Exploring the Distribution of Potential Energy HMC for Hierarchical Models Extensions of and Variations on HMC Discretization by Splitting: Handling Constraints and Other Applications Splitting the Hamiltonian Splitting to Exploit Partial Analytical Solutions Splitting Potential Energies with Variable Computation Costs Splitting According to Data Subsets Handling Constraints Taking One Step at a Time The Langevin Method Partial Momentum Refreshment: Another Way to Avoid Random Walks 150

6 Contents he Acceptance Using Windows of States Using Approximations to Compute the Trajectory Short-Cut Trajectories: Adapting the Stepsize without. Adaptation Tempering during a Trajectory 157 Acknowledgment 160 References Inference from Simulations and Monitoring Convergence 163 Andrew Gelman and Kenneth Shirley 6.1 Quick Summary of Recommendations Key Differences between Point Estimation and MCMC Inference Inference for Functions of the Parameters vs. Inference for Functions of the Target Distribution Inference from Noniterative Simulations Burn-In Monitoring Convergence Comparing between and within Chains Inference from Simulations after Approximate Convergence Summary 172 Acknowledgments 173 References Implementing MCMC: Estimating with Confidence 175 James M. Flegal and Galin L. Jones 7.1 Introduction Initial Examination of Output Point Estimates of Expectations Quantiles Interval Estimates of 0^ Expectations Overlapping Batch Means Parallel Chains Functions of Moments Quantiles Subsampling Bootstrap Multivariate Estimation Estimating Marginal 7.6 Terminating Densities 189 the Simulation Markov Chain Central Limit Theorems Discussion 194 Acknowledgments 195 References Perfection within Reach: Exact MCMC Sampling 199 Radu V. Craiu and Xiao-Li Meng 8.1 Intended Readership Coupling from the Past Moving from Time-Forward to Time-Backward 199

7 x Contents Hitting the Limit Challenges for Routine Applications Coalescence Assessment Illustrating Monotone Coupling Illustrating Brute-Force Coupling General Classes of Monotone Coupling Bounding Chains Cost-Saving Strategies for Implementing Perfect Sampling Read-Once CFTP Fill's Algorithm Coupling Methods Splitting Technique Coupling via a Common Proposal Coupling via Discrete Data Augmentation Perfect Slice Sampling Swindles Efficient Use of Exact Samples via Concatenation Multistage Perfect Sampling Antithetic Perfect Sampling Integrating Exact and Approximate MCMC Algorithms Where Are the Applications? 223 Acknowledgments 223 References Spatial Point Processes 227 Mark Ruber 9.1 Introduction Setup Metropolis-Hastings Reversible Jump Chains Examples Convergence Continuous-Time Spatial Birth-Death Chains Examples Shifting Moves with Spatial Birth and Death Chains Convergence Perfect Sampling Acceptance/Rejection Method Dominated Coupling from the Past Examples Monte Carlo Posterior Draws Running Time Analysis Running Time of Perfect Simulation Methods 248 Acknowledgment 251 References The Data Augmentation Algorithm: Theory and Methodology 253 James P. Hobert 10.1 Basic Ideas and Examples 253

8 Contents xi of the DA Markov Chain Properties Basic Regularity Conditions Basic Convergence Properties Geometric Ergodicity Central Limit Theorems Choosing the Monte Carlo Sample Size Classical Monte Carlo Three Markov Chains Closely Related to X Minorization, Regeneration and an Alternative CLT Simulating the Split Chain A General Method for Constructing the Minorization Condition Improving the DA Algorithm The PX-DA and Marginal Augmentation Algorithms The Operator Associated with a Reversible Markov Chain A Theoretical Comparison of the DA and PX-DA Algorithms Is There a Best PX-DA Algorithm? 288 Acknowledgments 291 References Importance Sampling, Simulated Tempering, and Umbrella Sampling 295 Charles ]. Geyer 11.1 Importance Sampling Simulated Tempering Parallel Tempering Update Serial Tempering Update Effectiveness of Tempering Tuning Serial Tempering Umbrella Sampling Bayes Factors and Normalizing Constants Theory Practice Setup Trial and Error Monte Carlo Approximation Discussion 309 Acknowledgments 310 References Likelihood-Free MCMC 313 Scott A. Sisson and Yanan Fan 12.1 Introduction Review of Likelihood-Free Theory and Methods Likelihood-Free Basics The Nature of the Posterior Approximation A Simple Example Likelihood-Free MCMC Samplers Marginal Space Samplers Error-Distribution Augmented Samplers 320

9 xii Contents Potential Alternative MCMC Samplers A Practical Guide to Likelihood-Free MCMC An Exploratory Analysis The Effect of The Effect of the Weighting Density The Choice of Summary Statistics Improving Mixing Evaluating Model Misspecification Discussion 331 Acknowledgments 333 References 333 Part II Applications and Case Studies 13. MCMC in the Analysis of Genetic Data on Related Individuals 339 Elizabeth Thompson 13.1 Introduction Pedigrees, Genetic Variants, and the Inheritance of Genome Conditional Independence Structures of Genetic Data Genotypic Structure of Pedigree Data Inheritance Structure of Genetic Data Identical by Descent Structure of Genetic Data ibd-graph Computations for Markers and Traits MCMC Sampling of Latent Variables Genotypes and Meioses Some Block Gibbs Samplers Gibbs Updates and Restricted Updates on Larger Blocks MCMC Sampling of Inheritance Given Marker Data Sampling Inheritance Conditional on Marker Data Monte Carlo EM and Likelihood Ratio Estimation Importance Sampling Reweighting Using MCMC Realizations for Complex Trait Inference a Estimating Likelihood Ratio or lod Score Uncertainty in Inheritance and Tests for Linkage Detection Localization of Causal Loci Using Latent p-values Summary 358 Acknowledgment 359 References An MCMC-Based Analysis of a Multilevel Model for Functional MRI Data 363 Brian Caffo, DuBois Bowman, Lynn Eberly, and Susan Spear Bassett 14.1 Introduction Literature Review Example Data Data Preprocessing and First-Level Analysis A Multilevel Model for Incorporating Regional Connectivity Model 368

10 Contents Simulating the Markov Chain 14.4 Analyzing the Chain Activation Results 14.5 Connectivity Results Intra-Regional Connectivity Inter-Regional Connectivity 14.6 Discussion References 15. Partially Collapsed Gibbs Sampling and Path-Adaptive Metropolis-Hastings in High-Energy Astrophysics David A. van Dyk and Taeyonng Park 15.1 Introduction 15.2 Partially Collapsed Gibbs Sampler 15.3 Path-Adaptive Metropolis-Hastings Sampler 15.4 Spectra] Analysis in High-Energy Astrophysics 15.5 Efficient MCMC in Spectral Analysis 15.6 Conclusion Acknowledgments References 16. Posterior Exploration for Computationally Intensive Forward Models... David Higdon, C. Shane Reese,}. David Moulton, Jasper A. Vrugt, and Colin Fox 16.1 Introduction An Inverse Problem in Electrical Impedance Tomography Posterior Exploration via Single-Site Metropolis Updates 16.3 Multivariate Updating Schemes Random-Walk Metropolis Differential Evolution and Variants 16.4 Augmenting with Fast, Approximate Simulators Delayed Acceptance Metropolis An Augmented Sampler 16.5 Discussion Appendix: Formulation Based on a Process Convolution Prior Acknowledgments References 17. Statistical Ecology Ruth King 17.1 Introduction 17.2 Analysis of Ring-Recovery Data Covariate Analysis Posterior Conditional Distributions Results Mixed Effects Model Obtaining Posterior Inference Posterior Conditional Distributions Results

11 xiv Contents Model Uncertainty Model Specification Reversible jump Algorithm Proposal Distribution Results Comments Analysis of Count Data State-Space Models System Process Observation Process Model Obtaining Inference Integrated Analysis MCMC Algorithm Results Model Selection Results Comments Discussion 444 References Gaussian Random Field Models for Spatial Data 449 Murali Haran 18.1 Introduction Some Motivation for Spatial Modeling MCMC and Spatial Models: A Shared History Linear Spatial Models Linear Gaussian Process Models MCMC for Linear GPs Linear Gaussian Markov Random Field Models MCMC for Linear GMRFs Summary Spatial Generalized Linear Models The Generalized Linear Model Framework Examples Binary Data Count Data Zero-Inflated Data MCMC for SGLMs Langevin-Hastings MCMC Approximating an SGLM by a Linear Spatial Model Maximum Likelihood Inference for SGLMs Summary Non-Gaussian Markov Random Field Models Extensions Conclusion 471 Acknowledgments 473 References 473

12 Contents xv 19. Modeling Preference Changes via a Hidden Markov Item Response Theory Model 479 Jong Hee Park 19.1 Introduction Dynamic Ideal Point Estimation Hidden Markov Item Response Theory Model Preference Changes in US Supreme Court Justices Conclusions 490 Acknowledgments 490 References Parallel Bayesian MCMC Imputation for Multiple Distributed Lag Models: A Case Study in Environmental Epidemiology 493 Brian Caffo, Roger Peng, Francesca Dominici, Thomas A. Louis, and Scott Zeger 20.1 Introduction The Data Set Bayesian Imputation Single-Lag Models Distributed Lag Models Model and Notation Prior and Hierarchical Model Specification Bayesian Imputation Sampler A Parallel Imputation Algorithm Analysis of the Medicare Data Summary 507 Appendix: Full Conditionals 509 Acknowledgment 510 References MCMC for State-Space Models 513 Paul Fearnhead 21.1 Introduction: State-Space Models Bayesian Analysis and MCMC Framework Updating the State Single-Site Updates of the State Block Updates for the State Other Approaches Updating the Parameters Conditional Updates of the Parameters Reparameterization of the Model Joint Updates of the Parameters and State Discussion 527 References 527

13 xvi Contents 22. MCMC in Educational Research 531 Roy Levy, Robert ]. Mislevy, and John T. Behrens 22.1 Introduction Statistical Models in Education Research Historical and Current Research Activity Multilevel Models Psychometric Modeling Continuous Latent and Observable Variables Continuous Latent Variables and Discrete Observable Variables Discrete Latent Variables and Discrete Observable Variables Combinations of Models NAEP Example Discussion: Advantages of MCMC Conclusion 542 References Applications of MCMC in Fisheries Science 547 Russell B. Millar 23.1 Background The Current Situation Software Perception of MCMC in Fisheries ADMB Automatic Differentiation Metropolis-Hastings Implementation Bayesian Applications to Fisheries Capturing Uncertainty State-Space Models of South Atlantic Albacore Tuna Biomass Implementation Hierarchical Modeling of Research Trawl Catchability Hierarchical Modeling of Stock-Recruitment Relationship Concluding Remarks 560 Acknowledgment 561 References Model Comparison and Simulation for Hierarchical Models: Analyzing Rural-Urban Migration in Thailand 563 Filiz Garip and Bruce Western 24.1 Introduction Thai Migration Data Regression Results Posterior Predictive Checks 569

14 Contents xvii 24.5 Exploring Model Implications with Simulation Conclusion 572 References 574 Index 575