Markov Chain Monter rlo Simulations and Their Statistical Analvsis With Web-Based Fortran Code Bernd A. Berg Florida State Univeisitfi USA World Scientific NEW JERSEY + LONDON " SINGAPORE " BEIJING " SHANGHAI " HONG KONG + TAIPEI " CHENNAI
Contents 1. Sampling, Statistics and Computer Code 1 1.1 Probability Distributions and Sampling........... I 1.1.1 Assignments for section 1.1.............. 5 1.2 Random Numbers....................... 6 1.2.1 Assignments for section 1.2.............. 12 1.3 About the Fortran Code.................... 13 1.3.1 CPU time measurements under Linux........ 22 1.4 Gaussian Distribution..................... 23 1.4.1 Assignments for section 1.4.............. 25 1.5 Confidence Intervals...................... 26 1..5.1 Assignment for section 1.5.............. 30 1.6 Order Statistics and 1"IeapSort................. 30 1.6.1 Assignments for section 1.6.............. 34 1.7 Functions and Expectation Values.............. 35 1.7.1 Moments and Tchebychev's inequality........ 36 1.7.2 The sum of two independent random variables.... 40 1..7.3 Characteristic functions and sums of N independent random variables.................... 41 1.7.4 Linear transformations, error propagation and covariance........................... 43 1.7.5 Assignments for section 1.7.............. 46 1.8 Sample Mean and the Central Limit Theorem........ 47 1.8.1 Probability density of the sample mean......... 47 1.8.2 The central limit theorem............... 50
xii Markov Chain Monte Carlo Simulations and Their Statistical Analysis 1.8.2.1 Counter example............... 51 1.8.3 Binning......................... 52 1.8.,4 Assignments for section 1.8.............. 53 2. Error Analysis for Independent Random Variables 54 2.1 Gaussian Confidence Intervals and Error Bars........ 54 2.1.1 Estimator of the variance and bias.......... 56 2.1.2 Statistical error bar routines (step)..,....... 57 2.1.2.1 Ratio of two means with error bars..... 60 2.1.3 Gaussian difference test................ 60 2.1.3.1 Combining more than two data points... 62 2.1.4 Assignments for section 2.1.............. 64 2.2 The x2 Distribution...................... 66 2.2.1 Sample variance distribution.............. 67 2.2.2 The Xz distribution function and probability density 70 2.2.3 Assignments for section 2.2.............. 72 2.3 Gosset's Student Distribution................. 73 2.3.1 Student difference test................. 77 2.3.2 Assignments for section 2.3.............. 81 2.4 The Error of the Error Bar.................. 81 2.4.1 Assignments for section 2.4.............. 84 2.5 Variance Ratio Test (P-test)................. 85 2.5.1 F ratio confidence limits................ 88 2.5.2 Assignments for section 2.5.............. 89 2.6 When are Distributions Consistent?............. 89 2.6.1 x2 Test.......................... 89 2.6.2 The one-sided Kolmogorov test............ 92 2.6.3 The two-sided Kolmogorov test............ 98 2.6.4 Assignments for section 2.6.............. 101 2.7 The Jackknife Approach.................... 103 2.7.1 Bias corrected estimators................ 206 2.7.2 Assignments for section 2.7.............. 108 2.8 Determination o Parameters (Fitting)............ 109 2.8.1 Linear regression.................... 111 2.8.1.1 Confidence limits of the regression line... 11.4 2.8.1.2 Related functional forms........... 115 2.8.1.3 Examples................... 3.17 2.8.2 Levenberg-Marquardt fitting.............. 12]. 2.8.2.1 Examples................... 125
. Contents xiii 2.8.3 Assignments for section 2.8.............. 127 3. Markov Chain Monte Carlo 128 3.1 Preliminaries and the Two-Dimensional Ising Model.... 129 3.1.1 Lattice labeling..................... 133 3.1.2 Sampling and Re-weighting.............. 138 3.1.2.1 Important configurations and re-weighting range...................... 141 3.1.3 Assignments for section 3.1.............. 142 3.2 Importance Sampling...................... 142 3.2.1 The Metropolis algorithm................ 147 3.2.2 The O(3) a-model and the heat bath algorithm... 148 3.2.3 Assignments for section 3.2.............. 152 3.3 Potts Model Monte Carlo Simulations............ 152 3.3.1 The Metropolis code.................. 156 3.3.1.1 Initialization.................. 158 3.3.1.2 Updating routines............... 160 3.3.1.3 Start and equilibration............ 163 3.3.1.4 More updating routines............ 164 3.3.2 Heat bath code.................. ;.. 165 3.3.3 Timing and time series comparison of the routines. 168 3.3.4 Energy references, data production and analysis code 169 3.3.4.1 2d Ising model................. 171 3.3.4.2 Data analysis................. 173 3.3.4.3 2d 4-state and 10-state Potts models.... 174 3.3.4.4 3d Ising model................. 177 3.3.4.5 3d 3-state Potts model............ 177 3.3.4.6 4d Ising model with non-zero magnetic field 178 3.3.5 Assignments for section 3.3.............. 179 3.4 Continuous Systems...................... 181 3.4.1 Simple Metropolis code for the 0(n) spin models.. 182 3.4.2 Metropolis code for the XY model.......... 186 3.4.2.1 Timing, discretization and rounding errors. 187 3.4.2.2 Acceptance rate................ 189 3.4.3 Heat bath code for the O(3) model.......... 192 3.4.3.1 Rounding errors................ 194 3.4.4 Assignments for section 3.4.............. 194 4. Error Analysis for Markov Chain Data 196
xiv Markov Chain Monte Carlo Simulations and Their Statistical Analysis 4.1 Autocorrelations........................ 197 4.1.1 Integrated autocorrelation time and binning.... 202 4.1.2 Illustration : Metropolis generation of normally distributed data...................... 205 4.1.2.1 Autocorrelation function........... 205 4.1.2.2 Integrated autocorrelation time....... 207 4,1.2.3 Corrections to the confidence intervals of the binning procedure............... 210 4.1.3 SeIf-consistent versus reasonable error analysis... 211 4.1.4 Assignments for section 4.1.............. 213 4.2 Analysis of Statistical Physics Data.............. 214 4.2.1 The d = 2 Ising model off and on the critical point. 214 4.2.2 Comparison of Markov chain MC algorithms.... 218 4.2.2.1 Random versus sequential updating..... 218 4.2.2.2 Tuning the Metropolis acceptance rate... 219 4.2.2.3 Metropolis versus heat bath : 2d q = 10 Potts 221 4.2.2.4 Metropolis versus heat bath : 3d Ising.... 222 4.2.2.5 Metropolis versus heat bath : 2d O(3) o- model 223 4.2.3 Small fluctuations.................... 224 4.2.4 Assignments for section 4.2.............. 227 4.3 Fitting of Markov Chain Moate Carlo Data......... 229 4.3.3. One exponential autocorrelation time......... 230 4.3.2 More than one exponential autocorrelation time.. 233 4.3.3 Assignments for section 4.3.............. 235 5. Advanced Monte Carlo 236 5.1 Multicanonical Simulations................., 236 5.1.1 Recursion for the weights............,.. 239 5.1.2 Fortran implementation..........,..... 244 5.1.3 Example runs...................... 247 5.1..4 Performance....................... 250 5.1.5 Re-weighting to the canonical ensemble........ 251 5.1.6 Energy and specific heat calculation......... 254 5.1.7 Free energy and entropy calculation....,..... 261 5.1.8 Time series analysis................... 264 5.1.9 Assignments for section 5,1.,............ 267 5.2 Event Driven Simulations................... 268 5.2.1 Computer implementation............... 270 5.2.2 MC runs with the EDS code.............. 276
Contents xv 5.2.3 Assignments for section 5.2.............. 278 5.3 Cluster Algorithms....................... 279 5.3.1 Autocorrelation times................. 284 5.3.2 Assignments for section 5.3.............. 286 5.4 Large Scale Simulations.................... 287 5.4.1 Assignments for section 5.4.............. 289 6. Parallel Computing 292 6.1 Trivially Parallel Computing................. 292 6.2 Message Passing Interface (MPI)............... 294 6.3 Parallel Tempering....................... 303 6.3.1 Computer implementation............... 305 6.3.2 Illustration for the 2d 10-state Potts model..... 310 6.3.3 Gaussian Multiple Markov chains........... 315 6.3.4 Assignments for section 6.3.............. 316 6.4 Checkerboard algorithms.................... 316 6.4.1 Assignment for section 6.4.............. 318 7. Conclusions, History and Outlook 319 Appendix A Computational Supplements 326 A.1 Calculation of Special Functions............... 326 A.2 Linear Algebraic Equations.................. 328 Appendix B More Exercises and some Solutions 331 B.1 Exercises............................ 331 B.2 Solutions........................... 333 Appendix C More Fortran Routines 338 Bibliography 339 Index 349