The Monte Carlo Method An Introduction to Markov Chain MC Sachin Shanbhag Department of Scientific Computing, Florida State University
Sources. Introduction to Monte Carlo Algorithms : Werner Krauth. Available as a pdf download at http://lps.ens.fr/~krauth. This is a great introduction to MC methods, geared towards applications in physical sciences, through puzzles and simple non-technical problems. 2. David Kofke s page on Molecular Simulation.
system with three states Markov Chains Transition-Probability Matrix Example "!! 2! 3 # " 0. 0.5 0.4# & ' $!!! % = $ 0.9 0. 0.0 % 2 22 23 $! 3! 32! % $ 33 0.3 0.3 0.4% ( ) ( ) If in state, will stay in state with probability 0. If in state, will move to state 3 with probability 0.4 Never go to state 3 from state 2 Requirements of transition-probability matrix all probabilities non-negative, and no greater than unity sum of each row is unity probability of staying in present state may be non-zero
Distribution of State Occupancies Consider process of repeatedly moving from one state to the next, choosing each subsequent state according to Π 2 2 3 2 2 3 3 2 3 etc. Histogram the occupancy number for each state n = 3 p = 0.33 n 2 = 5 p 2 = 0.42 n 3 = 4 p 3 = 0.25 2 3 After very many steps, a limiting distribution emerges
The Limiting Distribution Consider the product of Π with itself "!! 2! 3 # "!! 2! 3 # 2 & ' $!!! % ( $!!! % 2 22 23 2 22 23 $! 3! 32! % $ 33! 3! 32! % ) * ) 33 * "!! +! 2! 2 +! 3! 3!! 2 +! 2! 22 +! 3! 32 etc. # = $!! +!! +!!!! +!! +!! etc. % 2 22 2 23 3 2 2 22 22 23 32 $! 3! +! 32! 2 +! 33! 3! 3! 2 +! 32! 22 +! 33! 32 etc. % ) * In general! n is the n-step transition probability matrix probabilities of going from state i to j in exactly n steps All ways of going from state to state 2 in two steps Probability of going from state 3 to state 2 in two steps " ( n) ( n) ( n)!! 2! # 3 $ % n ( n) ( n) ( n) & ' $! 2! 22! 23 % ( n) ( n) ( n) $! 3! 32! % ( 33 ) defines ( n)! ij
The Limiting Distribution Define! (0) i as a unit state vector ( 0 0) ( 0 0) ( 0 0 ) (0) (0) (0) 2 3! =! =! =!! " # n ( ) (0) Then i n i is a vector of probabilities for ending at each state after n steps if beginning at state i " ( n) ( n) ( n)!! 2! # 3 $ % n! =! & ' 0 0 $!!! % =!!! ( n) ( n) ( n) $! 3! 32! % ( 33 ) The limiting distribution corresponds to n independent of initial state ( ) ( ) ( n) (0) ( n) ( n) ( n) ( n) ( n) ( n) 2 22 23 2 3 (") (") (") 2 3! =! =! #!
Stationary property of π The Limiting Distribution (0) n! = lim %! & i ' ( ) n"# ( (0) n ) lim % $! & i = ' ' ( ) n"# =!' π is a left eigenvector of Π with unit eigenvalue such an eigenvector is guaranteed to exist for matrices with rows that each sum to unity Equation for elements of limiting distribution π! i = "! j!! ji = 0.! + 0.9! 2 + 0.3! 3 " 0. 0.5 0.4# j! 2 = 0.5! + 0.! 2 + 0.3! 3 e. g. & = $ 0.9 0. 0.0 % $! 3 0.4! 0.0! 2 0.4! 3 0.3 0.3 0.4% = + + ' (! +! 2 +! 3 =! +! 2 +! 3 not independent
Statistical Mechanics NVT N particles, position r i = (x,y,z) momentum p i = (p x, p y, p z ) {r i, p i } = 6N degrees of freedom A particular snapshot of {r i, p i } is called a microstate Total energy of the system H({r i, p i }) = U({r i }) + K({p i }) U = U({r i }) potential energy (often pairwise summation) K = Σ p i2 /2m i kinetic energy
Statistical Mechanics In an NVT ensemble, NVT "(r i,p i ) = Q exp(# H(r,p ) i i ) kt where Q = classical partition function, defined by Q = h 3N N! " dp " dp 2 L" dp N " dr " dr 2 L dr N " exp(# H(r,p ) i i ) kt this is like a effective volume of the phase space It is related to the free energy via A/kT = - ln Q(N,V,T) An average property (e.g. specific heat) may be obtained via, A = " dp " N dr N A(r i,p i )#(r i,p i )
Statistical Mechanics The kinetic energy factors can be integrated out, Q = Q = h 3N N! h 3N N! " dp N dr N " exp(# U({r }) + K({p }) i i ) kt " dr N exp(# U({r }) i kt ) " dp N $ p i 2 2m i Z N = configurational integral It is easy to show that, A = Q = Z N " 3N Why do we care? Z N N! " dr N A(r i )exp(# U kt ) In MC, unlike MD, we don t typically have access to momenta (only positions).
Statistical Mechanics This is an integration problem, I can use quadrature! NVT A = Z N N! " dr N A(r i )exp(# U kt ) One dimensional integrals I b =! f ( x) dx a Sum areas of shapes approximating shape of curve f ( x) Error δi ~ /n 2 (for D) δi ~ /n 2/D (for dd) x n b! a I " # x f ( x ) = f ( x ) $ $ i i= n i= n i
Where is Monte Carlo? This is an integration problem, I can use quadrature! A = Z N N! " dr N A(r i )exp(# U kt ) One dimensional integrals I b =! a f ( x) dx Error δi ~ /n 0.5 (for D) δi ~ /n 0.5 (for dd) I n b! a " # n i= f ( x ) i! ( x) For large dimensions, MC has better convergence n points selected from uniform distribution p(x) x
Where is Monte Carlo? A = Z N N! " dr N A(r i )exp(# U kt ) Typically, 3D space, say N=00 particles, DOF = 3N = 300 n=0 points between (-L, +L) (dx, for example) If each function evaluation is flop (lower-bound), need 0 300 function evaluations to get integral. Fastest computers (IBM BlueGene/L ~ 500 Tflops) Time required = 0 300 /5*0 4 ~ 0 278 years age of the universe 3 billion years IMPOSSIBLE
The Etymology of Monte Carlo Principality of Monaco (~ mile2 - UM) Famous for casinos, Formula- races, royalty Random numbers
The Ubiquity of Monte Carlo Random Examples Chip Design and Newpaper Panels floorplanning Finance and Social Sciences
Broadly Speaking Monte Carlo Simulation Equilibrium Monte Carlo Kinetic or Dynamic Monte Carlo i. Detailed balance i. Detailed balance ii. Lot of algorithmic freedom ii. Kinetics (or Model) today iii. Trial Moves restricted to be local
The Tradition of Monte Carlo (930-950) various attempts The Manhattan Project Ulam, Feynman, von Neumann Fermi
The Miracle of the Central Limit Theorem p(x ) x = x x 2 = + 2 p(x 2 ) x 2 p(x N ) The probability distribution of the average of independent random variables becomes increasingly Gaussian and increasingly narrower sigma ~ /sqrt(n) x N
Children s s game in Monaco Mathematically oriented kids of Monaco want to find the pi N red /(N red +N blue ) bench = A cir /A sq =pi*(a/2) 2 /a 2 bench bench = pi/4 pi = 4 * N red /(N red +N blue ) a=50 ft bench N = N red + N blue Here, pi = 4 *0/3 ~ 3.08
Children s s game in Monaco estimate of pi N = N red + N blue The invisible hand of the CLT operates to ensure that we converge to the true value of pi as the number of rocks thrown is increased. Error vanishes as /sqrt(n)
Older people s s game in Monaco Time spent planting flags in the circular region is proportional to the area of the circle plant flags on a helipad A CHOICES (i) Move to B. Plant a gray flag. Continue throwing. (ii) Stay at A. Don t plant flag. Throw again. (iii) Stay at A. Plant a flag. Continue throwing. B 500 ft Many MC simulations are wrong because they make the wrong choice. Condition of Detailed Balance will give us sound advice. a= mile pi = 4 * N red /(N red +N blue )
Some Concepts KID S GAME OLDER PEOPLE S VERSION Each sample (place where throw lands) independent Direct Sampling Converges faster (Think CLT, think independent samples) Sample dependent on previous position (is within radius of 500ft) Markov-chain Sampling Need more throws (samples) to attain same level of accuracy May not be feasible for realistic problems!
Coins in a Shoebox Problem Q: How do you generate random configurations of coins such that they don t overlap? Direct Sampling Markov-chain Sampling Start with a legit configuration Random Sequential Depositon (?) Trial: Move a randomly chosen coin just a little in a randomly chosen direction WRONG! yes Overlap? no phi RSD < phi CP Reject move Enact move Physicists (since 953) haven t found a direct sampling algorithm
So far Central Limit Theorem: The Engine Sampling: How do I generate trials? Direct Sampling Kid s game, better, not always possible Markov Chain Sampling Older people s game, more common Coin in a shoe-box problem Unsolved Mystery: What should you do if you throw the stone out of bounds in the adults game? Tricky question, full of consequences for world peace and prosperity. We said upcoming condition of detailed balance will resolve it.
Scrambling Game Objective: Create a perfect scrambling algorithm such that any possible configuration of the puzzle is generated with equal probability Toddler s algorithm equilibrium where each state is equiprobable. Rip the whole thing apart Put it back together
Scrambling Game Scrambling Game Older People Version Choose central square to go (N, E, W, S) each with probability ¼ (which is indeed correct) Move the empty square in a random direction, each timestep 5 3 6 7 8 2 4 5 3 6 7 2 8 4 5 3 6 7 8 2 4 0.25 5 3 6 7 8 2 4 5 6 7 3 8 2 4 0.25 0.25 0.25
Scrambling Game /3? 4 6 2 7 3 8 5 b F(a) = F(b) = F(c) = F(d) : at equilibrium F(a) = F(a) p(a a) + F(b) p(b a) + F(c) p(c a) + F(d) p(d a) 4 6 2 7 3 8 5 a /3? 4 6 2 3 8 7 5 c [- p(a a) ]F(a) = F(b) p(b a) + F(c) p(c a) + F(d) p(d a) p(a a) + p(a b) + p(a c) + p(a d) = /3? 4 6 2 7 3 8 5 d p(a b) + p(a c) + p(a d) = - p(a a) [p(a b) + p(a c) + p(a d) ]F(a) = F(b)p(b a) + F(c) p(c a) + F(d) p(d a)
Scrambling Game [p(a b) + p(a c) + p(a d) ]F(a) = F(b)p(b a) + F(c) p(c a) + F(d) p(d a) F(a) p(a b) + F(b) p(b a) + F(a) p(a c) + = F(c) p(c a) + F(a) p(a d) F(d) p(d a) (A) Impose a condition that terms of the same color on either side of the = are equal F(a) p(a b) = F(a) p(a c) = F(a) p(a d) = F(b) p(b a) F(c) p(c a) F(d) p(d a) This recipe for satisfying equation (A) is called the condition of detailed balance Sufficient, but not necessary condition
Scrambling Game Look at one of these carefully, p(a c)/p(c a) = F(c)/F(a) 4 6 c 2 3 8 7 5 0.25 4 6 a 2 7 3 8 5 Given: F(a) = F(c) p(a c) = p(c a) = 0.25 (not /3) Therefore algorithm is: (i) Pick {N, E, W, S} with equal probability (ii) Move blank square to corresponding direction, if possible. Otherwise, stay where you are, reject the move, and advance the clock.
Older people s s game in Monaco Time spent planting flags in the circular region is proportional to the area of the circle plant flags on a helipad A CHOICES B (i) Move to B. Plant a gray flag. Continue throwing. (ii) Stay at A. Don t plant flag. Throw again. (iii) Stay at A. Plant a flag. Continue throwing. Many MC simulations are wrong because they make the wrong choice. Condition of Detailed Balance will give us wise advice. a= mile 500 ft pi = 4 * N red /(N red +N blue )
Some More Concepts Concept of REJECTION arose from the condition of detailed balance p(a c)/p(c a) = F(c)/F(a) This method of rejection has been enshrined into the Metropolis Algorithm p(a c) = min[, F(c)/F(a)] Connection with Statistical Mechanics F(c) = exp(-u c /k B T) p(a c) = min[, exp[-(u c -U a )/k B T]] Boltzmann
The Metropolis Method (over 7500 citations between 988-2003) THE JOURNAL OF CHEMICAL PHYSICS VOLUME 2, NUMBER 6 JUNE, 953 Equation of State Calculations by Fast Computing Machines NICHOLAS METROPOLIS, ARIANNA W. ROSENBLUTH, MARSHALL N. ROSENBLUTH, AND AUGUSTA H. TELLER, Los Alamos Scientific Laboratory, Los Alamos, New Mexico AND EDWARD TELLER, * Department of Physics, University of Chicago, Chicago, Illinois (Received March 6, 953) A general method, suitable for fast computing machines, for investigating such properties as equations of state for substances consisting of interacting individual molecules is described. The method consists of a modified Monte Carlo integration over configuration space. Results for the two-dimensional rigidsphere system have been obtained on the Los Alamos MANIAC and are presented here. These results are compared to the free volume equation of state and to a four-term virial coefficient expansion. 087 Nicholas Metropolis with the MANIAC
Back to Statistical Mechanics A = Z N N! " dr N A(r i )exp(# U kt ) Observation 2 It may be possible to compute <A> without evaluating Z N? This function is sharply peaked. Observation In the area of the circle problem, we evaluated an area (like a partition function), and the probability distribution was uniform (obs and 2 inapplicable). In the scramble problem, the probability distribution was uniform (obs inapplicable)
Back to Statistical Mechanics A = Z N N! " dr N A(r i )exp(# U kt ) Metropolis helps us generate new microstates with the appropriate probability. Thus a simple average over all the microstates gives us <A> Samples the important region of phase space
Detailed Balance Eigenvector equation for limiting distribution A sufficient (but not necessary) condition for solution is! i! ij =! j! ji Thus detailed balance or microscopic reversibility! i = "! j! ji j! =!! i j ji j = " " j!! i " ij =!! =! i ij i j! = " 0. 0.5 0.4# $ 0.9 0. 0.0% $ % & 0.3 0.3 0.4' For a given Π, it is not always possible to satisfy detailed balance; e.g. for this!! "!! Π 3 32 2 23 zero
Tutorial: Ising Model 924: Ising solved D 944: Onsager solved 2D Simple model for understanding magnetism Ferromagnets: Fe, Ni - magnetic even in the absence of an electric field Unpaired electrons = origin of magnetism Paramagnets: Al, Ba - magnetic only in the presence of an electric field Up-state Down state
Tutorial: Ising Model (a) Energy E({"}) = #J $ " i " j (b) Magnetization M({"}) = #" i N
Tutorial: Markov Chains
Tutorial: Markov Chains