Chapter 4: Monte Carlo Methods Paisan Nakmahachalasint
Introduction Monte Carlo Methods are a class of computational algorithms that rely on repeated random sampling to compute their results. Monte Carlo simulation methods are especially useful in studying systems with a large number of coupled degrees of freedom.
Introduction Monte Carlo methods describe a large and widely-used class of approaches which tend to follow a particular pattern: Define a domain of possible inputs. Generate inputs randomly from the domain using a certain specified probability distribution. Perform a deterministic computation using the inputs. Aggregate the results of the individual computations into the final result.
Uniform Random Number Generators Monte Carlo simulations are inherently probabilistic, and thus make frequent use of programs to generate random numbers. On a computer these numbers are not random at all -- they are strictly deterministic and reproducible, but they look like a stream of random numbers. For this reason such programs are more correctly called pseudorandom number generators.
Uniform Random Number Generators Essential Properties Repeatability -- the same sequence should be produced with the same seeds. Randomness -- should produce independent uniformly distributed random variables that pass all statistical tests for randomness. Long period The period should be much longer than the amount of random numbers needed. Insensitive to seeds -- period and randomness properties should not depend on the initial seeds.
Uniform Random Number Generators Additional Properties Portability -- should give the same results on different computers. Efficiency -- should be fast (small number of floating point operations) and not use much memory. Disjoint subsequences -- different seeds should produce long independent (disjoint) subsequences so that there are no correlations between simulations with different initial seeds. Homogeneity -- sequences of all bits should be random.
Uniform Random Number Generators Middle-Square Method Start with a four-digit number x 0, called the seed. Square it to obtain an eight-digit number (add a leading zero if necessary). Take the middle four digits as the next random number. n 0 1 2 3 4 5 6 7 8 x n 2041 1656 7423 1009 0180 1049 1004 80 64
Uniform Random Number Generators Linear Congruence Generator (LCG) Introduced by D. H. Lehmer in 1951. A great majority of pseudo-random numbers used today are based on this method. x n+1 = (ax n + b) (mod m) Example: if x 0 = 7, a = 1, b = 7, m = 10, we obtain the following sequence: 7, 4, 1, 8, 5, 2, 9, 6, 3, 0, 7, 4,
Uniform Random Number Generators Choice of the modulus, m. m should be as large as possible since the period can never be larger than m. One usually chooses m to be near the largest integer that can be represented. Choice of the multiplier, a It was proven by M.Greenberger in 1961 that the sequence will have period m if and only if b is a relative prime to m. a 1 is a multiple of p for all p m. 4 m 4 a 1.
Uniform Random Number Generators With b = 0, one cannot get the full period, but in order to get the maximum possible, the followings should be satisfied: x 0 is a relative prime to m. a is a primitive root modulo m. It is possible to obtain a period of length m 1, but usually the period is around m/4.
Uniform Random Number Generators A Disadvantage of LCG RANDU is a popular random number generator distributed by IBM in 1960 s with the algorithm x n+1 = 65539x n (mod 2 31 ). This generator was later found to have a serious problem all random numbers fall into hyperplanes.
Uniform Random Number Generators 5000 random points in 1 and 2 dimensions generated from RANDU. 1 60 50 40 30 20 10 0.2 0.4 0.6 0.8 1. 0.8 0.6 0.4 0.2 0 0 0.2 0.4 0.6 0.8 1 Histogram of 5000 random numbers. Distribution of 5000 points in 2 dimensions
Uniform Random Number Generators 5000 random points in 3 dimensions generated from RANDU viewed at two different view points.
Uniform Random Number Generators LCGs in rand() in various compilers. Source a b Output bits in rand() Borland C/C++ 22695477 1 16-30 Glibc (GCC) 1103515245 12345 0-30 ANSI C 1103515245 12345 16-30 Borland Delphi 134775813 1 32-63 Microsoft C/C++ 214013 2531011 16-30 In all above compilers, m = 2 32
Uniform Random Number Generators Lagged-Fibonacci Generator (LFG) Lagged Fibonacci pseudo-random number generators have become increasingly popular in recent years. These generators are so named because of their similarity to the familiar Fibonacci sequence. LFG uses the algorithm x n = x n l + x n k (mod m) where l > k > 0 with m initial values {x 0, x 1,, x l 1 }.
Uniform Random Number Generators For most applications of interest, m is a power of two; i.e., m = 2 M. With a proper choice of l, k and the initial values, the period P of LFG is (2 l 1) 2 M 1. Two commonly used LFGs. x n = x n 17 + x n 5 (mod 2 31 ) period 2 47 x n = x n 55 + x n 24 (mod 2 31 ) period 2 85
Uniform Random Number Generators Unlike LCG, the value of the modulus, m, does not by itself limit the period of the generator. LFG is computationally simple: an integer add, a logical AND (to accomplish the mod 2 M operation), and the advancing of two array pointers are the only operations required to produce a new random number.
Random Variables and Distribution A random variable can be thought of as an unknown value that may change every time it is inspected. Thus, a random variable can be thought of as a function mapping the sample space of a random process to the real numbers. A random variable can be discrete or continuous.
Random Variables and Distribution Example (Discrete RV) The sample space of tossing 2 fair coins is S = {HH, HT, TH, TT}. Let X = number of heads, then X = {(HH,2), (HT,1), (TH,1), (TT,0)}.
Random Variables and Distribution Example (Cont d) The probability function of X is given by The mean of X is The variance of X is 1 if x = 0 4 1 if x = 1 2 PX ( = x) = 1 if x 2 4 = 0 otherwise μ = E[ X] = x P( x ) = 0 + 1 + 2 = 1 X i i i 1 1 1 4 2 4 Var[ X] = E[( X μ ) ] = E[ X ] μ = X 2 2 2 1 X 2
Random Variables and Distribution Example: Binomial Distribution X ~ B(n,p) n x n x PX ( = x) = x p ( 1 p) x = 0,1,, n n E[ X] = xp( x) = np x= 0 n 2 2 Var[ ] ( ) 1 x= 0 ( ) ( ) X = x P x np = np p
Random Variables and Distribution Example: Continuous Uniform Distribution X ~ U(a,b) 2 Var[ ] ( ) X f X 1 for a x b b a ( x) = 0 otherwise 1 2 ( ) E[ X] = xf ( x) dx = a + b X a + b 1 X = x f x dx = b a 2 12 2 ( ) 2
Random Variables and Distribution Cumulative Distribution Function Example f X F ( x ) P ( X x ) f ( t ) dt X 1 for a x b b a ( x) = 0 otherwise 1.0 0.8 0.6 0.4 0.2 = = x X 0 x < a x a F ( x) = fora x b X b a 1 x > b 1.0 0.8 0.6 0.4 0.2-1 0 1 2 3 4 5-1 0 1 2 3 4 5
Random Variables and Distribution Example: X ~ U(0,1) and Y = X 2. 0 if y < 0 ( 2 F () y = P X y) = y if0 y 1 Y 1 if y > 1 4 3 f Y () y 1 y 2 y df () y if 0 1 Y = = y 0 otherwise 2 1-0.5 0.0 0.5 1.0 1.5
Random Variables and Distribution Example: X ~ U(0,1) and ( 1 X ) 2 0 if y < 0 ( 1 2 F () y = P ( X ) y) = 2 y if0 y 1 Y 2 1 1 if y > 4 Y = 2 8 f Y () y df () y if 0 y Y = = y 0 otherwise 1 1 y 4 6 4 2 0.0 0.2 0.4 0.6 0.8 1.0
Random Variables and Distribution Example: X ~ U(0,1), Y ~ U(0,1), Z = X+Y. 0 if z < 0 if 0 1 2 z z 2 = ( + ) = 2 (2 z ) < z F () z P X Y z Z 1 if 1 2 2 1 if z > 2 1.0 f Z () z df () z 1 1 z if 0 z 2 Z = = z 0 otherwise 0.8 0.6 0.4 0.2-0.5 0.0 0.5 1.0 1.5 2.0 2.5
Non-uniform Random Numbers Inverse CDF Method If X is a random variable having the probability density function f X (x) and the cumulative distribution function (CDF) F ( x x ) = f ( t ) dt X X then the random variable U = F X (X) is uniformly distributed on the interval [0,1]. Thus X = F 1 ( U) has the desired distribution. X
Non-uniform Random Numbers Example: An exponential distribution f X x λe λ for x > 0 ( x) = 0 for x 0 0 if x < 0 λx F ( x) = 1 e if 0 x 1 X 1 if x > 1 2.0 1.5 1.0 0.5 0.0 0.5 1.0 1.5 2.0 2.5 3.0 1.0 0.8 0.6 0.4 0.2 0.0 0.5 1.0 1.5 2.0 2.5 3.0 X lnu = where U λ U(0,1)
Non-uniform Random Numbers Example: A triangular distribution f F X X 2 2 4x for 0 x 1 ( x) = 0 otherwise 0 if x < 0 2 1 2x if 0 x 2 ( x) = 2 1 1 2(1 x) if < x 1 2 1 if x > 1 X if U = 1 ifu > U 1 2 2 1 U 1 2 2 where U U(0,1) 2.0 1.5 1.0 0.5-0.5 0.5 1.0 1.5 1.0 0.8 0.6 0.4 0.2-0.5 0.5 1.0 1.5
Non-uniform Random Numbers Simulated Result: A triangular distribution 3 N=10 4 N=10 5 N=10 6 N=10
Non-uniform Random Numbers Acceptance/Rejection Method X is a random variable with pdf f X (x). We want to generate a random variable Y with pdf f Y (x) and there is a constant c for which f ( x) cf ( x). Y X cf () x X f () x Y x
Non-uniform Random Numbers Acceptance/Rejection Algorithm Generate a random value x from X. Generate a random value u from U(0,1). f ( x) ( x) Y If u, then x is accepted. cf X Otherwise, x is rejected (and repeat with the step of the generation of x).
Non-uniform Random Numbers Box-Muller Method for Normal Distribution Let U ~ U(0,1) and generate u 1 and u 2 from U. Then x = cos(2 πu ) 2 ln u 1 1 2 x = sin(2 πu ) 2 ln u 2 1 2 are independently distributed as the standard normal distribution.
Central Limit Theorem Central Limit Theorem (CLT) The sum of a sufficiently large number of independent and identically distributed (i.i.d.) random variables, each with finite mean and variance, will be approximately normally distributed.
Central Limit Theorem Let X 1, X 2,..., X n be a sequence of n i.i.d. random variables each having finite values of expectation μ and variance σ 2 > 0. Let S n = X 1 + X 2 + + X n. If we define Z then Z n N(0,1). n = X n σ nμ n
Central Limit Theorem Example: If a fair coin is tossed 10 6 times, and we let X = number of heads, then E[ X] = np = 500, 000 Var[ X] = np(1 p) = 250, 000 X 500, 000 500 N(0,1) 0.0008 0.0006 0.0004 0.0002 499 000 500 000 501000 502000
Monte Carlo Simulation Monte Carlo simulation was named for Monte Carlo, Monaco, where the primary attractions are casinos containing games of chance. Games of chance such as roulette wheels, dice, and slot machines, exhibit random behavior. Monte Carlo simulation randomly generates values for uncertain variables over and over to simulate a model.
Monte Carlo Simulation Why Simulation? In some circumstances, it may not be feasible to observe the behavior directly or to conduct experiment. A system of elevators during rush hour. Automobile traffic controlling. In other situations, the system for which alternative procedures need to be tested may not even exist yet. Communication networks in an office building Locations of machines in a new industrial plant.
Monte Carlo Simulation In such instances, the modeler might simulate the behavior and test the various alternatives being considered to estimate how each affects the behavior. Build a scaled model. Drag force on a proposed submarine. Designs of a jet airplane. Monte Carlo simulation which is typically accomplished with the aid of a computer.
Monte Carlo Simulation Processes with an element of chance involved are called probabilistic, as opposed to deterministic. Behavior Probabilistic Deterministic Model Probabilistic Deterministic Monte Carlo simulation is a probabilistic model. It can also be used to approximate a deterministic behavior.
Monte Carlo Simulation Probabilistic Behavior: an Unfair Die Consider a probability model where each event is not equally likely. Assume a die is biased according to the following distribution. Roll value P(roll) Value of x i Assignment 1 0.1 [0, 0.1] ONE 2 0.1 (0.1, 0.2] TWO 3 0.2 (0.2, 0.4] THREE 4 0.3 (0.4, 0.7] FOUR 5 0.2 (0.7, 0.9] FIVE 6 0.1 (0.9, 1.0] SIX
Monte Carlo Simulation 0.1 0.1045 0.1044 0.104 0.120 0.150 6 0.2 0.2011 0.2012 0.201 0.184 0.280 5 0.3 0.3081 0.3082 0.308 0.320 0.350 4 0.2 0.1962 0.1962 0.192 0.199 0.210 3 0.1 0.0992 0.0992 0.099 0.099 0.080 2 0.1 0.0948 0.0948 0.094 0.098 0.130 1 Expected Results 40000 10000 5000 1000 100 Die Value
Monte Carlo Integration Let p be the probability of a die having a desired value. Let N be the number of total tosses, and n be the number of tosses with the desired value. E[ n] = pn Var[ n] = p(1 p) N
Monte Carlo Integration n ˆ, N 1 E[ pˆ ] = E[ n] = p N 1 Var[ pˆ ] = Var[ n] = 2 N If p = then Example: If p = 0.1, then p ( 1 p) N σ pˆ ( p) = Var[ pˆ ] = p 1 = 0.3 N N
Monte Carlo Integration Area under a curve area under curve area of rectangle number of points counted below curve total number of random points
Monte Carlo Integration The area under the curve y = cos x over the interval -π/2 x π/2. Number of points Approximation to area Number of points Approximation to Area 100 2.07345 2000 1.94465 200 2.13628 3000 1.97711 300 2.01064 4000 1.99962 400 2.12058 5000 2.01439 500 2.04832 6000 2.02319 600 2.09440 8000 2.00669 700 2.02857 10000 2.00873 800 1.99491 15000 2.00978 900 1.99666 20000 2.01093 1000 1.9664 30000 2.01186
Monte Carlo Integration Let the area under the curve be I, while the area of the rectangle is R. Suppose that n out of N points fall under the curve, then we estimate the integral to be ˆ n I = R N There is a probability I/R that each random point will fall under the curve.
Monte Carlo Integration Thus, the number of random points fall under the curve will be binomially distributed. I E[ n] = N R I I Var[ n] = N 1 R R
Monte Carlo Integration ˆ R E[ I] = E[ n] = I N 2 I R Var[ ˆ R I] = Var[ n] = N N ( I) Example: For π/2 π/2 cos xdx, I = 2, R= π. σ Iˆ ( π ) 2 2 ˆ 1.5 = Var[ I ] = N N
Monte Carlo Optimization Powerful applications of Monte Carlo simulation lie in numerical optimization. There are many Monte Carlo optimization methods including Genetic Algorithm and Simulated Annealing. The method is relatively simple, but can effectively solve many famous complex problems.
Monte Carlo Optimization Simulated Annealing Simulated annealing is an analogy with thermodynamics when liquids freeze and crystallize, or metals cool and anneal. At high temperatures, the molecules of liquids move freely. If the liquid is cooled slowly, thermal mobility is lost. Atoms line up to form a pure crystal, which is the state of minimum energy.
Monte Carlo Optimization If the liquid is cooled quickly or quenched, it will not reach the ground state but ends up in some other states having somewhat higher energy. The essence of the process is slow cooling, allowing ample time for redistribution of the atoms as they lose mobility. This is the technical definition of annealing.
Monte Carlo Optimization A system in thermal equilibrium at temperature T has its energy probabilistically distributed among all different energy states E according to the Boltzman distribution: PE ( ) e E/ kt where k is the Boltzman s constant.
Monte Carlo Optimization At low temperatures, the system can still be in a high energy state with a small probability. This allow the system to get out of a local minimum in favor of finding a better, more global, one. As the temperature is lowered, the system will be less likely to be in a higher energy state.
Monte Carlo Optimization In 1953, Metropolis et al propose that a system will change its configuration from energy E 1 to energy E 2 with probability PE ( 1 E2) = 1 if E < E ( E2 E1)/ kt e if E E 2 1 1 2 This general scheme is known as the Metropolis algorithm.
Monte Carlo Optimization To make use of the Metropolis algorithm in general systems, we must provide: Possible system configurations. Random changes in the configuration An objective function E to be minimized. A control parameter T and an annealing schedule. Could be accomplished with a physical insight or an trial-and-error experiments.
Monte Carlo Optimization The Traveling Salesman Problem A salesman visits N cities with given positions (x i,y i ), finally returning to the city of origin. Each city is to be visited only once, and the route is to be made as short as possible. The cities are number i = 1, 2,, N.
Monte Carlo Optimization Configuration: A configuration is the order in which the cities are visited; thus, it is a permutation of 1, 2,, N. Rearrangments: An efficient suggested moves are done by Reversal: A section of path is removed and replaced with the same cities in opposite order. Transport: A section of path is removed and replaced in between two cities on another randomly chosen part of the path.
Monte Carlo Optimization Objective Function: E is just the total length of journey N i= 1 ( ) ( ) 2 2 i i+ 1 i i+ 1 E = x x + y y where the city N+1 is just the city 1.
Monte Carlo Optimization Annealing Schedule: Require some experiments. First generate some random rearrangements to determine the range of ΔE, then proceed in multiplicative steps each amounting to a 10% decrease in T. Hold a new T for 100N reconfiguration or 10N successful reconfiguration, whichever comes first. Until efforts to reduce E further becomes discouraging.