Monte Crlo method in solving numericl integrtion nd differentil eqution Ye Jin Chemistry Deprtment Duke University yj66@duke.edu Abstrct: Monte Crlo method is commonly used in rel physics problem. The key ide of Monte Crlo is to simulte the result using lrge smpling, becuse rel physics experiment results re lwys the sttisticl verge of individul single processes. In this report, rndom number nd smpling method re mentioned first becuse the qulity of the rndom number genertor nd the smpling will ffect the result of Monte Crlo simultion significntly. After this, numericl integrtion nd solution of differentil eqution re studied. All results show tht Monte Crlo method is perfect option in such problems. Contents Problem description 2 2 Numericl method 2 2. Rndom number genertor................................. 2 2.2 Smpling method...................................... 3 2.3 Monte Crlo method in integrtion............................. 4 2.4 Monte Crlo method in solving Lplce eqution..................... 6 3 Experiments 7 3. Smpling.......................................... 7 3.2 Monte Crlo integrtion................................... 7 3.3 Lplce solution....................................... 8
ME555-3 Computtionl Mterils Science Ye Jin Problem description Monte Crlo method is now powerful tool in solving physics problems, especilly in sttistics mechnics nd quntum mechnics. It ws first invented for the design of tomic bomb in Second World Wr. This ide of this method is to simulte physics phenomenon bsed on rndom numbers. So the first problem is how to design n outstnding rndom number genertor. In quntum mechnics, the observtion cn hppen rndomly, however, in computer, nothing is rndom. Every result is given by specific progrm. In this report, I use rndom numbers insted of pseudorndom numbers just for convenience. If the numbers re given by certin progrm, the correltion between numbers should be smll enough. This is why we use sttisticl tools to test the qulity of rndom number genertors. After generting lrge smple of rndom numbers, we cn pply the Monte Crlo method to mny rel problems bsed on these numbers. In this reports, numericl integrtion nd solution to differentil eqution re studied. A fmous exmple of Monte Crlo integrtion method is the Buffon s needle problem, which, to my point of view, is the originl ide of Monte Crlo. In fct, Monte Crlo integrtion method in lower dimensionl integrtion is not so ccurte compred with the trditionl method, for exmple, the Newton-Cotes qudrture, which is wht we lerned in clss. The rel power of Monte Crlo is in higher dimensionl integrtion. This will be discussed in section 2.3. Monte Crlo ide cn lso be implemented in solving differentil equtions like Lplce eqution. This is fscinting ide becuse it comes from rel physics problem (electrochemicl growth). It will be discussed in section 2.4. All the works re done by myself. All codes re in C lnguge, lthough the suffix is.cpp. I rn them on visul studio so if you run them on linux, the hed file might need to be chnged. 2 Numericl method 2. Rndom number genertor Mny numericl clcultions need to be constructed by rndom numbers. A good rndom number genertor should hve smll correltion between numbers. And for Monte Crlo method, rndom numbers re lrgely used. As result, the speed of the lgorithm is lso importnt. In this study, 687 genertor is implemented. The bsic ide is using the liner congruentil method to give the rndom number series x n, which is given by I n+ = (I n + b) mod m (2.) x n = I n /m (2.2) The lowest stndrd genertor is clled 687 genertor, which is still widely used tody. This method sets = 7 5 = 687, b =, m = 2 3 = 247483647 (2.3) 2
ME555-3 Computtionl Mterils Science Ye Jin But it is hrd to get the module becuse m is quite lrge number which cn overflow the limit of the computer. So Schrge method is pplied to overcome this problem. In this method, m cn be represented s m = q + r (2.4) where q = [m/] nd r = m mod. For ny r < q nd < z < m, the module cn be represented s { (z mod q) r[z/q], if z mod m = (2.5) (z mod q) r[z/q] + m, otherwise The qulity of the generted rndom numbers cn be determined by sttisticl testing. The most common tests re homogeneity test nd independence test. Homogeneity cn be tested by counting the frequency in different subres nd clculting the chi-squre, χ 2 = K (n k m k ) 2 k= m k (2.6) where the χ 2 obeys the Person theorem, P (χ 2 x v) = 2 v/2 Γ(v/2) x t (v 2)/2 e t/2 dt (2.7) This eqution gives the probbility of χ 2 < x. Here v is the degree of freedom of the system. By checking the χ 2 tble, with given degree of freedom nd confidence limit α, we cn use this test to decide the homogeneity of rndom numbers. As of the independence, we cn test the independence of two neighbors by clculting the correltion coefficient. Smll correltion coefficient vlue shows high independence. The correltion function with intervl l is C(l) = x nx n+l x n 2 x 2 n x n 2 (2.8) However, smll correltion vlue only shows the property of liner correltion. It cnnot gurntee tht two series hve correltions in other functions. 2.2 Smpling method In clss, the smpling cn be equi-spced with weight. For exmple, if we wnt to do the numericl qudrture of polynomils, Vndermonde mtrix cn be pplied. However, for Monte Crlo method, rndom numbers re required. Generlly, in rel physics problem, the smpling cnnot be verge within n re. It lwys obeys probbility distribution density function. So we need n efficient method to generte the smpling. A direct smpling method cn be used here. We only tlk bout the consecutive vritions becuse we wnt to do the integrtion. Suppose tht x [, b] nd x follows the distribution function p(x). The 3
ME555-3 Computtionl Mterils Science Ye Jin cumultive distribution function (CDF) ξ(x) is thus ξ(x) = x p(x )dx (2.9) where p(x) hs been normlized here. For ech x, we cn clculte the ξ first nd do the inverse to get the smpling. One exmple is tht for prticle with rndom movement, the free pth distribution is The CDF is then As result, the inverse is ξ = x p(x) = λ e x/λ (2.) λ e t/λ dt = e x/λ (2.) x = λ ln( ξ) = λ ln ξ (2.2) The reson we replce ξ with ξ is becuse they re equl in smpling. In this sitution, we cn smple ξ by generting the rndom numbers (discussed in section 2.) nd generte x by eqution (2.2). However, we need more thn this. In Monte Crlo integrtion, we do not hve such distribution for the vrition x. In fct, the method is similr, we will further discuss it in the next section. 2.3 Monte Crlo method in integrtion Different from other numericl qudrture methods, in Monte Crlo method, the smpling x i cn be generted rndomly on the scle [, b]. According to integrtion verge theorem And the verge cn be derived by As result, f(x)dx = (b ) f (2.3) f N f(x i ) (2.4) i= f(x)dx b N f(x i ) (2.5) This eqution is similr with the method we tlked bout in clss, but (b )/N is not the width of the rectngulr spce, it is in fct the weight for rndom number x i. To get higher ccurcy in the integrtion, we need lrge smpling. The error for Monte Crlo method cn be strictly derived by sttistics. The lw of lrge numbers shows tht lim f i µ (2.6) N N i= 4 i=
ME555-3 Computtionl Mterils Science Ye Jin where µ is the expecttion of rndom series {f i }. The centrl limit theorem shows tht when N is not infinity, the error distribution is where Φ(β) is the norml distribution. As result, P { f µ σ f / < β} Φ(β) (2.7) N σ S = f µ σ f N = N f 2 f 2 (2.8) where σ f is the stndrd devition of f(x) nd σ S is the stndrd devition of the integrtion. The bove discussion shows two importnt spects of Monte Crlo integrtion method, () σ S chnges with / N. When the smpling is enlrged times, the error will be cut down times. To cquire n ccurcy limit, the smpling should increse s O(N 2 ). This is the wek point of Monte Crlo, which mens tht the convergence rte is slower compred with other methods. However, in higher dimension or singulr integrtion, Monte Crlo performs the best. (2) When σ f is flt, the ccurcy limit cn be incresed. However, if the function is δ(x), quite smll effective smpling cn be selected. The error will thus be very lrge. This is the common problem for not only Monte Crlo integrtion, but lso for every ppliction with Monte Crlo method. In higher dimension integrtion, Monte Crlo method hs the form dx 2 2 dx 2 n n dx n f(x, x 2,, x n ) = N [ n (b j j )] j= f(x i, x 2i,, x ni ) (2.9) i= For one smpling with N numbers, the error of Monte Crlo method is / N, while for settled grid, the error is N /2d > / N. This is why when the dimension is higher thn 4, no other method cn be effective thn Monte Crlo method. The bove discussion cn be done by direct smpling. However, in some sitution, for exmple, the integrtion of function which hs the similr shpe with gussin function, we cn use selective smpling method to do more effective integrtion. In generl, if distribution function g(x) hs similr shpe with f(x), the integrtion cn be reformulted f(x)dx = f(x) g(x)dx (2.2) g(x) where g(x)dx =. If x is not rndomly selected in [, b], but selected by the distribution g(x), then under this smpling, the Monte Crlo integrtion will turn to f(x)dx = f/g N 5 i= f(x i ) g(x i ) (2.2)
ME555-3 Computtionl Mterils Science Ye Jin We cn tret this method s simple method we hve lernt in clss. Suppose As result, dy = g(x)dx, y = f(x)dx = x f(x) g(x) g(x)dx = g(x )dx (2.22) f(y) dy (2.23) g(y) If g(x) is not generlized, for exmple, c = g(x)dx, the bove eqution will be f(x)dx = c f(y) dy (2.24) g(y) The interesting prt is tht in this wy, the simple smpling of y is just the smpling of x with the weight g(x). The exmple will be shown in the experiment prt. 2.4 Monte Crlo method in solving Lplce eqution In electrochemicl growth process, ll the copper ion will do rndom wlk under the potentil φ in Lplce eqution 2 φ = (2.25) As result, the eqution cn be solved by pplying rndom wlk method. I will not go into detils bout rndom wlk in this report. In fct, the probbility of prticle wlking in grid rndomly to the position (i, j) is P i,j = (P i,j + P i+,j + P i,j + P i,j+ )/4 (2.26) The Lplce eqution cn be solved by this rndom wlk ide. More generlly, we cn use this ide to solve the Poisson eqution 2 φ(x, y) = q(x, y), φ Γ = Φ (2.27) Lplce eqution is specil cse of Poisson eqution t q =. For equidistnt division squre grid with length h, the bove eqution hs the discrete form We cn rewrite it s φ i,j = (φ i,j + φ i+,j + φ i,j + φ i,j+ h 2 q i,j )/4 (2.28) φ = 4 p,k φ k h 2 q /4, p,k = /4 (2.29) k= where p,k is the probbility from to k. Suppose tht we strt the rndom wlk t (i, j) under the potentil φ, if it chooses one of the four grids m, then φ = φ m h 2 q /4. After this, the prticle goes to its neighbor n from m nd we obtin 6
ME555-3 Computtionl Mterils Science Ye Jin φ m = φ n h 2 q m /4. At now, we hve φ = φ n h 2 (q + q m )/4. Repet this process until the prticle reches the edge Γ t point s. Tke down the vlue Φ(s), we hve φ = Φ(s) h2 4 K k= q k (2.3) This is the result of one trjectory. If we repet this work N times, the verge is φ = N n= φ (n) = N n= {Φ(s (n) ) h2 4 K (n) k= q (n) k } (2.3) I m interesting in this problem becuse in physics, we cn study the rndom growth process t potentil φ. This ide cn be implemented to solve the differentil eqution. For Lplce eqution, the vlue t fixed point is the verge of Φ(s) tht this prticle cn rech through 2D rndom wlk. The detil will be discussed in the following chpter. 3 Experiments 3. Smpling The code of smpling is in Schrge.cpp. 687 genertor nd Schrge method re used to produce rndom numbers. I produced 2 5 rndom numbers nd counted them by grouping into mtrix for the chi-squre test. The correltion coefficient is lwys round ±.2, which shows tht this genertor cn produce higher independence rndom numbers. The chi-squre test shows tht the uniformity is gret. As result, this genertor cn be used for Monte Crlo method. 3.2 Monte Crlo integrtion By using the 687 rndom number genertor, we cn derive the Monte Crlo integrtion. The code is in Monte Crlo Integrtion.cpp. The first integrtion I choose is I = x + xdx = f(x)dx (3.32) I set the totl smpling number s 5. The result is compred Mthemtic result, which is.453. As we discussed in the lst chpter, if we wnt to do the selective smpling, we hve to decide weight function. This function is similr with x. By expnding it into Tylor series t x = nd only preserving the liner term, we hve x = + 2 (x ) = (x + ) = g(x) (3.33) 2 7
ME555-3 Computtionl Mterils Science Ye Jin So the integrtion turns into I = x + 3/4 xdx = f(y) dy (3.34) g(y) where y = x (x + ) = 2 4 x2 + x. And the up limit 3/4 is just substitute x with in the left eqution. 2 The result given by direct nd selective smpling Monte Crlo methods re shown in Tble 3.. Tble 3.: Direct smpling Monte Crlo integrtion results. Run 2 3 4 5 6 7 8 9 Direct.454.445.455.459.45.455.452.455.456.455 Selective.453.45.453.45.454.45.452.452.452.456 The stndrd devition of direct smpling is.36, while the selective smpling gives the stndrd devition.7. The ccurcy is incresed slightly. In this sitution, the weight function is not plying n importnt role. However, in some integrtion, for exmple, I = π dx (3.35) x 2 + cos 2 x the weight cn be extremely importnt. Tble 3.2 shows the result with nd without the weight function. The weight function I choose here is g(x) = e x becuse they hve similr shpes. The result given by Mthemtic is.582. Obviously, the weight function highly increse the ccurcy of the result. Tble 3.2: Direct smpling Monte Crlo integrtion results. Run 2 3 4 5 6 7 8 9 Direct.9476.9477.9476.9477.9476.9477.9476.9477.9477.9477 Selective.583.586.586.58.58.589.586.586.582.58 I lso tried the higher dimensionl integrtion. For exmple, I =.7 dx.8 dy.9 dz du. dv(6 x 2 y 2 z 2 u 2 v 2 ) (3.36) The integrtion result is 2.5598 ccording to eqution (2.9). This result is ccurte compred with the vlue clculted by Mthemtic. It is not rel physics problem so the integrtion seems esy. If we come cross rel sttisticl mechnic problems, Monte Crlo cn be extremely helpful becuse the integrtion will go over ll degrees of freedom of the system. 3.3 Lplce solution The code is in lplce.cpp. The first step is to decide the edge. In my code, I set the re s. Two mtrices re defined in the code. The boundry mtrix is Boole mtrix, if the point is the edge, 8
ME555-3 Computtionl Mterils Science Ye Jin then the vlue is, otherwise is. The other mtrix is the vlue mtrix, which defines the initil vlue t ech point. Here I set the potentil t (5,33) s 2 nd (5,66) s, while the edge is treted s infinity so the potentil is. For ech point, I repet the rndom wlk simultion 3 times nd clculte the verge. The time for simultion lsts for bout 3 minutes. I plot the results in Mthemtic (shown in Fig 3.). This result shows exctly the correct solution to this problem. As cn be seen in this figure, two singulr points hve sme positions s the initil condition nd the rtio is bout 2:. The solution looks flt nd continuously reches t the edge. In conclusion, Monte Crlo method cn give the solution to Lplce eqution with initil conditions. However, the speed is not so fst. I think there exists fster wy in generting the trjectory. 5.8.6.4.2. 5 Figure 3.: 3D point plot of the solution to Lplce eqution. Reference All of the exmples re mde on my own. The ide of Monte Crlo is from slides online. 9