Monte carlo methods for the solution of ODE's and PDE's

Transcription

1 University of Washington ME-535 Computational Techniques in Mechanical Engineering Monte carlo methods for the solution of ODE's and PDE's Authors: Tony Enslow David Jonsson June 9, 2014

2 1 Introduction The Monte Carlo method is an extremely powerful numerical method for high-dimensional and complex problems. It provides the tools necessary to both visualize the process, as well as computing the solution of many engineering problems.although it is not very ecient for lower order dierential equations, it is especially ecient when the computational complexity grows exponentially with increasing dimensions (PDE). The method uses random number generation to approximate numerical solutions. 2 Background The Monte Carlo method, as we know it today, dates back to the 1940s in the development in the atomic bomb. The name came from the casino industry because of the randomness of gambling and its connection to probability. It was therefore named after a popular place in Monaco, renowned for its gambling. The method of using randomness in solving mathematical problems can be traced back to the eighteenth century, to estimate the true value of pi. Another widely used application of the method is also to estimate the area of amorphous shapes. This can be done by enclosing the shape in a unit square and sampling random points. Then by determining the amount of points inside the shape, one can determine the area of the shape by the ratio of points inside and outside of the shape boundaries.this method has also been found to be very useful in stochastic processes. In particular, to estimate investments in nance by evaluating nancial derivatives and investment risks by evaluating the probability of success. This method also has wide applications in computational physics, and physical chemistry due to the randomness of small particles often studied in these elds. It is clear that this method has a wide variety of applications. Brownian motion Brownian motion is the random movement of particles suspended in a uid. It is caused by the collision of larger particles with atoms or molecules in the uid, causing them to alter their movement. This motion is a limit to the stochastic process,the random walk, is often used in Monte Carlo methods. Albert Einstein predicted that the Brownian motion of a particle in uid is characterized by the diusion coecient D = k b T b where b is the Boltzmann constant. The general diusion equation is given by δu δt = D δu δx 2 And from there we can nd the root mean square displacement at time t disp = 2Dt Due to the relationship of Brownian motion and the random walks, this motion is particularly important in many applications of Monte Carlo methods, including the solution of dierential equations 2,3.

3 Random Walks In many cases of engineering, closed form equations can often be hard to derive for a given problem. When this is the case, using a stochastic approach is often an attractive alternative. Using random numbers to calculate the behavior of the problem often gives a good approximation of the solution. A good example of this is the random walk problem of a drunk person walking home from a bar. If we simulate one walk, where a step in any direction can be taken with the same probability, we might end far from the bar or close to the starting point. On the other hand, if we simulate this walk many times, the average distance should give an expected quantitative result to the average displacement of the individual. This is called the method of random walks. The probability of each step will not always be equal and the use of Brownian motion will also come into play where a collision will alter your movement, but as can be seen in the examples below, this method is at the core of many Monte Carlo applications. 3 Method and Precision Any algorithm that uses random numbers to get a quantitative solution to a problem, is a Monte Carlo method. The idea is to initiate a random walk within a domain of a given equation. Probability of moving in each direction is then dened by the dierential equation that is being solved. The walk continues until the particle reaches the boundary of the domain. This can often be a good approximation, given that a sucient number of simulations are run. For low dimensional equations, this is not an ecient method, but it becomes increasingly ecient as the dimensions increase, compared to many other numerical methods. Although there are no general predened steps for a Monte Carlo method, most of them follow a general 4 step trend 1. Estimate the proper range of inputs. For example, if we are trying to estimate the density of air during a process, we would never input a negative number. 2. Generate random numbers in the desired range using either a uniform or non-uniform distribution of randomness. 3. Derive a deterministic function for the given problem, and use the random numbers as an input 4. Iterate and use either a ratio or the average value of all iterations. Continue to iterate until the solution becomes stable. Generally, the precision of the Monte Carlo method is strictly based on the number of iterations. Given a function f(x) with uniform random distribution for the value x 0 x 1, the error can be estimated by error = 1 N N f(x i ) I i=1 Where I is the estimated quantity. Simplifying this equation gives the error approximation error σ f N η(0, 1) where σf 2 = (f(x) I) 2 dx ω

4 and η(0, 1) is the standard random variable. It is worth noting that the Monte Carlo 1 method converges with the rate of N which is rather slow compared to other methods. 4 ODEs The Monte Carlo method is not ecient for standard ODEs with a known analytical solution, as was stated above. It becomes very good for stochastic dierential equations, higher dimension equations, and equations with special conditions or unknown equations. Due to the scope of this paper, this chapter will only cover the last case but the other two cases were briey mentioned above. For an equation with special conditions, we imagine an area with a boundary conditions. We start from some point inside the area and walk towards the boundary. At each step we can go in some direction and the probability of each direction is controlled by the dierential equation at that point. When the boundary is reached, the particle is absorbed or eliminated, depending on the boundary condition. We perform this movement n number of times using uniformly randomized set of numbers in a chosen interval and after those n walks, we will have the solution to our dierential equation inside that perimeter. Example 1 For clarity, let's do an engineering example for a colloid thruster, to demonstrate the application of the Monte Carlo method in the engineering realm. A colloid thruster is a space propulsion system that creates an electrospray of charged liquid droplets to produce propulsion. The problem with solving this system analtically is that the movement of the particles is random in nature, and therefore we cannot assume that they all travel in the same direction during propulsion. Consider the trajectory of a particle inside the electric eld, 0 x L,that can be described using a set of dierential equations dx dt = v x dy dt = v y dv x dt = A x(x) dv r dt = 0 Where v x is the velocity in the x direction and v y is the velocity in the y direction and all particles have dierent, random initial velocity distribution. A x = e dφ(x) is the acceleration induced by the electric eld with the boundary conditions for x m dx { A x0 (1 3x 2 + 2x 3 ), if x L. A x (x) = 0, if x > L. With the initial position of particles x(0) = y(0) = 0 and an initial velocity v x (0) = V 0 cosα 0 and v y (0) = V 0 sinα 0 where α is the angle of angle of the particle. When solving the sets of equation we assume that α is a uniform random variable. We run the code in appendix A and get the average velocities, v x, v y after the particle leaves the electric eld, given in chosen units v x = ± , v y = ±

5 When solving the set of equation we rst generated a random sample of the departure angle with a xed velocity. Then we analyzed every computation for each of the random samples. Finally we found the average velocities and their standard deviation. 5 PDEs Partial dierential equations are very important in many applications of engineering. Monte Carlo methods often give a good solution to these equations. As PDE's can often become very complex and dicult to solve analytically, applying a randomized method such as the Monte Carlo method, can be a good approach to solving them. In order to show the solution process, let's consider two examples. Example 1 We consider the Dirichlet solution to the Laplace equation. Let there be a bounded, connected, region S and a function F(x) which satises Laplace's equation everywhere inside of S where φ is given at the boundary k j=1 = δ2 φ j δx 2 j φ(x) = f(q) = 0 (1) Next we set up the problem by drawing a square around S and splitting it up into a grid. Then we set up our random walks in two dimensions. We start from the center and walk towards the boundary. We solve the dierential equation (1) for two dimensions, by substituting the known dierence equations into the ODEs, and get Which yields 0 = δ2 φ δx + δ2 φ 2 δy φ(p 1) + φ(p 2 ) + φ(p 3 ) + φ(p 4 ) 2 h 2 φ(p ) = φ(p 1) + φ(p 2 ) + φ(p 3 ) + φ(p 4 ) 4φ(P ) 4 From equation 2 we can simulate the solution to the problem. We walk from the center towards the boundary with a choice of going to 4 surrounding nodes (left,right,up,down). (2)

6 After N simulation of random walks, all ending somewhere at the boundary, the solution can be approximated using φ(x, y) N n i f(q i ) i=1 where n is the number of walks terminated at the boundary node Q i. Random walks do not necessarily have an equal probability to walk in each direction. Let's consider a general elliptic dierential equation in the same region S. Which has the dierence equation N δ 2 φ β 11 δx + 2β δ 2 φ 2 12 δxδy + φ β22δ2 δy + 2α δφ 2 1 δx + 2α δφ 2 δy = 0 (4) β 11 xx φ + 2β 12 xy φ + β 22 yy φ + 2α 1 x φ + 2α 2 y φ = 0 We approximate the dierence equation for ve points around (x,y) where each point is a possible destination for the next step. The probabilities for each point are found by deriving the dierence equation and arriving at the function for φ φ(x, y) = φ(x + h, y)p 1 + φ(x, y + h)p 2 + φ(x h, y)p 3 + φ(x, y h)p 4 + φ(x + h, y + h)p 5 Where Where P 1 = β α 1 h 2β 12, P 2 = β α 2 h 2β 12 D D P 3 = β 11 D, P 4 = β 22 D, P 5 = 2β12 D D = 2β11 2β β (α 1 + α 2 )h The solution from these random walks is then nally found using equation 3 above. We have now seen how the Monte Carlo method can be easily applied to two dimensional problems. Let's look at one of the most important partial dierential equation used by engineers; the diusion equation. Since the basic model for the diusion equation of heat uses the idea of random spread of particles in all direction, the Monte Carlo method is good for acquiring the solution as well as giving some insight into the individual movement of each particle. (3) Example 2 Let's consider the one dimensional diusion equation δu δt = δu δx 2 u(x, 0) = u 0 (x) Which we can describe with a the dierence equation u n+1 i = (1 2λ)u n i + λ(u n i+1 + u n i 1 (5) where u n i is the approximation to u(i x, n t). λ = t. The random walk of each x 2 heat particle can then be described by the dierence equation(5) for each time point. Each

7 particle has three possible actions, stay put, go left, and go right. The probability of each option is given respectively P 1 = 1 2λ, P 2 = λ, P 3 = λ Let's now apply this general approach to particles in a uid container. We put particles into a container lled with a dierent substance. We observe their behavior and their relative concentration in the uid and model it using the Monte Carlo method described above. We then compare the solution to the analytical solution of the problem 11. Figure 1: Particles in uid We see that the Monte Carlo method for time steps and random walks gives a good approximation to the diusion equation. The Matlab code for this gure is given in appendix A. 6 Conclusion The Monte Carlo methods demonstrate a good solution for many problems in engineering. In many cases it can be dicult to nd the closed form equation of a process. This was shown in the example of a thrusting power of a colloid thruster, where the particle departure angle is a random process which greatly aects the solution to the problem. We furthermore explored the domain of partial dierential equations where we showed the ecacy of the Monte Carlo method for solving boundary value problems using random walks. We then solved the one dimensional heat equation and demonstrating the usefulness of this method. For higher order equations, this method becomes increasingly attractive compared to other methods, especially when an analytical solution is hard to derive.

8 7 Reference 1. Lecture notes on numerical methods for ODE's, MIT, 2012 http : //web.mit.edu/course/16/16.90/bac 2. Petar KoÄ oviä, Fakultet za obrazovanje rukovodecih kadrova u privredi, Novi Sad,Brownian motion development for Monte Carlo method applied on european style option price forecasting,2011, International journar of economics and law 3. Brownian motion http : //en.wikipedia.org/wiki/brownian_motion 4. Anders à sterling, Diusion equation and Monte Carlo. 5. David Bindel and Jonathan Goodman,2006, Principles of Scientiï c Computing Monte Carlo methods 6. Bernard Lapeyre,Introduction to Monte-Carlo Methods, Monte Carlo methodshttp : //en.wikipedia.org/wiki/monte C arlo m ethod 8. Branislav K. Nikoli Ä, From Random Numbers to Monte Carlo 9. Rubin H Landau, Random Walks 10. Anne GILLE-GENEST,2012,Introduction to the Monte Carlo Methods 11. Moroko, William J., and Russel E. Caisch. "A quasi-monte Carlo approach to particle simulation of the heat equation." SIAM Journal on Numerical Analysis 30.6 (1993): Peter M orters and Yuval Peres, 2008,Brownian Motion 13. Gregory F. Lawler,Random Walk and the Heat Equation 14. Matthew. N. O. Sadiku et.al, 2007, Direct Monte Carlo simulation of time-dependent problems 15. Dr. Eric Ayars, Computational Physics With Python

9 Appendix A Matlab code for colloid thruster 1 function [vx, vr] = thruster(v0, a0, Ax0, L, dt) % initial condition x = 0; r = 0; vx = V0*cos(a0); vr = V0*sin(a0); % time integration using Forward Euler while (x â L) x = x + dt*vx; r = r + dt*vr; vx = vx + dt*ax0*(1-3*x.ë 2 + 2*xx.Ë 3); end % Deterministic (non-random) parameters V0 = 0.1; Ax = 1.0; L = 1.0; dt = 0.001; a0max = 60. * pi / 180.; % 1. Generate a random sample of the input parameters a0mc = rand(10000, 1) * a0max; % Array to store Monte Carlo outputs vxmc = []; vrmc = []; % 2. Analyze (deterministically) each set of inputs in the sample for i = 1:10000 [vx, vr] = thruster(v0, a0mc(i), Ax, L, dt); vxmc = [vxmc; vx]; vrmc = [vrmc; vr]; forward_propulsion(i) = vx; side_propulsion(i) = vr; end % mean muvx = mean(vxmc); muvr = mean(vrmc); % standard deviation sigmavx = std(vxmc); sigmavr = std(vrmc);

10 Matlab code for particles in uid %========================================================================= %******** To do Monte Carlo simulation of particle diffusion ************* %************************ By Dr. Mahesha MG ****************************** % Date: 17/11/2012 %========================================================================= clear clc a1=fopen('particle.dat','w'); %Data will be stored in this file t=0:50000; nop=10000; % Total number of particles x=[nop]; nl=nop; % Asssumed that all particles are on left side of the box at t=0 for i=1:50000 lambda = nl/nop; if rand()<lambda nl=nl-1; else nl=nl+1; end fprintf(a1,'%d\t%d\n',t(i),nl); x=[x;nl]; end y=(nop/2)*(1+exp(-2*t/nop)); %Analytic solution plot(t,x,'b',t,y,'r') legend('monte-carlo','analytic') xlabel('time in s') ylabel('no. of particles on left side') fclose(a1);