Introduction to Numerical Methods for Solving Partial Differential Equations

Transcription

1 Introduction to Numerical Methods for Solving Partial Differential Equations Benson Muite benson 25 March 2015

2 Outline Review Aim The heat equation Methods for solving the heat equation Fourier Series The Allen Cahn Equation Summary References

3 Ordinary Differential Equations: Numerical Schemes Forward Euler method y n+1 y n = f ( y n) δt Backward Euler method Implicit Midpoint rule y n+1 y n δt y n+1 y n δt Crank Nicolson Method y n+1 y n δt = f (y n+1) ( y n+1 + y n ) = f 2 = f ( y n+1) + f (y n ) 2 Other Methods: Runge Kutta, Adams Bashforth, Backward differentiation, splitting

4 The Heat Equation Used to model diffusion of heat, species, 1D 2D 3D u t u t u t = 2 u x 2 = 2 u x u y 2 = 2 u x u y u z 2 Not always a good model, since it has infinite speed of propagation Strong coupling of all points in domain make it computationally intensive to solve in parallel

5 Linear Schrödinger Equation Used to model quantum mechanical phenomena and often appears in simplified wave propagation models 1D 2D 3D i u t i u t i u t = 2 u x 2 = 2 u x u y 2 = 2 u x u y u z 2 Looks like a heat equation with imaginary time Strong coupling of all points in domain make it computationally intensive to solve in parallel

6 The Wave Equation Used to model propagation of sound, light 1D 2 u t 2 = 2 u x 2 2D 2 u t 2 = 2 u x u y 2 3D 2 u t 2 = 2 u x u y u z 2 Has finite speed of propagation Finite signal propagation speed sometimes useful in parallelization since there is no coupling between grid points that are far apart, hence smaller communication requirements If propagation speed is very fast, communication requirements still important

7 The Poisson Equation Used to model static deflection of a drumhead 1D 2D 3D f (x) = 2 u x 2 f (x, y) = 2 u x u y 2 f (x, y, z) = 2 u x u y u z 2 No time dependence, but also arises in time discretizations of time dependent partial differential equations

8 Burgers Equation Simple model for gas dynamics, also traffic Inviscid form (can develop shocks) Viscous form u t = uu x u t = uu x + ɛu xx

9 Navier-Stokes Equations Models motion of fluids Consider incompressible case only (low speed motion relative to speed of sound) ( ) u ρ t + u u u = 0. p pressure, µ viscosity, ρ, density 2D u(x, y) = (u(x, y), v(x, y)) = p + µ u 3D u(x, y, z) = (u(x, y, z), v(x, y, z), w(x, y, z))

10 Maxwells Equations Model propagation of electromagnetic waves D t H = 0 B t + E = 0 with the relation between the electromagnetic components given by the constitutive relations: D = ε o (x, y, z) E B = µ o (x, y, z) H

11 Summary on Models Many diverse models In 2D and 3D, parallel computing is very useful for getting numerical solutions in reasonable time Typical applications are in physics, chemistry, biology and engineering Related models also appear in social sciences, though usefulness for predictability of real world data is less clear

12 Aim to simulate this, Visualization around Formula 1 Race Car by Renato N. Elias, Rio de Janeiro, Brazil

13 or this Visualization around Formula 1 Race Car by Renato N. Elias, Rio de Janeiro, Brazil

14 Maybe even this

15 A simple model: The Heat Equation

16 The Heat Equation How does the temperature of the rod vary with time?

17 The Heat Equation: Model 1 Middle of rod is initially hot due to previous heating (eg. oven in house that has just been turned off) Temperature on the two sides is 0 (winter and cold outside the house) Assume discrete uniformly space time, and discrete space with molecules at each coordinate point Assume temperature in bar is directly proportional to heat, with the same proportionality constant throughout the bar At next time step, a molecule will with probability 0.5 not transfer its current heat with probability 0.25 transfer half of its heat to its left neighbor with probability 0.25 transfer half of its heat to its right neighbor

18 The Heat Equation: Model 1 Let Ti n denote the temperature at position i at time n. Particles at the end have a fixed temperature/heat they can transfer, but they always remain at the same temperature. T n+1 i = Not transfer heat 0.0T n i 1 + T n i 0.5T n i 1 + T n i 0.5T n i 1 + T n i 0.0T n i 1 + T n i Transfer heat left 0.0T n i T n i 0.5T n i T n i 0.5T n i T n i 0.0T n i T n i Transfer heat right 0.0T n i T n i 0.5T n i T n i 0.5T n i T n i 0.0T n i T n i + 0.5T n i+1 probability T n i+1 probability T n i+1 probability T n i+1 probability T n i+1 probability T n i+1 probability T n i+1 probability T n i+1 probability T n i+1 probability T n i+1 probability T n i+1 probability T n i+1 probability

19 DEMO The Heat Equation: Model 1

20 The Heat Equation: Model 2 Probability is hard! It takes a long time May only interested in average temperature/heat, not instantaneous fluctuations Take averages T n+1 i = ( )Ti 1 n n + ( )Ti + ( )Ti+1 n T n+1 i or T n+1 i = 0.125Ti 1 n T i n Ti+1 n = T n i + T i 1 n 2T i n + Ti+1 n 8 The last line looks like a difference approximation for a differential equation

21 The Heat Equation: Model 3 Let us find a differential equation! Make the time increment small T n+1 i T n+1 i = T n i T n i δt + T i 1 n 2T i n + Ti+1 n 8 = T i 1 n 2T i n 8δt + T n i+1

22 The Heat Equation: Model 3 Let us find a differential equation! Make the space increment small T n+1 i T n+1 i T n i δt T n i δt = T i 1 n 2T i n + Ti+1 n 8δt = (δx)2 8δt Ti 1 n T i n δx T i n Ti+1 n δx δx Let δx 0 and δt 0 such that (δx) 2 = δt we get T t = 1 2 T 8 x 2 Constant in continuum formulation depends on physics and is usually measured experimentally, or determined from microscopic (many particle) simulations.

23 The Heat Equation: Numerical Solution Methods Probabalistic Finite Difference Finite Volume Finite Element Spectral

24 Finite Difference Method Approximate derivatives by difference quotients Simple method to derive and implement Convergence rates tend not to be great Difficult to use for complicated geometries Tends to scale well since communication requirements are low

25 Finite Difference Method for Heat Equation u t = 8u xx Using backward Euler time stepping: u n+1 i u n i δt = 8 un+1 i 1 2un+1 i + u n+1 i+1 (δx) 2 Using forward Euler time stepping (strong stability restrictions): u n+1 i u n i δt = 8 un i 1 2un i + u n i+1 (δx) 2

26 Finite Difference Method for Heat Equation Simple method to derive and implement Hardest part for implicit schemes is solution of resulting linear system of equations Explicit schemes typically have stability restrictions or can always be unstable Convergence rates tend not to be great to get an accurate solution, a large number of grid points are needed Difficult to use for complicated geometries Tends to scale well since communication requirements are low

27 Finite Volume Method for Heat Equation u = v and t = v x Consider a cell averaged integral, then use implicit midpoint rule xi+1 x i u x dx = x i+1 x i vdx u i+1 u i = x i+1 x i vdx u t = 2 u or u x 2 x u n+1 i+1 +un i+1 un+1 u i i n 2 = v n+1 i+1 +v i+1 n n+1 +v +v i i n xi+1 xi+1 x i u n+1 i+1 +un+1 u i i+1 n un i x i u t dx = x i+1 v x dx x i u t dx = v i+1 v i 2δt δx = v n+1 2 δx i+1 +v i+1 n n+1 v v i i n 2 Several ways of approximating the integrals. The one above is a little unusual, most finite volume schemes use left sided or right sided approximations.

28 Finite Volume Method for Heat Equation For implicit schemes, hardest part is solving the system of equations that results Explicit schemes parallelize very well, however a large number of grid points are usually needed to get accurate results Automated construction of simple finite volume schemes is possible, making them popular in packages No convergence theory for high order finite volume schemes Tricky to do complicated geometries accurately

29 Finite Element Method for Heat Equation u t = 2 u x 2 Assume u(x, t) ũ(x, t) = i φ i(x)t i (t), where φ i (x) is zero for x > x i+1 and x < x i 1. A simple choice is ψ is the triangle hat function, x i+1 x for x [x i, x i+1 ] and x x i 1 for x [x i 1, x i ] Now need to find T i (t), which will be evaluated using finite differences.

30 Finite Element Method for Heat Equation Consider multiplying the heat equation by a polynomial, φ j that is only non zero over a few grid points, then integrate by parts, xi+1 x i 1 xi+1 x i 1 ũ t φ idx = x i+1 2 ũ x i 1 x 2 φ i dx ũ t φ idx = x i+1 ũ φ i x i 1 x x dx

31 Finite Element Method for Heat Equation Then use implicit midpoint rule xi+1 x i 1 xi+1 ũ x i 1 t φ idx = x i+1 ũ n+1 ũ n δt φ i dx = x i+1 1 x i 1 2 ũ x ( ũ n x x i 1 φ i x dx + ũn+1 x ) φi x dx [ 1 ũn+1 2 = i+1 ũn i+1 δt [ 1 2 [ 1 2 ( ũ n+1 x ( ũ n+1 x φ i xi+1 + ũn+1 i 1 ũn i 1 ] δt φ i xi 1 2δx ) ] + ũn φ xi+1 x i xi+1 x 2δx xi+1 ) ] + ũn 2δx xi 1 xi 1 xi 1 Since φ are known before hand, can re-write this as matrix vector products, so need to solve a linear system at each time step. x φ i x

32 Finite Element Method for Heat Equation Several other ways of approximating the integrals, can extend to multiple dimensions. Weak formulation allows for solution of equations where second derivative is not naturally defined Large mathematical community developing convergence theory for these methods Well suited to complicated geometries Rather difficult to implement compared to other schemes because of integrals that need to be computed Used in many codes, but typically codes are hand written to obtain high efficiency For implicit time discretizations, solving the linear system of equations that results can be most time consuming part

33 Fourier Spectral Method for Heat Equation

34 Fourier Series: Separation of Variables 1 dy dt = y dy = dt y dy = dt y ln y + a = t + b e ln y+a = e t+b e ln y e a = e t e b y = eb e a et y(t) = ce t

35 Fourier Series: Separation of Variables 2 Suppose u = X(x)T (t) u t = 2 u x 2 dt dt (t) d2x T (t) = (x) dx 2 X(x) = C, Solving each of these separately and then using linearity we get a general solution α n exp( C n t) sin( C n x) + β n exp( C n t) cos( C n x) n=0

36 Fourier Series: Separation of Variables 3 How do we find a particular solution? Suppose u(x, t = 0) = f (x) Suppose u(0, t) = u(2π, t) and u x (0, t) = u x (2π, t) then recall 2π 0 2π 0 2π 0 { π m = n sin(nx) sin(mx) = 0 m n, { π m = n cos(nx) cos(mx) = 0 m n, cos(nx) sin(mx) = 0.

37 Fourier Series: Separation of Variables 4 So if then and u(x, t) = f (x) = α n sin(nx) + β n cos(nx). n=0 α n = β n = 2π 0 f (x) sin(nx)dx 2π 0 sin 2 (nx)dx 2π 0 f (x) cos(nx)dx 2π 0 cos 2 (nx)dx. exp( n 2 t) [α n sin(nx) + β n cos(nx)] n=0

38 Fourier Series: Separation of Variables 5 The Fast Fourier Transform allows one to find good approximations to α n and β n when the solution is found at a finite number of evenly spaced grid points By rescaling, can consider intervals other than [0, 2π) Fourier transform also works on infinite intervals, but require function to decay to the same constant value at ±

39 Complex Fourier Series By using Euler s formula, one can get a simpler expression for a Fourier series where sine and cosine are combined u = n= γ n exp(inx) x [0, 2π) Source:

40 A Computational Algorithm for Computing An Approximate Fourier Transform 1 Analytic method of computing Fourier transform can be tedious Can use quadrature to numerically evaluate Fourier transforms O(n 2 ) operations Gauss and then Cooley and Tukey found O(n log n) algorithm Key observation is to use factorization and recursion Modern computers use variants of this idea that are more suitable for computer hardware where moving data is more expensive than floating point operations

41 A Computational Algorithm for Computing An Approximate Fourier Transform 2 Example pseudo code to compute a radix 2 out of place DFT where x has length that is a power of 2 1: procedure X 0,...,N 1 (ditfft2(x,n,s)) 2: DFT of (x 0, x s, x 2 s,..., x (N 1)s ) 3: if N=1 then 4: trivial size-1 DFT of base case 5: X 0 X 0 6: else 7: DFT of (x 0, x 2s, x 4s,...) 8: X 0,...,N/2 1 ditfft2(x, N/2, 2s) 9: DFT of (x s, x s+2s, x s+4s,...) 10: X N/2,...,N 1 ditfft2(x + s, N/2, 2s) 11: Combine DFTs of two halves into full DFT 12: for k = 0 N/2 1 do 13: t X k 14: X k t + exp( 2πik/N)X k+n/2 15: X k+n/2 t exp( 2πik/N)X k+n/2 16: end for 17: end if 18: end procedure Sources:

42 The Heat Equation: Finding Derivatives and Timestepping Let u(x) = û k exp(ikx) k then d ν u dx ν = (ik) ν û k exp(ikx). Consider u t = u xx, which is approximated by û n+1 k û k t û n k δt = (ik) 2 û k û n+1 k [1 δt(ik) 2 ] = û n k û n+1 k = = (ik) 2 û n+1 k û n k [1 δt(ik) 2 ].

43 The Heat Equation: Finding Derivatives and Timestepping Python demonstration

44 The Allen Cahn Equation: Implicit-Explicit Method Consider u t = ɛu xx + u u 3, which is approximated by backward Euler for the linear terms and forward Euler for the nonlinear terms û t û n+1 û n δt = ɛ(ik) 2 û + û û3 = ɛ(ik) 2 u n+1 ˆ + uˆ n (u ˆn ) 3

45 The Allen Cahn Equation: Implicit-Explicit Method Python demonstration

46 Key Concepts Differential equations used to model physical phenomena - approximate model Microscopic model can suggest ways to find approximate solutions on computers Several methods introduced, probabilistic, finite difference, finite element, spectral (Fourier transform) Need to check simulations for discretization errors, then before using for prediction, understand modeling errors Time stepping, Forward Euler, Backward Euler, Implicit Midpoint Rule, Implicit-Explicit

47 References Barth, T. and Ohlberger, M. Finite Volume Methods: Foundation and Analysis Boyd, J.P. Chebyshev and Fourier Spectral Methods 2nd. edition http: //www-personal.umich.edu/ jpboyd/book_spectral2000.html Chen G., Cloutier B., Li N., Muite B.K., Rigge P. and Balakrishnan S., Souza A., West J. Parallel Spectral Numerical Methods petascale/materials/upmodules/parallel_spectral_methods/ Corral M. Vector Calculus Cooley J.W. and Tukey J.W. An algorithm for the machine calculation of complex Fourier series Math. Comput. 19, , (1965) Eymard, R. Gallouët, T. and Herbin, R. Finite Volume Methods herbin/book/bookevol.pdf Greenbaum A. and Chartier T.P. Numerical Methods Princeton University Press (2012) Heideman M.T., Johnson D.H. and Burrus C.S. Gauss and the History of the Fast Fourier Transform IEEE ASSP Magazine 1, 14 21, (1984) Ketcheson, D. Hyperpython Lawler, G.F. Random Walk and the Heat Equation American Mathematical Society (2010) Macqueron, C. Computational Fluid Dynamics Modeling of a wood-burning stove-heated sauna using NIST s Fire Dynamics Simulator

48 References McGrattan, K. et al. Fire Dynamics Simulator Petersen W.P. and Arbenz P. Introduction to Parallel Computing Oxford University Press (2004) Quarteroni, A. Sacco, R. and Saleri, F. Numerical Mathematics Sauer T. Numerical Analysis Pearson (2012) Süli, E. Finite element methods for partial differential equations people.maths.ox.ac.uk/suli/fem.pdf Trefethen, L.N. Finite Difference and Spectral Methods for Ordinary and Partial Differential Equations Vainikko E. Scientific Computing Lectures Vainikko E. Fortran 95 Ja MPI Tartu Ülikool Kirjastus (2004) eero/pc/f95jampi.pdf

49 Acknowledgements Oleg Batra sev Leonid Dorogin Michael Quell Eero Vainikko The Blue Waters Undergraduate Petascale Education Program administered by the Shodor foundation The Division of Literature, Sciences and Arts at the University of Michigan

50 Extra Slides

51 Fun with Fourier Series A Fourier series can represent a regular k-gon in the complex plane f (t) = + n= (1 + kn) 2 exp [i(1 + kn)t] t ( π, π) Robert, A Fourier Series of Polygons Amer. Math. Monthly 101(5) pp (1994) Schonberg, IJ The Finite Fourier Series And Elementary Geometry Amer. Math. Monthly 57(6) pp (1950)

52 Monte Carlo Method: A Probabilistic Way to Calculate Integrals Recall Hence given f 1 b f = f (x) dx b a a b a f (x)dx = (b a) f Doing the same in 2 dimensions and estimating the error using the standard deviation f f (x, y) da A(R) f ± A(R) 2 ( f ) 2, N R Approximate f by random sampling N f i=1 f (x i, y i ) N and f N 2 i=1 (f (x i, y i )) 2 N

53 # we scale the i n t e g r a l by the t o t a l area and d i v i d e by the number of # p o i n t s used I2d=I2d / numpoints I2dsquare=I2dsquare / numpoints EstimError =4.0 numpy. sqrt ( ( I2dsquare I2d 2)/ numpoints ) # estimated e r r o r I2d=I2d 4.0 p r i n t Value : %f %I2d p r i n t Error estimate : %f %EstimError Listing 1: A Python program which demonstrates how to use the Monte Carlo method to calculate the volume below z = x 2 + 2y 2, with (x, y) (0, 1) (0, 4). Monte Carlo Method: Python Program A program to approximate an i n t e g r a l using a Monte Carlo method This could be made faster by using vectorization, however i t is kept as simple as possible for c l a r i t y and ease of translation into other languages import math import numpy import time numpoints=65536 # number of random sample p o i n t s I2d =0.0 # i n i t i a l i z e value I2dsquare =0.0 # i n i t i a l i z e to allow for calculation of variance for n in xrange ( numpoints ) : x=numpy. random. uniform ( ) y=4.0 numpy. random. uniform ( ) I2d=I2d+x x+2.0 y y I2dsquare=I2dsquare +( x x+2.0 y y) 2

54 Sample Results of Monte Carlo Program N Value Error Estimate / / / / See Spectral_Numerical_Methods/Introduction_to_ Parallel_Programming for more information