Lecture 1 Numerical methods: principles, algorithms and applications: an introduction

Transcription

1 Lecture 1 Numerical methods: principles, algorithms and applications: an introduction Weinan E 1,2 and Tiejun Li 2 1 Department of Mathematics, Princeton University, weinan@princeton.edu 2 School of Mathematical Sciences, Peking University, tieli@pku.edu.cn No.1 Science Building, 1575

2 Outline

3 1. Solar system (two-body problem) Schematics of two body problem V0 Sun Planet

4 1. Solar system (two-body problem) Governing equations: Hamiltonian system dx dt = v, d(mv) dt = V (x). Simply take V (x) = GmM. Define Hamiltonian r then dh dt = 0. Numerical solution of ODEs H = 1 2 mv2 + V (x)

5 1. Solar system (two-body problem) Long time numerical integration (T 1) 1. Bad scheme (Forward Euler) Good scheme (Symplectic scheme)

6 2. Transportation problem Schematics of transportation problem a 1 C ij b 1 a 2 b 2 a m Origin b n 1 b n Destination

7 2. Transportation problem Formulation: m n min s = i=1 j=1 c ijx ij subject to the condition m x ij = b j, j = 1,..., n i=1 n x ij = a i, i = 1,..., m j=1 x ij 0, i = 1,..., m; j = 1,..., n. where a i is the supply of the i-th origin, b j is the demand of the j-th destinations, x ij is the amount of the shipment from source i to destination j and c ij is the unit transportation cost from i to j. Optimization problem (Simplex method)

8 3. Image processing Image restoring inf u L E(u) = 1 2 Ω u 0 Ru 2 dx + λ φ( u )dx Ω Here Ω is the domain, u 0 is the degradated image, R is the blurring operator which is known a priori, φ is a particularly chosen function. u will be the recovered image. Nonlinear optimization problem or nonlinear PDE problem

9 3. Image processing Total variation denoising φ( u ) = u A blurred image with 80 lost packets Deblurring and error concealment by TV inpainting Figure 1: TV inpainting for the error concealment of a blurry image. objects J s on all the connected components 0 s of 0 ˆ. Thus in both the

10 4. Stochastic simulations Ising model for mean field ferromagnet modeling 1 2 Spin system M

11 4. Stochastic simulations Ising model in statistical physics Define the Hamiltonian H(σ) = J <ij> σ iσ j, where σ i = ±1, < ij > means to take sum w.r.t all neighboring spins i j = 1. The internal energy per site U M = 1 M σ H(σ) exp{ βh(σ)} Z M = 1 M H(σ) = 1 M ln Z M, β where Z M = σ exp{ βh(σ)} is the partition function and β = (k BT ) 1.

12 4. Stochastic simulations Finding the critical temperature T c for 2D Ising model (Metropolis algorithm)

13 4. Stochastic simulations Biological network Suppose there are N s species of molecules S i, i = 1,..., N s, and M R reaction channels R j, j = 1,..., M R. x i is the number of molecules of species S i. Then the state of the system is given by x = (x 1, x 2,..., x Ns ). Each reaction R j is characterized by a rate function a j(x) and a vector ν j that describes the change of state due to reaction (after the j th reaction, x x + ν j). In shorthand denote R j = (a j, ν j) How to simulate this biological process?

14 4. Stochastic simulations Stochastic simulation (Kinetic Monte Carlo) 4370 (Number of molecules vs Time) 4360 Direct SSA Nested SSA time FIG. 2: Extracted from a paper on stochastically simulating the number of one kind of molecules in a chemical reaction.

15 5. Signal processing Filtering Given a discrete time signal {u j} N 1 j=0, analyze the high frequency and low frequency part. Discrete Fourier Transform û k = N 1 j=0 jk 2πi u je N, k = 0, 1,..., N 1. Basic technique: FFT A polluted signal

16 5. Signal processing High pass and low pass filter (signal and noise)

17 Three approaches in scientific research Schematics for the relation of theory, computation and experiment Experiment Theory Computation

18 Top 10 algorithms in 20th century from Computing in Science and Engineering 1. Metropolis algorithm 2. Simplex method 3. Krylov subspace iteration methods 4. Matrix decomposition approach 5. Fortran Compiler 6. QR algorithm 7. Quicksort algorithm 8. Fast Fourier Transform (FFT) 9. Integer relation detector 10. Fast multipole method The algorithm 1,2,4,6,8 will be taught in this course.

19 Three different eras of computing Numerical analysis Scientific computing Computational science

20 Outline

21 Qin Jiushao algorithm (Horner s algorithm) Compute the value of polynomial p(x) = a nx n + a n 1x n a 0 naively. The computational efforts will be Nested multiplication (n + 1)(n + 2) 2 multiplications and n additions p(x) = ((a nx + a n 1) x + a n 2) )x + a 0 The computational efforts will be n multiplications and n additions

22 Solving linear system Solving the linear system Ax = b. Cramer s rule From Cramer s rule, if det(a) 0 it is solvable and the solution can be explicitly represented as x i = det(ai) det(a) and A i is the matrix with i-th column replaced by b. Working load for computing N-determinant with definition: (N 1) N! multiplications and N! 1 additions. It is an astronomy number! When N = 20, N! If the computer power is 10Gflops/s, we need 200 years at least, which is impossible.

23 Computational efficiency Theoretically solving a problem is NOT equivalent that it could be solved with computer because of the computational efficiency! In general, an O(n 4 ) algorithm is unacceptable!

24 Floating point arithmetic Binary floating point system F = {±0.d 1d 2... d t 2 m } {0} where d 1 = 1, d j = 0 or 1(j > 1). t is called precision, L m U. For any x R, denote fl(x) the floating point representation in computer. We have relative error fl(x) x 2 t := ɛ mach x The floating point system has Underflow limit UF L = 2 L 0.1 Overflow limit OF L = 2 U infinity Inf and not a number Nan.

25 Some issues for floating point arithmetic Cancellation = Difference between two approximately equal reals cause loss of significants! sin(x + ɛ) sin(x) = 2 cos(x + ɛ 2 ) sin ɛ 2 The right hand side is more suitable for computing than the left side hand. Summation The first order The second order 1 + n i=1 1 n = n = 1 in computer n ( 1 n + 1 n ) + + ( 1 n + 1 n ) + 1 = ( 2 n + 2 n ) + + ( 2 n + 2 n ) + 1 = + 1 = 2

26 Outline

27 Course plan 3 hours powerpoint teaching per week; 2 hours (or more) computer work per week (50 points); Homework 10 points; Final exam 40 points; Notes to be downloaded from tieli with Account and Password: lnm2005.

28 References 1. A. Quarteroni and F. Saleri, Scientific computing with MATLAB, Berlin and New York, Springer Verlag, D. Kahaner, C. Moler and S. Nash, Numerical methods and software, Prentice Hall, E. Süli and D.F. Mayers, An introduction to numerical analysis, Cambridge and New York, Cambridge University Press, S.D. Conti and C. de Boor, Elementary numerical analysis: an algorithmic approach, New York, McGraw-Hill, G. Dahlquist and A. Bjork, Numerical methods, Prentice Hall, G.H. Golub and C.F. Van Loan, Matrix computation, Johns Hopkins University Press, 1996.

29 Homework assignment 1 Familiarize software MATLAB about the basic definition of variables, arrays, function definition, subroutine definition, basic linear algebra operations and graphical operations.