The equation does not have a closed form solution, but we see that it changes sign in [-1,0]

Transcription

1 Numerical methods

2 Introduction Let f(x)=e x -x 2 +x. When f(x)=0? The equation does not have a closed form solution, but we see that it changes sign in [-1,0] Because f(-0.5)= , the root is in [-0.5,0] Because f(-0.25)=0.4663, the root is in [-0.5,-0.25]...

3 Introduction itercount = 0 x3 = (x2+x1)/2.0d0 err = f(x3) DO WHILE ((ABS(err) > tolerance).and. & (itercount < maxiter)) IF ((f(x1) < 0.0d0).AND. (f(x2) > 0.0d0)) THEN IF (f(x3) < 0.0d0) THEN x1 = x3 ELSE x2 = x3 END IF ELSE IF (f(x1) > 0.0d0.AND. f(x2) < 0.0d0) THEN IF (f(x3) < 0.0d0) THEN x2 = x3 ELSE x1 = x3 END IF END IF WRITE (*,'(A,I6,A,F10.6,A,E10.4)') & 'Iteration: ', itercount, ' & x*=', x3, ' f= ', err itercount = itercount + 1 x3 = (x2+x1)/2.0d0 err = f(x3) END DO Iteration: 0 x*= f= E+00 Iteration: 1 x*= f= E+00 Iteration: 2 x*= f= E+00 Iteration: 3 x*= f= E-01 Iteration: 4 x*= f= E-01 Iteration: 5 x*= f= E-01 Iteration: 6 x*= f= E-02 Iteration: 7 x*= f= E-02 Iteration: 8 x*= f= E-02 Iteration: 9 x*= f= E-03 Iteration: 10 x*= f= E-03 See bisection.f90

4 Numerical methods This was an example of a numerical method: the bisection algorithm for root finding We may solve mathematical problems besides algebraically, also numerically: Numerical methods In general, a mathematical problem cannot be solved exactly with a finite number of operations We may approximate the solution (sufficiently enough) with a finite number of operations The remaining error is referred to as the truncation error In addition, a computer handles real numbers with limited precision -> rounding error

5 When solving mathematical problems numerically Infinite becomes finite Continuous becomes discrete Often also non-linear problems are linearized Keep in mind The mathematical background of the model Into which kind of problems a method is good for How the chosen method scales when the problem size increases How the truncation and rounding errors behave as computing progresses: robustness A method is referred to as stable, if the solution is not affected by the errors in the initial values more than the actual problem does Result verification

6 Algorithms An algorithm is a finite set of operations performed in a certain order It states how the problem is solved and the errors handled Complexity: what is the order of magnitude of the required operations for a given system size O-notation: e.g. matrix-vector multiplication O(N 2 ) Also memory requirements usually of interest

7 Literature recommendations Numerical Recipes Haataja et al., Numeeriset menetelmät käytännössä (CSC 2002) In Finnish, PDF free of charge Available at

8 Root finding

9 Newton's method The bisection method is inefficient as it converges slowly It is also inapplicable to multivariate functions Newton's method: utilize the derivative to improve convergence find limits for the root x*[a,b] x 1 =(a+b)/2 k=1 do while( d <) d = f(x k )/f'(x k ) x k+1 = x k - d if x k+1 [a,b] exit k = k + 1 end do See newton.f90

10 Newton's method Shortcomings If for some k f'(xk)=0, the method will not work If for some k f'(x*)=0, the method is inefficient If there are multiple roots, the convergence is only linear Usable also for complex roots

11 Secant method The secant method is defined as a recurrence relation x n1 =x n x n x n 1 f x n f x n 1 f x n We need two initial points x 0 and x 1 that ideally should lie close to the root

12 Polynomial equations The fundamental theorem of algebra: Every nonzero single-variable polynomial, with complex coefficients, has exactly as many complex roots as its degree, if each root is counted up to its multiplicity For polynomials of order 4 or less, use the closed form expressions for the roots Implementations can be found e.g. from NetLIB, For higher order polynomials, use the specific polynomial solvers Müller or Laguerre methods Jenkins-Traub algorithm

13 Interpolation and approximation

14 Approximation Making an approximate description on the behavior of some quantity Continuous approximation For example: constructing a simple function that resembles a more complicated function Discrete approximation For example: the original function is known only in a finite set of points {x i }. Now we have to come up with a function that describes the set of points {(x i ),(f i )}. Usually we need to approximate multivariate functions

15 Interpolation In discrete approximation, we fitted a function into a given set of data {x i } such that the function is close to all of the points If the function is required to go exactly go through the datapoints, we call this interpolation In interpolation problems, we are interested in how the function behaves between the known datapoints Note the connection to analyzing experimental data If we use the interpolating function outside the range [min{x i },max{x i }], the process is called extrapolation

16 Linear approximation Assume we have data available on a discrete set of points {x 1, x 2,...,x N } with corresponding values {f(x 1 ),f(x 2 ),...,f(x N )} On many occasions, the approximating function is constructed as a linear combination of some basis functions N f x p N x= k=0 a k k x For example, let k =x k, k=0,..., N then we have the following representation at a known pair (xi, f(xi)) f x i =a 0 a 1 x i a 2 x 2 N i...a N x i

17 Linear approximation Altogether we have N+1 such equations for the N+1 unknowns a i This system of equations can be recast in a matrix form [ 2 N 1 x0 x0... x 0 2 N 1 x 1 x 1... x x N x N... x N N][ ]=[ a0 a 1... a N f x0 f x 1 ]... f x N or, Va=f. V is referred to as the Vandermonde matrix We could invert V and thus obtain a Matrix inversion is O(N 3 ) in complexity, and more efficient approximation methods exist

18 Weierstrass theorem Polynomial approximation is justified by the Weierstrass theorem: Let f(x) be continuous in a finite interval [a,b] and > 0. Then a polynomial p(x) for which f(x)-p(x) < x [a,b]. In practice, finding the optimal polynomial very difficult Also pathological cases can be constructed, cf. the Runge Phenomenon

19 Newton interpolation In the Newton interpolation k 1 method, the basis is changed to x x0 This basis is still a polynomial basis, but the coefficient matrix becomes now a lower triangular matrix The inversion of lower triangular matrix requires only O(N 2 ) operations k x= i= 0

20 Lagrangian interpolation In Lagrangian interpolation method the basis is constructed of Lagrangian polynomials h k x= x x i x k x i,i=0,1,..., N ;i k that is a polynomial of degree N and satisfies h i x j = ij See lagrange.f90 and mpi-lagrange.f90

21 Least squares fitting In many practical circumstances, we need to fit a curve into a set of experimental data points {y i } that do not follow any obvious function E.g. fitting a measurement that should follow a quadratic trend into a parabola In the least squares method we aim at finding the linear combination coefficients {a k } such that the residual R a 0, a 1,...,a n = k=0 w i [f x i y i ] 2 is minimized m

22 Least squares algorithm Aim: finding the least squares polynomial where q k (x) is a orthonormal polynomial that satisfies a recursion given below The algorithm 0 = i=1 do j = 1, n-1 end do n f x k= 0 a k q k x m wi ;q 0 x=1 ;q 1 x= x 1 m 0 i=1 m j = i=1 j j = j 1 wi q j 2 x m j1 = 1 j i=1 w i x i q 2 j x wi x i q j1 x=x q j x j1 q j x j q j 1 x; j=1,...,n 1 m is the number of grid points, n the degree of the fitting polynomial

23 Least squares algorithm Having constructed all the polynomials qk(x) we can compute the coefficients from a k = 1 k i=1 m wi q k x i y i ;k=0,1,...,n and finally the least-squares polynomial

24 MPI point-to-point communication

25 Sending operation in MPI MPI_Send(buf, count, datatype, dest, tag, comm, rc) Parameters buf The data that is sent count Number of elements in buffer datatype Type of each element in buf dest The rank of the receiver tag An integer identifying the message comm Communicator rc An error / return code

26 Basic datatypes in MPI MPI type MPI_CHARACTER MPI_INTEGER MPI_REAL MPI_REAL8 MPI_DOUBLE_PRECISION MPI_COMPLEX MPI_DOUBLE_COMPLEX MPI_LOGICAL MPI_BYTE MPI_PACKED Fortran type CHARACTER INTEGER REAL REAL*8(non standard) DOUBLE PRECISION COMPLEX DOUBLE COMPLEX LOGICAL

27 Sending operation in MPI Argument types <MPI datatype> buffer(*) integer count, datatype, dest, tag, comm, rc Special values for parameters dest MPI_PROC_NULL No operation takes place. comm rc MPI_COMM_WORLD Default communicator, this includes all processes MPI_SUCCESS Value if operation was successful

28 Receiving operation in MPI MPI_Recv(buf, count, datatype,source, tag, comm, status, rc) Parameters buf Buffer for storing received data count Number of elements in buffer (not the number of elements that are actually received) datatype Type of each element in buf source Sender of the message tag Number identifying the message comm Communicator status Information on the received message rc As for the send operation

29 Receiving operation in MPI Argument types <MPI datatype> buf(*) integer count, datatype, source, tag, comm, status(mpi_status_size), rc Special values for parameters source MPI_ANY_SOURCE Receive from any sender MPI_PROC_NULL No operation takes place tag MPI_ANY_TAG Receive messages with any tag status MPI_IGNORE_STATUS Do not store any status

30 Status parameter of MPI_Recv Number of received elements Use the MPI_GET_COUNT function to extract this information MPI_GET_COUNT(STATUS, DATATYPE, COUNT, RC) INTEGER STATUS(MPI_STATUS_SIZE), DATATYPE, COUNT, RC status Return status of receive operation datatype datatype of each receive buffer element count number of received elements rc return value Tag of received message STATUS(MPI_TAG) Rank of the sender STATUS(MPI_SOURCE)

31 Blocking MPI_Send and MPI_Recv routines are blocking Completion depends on other processes Risk for deadlocks the program is stuck forever MPI_Send Exits once the send buffer can be safely read and written to MPI_Recv Exits once it has received the message in the receive buffer

32 Combined send&receive MPI_Sendrecv(sendbuf, sendcount, sendtype, dest, sendtag, recvbuf, recvcount, recvtype, source, recvtag, comm, status ) Parameters as for MPI_Send and MPI_Recv combined Sends one message and receives another one, with one single command Reduces risk for deadlocks Destination rank and source rank can be different, or same Usage examples: Two process exchange data with each other Pipe or ring of processes exchanging data