1 Capter 4: Numerical Metods for Common Matematical Problems Interpolation Problem: Suppose we ave data defined at a discrete set of points (x i, y i ), i = 0, 1,..., N. Often it is useful to ave a smoot function y(x) tat passes troug all tese points so tat we can calculate y at intermediate values of x. Lagrange Polynomial Interpolation Tere is a UNIQUE polynomial of degree N passing troug te N+1 points (x i, y i ), i = 0, 1,..., N. Formula P (x) = (x x 1 )(x x 2 )...(x x N ) (x 0 x 2 )(x 0 x 3 )...(x 0 x N ) y 0 + (x x 0)(x x 2 )...(x x N ) (x 1 x 0 )(x 1 x 2 )...(x 1 x N ) y 1 +... + (x x 0)(x x 1 )...(x x N 1 ) (x N x 0 )(x N x 1 )...(x N x N 1 ) y N Tis is known as te LAGRANGE INTERPOLATING POLYNOMIAL. Note tat te denominators are constants and te numerators are all polynomials of degree N. Ceck P (x 0 ) = (x 0 x 1 )(x 0 x 2 )...(x 0 x N ) (x 0 x 2 )(x 0 x 3 )...(x 0 x N ) y 0 + (x 0 x 0 )(x 0 x 2 )...(x 0 x N ) (x 1 x 0 )(x 1 x 2 )...(x 1 x N ) y 1 Similarly P (x 1 ) = y 1, P (x 2 ) = y 2, etc. +... + (x 0 x 0 )(x 0 x 1 )...(x 0 x N 1 ) (x N x 0 )(x N x 1 )...(x N x N 1 ) y N = 1.y 0 + 0.y 1 +... + 0.y N = y 0 as claimed! Linear and Cubic Interpolation Sometimes N is so large tat te Lagrange polynomials become impractical. include LINEAR and CUBIC interpolation. Linear: For linear interpolation, draw a straigt line between adjacent points (see diagram). In tis case, alternatives Cubic: For cubic interpolation, use Lagrange polynomials to interpolate between surrounding 4 points. i.e. if desired x [x j, x j+1 ] find te Lagrange polynomial interpolating (x j 1, y j 1 ), (x j, y j ), (x j+1, y j+1 ), (x j+2, y j+2 ). Note tat if x [x 0, x 1 ] ten you need to use (x 0, y 0 ), (x 1, y 1 ), (x 2, y 2 ), (x 3, y 3 ) and if x [x N 1, x N ] use (x N 3, y N 3 ), (x N 2, y N 2 ), (x N 1, y N 1 ), (x N, y N ).
2 Lagrange Interpolating Polynomial X X X X X X X 1 2 3 4 5 6 7 X X X X X X X 1 2 3 4 5 6 7 Linear Interpolation (Straigt lines between adjacent points) Cubic Interpolation (Use nearest four neigbours) X X X X X X X 1 2 3 4 5 6 7 Figure: Illustrating different interpolation metods.
3 Root Finding Problem: To solve a nonlinear equation f(x) = 0 (e.g. find te roots of a polynomial, or te solution of cos x x 3 =0). Advantages: Fast, Easy to Implement Newton s Metod Disadvantages: Need to be able to calculate f (x), sometimes goes wild and gives wrong answer 1. Start at a point x 1 opefully near te root at x = r, (f(r) = 0). 2. Calculate f 1 = f(x 1 ) and f 1 = f (x 1 ) 3. Draw tangent line between (x 1, f 1 ) and new point (x 2, 0). Use equation of line in form y 2 y 1 = m (gradient) in our case tis gives or rearranging 4. Repeat to get a sequence x 3, x 4,... using 0 f 1 = f 1, x 2 = x 1 f 1 f 1. x n+1 = x n f n f n 5. Stop wen x n+1 x n < δ for some tolerance δ, e.g. δ = 10 6. Example: Find 3 witout a calculation a = 3, a 2 3 = 0. Tis suggests so Newton s metod gives Start nearby, e.g. x 1 = 2. f(x) = x 2 3, f (x) = 2x x n+1 = x n x2 n 3 2x n or x n+1 = x2 n + 3 2x n. Difficulties: x 2 = 4 + 3 4 = 7 4, x 3 = Consider p(x) = x 3 x, tis as roots at a = 1, 0, 1. Try a starting point x 1 = 1/2 x 2 = x 1 f 1 f 1 = x 1 x3 1 x 1 ( 7 ) 2 4 + 3 7 2 3x 2 1 1 = 1 2 1, x 4 = etc. 8 1 2 3 4 1 = 1
4 f(x) r x 2 x 1 x Figure: Illustrating te Newton-Rapson algoritm. tis means we ave found a root we were not even near!!! Problem arises wenever f (x n ) is small...te tangent is nearly orizontal, and intersects te origin far from te starting point. Secant Metod Advantages: Fast (but sligtly slower tan Newton s metod), no need to calculate f (x). Disadvantages: Sometimes goes wild and gives wrong answer, need to make two initial guesses. Crosses te x-axis at: Te equation is terefore Slope of secant: m = f(x 2) f(x 1 ) f(x 2 ) f(x 1 ) = 0 f(x 2) x 3 x 2 rearranging x 3 = x 2 f(x 2 ) ( ) x n x n 1 x n+1 = x n f(x n ) f(x n ) f(x n 1 ) and tis can be used to generate x 4, x 5, x 6, etc. as before. ( ) f(x 2 ) f(x 1 )
5 Bisection Metod Advantages: Very robust, always finds te root in te interval. Disadvantages: Relatively slow to converge. Need starting points eiter side of root. 1. Start wit points a, b known to be eiter side of te root r. Tis means tat Set n = 1. 2. If not, define f(a)f(b) 0, and if f(a)f(b) = 0, ten root r is at a or b. x n = a + b 2, and compute f(x n). If f(x n ) = 0 we ave found te root. Stop! 3. Ceck sign of f(a)f(x n ): if f(a)f(x n ) < 0 root is between a and x n so set b = x n, n n + 1., if f(a)f(x n ) > 0 root is between x n and b so set a = x n, n n + 1. 4. Ceck a b. If a b > δ (tolerance) return to step 2. Oterwise, te final estimate for te root is x n = a + b 2. How many iterations are needed? Let L 0 = b a be te initial interval size and δ be te tolerance. After N iterations te lengt of te interval is L N = 2 N L 0, as it as been copped in alf N times. Need to stop te iteration wen 2 N L 0 < δ, 2 N > L 0 δ, N > log 2 ( L0 δ ).
6 Numerical Differentation Problem: Consider a dataset (x 1, y 1 ), (x 2, y 2 ),..., (x N, y N ) were te x i may not be evenly spaced. Te aim is to find good approximate expressions for te first and second derivatives. Tat is, we assume tat tere exists a smoot function f(x) so tat y i = f(x i ), and look for approximations to df/dx and d 2 f/dx 2. First Derivatives Starting wit te definition Formulae for Numerical Derivatives df (x) = lim dx 0 f(x + ) f(x), it is straigtforward to make two approximations for df/dx(x i ). f + (x i) = f(x i+1) f(x i ) x i+1 x i and f (x i) = f(x i) f(x i 1 ) x i x i 1. Notice tat f + is a rigt-sided derivative and f is left-sided. A better approximation is given by te average of f + and f (centred derivative) Second Derivatives f (2) (x i) = 1 2 ( f + + f ). Te rigt-sided derivative f + (x i) equals te exact derivative of f at some point [x i, x i+1 ]. Can guess tat tis is te midpoint (x i+1 + x i )/2, i.e. f + (x i) df ( x = x ) i+1 + x i dx 2 Similarly we guess From tese we can write Uniform Grid f (x i ) df ( x = x ) i + x i 1 dx 2 d 2 f dx 2 (x i) f (2) = f +(x i ) f (x i ) x i+x i+1 2 xi+xi 1 2 Often we ave a uniform grid in x, i.e. x i = x 0 + i. Wat do our forumlae above become? and f + (x i) = f(x i+1) f(x i ), f +(x i ) = f(x i) f(x i 1 ), f (2) (x i) = f(x i+1) f(x i 1 ), 2 f (2) (x i) = f +(x i ) f (x i ) = f(x i+1) 2f(x i ) + f(x i 1 ) 2.
7 Slope of Actual derivative f (x i ) Slope of Numerical Derivative f + Slope of Numerical Derivative f = f( x )- f( x ) - i i-1 x x i - i-1 x x x x i-2 i-1 i i+1 x i+2 x i+3 Figure: Illustrating left f = (f(x i) f(x i 1))/(x i x i 1) and rigt f + = (f(x i+1) f(x i))/(x i+1 x i) numerical derivatives. How accurate are te above approximations? Consider te simple function f(x) = x N. We ave f (x) = Nx N 1, f (x) = N(N 1)x ( N 2). Can compare wit numerical derivatives. Take x 0 = 0, x 0 = 1, x 0 = 2. Ten f +(x 1 ) = f(x 2) f(x 1 ) f (x 1) = f(x 1) f(x 0 ) x 1 x 0 = 2N 1 N 1 = 1N 0 N 1 = 2 N 1. = 1. wereas te exact value is f (x 1 ) = N. Tis is equal to f +(x 1 ) (and f (x 1 )) only for N = 1. Rigt-sided (f + ) and left-sided (f ) numerical derivatives are exact only for first order polynomials. Try centred derivatives Tis equals f (x 1 ) for bot N = 1 and 2. f (2) (x 1) = f(x 2) f(x 0 ) = 2N 0 N 2 = 2 N 1. Centred (f(2) ) numerical derivatives are exact for second order polynomials. Comment: Can prove te above statements more generally by taking nx m 1 = x m, x m, and x m+1 = x m +.
8 Ordinary Differential Equations Problem: Te aim is to generate a numerical solution for te INITIAL VALUE PROBLEM consisting of te ordinary differential equation ( ) and te initial condition ( ) dx = f(y, x) ( ), y(x 0) = Y ( ). Te numerical solution will be defined only at a discrete set of points (x i, y i ), wit y i y(x i ) (were y(x) is te true solution). Note tat interpolation may be used to estimate te solution at an intermediate point. Reminder: It is straigtforward to find analytic solutions of (*) wen f(y, x) is separable (f(y, x) = f 1 (x)f 2 (y), use SEPARATION) or f(y, x) is linear in y, (f(y, x) = F 1 (x)y + F 2 (x), use INTEGRATING FACTOR). Numerical solutions are often needed wen f(y, x) is nonlinear and non-separable. Example 1: separation dx = x2 y 3 y(1) = 1. y 3 = x 2 dx 1 2y 2 = 1 3 x3 + c y(1) = 1, so c = 5 6 3 y = 5 2x 3 Example 2: integrating factor multiply equation by q(x) use definition of integrating factor gives integrating dx = yx + x3 y(0) = 1. q dx yqx = qx3. dq dx = qx, solving q(x) = exp x2 /2. q dx + dq dx y = d dx (yq) = d ( ) ye x2 /2 = x 3 e x2 /2. dx ye x2 /2 = (2 + x 2 )e x2 /2 + c y(0) = 1, so c = 2. y = 2 x 2 + 2e x2 /2. Metod 1: Euler s Metod Aim is to get a solution on a regular grid x i = x 0 + i ( is defined to be te stepsize). Simplest metod is to use te RIGHT-SIDED numerical derivative defined above. Tat is, we replace ( ) wit y i+1 y i = f(y i, x i ). rearranging we get y i+1 = y i + f(y i, x i ) wit y 0 = Y. Euler s Metod Euler s metod is used to generate a sequence {y i } tat gives te numerical solution of te equation at eac point x i (see diagram).
9 y(t) Numerical Solution t0 t1 t2 Figure: Illustrating te actual and numerical solutions of an ordinary differential equation. Metod 2: Midpoint Metod, or Second-order Runge-Kutta Metod (RK2) As CENTRED numerical derivatives are more accurate tan rigt-sided ones, tese may be used to derive a more accurate metod for solving te initial value problem ( ) + ( ), i.e. y i+1 y i = f(y i+1/2, x i+1/2 ). Problem ere is tat we don t know te numerical solution at y i+1/2 y(x i+1/2 ) (note tat x i+1/2 = x i + /2). However, we can estimate tis using Euler s metod by setting so explicitly, we ave, y i+1/2 = y i + 2 f(y i, x i ) y i+1 = y i + f(y i + 2 f(y i, x i ), x i + /2) y 0 = Y Midpoint (RK2) Metod Tis formula can be used to generate a sequence {y i }, tat can be sown to ave iger accuracy in te limit of small compared wit Euler s metod. Systems of ODEs Note tat two coupled ordinary differential equations (+initial conditions) dx dz dx = f(y, z, x), y(x 0 ) = Y = g(y, z, x), z(x 0 ) = Z
10 can be solved togeter straigforwardly (e.g. using Euler s metod) y i+1 = y i + f(y i, z i, x i ), y 0 = Y z i+1 = z i + g(y i, z i, x i ), z 0 = Z. Tese generate sequences {y i } and {z i }, te numerical solutions of te equations. Te same principle can be extended to a system of N equations. Higer order ODEs Higer order differential equations may be solved by tranforming tem into systems of first order equations like tose above. We will illustrate tis by considering te general second order equation (wit 2 initial conditions) d 2 y = f( dx2 dx, y, x) y(x 0) = Y, dx (x 0) = Z. To solve tis equation we set te auxillary variable z = /dx, and replace te above problem wit two equivalent first order equations in y and z. dx dz dx = z y(x 0 ) = Y = f(z, y, x) z(x 0 ) = Z. Tis is an example of a system of first order ODEs like tose described above and can be solved in te usual way, e.g. using Euler s metod y i+1 = y i + z i, y 0 = Y, z i+1 = z i + f(y i, z i, x i ), z 0 = Z. Tese generate sequences {y i } - te numerical solution of te equation (y i z i /dx (x i ). y(x i )) and {z i }, were