Integration, differentiation, and root finding Phys 420/580 Lecture 7
Numerical integration Compute an approximation to the definite integral I = b Find area under the curve in the interval Trapezoid Rule: simplest geometric approximation a f(x) dx f(x) [a, b] f(x) f(b)+f(a) I (b a) 1 2 a b x
Numerical integration What is the quality of the approximation? Construct a power series from the left: f(x) =f(a)+f (a)(x a)+ f (a) 2! = f(a)+f (a)(x a)+ f (ξ) 2! (x a) 2 + f (a) 3! (x a) 2 (x a) 3 + exact equality true for some choice of ξ [a, b]
Numerical integration Integrating term by term (with H = b a ) gives I = f(a)h + f (a) 2! H 2 + f (ξ) 3! H 3 I trap = f(a) 2 H + f(b) 2 H error: I I trap = O(H 3 ) = f(a) 2 H + 1 f(a)+f (a)h + f H (ξ) H 2 2 2! = f(a)h + f (a) 2 H2 + f (ξ) H 3 8
Numerical integration How to improve the estimate? 1. Find an approximation that matches to higher order: e.g., trapezoid is the best linear fit; Simpson s rule is the best quadratic fit through three points I Simp = H a + b f(a)+4f 6 2 I I Simp = O(H 5 ) + f(b) 2. Make H small by subdividing the interval
Polynomial fitting Newton-Cotes formulas: H (f(b) f(a)) 2 H a + b f(a)+4f + f(b) 6 2 H a +2b 2a + b f(a)+3f +3f + f(b) 8 3 3 H 3a + b a + b a +3b 7f(a) + 32f + 12f + 32f 90 4 2 4 +7f(b) Polynomial fitting over a uniformly spaced set of points At high order, prone to Runge s phenomenon
Runge s phenomenon order 9 polynomial fit through 10 points!"#$#%&' ()(%'*+,-&.(/.0(1*,0()!-2#3#4(/ f(x) = 2 2+(2x 11) 2
Interpolation error Suppose n +1 ordered points x 0 <x 1 < <x n evaluate to f(x i )=y i Order- n interpolating polynomial satisfying P (x i )=y i is related to the original function by f(x) =P (x)+ f (n+1) (ξ) (n + 1)! n (x x i ) i=0 for some ξ [x 0,x n ]
Interpolation error Overall fitting error controlled by E = xn Formal minimization x i = x 0 dx (x) 2 (x) = gives n (x x i ) i=0 E/ x i =0 xn dx x x 0 j=i (x x j ) 2 xn dx x 0 k=i (x x k ) 2 Chebyshev nodes provide a good approximate solution: x i = 1 2 x 0 + x n (x n x 0 ) cos (i +1/2)π n +1
Integrating piecewise Break the integral into I = = b a x1 a f(x) dx a x2 N disjoint intervals covering b f(x) dx + f(x) dx + + f(x) dx x 1 x N 1 Treat the integrals piecewise using, e.g., trapezoid rule x i [a, b] b global error: H N O N 3 = O 1 N 2 Becomes exact in the limit N
Romberg integration Break the interval [a, b] of width H = b a into subintervals of width h n = H/2 n 2 n By recursive construction, define R 0,0 = H 2 (f(a)+f(b)) R n,0 = 1 2 2 R n 1 n 1,0 + h n j=1 R n,m = 4m R n,m 1 R n 1,m 1 4 m 1 f(a +(2k 1)h n )
Romberg integration R 0,0 R 1,0 R 1,1 R 2,0 R 2,1 R 2,1 R 3,0 R 3,1 R 3,2 R 3,3 2 1 dx e x2 π 0. =0.8427007929497149.... 0.77174333 0.82526296 0.84310283 0.83836778 0.84273605 0.84271160 0.84161922 0.84270304 0.84270083 0.84270066 0.84243051 0.84270093 0.84270079 0.84270079 0.84270079
Troublesome cases There are additional complications if the region of integration is infinite or semi-infinite the integrand is otherwise badly behaved: e.g., 1. it diverges 2. has discontinuities 3. oscillates infinitely often in some finite region
Semi-infinite integral Choose a monotonic increasing function φ :[0, ] [0, 1] 1-1 map between R + and the unit interval E.g., y = φ(x) = x 1+x, x = φ 1 (y) = y 1 y Conventional integration in transformed coordinates: y i [0, 1] 0 dx f(x) = 1 0 dy f(φ 1 (y)) φ 1 (y) x i R +
Nonuniform mesh Many problems can be addressed by the proper choice of mesh the points a, x 1,x 2,...,x N 1,b do not have to be uniformly spaced: Gaussian quadrature dx f(x). = i f(x i )w i Clenshaw-Curtis quadrature choose an adaptive mesh to keep the piecewise areas roughly constant
Adaptive mesh Adaptive mesh near a singularity: f(x) a b x Rule of thumb: (x i+1 x i ) 1 2 f(xi )+f(x i+1 ) const.
Numerical Calculus Most physical phenomena evolve continuously and are described in terms of time rates of change and spatial gradients t, 2 t 2,, 2,... On the computer we have only floating point approximations to real numbers and no proper sense of a continuous function and its derivatives
Numerical Calculus Recall: integration can be implemented as a limiting process of ever smaller discretization Numerical differentiation can be done in similar fashion E.g., breaking the real line into a fine grid x 1,x 2,x 3,... leads to the lowest-order approximation f (x i ) = lim δx 0 f(x i + δx) f(x i ) δx f(x i+1) f(x i ) x i+1 x i
Numerical Calculus We must always consider how the finite-difference scheme scales with the grid spacing h = x i+1 x i Taylor expansion around x i yields f(x i+1 ) f(x i )=f (x i )h + 1 2! f (x i )h 2 + 1 3! f (x i )h 3 + We find that the error scales rather badly f (x i )= f(x i+1) f(x i ) h + O(h) asymmetric
Numerical Calculus Instead, expand to the right and left around x i f(x i+1 )=f(x i )+f (x i )h + 1 2! f (x i )h 2 + f(x i 1 )=f(x i ) f (x i )h + 1 2! f (x i )h 2 + Difference gives the symmetric three-point formula f (x i )= f(x i+1) f(x i 1 ) 2h + O(h 2 )
Numerical Calculus Procedure can be generalized to higher order A five-point formula can be derived by including expansions for f(x i+2 ),f(x i 2 ): f(x i+1 ) f(x i 1 )=2hf (x i )+ 1 3 f (x i )h 3 + O(h 5 ) f(x i+2 ) f(x i 2 )=4hf (x i )+ 8 3 f (x i )h 3 + O(h 5 ) Eliminating the f (x i )= 1 12h f term yields f i 2 8f i 1 +8f i+1 f i+2 + O(h 5 )
Richardson extrapolation Method to automate these order-by-order improvements Numerical derivatives built from function evaluations at x, x + h, x + h/ξ, x+ h/ξ 2,... for ξ > 1 Recursive definition: D (1) (h) = f(x + h) f(x) h D (n+1) (h) = ξn D (n) (h/ξ) D (n) (h) ξ n 1
Numerical Calculus The three-point formula is often good enough Sometimes necessary to take h small rather than to use f(x) a high-order multi-point finite difference formula A common problem is that i 3 i 1 i+1 i+3 high-order formulas are illdefined near boundaries f (x i ) x i 4 i 2 i i+2 i+4
Numerical Calculus Finite difference is a well- i 3 i 2 i 1 i i+1 i+2 i+3 defined shift and subtract operation Derivatives at arbitrary order have weights that are binomial coefficients: n [f(x i )] = n n ( 1) k f(x i+n 2k ) k 1 1 1 1 1 2 1 1 2 1 1 2 1 1 3 3 1 k=0
Root finding How to find the solution(s) of the equation f(x) =0? Choice of method depends on whether analytically f (x) is known If we have knowledge of the derivative, a common and efficient scheme is the Newton-Raphson method
Root finding Suppose there is a root at x and we guess that its position is x n Expansion in terms of the deviation x n = x n x yields f(x )=f(x n ) f (x n ) x n + O [ x n ] 2 =0 View approximate solution as a refined guess, : x x n+1 x n f(x n) f (x n ) x n+1
Newton-Raphson iterations Example of a successful Newton-Raphson search f(x) Each iteration brings us closer to the true root x 1 x 3 x If f(x) is smooth and the initial guess is close to the root, convergence is very fast lim x n = x n x 2
Convergence region f(x) Failure to find the root can result if the initial guess is x 1 x not sufficiently close to the true root Linear extrapolation from the local slope may be a bad x 2 approximation
Convergence region f(x) The region in which initial guesses converge to the x 1 x correct solution can be expanded with a quadratic estimate of the y-axis crossing x 2 x n+1 := x n f(x n) f (x n ) 1 f (x n )f(x n ) 2(f (x n )) 2 1
Common variants Infer the direction from the slope but use a much smaller step size than demanded by Newton-Raphson x n+1 := x n sgn f(xn ) f (x n ) h h f(x n ) f (x n ) Use a randomly generated distribution of step sizes