SKMM 3023 Applied Numerical Methods

Transcription

1 UNIVERSITI TEKNOLOGI MALAYSIA SKMM 3023 Applied Numerical Methods Numerical Differentiation ibn Abdullah Faculty of Mechanical Engineering Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 1 / 57

2 Outline 1 Introduction 2 Engineering Applications 3 Finite Difference Approximation Derivative Defined Taylor Series & Difference Formulae Centred Difference Formulae Richardson Extrapolation 4 Finite Difference in PDE 5 Curve-Fitting & Interpolation Collocation-Polynomials Fit Lagrange Polynomials Least Square Curve Fitting Divided Difference Polynomials 6 Bibliography Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 2 / 57

3 Introduction Why/When? 1 Finite Difference Approximation complex differential equations in many engineering applications, e.g. wave equation 1 2 u c 2 t = 2 u 2 x + 2 u 2 y + 2 u 2 z = 2 2 u may require numerical solutions due to presence of complex geometry, complicated boundary conditions, nonlinearity, large number of simultaneous differential equations, etc. 2 Polynomial Interpolation differentiation of discrete data, e.g. from experimental results tabulated below, requires an approximate numerical procedure. x y x y x y Exact Differentiation of known functions through differential calculus, e.g. distance traversed at a constant acceleration s = ut at2 where ds dt = v and d 2 s dt = dv 2 dt = a Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 3 / 57

4 Introduction Finite Difference Approximation Differential equations in many engineering applications require numerical solutions due to presence of complex geometry and/or complicated boundary conditions nonlinearity and/or large number of simultaneous differential equations, etc. as in 1 2 u c 2 t = 2 u 2 x + 2 u 2 y + 2 u 2 z = 2 2 u Derivatives in these differential equations are replaced by their discrete forms, known as finite-difference approximations. Use of finite-differences transforms an ordinary differential equation into an algebraic equation Taylor series used in computing derivatives using discrete methods is cumbersome for approximations of increasingly high accuracy or higher-order derivatives Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 4 / 57

5 Introduction Polynomial Interpolation If a function is very complicated or known only from values in a table (e.g. from experiments), it may be necessary to resort to numerical differentiation. In cases especially where data are obtained experimentally it is best to perform a least squares curve fit to data and then find the resulting polynomial. Formulae for numerical differentiation may easily be obtained by differentiating interpolation polynomials. The essential idea is that the derivatives f, f,... of a function are represented by the derivatives P n, P n,... of the interpolating polynomial P n more on this towards the end! Figure 1: Interpolating polynomial. Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 5 / 57

6 Introduction Errors in Numerical Differentiation Numerical differentiation is avoided wherever it is possible because of several inherent difficulties: Integration describes an overall property of a function, whereas differentiation describes the slope of a function at a point. Integration is not sensitive to minor changes in the shape of a function, whereas differentiation is. Any small changes in a function can easily create large changes in its slope in the neighborhood of the change. Compared to integration, numerical differentiation is less reliable. Large errors may also occur in numerical differentiation based on Taylor series, which is cumbersome for approximations of high accuracy or higher-order derivatives. Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 6 / 57

7 Engineering Applications Example 1 Problem Statement: The barrel and the recoil mechanism of a cannon, Figure 2, have a mass of 500 kg with a recoil spring stiffness of N/m. The response of the cannon, with a critically damped recoil mechanism, is given by x(t) = (c 1 + c 2t) ωnt (1) where c 1 = x 0, c 2 = ẋ 0 + ω nx 0, x 0 is the initial displacement, ẋ 0 is the initial velocity, and ω n = p k/m is the undamped natural frequency of the system. Figure 2: Recoil mechanism of a cannon. Find the recoil velocity of the cannon using numerical differentiation with x 0 = 0 and ẋ 0 = 5 m/s. Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 7 / 57

8 Engineering Applications Example 2 Figure 3: Tilt thrust bearing. Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 8 / 57

9 Engineering Applications Example 2 Problem Statement: The load carrying capacity of a tilt thrust bearing (P), shown in Figure 3, is given by j» ff 6µ(U1 U2)lb2 1 2(m 1) P = ln m (2) h 2 2 (m 1) 2 m + 1 where U 1 = surface velocity of the supporting plate, U 2 = surface velocity of the pad, m = h 1 /h 2 = tilt ratio, b = length of the lubricant flow path, h 1 = film thickness at entrance, h 2 = film thickness at exit, µ = viscosity of the lubricant, l = length of the bearing. The change in the load due to a small change in the tilt ratio, given by dp/dm, will be of interest from a design point of view. Find the value of dp/dm for the following data: b = 3.0 in h 1 = 0.10 in P = 5000 lb U 1 U 2 = 150 in/sec l = 1.5 in h 2 = 0.05 in µ = reyn Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 9 / 57

10 Engineering Applications Example 3 Problem Statement: The pressure (p) specific volume (v) relationship of superheated water vapour at 350 C is given by van der Waals equation where p = RT v b a v 2 (3) T = absolute temperature = K a = R = specific gas constant = kj/kg-k b = Expand the pressure in Taylor s series and estimate the value of p at v = 0.051, 0.054, and assuming that the value of p and its derivatives are known at v = Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 10 / 57

11 Finite Difference Approximation Derivative Defined Derivative of a function f(x) is defined as df(x) = f f(x + h) f(x) (x) = lim dx h 0 h x (4) where h is the step size, or a small increment x, and does not approach zero but remains a finite quantity. As h = x, Eq. (4) can also be written as f (x) f(x + x) f(x) x (5) Figure 4: Derivative defined. Given a function f(x), we wish to approximate its derivatives at a point x using the Taylor Series. Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 11 / 57

12 Finite Difference Approximation Taylor Series Expansions For positive step sizes +h and +2h, forward Taylor series expansion gives f(x + h) = f(x) + h 1! f (x) + h2 2! f (x) + h3 3! f (x) + h4 4! f (x) +... (6) f(x + 2h) = f(x) + (2h) 1! f (x) + (2h)2 2! f (x) + (2h)3 3! f (x) + (2h)4 f (x) +... (7) 4! For negative step sizes h and 2h, backward Taylor series expansion gives f(x h) = f(x) + ( h) 1! f(x 2h) = f(x) + ( 2h) 1! f (x) + ( h2 ) 2! f (x) + ( 2h)2 2! f (x) + ( h3 ) 3! f (x) + ( 2h)3 3! f (x) + ( h4 ) f (x) +... (8) 4! f (x) + ( 2h)4 f (x) ! (9) Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 12 / 57

13 Finite Difference Approximation Forward and backward FD of First Derivative to O(h) Accuracy Solving Eq. (6) for (df(x)/dx) i,j df(x) dx = f (x) = = f(x + h) f(x) f f (h) {z h } 2! 3! (h)2... {z } finite difference truncation error, O(h) f(x + h) f(x) h + O(h) For h = x and ignoring O(h), we get a first-order accurate (i.e. O(h)) forward difference formula of the first derivative as f (x) f(x + h) f(x) h (10) Figure 5: Forward difference formulation. Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 13 / 57

14 Finite Difference Approximation Forward and backward FD of First Derivative to O(h) Accuracy Solving Eq. (8) for (df(x)/dx) i,j df(x) dx = f (x) = = f(x h) f(x) f f (h) {z h } 2! 3! (h)2... {z } finite difference truncation error, O(h) f(x h) f(x) h + O(h) For ( h) = ( x) and ignoring O(h), we get a first-order accurate (i.e. O(h)) backward difference formula of the first derivative as f (x) f(x) f(x h) h (11) Figure 6: Backward difference formulation. Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 14 / 57

15 Finite Difference Approximation Forward and backward FD of First Derivative to O(h 2 ) Accuracy The common practice is to use forward and backward expressions of O(h 2 ). We can derive them by retaining more terms in the Taylor series. Multiply Eq. (6) by 4 and subtract it from Eq. (7) to eliminate f (x) or f(x + 2h) 4 f(x + h) = 3f(x) 2h f (x) + 2h3 3 f (x) +... (12) f (x) = f(x + 2h) + 4f(x + h) 3f(x) 2h + O(h 2 ) (13) Eq. (13) is the second-order accurate forward difference formula for f (x). The second-order accurate backward difference formula for f (x) may be obtained through similar process by expanding f(x h) and f(x 2h) using the Taylor series see Eqs.(8) and (9) resulting in f (x) = f(x 2h) 4f(x h) + 3f(x) 2h + O(h 2 ) (14) Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 15 / 57

16 Finite Difference Approximation Centred FD of First Derivative to O(h 2 ) Accuracy If we substract Eq. (8) from Eq. (6) we get the second-order accurate centred difference formula for f (x) f f(x + h) f(x h) (x) = f (x) h h 3! f(x + h) f(x h) 2h (15) Figure 7: Central difference formulation. Adding Eq. (8) to Eq. (6) leads to the second-order accurate centred difference formula for f (x) Note:Accuracy of Eqs. (10) (11) and (15) can be improved by retaining the second-derivative term, f (x), and substituting it with Eq. (16). f f(x + h) 2f(x) + f(x h) (x) = h 2 f(x + h) 2f(x) + f(x h) h 2 (16) f iv (x) h ! Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 16 / 57

17 Finite Difference Approximation Centred FD of Higher Derivatives to O(h 2 ) and O(h 4 ) Accuracy Centred Difference Formulae of Order O(h 2 ) f f(x + h) f(x h) (x) (17) 2h f f(x + h) 2f(x) + f(x h) (x) h 2 (18) f (x) f (4) (x) f(x + 2h) f(x + h) + 2f(x h) f(x 2h) 2h 3 (19) f(x + 2h) 4f(x + h) + 6f(x) 4f(x h) + f(x 2h) h 4 (20) Centred Difference Formulae of Order O(h 4 ) f f(x + 2h) + 8f(x + h) 8f(x h) + f(x 2h) (x) (21) 12h f f(x + 2h) + 16f(x + h) 30f(x) + 16f(x h) f(x 2h) (x) 12h 2 (22) f (x) f (4) (x) f(x + 3h) + 8f(x + 2h) 13f(x + h) + 13f(x h) 8f(x 2h) + f(x 3h) 8h 3 (23) f(x + 3h) + 12f(x + 2h) 39f(x + h) + 56f(x) 39f(x h) + 12f(x 2h) f(x 3h) 6h 4 (24) Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 17 / 57

18 Finite Difference Approximation Example 1 Effect of step size h Problem Statement: Let f(x) = sin(x), where x is measured in radians. Calculate approximations to f (0.8) using the centred difference formula of order O(h 2 ) with x = h = 0.1, x = h = 0.01, x = h = Carry eight or nine decimal places. Compare with the value f (0.8) = cos(0.8). Solution: Work through the example. Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 18 / 57

19 Finite Difference Approximation Example 2 Effect of O(h 2 ) and O(h 4 ) Problem Statement: Compare the numerical differentiation by centred difference formulae of order O(h 2 ) and order O(h 4 ). Let f(x) = x 3 and find approximations for f (2). Use order O(h 2 ) with x = h = 0.05 Use order O(h 4 ) with x = h = 0.05 Solution: Work through the example. Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 19 / 57

20 Finite Difference Approximation Centred Difference Formulae Programming Example Problem Statement: Take a look at ܽ ½º to study sample computation of the derivative of a function using forward difference formula Take a look at ܾ ½º to study sample computation of the derivative of a function using central difference formula Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 20 / 57

21 Richardson Extrapolation Two approaches normally used to improve derivative estimates decrease step size h use higher-order formula that employs more points There is also a third approach Richardson extrapolation a method often used to improve the results of a numerical method, from a method of order O(h n ) it gives us a method of order O(h n+1 ). Richardson extrapolation uses two derivatives to compute a third, more accurate approximation. Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 21 / 57

22 Richardson Extrapolation In integration, Richardson extrapolation provides a means to obtain an improved integral estimate I by formula I I(h 2) + h1 1 «2 2) I(h 1)] (25) 1[I(h h 2 where I(h 1) and I(h 2) are integral estimates using two step sizes h 1 and h 2. Eq. (25) is usually written for the case where h 2 = 1 h1 to give 2 I 4 3 I(h2) 1 3 I(h1) (26) Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 22 / 57

23 Richardson Extrapolation In similar fashion, Eq. (26) can be written for derivatives, D, as D 4 3 D(h2) 1 3 D(h1) (27) for centred-difference approximations with O(h 2 ). The application of this formula will yield a new derivative estimate of O(h 4 ) Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 23 / 57

24 Richardson Extrapolation Example 3 Problem Statement: Using the function f(x) = 0.1x x 3 0.5x x estimate the first derivative at x = 0.5 employing step sizes h 1 = 0.5 and h 2 = Compute an improved estimate with Richardson extrapolation. As a comparison, the true value is Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 24 / 57

25 Finite Difference Approximation Finite Difference in PDE So far we dealt with total or ordinary derivatives where function f depends on a single independent variable x. In many engineering problems, function f depends on two or more independent variables. We use notation (i, j) to designate the pivot point if there are two independent variables, (i, j, k) for three see Figure 8 and so on as the respective counters in x, y and z directions. Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 25 / 57

26 Finite Difference Approximation Finite Difference in PDE (a) (b) Figure 8: Two- and three-dimensional grid systems. Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 26 / 57

27 Finite Difference Approximation Finite Difference in PDE Consider f(x, y). Finite-difference approximation for the partial derivative f(x, y) x at (x = x i, y = y j ) can be found by fixing value of y at y j and treat f(x, y j ) as a one-variable function. Forward-difference approximation of f/ x is f f(x i + x, y j ) f(x i, y j ) (28) x i,j x Backward-difference approximation of f/ x is f f(x i, y j ) f(x i x, y j ) (29) x i,j x Central-difference approximation of f/ x is f f(x i + x, y j ) f(x i x, y j ) x 2 x i,j (30) Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 27 / 57

28 Finite Difference Approximation Finite Difference in PDE Similarly, finite-difference approximation for the partial derivative f(x, y) y at (x = x i, y = y j ) can be found by fixing value of x at x i and treat f(x i, y) as a one-variable function. Forward-difference approximation of f/ y is f f(x i, y j + y) f(x i, y j ) (31) y i,j y Backward-difference approximation of f/ y is f f(x i, y j ) f(x i, y j y) (32) y i,j y Central-difference approximation of f/ x is f f(x i, y j + y) f(x i, y j y) y 2 y i,j (33) Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 28 / 57

29 Finite Difference Approximation Finite Difference in PDE Central-difference approximation of 2 f/ x 2 2 f f(x i + x, y j ) 2f(x i, y j ) + f(x i x, y j ) (34) x 2 i,j ( x) 2 Central-difference approximation of 2 f/ y 2 2 f f(x i, y j + y) 2f(x i, y j ) + f(x i, y j y) (35) y 2 i,j ( y) 2 Central-difference approximation of 2 f/ x y 2 f `f(xi + x, y j + y) f(x i + x, y j y) f(x i x, y j + y) + f(x i x, y j y) x y i,j (4 x y) (36) Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 29 / 57

30 Curve-Fitting & Interpolation The function f(x), which is to be differentiated, may be a known function, or a set of discrete data. In general, known functions can be differentiated exactly. Differentiation of discrete data, however, requires an approximate numerical procedure. To perform numerical differentiation, an approximating function (normally a polynomial) curve is fit to the discrete data, or a subset of the discrete data, and the approximating function is differentiated. The best-fitting curve can be obtained through various methods and we shall look at approximating function obtained through collocation-polynomials fit, Lagrange interpolation formula, least-squares regression, and divided difference fit which can be applied to both unequally spaced data and equally spaced data. Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 30 / 57

31 Curve-Fitting & Interpolation Collocation-Polynomials Fit Also known as the method of direct fit polynomials. Based on fitting the data directly by a polynomial, which is given by P n(x) = a 0 + a 1x + a 2x a nx n (37) where P n(x) is determined by one of the following methods: Given N = n + 1 points, [x i, f(x i )], determine the exact nth-degree polynomial that passes through the data points. Given N > n + 1 points, [x i, f(x i )], determine the least squares nth-degree polynomial that best fits the data points a much detailed coverage on this later! Derivatives obtained by differentiating the approximating polynomial f P n = a 1 + 2a 2x + 3a 3x f P n = 2a 2 + 5a 3x +... (38a) (38b) Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 31 / 57

32 Curve-Fitting & Interpolation Lagrange Polynomials Can be used for both unequally spaced data and equally spaced data. It is based on fitting second degree Lagrange polynomial, Eq. (39), to a (sub)set of three discrete data pairs, e.g. (a, f(a)), (b, f(b)), and (c, f(c)), such that P 2(x) = (x b)(x c) (a b)(a c) Differentiating Eq. (39) yields f P 2(x) = f(a) + (x a)(x c) (b a)(b c) (x a)(x b) f(b) + f(c) (39) (c a)(c b) 2x (b + c) 2x (a + c) 2x (a + b) f(a) + f(b) + (a b)(a c) (b a)(b c) (c a)(c b) f(c) (40a) f P 2f(a) 2 (x) = (a b)(a c) + 2f(b) (b a)(b c) + 2f(c) (c a)(c b) (40b) Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 32 / 57

33 Curve-Fitting & Interpolation Least Square Curve Fitting The methods of least squares assumes that the best-fit curve of a given type is the curve that has the minimal sum of the deviations squared (least square error) from a given set of data. Suppose that the data points are (x 1, y 1), (x 2, y 2),..., (x n, y n) where x is the independent variable and y is the dependent variable. The fitting curve f(x) has the deviation (error) d from each data point, i.e., d 1 = y 1 f(x 1), d 2 = y 2 f(x 2),... d n = y n f(x n). According to the method of least squares, the best fitting curve has the property that: Π = d d d 2 n = d 2 i = [y i f(x i )] 2 = a minimum Polynomials are one of the most commonly used types of curves in curve fitting. Methods of least squares curve fitting using polynomials include: least-squares line (i.e. 1 st degree polynomial) method, least-squares parabola (i.e. 2 nd degree polynomial) method, least-squares m th degree polynomials method multiple regression least-squares Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 33 / 57

34 Curve-Fitting & Interpolation Least Square Curve Fitting Least-squares Line Method To approximate the given set of data, (x 1, y 1), (x 2, y 2),..., (x n, y n) where n 2, the least-squares line uses a straight line y = a + bx The best fitting curve f(x) has the least square error, i.e., Π = = [ y i f(x i )] 2 [ y i (a + bx i )] 2 = a minimum where a and b are unknown coefficients while all x i and y i are given. Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 34 / 57

35 Curve-Fitting & Interpolation Least Square Curve Fitting Least-squares Line Method To obtain the least square error, the unknown coefficients a and b must yield zero first derivatives Π a = 2 xi 0 [ y i (a + bx i )] = 0 Π b = 2 xi 1 [ y i (a + bx i )] = 0 Expanding the above equations, we have: xi 0 y i = a xi 1 y i = a xi 0 + b xi 1 + b x 1 i x 2 i The unknown coefficients a, and b can hence be obtained by solving the above system of linear equations. Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 35 / 57

36 Curve-Fitting & Interpolation Least Square Curve Fitting Least-squares Parabola Method To approximate the given set of data, (x 1, y 1), (x 2, y 2),..., (x n, y n) where n 3, the least-squares line uses a second order polynomial y = a + bx + cx 2 The best fitting curve f(x) has the least square error, i.e., Π = = [ y i f(x i )] 2 [ y i (a + bx i + cxi 2 )] 2 = a minimum where a, b and c are unknown coefficients while all x i and y i are given. Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 36 / 57

37 Curve-Fitting & Interpolation Least Square Curve Fitting Least-squares Parabola Method To obtain the least square error, the unknown coefficients a, b and c must yield zero first derivatives Π a = 2 xi 0 [ y i (a + bx i + cxi 2 )] = 0 Π b = 2 xi 1 [ y i (a + bx i + cxi 2 )] = 0 Π c = 2 xi 2 [ y i (a + bx i + cxi 2 )] = 0 Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 37 / 57

38 Curve-Fitting & Interpolation Least Square Curve Fitting Least-squares Parabola Method Expanding the above equations, we have: xi 0 y i = a xi 1 y i = a xi 2 y i = a xi 0 + b xi 1 + b xi 2 + b xi 1 + c xi 2 + c xi 3 + c The unknown coefficients a, b, and c can be obtained by solving the above system linear equations. x 2 i x 3 i x 4 i Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 38 / 57

39 Curve-Fitting & Interpolation Least Square Curve Fitting Least-squares m th Polynomial Method To approximate the given set of data, (x 1, y 1), (x 2, y 2),..., (x n, y n) where n m + 1, the least-squares line uses an m th degree polynomial y = a 0 + a 1x + a 2x a mx m The best fitting curve f(x) has the least square error, i.e., Π = = [ y i f(x i )] 2 [ y i (a 0 + a 1x i + a 2xi a mxi m )] 2 = a minimum where a 0, a 1, a 2... and a m are unknown coefficients while all x i and y i are given. Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 39 / 57

40 Curve-Fitting & Interpolation Least Square Curve Fitting Least-squares m th Polynomial Method To obtain the least square error, the unknown coefficients a 0, a 1, a 2... and a m must yield zero first derivatives Π a = 2 xi 0 [ y i (a 0 + a 1x i + a 2xi a mxi m )] = 0 Π b = 2 xi 1 [ y i (a 0 + a 1x i + a 2xi a mxi m )] = 0 Π b = 2 xi 2 [ y i (a 0 + a 1x i + a 2xi a mxi m )] = Π b = 2 xi m [ y i (a 0 + a 1x i + a 2xi a mxi m )] = 0 Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 40 / 57

41 Curve-Fitting & Interpolation Least Square Curve Fitting Least-squares m th Polynomial Method Expanding the above equations, we have: xi 0 y i = a 0 xi 1 y i = a 0 xi 2 y i = a xi m y i = a 0 xi 0 + a 1 xi 1 + a 1 xi 2 + a 1 xi 1 + a 2 xi 2 + a 2 xi 3 + a 2 xi a m xi a m x 4 i a m xi m + a 1 x m+1 i + a 2 x m i x m+2 i a m x m+1 i x m+2 i x m+m i The unknown coefficients a 0, a 1, a 2,..., and a m can be obtained by solving the above system of linear equations. Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 41 / 57

42 Curve-Fitting & Interpolation Least Square Curve Fitting Multiple Regression This method estimates the outcomes (dependent variables) which may be affected by more than one control parameter (independent variables) or there may be more than one control parameter being changed at the same time. To approximate the given set of data of two independent variables x and y and one dependent variable z, e.g. (x 1, y 1, z 1), (x 2, y 2, z 2),..., (x n, y n, z n), where n 3, in the linear relationship case: z = a + bx + cy the best fitting curve f(x) has the least square error, i.e., Π = = [ z i f(x i, y i )] 2 [ z i (a + bx i + cy i )] 2 = a minimum where a, b and c are unknown coefficients while all x i and y i are given. Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 42 / 57

43 Curve-Fitting & Interpolation Least Square Curve Fitting Multiple Regression To obtain the least square error, the unknown coefficients a, b and c must yield zero first derivatives Π a = 2 xi 0 [ z i (a + bx i + cy i )] = 0 Π b = 2 xi 1 [ z i (a + bx i + cy i )] = 0 Π c = 2 yi 1 [ z i (a + bx i + cy i )] = 0 Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 43 / 57

44 Curve-Fitting & Interpolation Least Square Curve Fitting Multiple Regression Expanding the above equations, we have: xi 0 z i = a xi 1 z i = a yi 1 z i = a xi 0 + b xi 1 + c xi 1 + b xi 2 + c yi 1 + b x i y i + c y 1 i x i y i The unknown coefficients a, b, and c can be obtained by solving the above system linear equations. y 2 i Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 44 / 57

45 Polynomial Interpolation Divided Difference Polynomials Can be used for both unequally spaced data and equally spaced data. Given the points (x 0, y 0), (x 1, y 1), (x 2, y 2), (x 3, y 3), we construct a divided difference table: x i y i = f (0) i x 0 y 0 = f (0) 0 x 1 y 1 = f (0) 1 x 2 y 2 = f (0) 2 x 3 y 3 = f (0) 3 f (1) 0 f (1) 1 f (1) 2 f (1) i f (2) i f (3) i = f(0) f (0) 1 0 x 1 x 0 f (2) = f(1) f (1) x 2 x 0 = f(0) f (0) 2 1 f (3) = f(2) f (2) 1 0 x 2 x 1 0 x 3 x 0 f (2) = f(1) f (1) x 3 x 1 = f(0) f (0) 3 2 x 3 x 2 from which we derive the divided difference polynomial representing these points: P n(x) = f (0) i + (x x 0) f (1) i + (x x 0)(x x 1) f (2) i + (x x 0)(x x 1)(x x 2) f (3) i (41) Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 45 / 57

46 Polynomial Interpolation Divided Difference Polynomials Differentiating Eq. (41) yields f P n(x) = f (1) i + [2x (x 0 + x 1)] f (2) i + [3x 2 2(x 0 + x 1 + x 2) x + (x 0 x 1 + x 0 x 2 + x 1 x 2)] f (3) i (42a) f P n (x) = 2f (2) i + [6x 2(x 0 + x 1 + x 2)] f (3) (42b) Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 46 / 57

47 Curve-Fitting & Interpolation Example 1 Problem Statement: Evaluate the derivatives by numerical differentiation formulae developed by fitting direct fit polynomial, Lagrange polynomial, and divided difference polynomial to the following set of discrete data: x y The exact derivatives at x = 3.5 are f (3.5) = and f (3.5) = Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 47 / 57

48 Curve-Fitting & Interpolation Example 1 Solution: Fit the quadratic polynomial, P 2(x) = a 0 + a 1x + a 2x 2 2, to the three data points: = a 0 + a 1(3.4) + a 2(3.4) 2 (43a) = a 0 + a 1(3.5) + a 2(3.5) 2 (43b) = a 0 + a 1(3.6) + a 2(3.6) 2 (43c) Solving for a 0, a 1, and a 2 by Gauss elimination gives a 0 = , a 1 = , and a 2 = Substituting these values into Eqs. (5.7a) and (5.7b) and evaluating at x = 3.5 yields the solution for direct fit polynomial: P 2 (3.5) = ( )(3.5) = (43d) P 2 (3.5) = (43e) Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 48 / 57

49 Curve-Fitting & Interpolation Example 1 Solution: continued... Substituting the tabular values into Eqs. (40a) and (40b) and evaluating at x = 3.5 yields the solution for the Lagrange polynomial P 2(3.5) ( ) 2(3.5) ( ) 2 (3.5) = ( ) + ( )( ) ( )( ) ( ) 2(3.5) ( ) + ( ) = (44a) ( )( ) P 2 (3.5) = 2( ) ( )( ) + 2( ) ( )( ) 2( ) + = (44b) ( )( ) Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 49 / 57

50 Curve-Fitting & Interpolation Example 1 Solution: continued... Construct a divided difference table for the tabular data to use the divided difference polynomial: x i f (0) f (1) f (2) i i i Substituting these values into Eqs. (42a) and (42b) yields the solution for the divided difference polynomial: P 2 (3.5) = [2(3.5) ( )]( ) = (45a) P 2 (3.5) = 2( ) = (45b) Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 50 / 57

51 Curve-Fitting & Interpolation Example 1 Solution: continued... The results obtained by the three procedures are identical since the same three points are used in all three procedures. The error in f (3.5) is Error = f (3.5) P 2 (3.5) = ( ) = The error in f (3.5) is Error = f (3.5) P 2 (3.5) = = Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 51 / 57

52 Curve-Fitting & Interpolation Example 2: Using Matlab Problem Statement: An experiment yields the following tabulated x-y pairs: x y x (cont... ) y (cont... ) Perform a curve fit to the data and find the approximating polynomial. Determine the derivative at x = 0.5. Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 52 / 57

53 Curve-Fitting & Interpolation Example 2: Using Matlab Solution: To find a polynomial that fits at a set of data we use the command ÔÓÐÝ Ø Ü Ý Òµ, where Ü is a vector containing x-axis values, Ý is a vector containing y-axis values and Ò is the order of the polynomial that we want to fit. Matlab Session Ü ¼º¼¼¼ ¼º½¼¼ ¼º¾¼ ¼º ¼ ¼º ¼ ¼º ¼ ¼º ¼ ¼º ¼ ¼º ¼ ¼º ¼ ½º¼¼ Ý ¹¼º ½º º¾ º½ º¼ º º º º º ¼ ½½º¾ Ò ¾ È ÔÓÐÝ Ø Ü Ý Òµ Next we differentiate the polynomial Matlab Session È ÔÓÐÝ Ö Èµ Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 53 / 57

54 Curve-Fitting & Interpolation Example 2: Using Matlab Solution: continued... and compute the slope at x = 0.5 Matlab Session ÐÓÔ Ó ÔÓÐÝ ÔÓÐÝÚ Ð È ¼º µ We could easily plot the approximating polynomial over the original x-y points to see effect of the fitting on the approximation of derivatives Matlab Session Ü ¼º¼ ¼º¼½ ½º¼ Ý ÔÓÐÝÚ Ð È Ü µ ÔÐÓØ Ü Ý ³Ó³ Ü Ý ³Ö¹³µ Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 54 / 57

55 Curve-Fitting & Interpolation Example 3: Using Matlab Problem Statement: An experiment yields unequally spaced data in x and y x y x y x y Determine the differences between adjacent elements of both x and y vectors and compute divided-difference approximations of the derivative. Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 55 / 57

56 Curve-Fitting & Interpolation Example 3: Using Matlab Solution: To differentiate unequally spaced data in x and y we use the command ܵ and ݵ, where Ü is a vector containing x values, Ý is a vector containing y values Matlab Session ܵ ݵ To compute divided-difference approximations of the derivative we perform vector division of y differences by x differences Matlab Session Ý Ü Ýµº» ܵ Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 56 / 57

57 Bibliography 1 STEVEN C. CHAPRA & RAYMOND P. CANALE (2009): Numerical Methods for Engineers, 6ed, ISBN , McGraw-Hill 2 SINGIRESU S. RAO (2002): Applied Numerical Methods for Engineers and Scientists, ISBN X, Prentice Hall 3 DAVID KINCAID & WARD CHENEY (1991): Numerical Analysis: Mathematics of Scientific Computing, ISBN , Brooks/Cole Publishing Co. 4 STEVEN C. CHAPRA (2012): Applied Numerical Methods with MATLAB for Engineers and Scientists, 3ed, ISBN , McGraw-Hill 5 JOHN H. MATHEWS & KURTIS D. FINK (2004): Numerical Methods Using Matlab, 4ed, ISBN , Prentice Hall 6 WILLIAM J. PALM III (2011): Introduction to MATLAB for Engineers, 3ed, ISBN , McGraw-Hill Òº ÙÐÐ ÚºÒÙÐÐ ¾¼½ SKMM 3023 Applied Numerical Methods Numerical Differentiation 57 / 57