Chapter 4: Interpolation and Approximation. October 28, 2005

Transcription

1 Chapter 4: Interpolation and Approximation October 28, 2005

2 Outline Linear Interpolation Lagrange Interpolation Newton Interpolation and Divided Differences Interpolation Error Piecewise Polynomial Interpolation An Introduction To Splines Least Squares Concepts in Approximation

3 Linear Interpolation Interpolation: estimating intermediate values of the function if we only know its values at certain points x k (k = 0,..., n) Graphs produced by computers are generally obtained by piecewise linear polynomial interpolation, in which the machine draws a very large number of straight lines to represent the curve. Given a set of data points x k (nodes), we say that the polynomial p interpolates the function f at these nodes if p(x k ) = f (x k ) We want to see to what extent the approximation is accurate. p f

4 First, we will look at the case when p(x) is a linear polynomial We know that every straight line is determined by two points. We will assume that we are given two nodes x 0, x 1 and a function f. The equation of the line passing through the points (x 0, f (x 0 )) and (x 1, f (x 1 )) is p 1 (x) = x 1 x x 1 x 0 f (x 0 ) + x x 0 x 1 x 0 f (x 1 ) Easy to check: p 1 (x 0 ) = f (x 0 ) and p 1 (x 1 ) = f (x 1 ).

5 Example Suppose the data points are (1,1) and (4,2). The linear polynomial p 1 (x) is: p 1 (x) = 4 x x = x This polynomial will in fact approximate the function y = x from which the data points were taken.

6 Question: What is the accuracy of approximating f (x) by p 1 (x)? We will use Rolle s Theorem to get an answer. Define E(x) = f (x) p 1 (x) w(x) = (x x 0 )(x x 1 ) G(x) = E(x) w(x) w(t) E(t) where t is some fixed value in (x 0, x 1 ).

7 First, we notice that G(x 0 ) = 0, G(x 1 ) = 0, G(t) = 0 By Rolle s Theorem, there are points β 0 between x 0 and t, and β 1 between t and x 1, such that G (β 0 ) = 0, G(β 1 ) = 0. Using Rolle s Theorem again, we see that there is a point ξ between β 0 and β 1 such that G (ξ) = 0. However, so that G (x) = f (x) 2 w(t) E(t) G (ξ) = 0 f (ξ) 2 w(t) = 0.

8 The consequence of this computation is that we have f (t) p 1 (t) = 1 2 (t x 0)(t x 1 )f (ξ) for any point t [x 0, x 1 ] Then, for the error of this linear polynomial approximation, we get the following upper bound: f (x) p 1 (x) 1 2 (x x 0)(x x 1 ) max x 0 t x 1 f (t) 1 2 ( max x 0 t x 1 (t x 0 )(t x 1 ) )( max x 0 t x 1 f (t) ) So, in order to find an upper bound for the error, we need to maximize the function g(x) = (x x 0 )(x x 1 ) = (x 1 x)(x x 0 ) on the interval [x 0, x 1 ]

9 The critical point for this function is the midpoint of the interval x c = x 0 + x 1 2 Since the values at the endpoints are g(x 0 ) = g(x 1 ) = 0, the maximum absolute value is at x c, and it is equal to: g(x c ) = (x 1 x 0 ) 2. 4 Substituting this maximum value into an earlier formula for the error, we get: f (x) p 1 (x) 1 8 (x 1 x 0 ) 2 ( max x 0 x x 1 f (x) ).

10 Theorem (Linear Interpolation Error) Let f have the first two derivatives on [x 0, x 1 ] and let p 1 (x) be the linear polynomial that interpolates f at x 0 and x 1. Then, for all x [x 0, x 1 ], we have f (x) p 1 (x) 1 8 (x 1 x 0 ) 2 ( max x 0 x x 1 f (x) ).

11 Example Obtain the estimate of e using the function values e 0.82 = , e 0.83 = The linear interpolating polynomial is and p 1 (x) = (0.83 x) ( ) + (x 0.82) ( ) 0.01 p 1 (0.826) = Since f (x) = e x, the error estimate will be e p 1 (0.826) 1 8 (0.01)2 e 0.83 =

12 Example We want to construct a piecewise linear approximation to f (x) = log 2 (x) using the nodes: 1/4, 1/2, 1. We construct separate linear polynomials over each pair of adjacent nodes Q 1 (x) = ( 1 2 x) log 2 ( 1 4 ) + (x 1 4 ) log 2 ( 1 2 ) = 4x Q 2 (x) = (1 x) log 2 ( 1 2 ) + (x 1 2 ) log 2 (1) = 2x 2 1 2

13 So, the piecewise interpolating polynomial function is: { 4x 3, 1 q(x) = 4 x x 2, 2 x 1 To estimate the error, we calculate the errors of individual polynomials: log 2 (x) Q 1 (x) 1 8 (1 4 )2 max t [ 1 4, 1 2 ] log 2 (e)t 2 = log 2 (x) Q 2 (x) 1 8 (1 2 )2 max t [ 1 2,1] log 2 (e)t 2 = so the maximum error will be the maximum of the two errors (which are the same in this example) log 2 (x) q(x)

14 Quadratic Interpolation Most sets of data arise from the functions whose graphs are curved rather than straight lines. In order to approximate such functions better we need to use polynomials of degree greater than one. Assume we are given three different data points: (x 0, f (x 0 )), (x 1, f (x 1 )), (x 2, f (x 2 )). We want to find three quadratic polynomials L 0 (x), L 1 (x), and L 2 (x), such that our quadratic interpolating polynomial is p 2 (x) = y 0 L 0 (x) + y 1 L 1 (x) + y 2 L 2 (x)

15 One such choice for quadratic polynomials is: L 0 (x) = (x x 1)(x x 2 ) (x 0 x 1 )(x 0 x 2 ) L 1 (x) = (x x 0)(x x 2 ) (x 1 x 0 )(x 1 x 2 ) L 2 (x) = (x x 0)(x x 1 ) (x 2 x 0 )(x 2 x 1 ) Each polynomial L i (x) has degree 2, and so the polynomial p 2 (x) will be of degree at most 2. In addition, L i (x j ) = 0, L i (x i ) = 1 i j

16 We can write this succinctly as L i (x) = δ ij where δ ij is Kronecker s delta function defined as δ ij = Now, it is easy to show that { 1, i = j 0, i j p 2 (x i ) = f (x i ), i = 0, 1, 2

17 Example Given the data points (0,-1), (1,-1), and (2,7), the interpolating quadratic polynomial is p 2 (x) = (x 1)(x 2) ( 1) + 2 x(x 2) ( 1) + 1 x(x 1) (7) 2

18 In fact, using some facts from abstract algebra, one can show that the polynomial p 2 (x) is the unique quadratic polynomial p(x) such that p(x i ) = f (x i ), i = 0, 1, 2 Namely, if there were other such polynomial q(x), then the polynomial p 2 (x) q(x) is a non-zero polynomial of degree 2 which has three distinct roots: x 0, x 1, and x 2, which is impossible.

19 Example Now, we will try to find another approximation to e 0.826, this time using a quadratic interpolating polynomial, if we are given the values e 0.82 = , e 0.83 = , e 0.84 = Using the formula for p 2 (x) and the given set of data, we have: p 2 (0.826) = This is a significant improvement over the value since the actual value is: p 1 (0.826) = e

20 General Case Suppose we are given n + 1 distinct data points: (x 0, f (x 0 )), (x 1, f (x 1 )),..., (x n, f (x n )) We are looking for polynomials L i (x) of degree n such that p n (x) = f (x 0 )L(x) + f (x 1 )L(x 1 ) f (x n )L(x n ). These polynomials of degree n are given by L i (x) = (x x 0)... (x x i 1 )(x x i+1 )... (x x n ) (x i x 0 )... (x i x i 1 )(x i x i+1 )... (x i x n )

21 As before, it is easy to check that L i (x j ) = δ ij, 0 j n Direct computation shows that p n (x j ) = f (x j ), j = 0, 1,..., n This is called Lagrange s formula for the interpolating polynomial. Again, one can easily show that this is the unique interpolating polynomial of degree n for these n + 1 data points.

22 Theorem (Polynomial Interpolation - Existence and Uniqueness) Let the nodes x i (i = 0, 1,..., n) be given. As long as all x i are distinct, there is a unique polynomial p n of degree n such that p n (x i ) = f (x i ), 0 i n for a given function f.

23 Example Suppose f (x) = e x and consider the interval [-1,1]. We want to construct an approximation using the nodes x 0 = 1, x 1 = 1 2, x 2 = 0, x 3 = 1 2, x 4 = 1

24 p 4 (x) = (x )(x 0)(x 1 2 )(x 1) ( )( 1 0)( )( 1 1)e 1 (x + 1)(x 0)(x 1 2 )(x 1) + ( )( 1 2 0)( )( 1 2 1)e + (x + 1)(x )(x 1 2 )(x 1) (0 + 1)( )(0 1 e0 2 )(0 1) + (x + 1)(x )(x 0)(x 1) 1 ( )( )( 1 2 0)( 1 2 1)e 2 + (x + 1)(x )(x 0)(x 1 2 ) (1 + 1)( )(1 0)(1 1 2 ) e1 1 2

25 This polynomial simplifies to p 4 (x) = x x x x + 1

26 Question: What is the error when approximating f (x) by p n (x)? (will be answered later) Problem: Suppose we want to add a node to the set of data. Then, all L i (x) need to be recalculated, which is rather cumbersome, so Lagrange form of the interpolating polynomial is not suitable for actual computations. There is a far superior alternative construction of p n (x) which will be given in 4.2.

27 Newton Interpolation As we have seen in 4.1, given n + 1 distinct data points (x 0, f (x 0 )), (x 1, f (x 1 )),..., (x n, f (x n )) there is a unique polynomial of degree n that interpolates them. Lagrange interpolation formula doesn t work too well in practical computations. The reason for this is that, if we add another node we have to recompute all L i.

28 An alternative form of the polynomial, which is called the Newton form of the interpolation polynomial, avoids this problem and computes p n+1 (x) recursively from p n (x). Theorem (Newton Interpolation Construction) Let p n be the polynomial that interpolates f at the nodes x 0, x 1,..., x n and let p n+1 be the polynomial that interpolates f at teh nodes x 0, x 1,..., x n, x n+1. Then, p n+1 (x) = p n (x) + a n+1 w n (x) where w n (x) = a 0 = f (x 0 ) n (x x i ) i=0 a n+1 = f (x n+1) p n (x n+1 ) w n (x n+1 )

29 To show that this is true, it suffices to show that p n+1 (x 0 ) = f (x 0 ), p n+1 (x 1 ) = f (x 1 ),..., p n+1 (x n+1 ) = f (x n+1 ). For x 0, x 1,..., x n, we see that w n (x i ) = 0, (i = 0, 1,..., n) and p n+1 will interpolate f at those points. What is left to check is that p n+1 will interpolate f at x n+1 : p n+1 (x n+1 ) = p n (x n+1 ) + a n+1 w n (x n+1 ) = p n (x n+1 ) + f (x n+1 ) p n (x n+1 ) = f (x n+1 )

30 Newton form of the interpolating polynomial can be used to construct and evaluate interpolating polynomials in a much more efficient fashion. From the theorem, it follows that p n (x) = a 0 + a 1 (x x 0 ) + a 2 (x x 0 )(x x 1 ) a n (x x 0 )... (x x n 1 ) Another way to view this formula is as: p n (x) = where we take w 1 (x) = 1. n a k w k 1 (x), k=0 This formula can also be interpreted as the Horner s Rule for evaluating polynomials: p n (x) = a 0 +(x x 0 )(a 1 +(x x 1 )(a (x x n 1 )a n )...)).

31 Example Construct a quadratic polynomial that interpolates the function f (x) = sin x on the interval [0, π] using equally spaced nodes. Since the nodes are equally spaced,. Then, x 0 = 0, x 1 = π 2, x 2 = π a 0 = f (x 0 ) = 0 p 0 (x) = 0 a 1 = f (x 1) p 0 (x 1 ) w 0 (x 1 ) a 2 = f (x 2) p 1 (x 2 ) w 1 (x 2 ) = 1 0 π 2 0 = 2 π p 1(x) = π x 0 2 = (π 0)(π π 2 ) = 4 π 2

32 So, the interpolating polynomial is p 2 (x) = 2 π x 4 π 2 x(x π 2 )

33 Divided Differences The coefficients a k in Newton interpolation formula are called divided differences. Now, we will give a slightly different way of evaluating these coefficients which is suitable for computations done by hand. This approach is called constructing the Newton interpolation polynomial using the divided-difference table

34 The zeroth divided differences are simply the values of the function at the interpolating nodes: f 0 (x k ) = f (x k ), (k = 0, 1,..., n) The first divided differences are formed recursively from the zeroth ones as: f 1 (x k ) = f 0(x k+1 ) f 0 (x k ) x k+1 x k The second divided differences are then computed recursively as: f 2 (x k ) = f 1(x k+1 ) f 1 (x k ) x k+2 x k The general formula for the j-th divided difference is: f j (x k ) = f j 1(x k+1 ) f j 1 (x k ) x k+j x k

35 Example Suppose we wanted to interpolate the function using the three nodes f (x) = log 2 x, x 0 = 1, x 1 = 2, x 2 = 4. k x k f 0 (x k )

36 Next, we calculate the first divided differences: f 1 (x 0 ) = f (x 1) f (x 0 ) x 1 x 0 = = 1 f 1 (x 1 ) = f (x 2) f (x 1 ) x 2 x 1 = = 1 2 We enter these values into our divided difference table: k x k f 0 (x k ) f 1 (x k )

37 Finally, we calculate the second divided difference: f 2 (x 0 ) = f 1(x 1 ) f 1 (x 0 ) x 2 x 0 = = 1 6. k x k f 0 (x k ) f 1 (x k ) f 2 (x k )

38 Now, p 2 (x) can be derived in two ways from the table: If we use f 0 (x 0 ), f 1 (x 0 ), and f 2 (x 0 ) (the top diagonal row): p 2 (x) = (x 1) 1 6 (x 1)(x 2) = 1 (x 1)(x 8) 6 If we use the bottom diagonal row, i.e. the values f 0 (x 2 ), f 1 (x 1 ), and f 0 (x 0 ), we get: p 2 (x) = (x 4) 1 6 (x 4)(x 2) = 1 (x 1)(x 8) 6 Then, p 2 (1) = 0, p 2 (2) = 1, p 2 (4) = 2, so this is the correct interpolating polynomial.

39 Now, suppose we want to increase the accuracy of the polynomial by adding an additional node, for example, x 3 = 1 2, for which f (x 3 ) = 1. We can add that extra node to the table: k x k f 0 (x k ) f 1 (x k ) f 2 (x k )

40 Then, we compute f 1 (x 2 ), f 2 (x 1 ) and f 3 (x 0 ), and place them into the table: k x k f 0 (x k ) f 1 (x k ) f 2 (x k ) f 3 (x k ) So, the interpolating polynomial is p 3 (x) = (x 1) 1 6 (x 1)(x 2) + 1 (x 1)(x 2)(x 4) 7

41 Interpolation Error We have seen in 2.4 that the error of the linear interpolation polynomial can be estimated as f (x) p 1 (x) 1 8 (x 1 x 0 ) 2 max x 0 x x 1 f (x) Question: Is there a similar estimate for interpolation polynomials of arbitrary degree p n (x)?

42 Theorem (Polynomial Interpolation Error Theorem) Let f be a function with n + 1 derivatives on [a, b] and let the nodes x k [a, b], for k = 0, 1,..., n. Then, for each x [a, b], we can find a ξ [a, b] such that f (x) p n (x) = w n(x) (n + 1)! f (n+1) (ξ) where n w n (x) = (x x k ) k=0 The problem is that, often, we don t know the formula for f (x) so it is impossible to estimate the maximum of f (n+1) (x) on [a, b].

43 It would be useful to have a way of measuring the size of a function on [a, b]. This is done using the norm of a function. Definition (Function Norm) A function norm is any function that assigns to a function on [a, b] a non-negative number f so that: 1 f > 0 for any function that is not the constant-zero function on [a, b] 2 af = a f, for any constant a. 3 f + g f + g, for any two functions f and g.

44 Examples 1 The infinity (pointwise) norm: f = max f (x) x [a,b] 2 The 2-norm : b f 2 = a [f (x)] 2 dx

45 Example If f (x) = cos x on [0, π], then f = max f (x) = 1 x [0,π] π π ( ) 1 + cos(2x) π f 2 = cos 2 (x)dx = dx =

46 As we have already seen, f (x) p 1 (x) 1 8 (x 1 x 0 ) 2 max x 0 x x 1 f (x) Using the infinity norm, this can be rewritten as f (x) p 1 (x) 1 8 (x 1 x 0 ) 2 f Let s try to find an upper bound for the error of the quadratic interpolating polynomial.

47 By the theorem from the beginning of this section, f (x) p 2 (x) = 1 6 (x x 0)(x x 1 )(x x 2 )f (ξ) To get a reasonable upper bound, we will assume that the nodes are equally spaced: for some h > 0. Then, x 1 x 0 = x 2 x 1 = h (x x 0 )(x x 1 )(x x 2 ) = ((x x 1 )+h)(x x 1 )((x x 1 ) h) = t(t 2 h 2 where t = x x 1. Therefore, f (x) p 2 (x) 1 6 ( max h t h t(t2 h 2 ) ) max x 0 t x 2 f (t)

48 In order to maximize this estimate, we need to maximize on [ h, h]. t(t 2 h 2 ) We can either use the maximize command in Maple, or examine the critical points (by hand) and we get max h t h t2 (t h 2 ) = h3. So, the error of quadratic interpolation is bounded above by or f (x) p 2 (x) h3 max x [x 0,x 2 ] f (t) f (x) p 2 (x) h3 f where h is the spacing between the consecutive nodes.

49 Example What is the error in quadratic interpolation to f (x) = x using equally spaced nodes on the interval [ 1 4, 1]? Solution: Since the nodes are equally spaced, h = = The third derivative of the function is: so that Then, f = f (x) = 3 8 x 5 2 max f (x) = 3 x [1/4,1] 8 (1 4 ) = 32 = f (x) p 2 (x) ( ) =

50 In a similar fashion, one can show that the error of the cubic interpolating polynomial on [a, b], with equally spaced nodes is: f (x) p 3 (x) 1 24 h4 f (4)

51 Piecewise Polynomial Approximation Interpolation might not be the best tool for approximation if we let n, the degree of the interpolation polynomial p n (x) become large. One way to avoid that problem is to keep the order of the polynomial fixed and use different polynomials over different intervals. If we let the lengths of subintervals become increasingly smaller, this way of approximating the function will be very accurate.

52 The only problem with this approach (using piecewise interpolation) is that we are dealing with different polynomials. So, in addition to being given a certain set of nodes, we have to select a subset of the se nodes which will be the endpoints of the subintervals over which the polynomials will differ. This subset of the set of all nodes are called knots of piecewise interpolation. If all the polynomials over different subintervals have the fixed degree d, the knots will be x dj, 0 j m where m is the number of different polynomials we will use.

53 Example Suppose we want to construct a piecewise quadratic approximation to f (x) = sin(πx) using the nodes x k = k, k = 0, 1, 2,..., 6 6 Since every quadratic polynomial requires three nodes in this partition, there will be three separate polynomial approximations with the knots x 0, x 2, x 4, x 6 next, we construct the three quadratic polynomials.

54 The interval [x 0, x 2 ]: x k f 0 (x k ) f 1 (x k ) f 2 (x k ) / / The interpolating polynomial is p 2,1 (x) = x x(x 1 6 )

55 The interval [x 2, x 4 ]: x k f 0 (x k ) f 1 (x k ) f 2 (x k ) 1/ / / The interpolating polynomial is p 2,2 (x) = (x 1 3 ) (x 1 3 )(x 1 2 )

56 The interval [x 4, x 6 ]: x k f 0 (x k ) f 1 (x k ) f 2 (x k ) 2/ / The interpolating polynomial is p 2,3 (x) = (x 2 3 ) (x 2 3 )(x 5 6 )

57 The piecewise polynomial will be: x x(x 1 6 ), 0 x 1 3 q 2 (x) = (x 1 3 ) (x 1 3 )(x 1 2 ), 1 3 x (x 2 3 ) (x 2 3 )(x 5 6 ) 2 3 x 1

58 Error Estimate Theorem (Piecewise Polynomial Interpolation Error Estimate) Let f have a sufficient number of derivatives on [a, b] and let q d be the piecewise polynomial interpolant of degree d for f on [a, b], using n + 1 equally spaced nodes, where n = md. Then f q d C d h d+1 f (d+1), d = 1, 2, 3 where C 1 = 1/8, C 2 = 1/9 3, C 3 = 1/24

59 Definition of the Problem We have seen that piecewise polynomial interpolants produce fairly accurate approximations to the function, but their graphs may lack smoothness" at the nodes. This is of importance in e.g. computer graphics, since the graphs are generally generated using piecewise polynomial interpolation through a finite number of sample nodes. The piecewise function q d is continuous but its derivatives may not be. So, the graph may have kinks" at the points where two different polynomials are joined (knots) Splines provide a way to rectify this.

60 We are given a set of nodes x 0, x 1,..., x n We want all the polynomials to have the same degree d, which will be the degree of the approximation of the spline. We want to find a piecewise polynomial function q d which will satisfy three properties.

61 Properties of a spline: 1 Interpolation : q d (x k ) = f (x k ), 0 k n 2 Smoothness : lim x x k q (i) d (x) = lim x x + k q (i) d (x k) for 0 i N. This N is called the degree of smoothness of the spline. 3 q d is a polynomial of degree d on each subinterval [x k 1, x k ]

62 Question: How should we choose N, the degree of smoothness? By analyzing the properties of the spline, one can show that a good choice for the degree of smoothness is N = d 1 i.e. one less than the degree of the polynomial we use. For instance, if we use cubic polynomials (d = 3), we would have N = 2, so the first and the second derivatives of polynomials on neighbouring intervals have to agree at the junction nodes x k. We will also need to impose N extra conditions on the derivatives but we will deal with that later.

63 Example For what value of k is the following a spline function? { x q(x) = 3 + 3x 2 + 1, 1 x 0 x 3 + kx 2 + 1, 0 x 1 Solution: First, the values of the two polynomials have to agree at the junction point x = 0: lim x 0 (x 3 + 3x 2 + 1) = lim x 0 +( x 3 + kx 2 + 1) which is true for every value of k. Since d = 3 we have N = 2, and the limits of the first and the second derivatives have to agree at x = 0: lim x 0 (3x 2 + 6x) = lim x 0 +( 3x 2 + 2kx) lim x 0 + 6) = lim (6x +( 6x + 2k) x 0

64 The first equality is always true, while from the second equation we have k = 3 So, k = 3 will make q(x) a spline function.

65 Cubic Splines We will now show two different constructions for a cubic spline interpolation through the set of n + 1 nodes (x 0, y 0 ), (x 1, y 1 ),..., (x n, y n ) Suppose that cubic spline is q(x). Introduce n new variables M 1, M 2,..., M n as M i := q (x i ), (i = 1, 2,..., n) We want to produce a linear system for calculating M 1, M 2,..., M n.

66 On each interval [x j 1, x j ], the function q (x) is linear, and we can use linear interpolation to write the formula for q (x) on that interval: q (x) = (x j x)m j 1 + (x xj 1)M j x j x j 1 Then, we can integrate this linear function twice and use the interpolating conditions to get q(x j 1 ) = y j 1, q(x j ) = y j q(x) = (x j x) 3 M j 1 + (x x j 1 ) 3 M j 6(x j x j 1 ) for x j 1 x x j. + (x j x)y j 1 + (x x j 1 )y j x j x j (x j x j 1 )[(x j x)m j 1 + (x x j 1 )M j ]

67 To ensure the smoothness of the curve, we differentiate this last formula for q(x) to get q (x) and make sure that the values of q (x) on two adjacent intervals [x j 1, x j ] and [x j, x j+1 ] agree at x j. After some work, it can be shown that we have x j x j 1 M j x j+1 x j 1 3 M j + x j+1 x j 6 M j+1 = y j+1 y j x j+1 x j y j y j 1 x j x j 1 for j = 1, 2,..., n 1.

68 As we have mentioned earlier, since for a cubic spline N = 2, we need two extra conditions in order to be able to solve this system (the system has n 1 equations and n + 1 variables M j ) These two conditions can be imposed in two different ways: 1 Natural Spline: M 0 = M n = 0 (these two conditions will produce a tridiagonal system) 2 Complete Spline: q (x 0 ) = f (x 0 ) and q (x n ) = f (x n ) We prefer to use the natural spline, since, sometimes, we may not know the formula for f (x), so we cannot use f

69 Example Calculate the natural cubic spline interpolating the data (1, 1), (2, 1 2 ), (3, 1 3 ), (4, 1 4 ) Solution: The number of points is 4, and all x j x j 1 = 1, j = 1, 2, 3 The system of equations becomes 1 6 M M M 2 = M M M 3 = 1 12 Since M 0 = M 3 = 0, we reduce the system to: 2 3 M M 2 = M M 2 = 1 12

70 The solution is M 1 = 1 2, M 2 = 0 The resulting cubic spline is: q(x) = 1 12 x x x x x x x x x x 4

71 Example Construct a natural cubic spline approximation to f (x) = sin(πx) on the interval [0, 1/2] using the nodes x 0 = 0, x 1 = 1/4, and x 2 = 1/2. Solution: Here, n = 2, and we have the single equation: Since the spline is natural: 1 24 M M M 2 = M 0 = M 2 = 0 and M 1 = = 24(1 2).

72 Substituting M 0 = 0, M 1 = 24(1 2), M 2 = 0into the formula for q(x), we get: 16( 2 1)x 3 + (3 2 1)x 0 x 1/4 q(x) = 16( 2 1)x 3 8( 2 1)x 2 +( 2 + 1)x + ( 2 1)/2 1/4 x 1/2

73 Error of Spline Approximation Theorem (Spline Approximation) Suppose f has continuous first four derivatives on [a, b], and q interpolates f through the nodes a = x 0 < x 1 < x 2 <... < x n 1 < x n = b with max(x i x i 1 ) h for all i. Then, f q h4 f (4)

74 Example In our previous example, f (x) = sin(πx) on [0, 1/2]. Since h = 1/4 (the nodes are equally spaced) and f (4) (x) = π 4 sin(πx) we have so that f q f (4) = π 4, ( ) 1 4 π 4 =

75 An Introduction to Data Fitting Often, when we want to find a function representing the results of some experiment, we do not want to have a curve passing through every data point. The reason for this is that the experimental data is not entirely precise; there is some error due to the measurement, etc. Rather, we ant to get some sense of how the data points are distributed: is the pattern linear, quadratic, cubic, etc? The most common approach is least squares data fitting

76 We will start with the simplest case: suppose we are given a set of data (x k, y k ), k = 1, 2,..., n for which the plot would indicate that it s distributed linearly. We want to find a linear function y = mx + b which would fit this data best. How do we measuer how well this line would represent the given points? We try to minimize the sum of the squared distances between the points and the line: F(m, b) = n (y k (mx k + b)) 2 k=1 This is a function of two variables m and b which needs to be minimized for the right choice of these two parameters.

77 Take the partial derivatives with respect to both variables: F n m = 2 (y k (mx k + b))x k = 0 k=1 F n b = 2 (y k (mx k + b)) = 0 k=1 This simplifies to the system of two linear equations in m and b: n (y k (mx k + b))x k = 0 k=1 n (y k (mx k + b)) = 0 k=1

78 This is equivalent to: ( n ) ( n ) m + b x k = k=1 x 2 k k=1 ( n ) ( n ) m x k + b 1 = k=1 k=1 This gives us the formulas for m and b: m = n x k y k ( x k )( y k ) n( x 2 k ) ( x k ) 2 n x k y k k=1 n k=1 x k b = ( x 2 k )( y k ) ( x k )( x k y k ) n( x 2 k ) ( x k ) 2

79 Example Suppose we are given data x k y k Find the least squares linear linear fit to this data. Solution: We compute x k = 15, y k = 367, xk 2 = 55, x k y k = 1303 k=1 Therefore, The linear fit is: k=1 k=1 k=1 6(1303) m = 6(55) (15) 2 = b = 6(55) (15) 2 = y = x

80 Instead of using a linear function for the least squares approximation, one can use a polynomial of any other degree. Sometimes, this is necessary, since the distribution of data may not be linear. We will sketch how to find a quadratic least squares approximation for a given set of data points. You will use this in Exercises 1 and 2 from this section.

81 Suppose we are given n data points (x k, y k ), k = 1, 2,..., n which is clustered around the graph of some quadratic function y = ax 2 + bx + c Again, we try to minimize the sum of squared distances from the points to this parabola: F(a, b, c) = n (y k (axk 2 + bx k + c)) 2 k=1 This is a function of three variables a, b, and c, so we want to make the partial derivatives F a = F b = F c = 0

82 Taking the derivatives and simplifying, we get: n 2 (y k (axk 2 + bx k + c))xk 2 = 0 2 k=1 n (y k (axk 2 + bx k + c))x k = 0 k=1 2 n (y k (axk 2 + bx k + c)) = 0 k=1 This yields the system in a, b and c: a( x 4 k ) + b( x 3 k ) + c( x 2 k ) = x 2 k y k a( x 3 k ) + b( x 2 k ) + c( x k ) = x k y k a( x 2 k ) + b( x k ) + nc = y k

83 Least Squares Approximation and Orthogonal Polynomials Given two vectors x and y in R n, we defined their inner product as x y = x 1 y 1 + x 2 y x n y n This inner product can be generalized to functions on some fixed closed interval [a, b]

84 Definition (Inner Product for Real Functions) If f and g are two functions on an interval [a, b], then their inner product (f, g) is any function which has these three properties: 1 (f, f ) > 0 for any nonzero function f. 2 (f, αg 1 + βg 2 ) = α(f, g 1 ) + β(f, g 2 ) for any two constants α and β and any functions f, g 1, g 2. 3 (f, g) = (g, f ) for any two functions f and g.

85 Example Our main example is the weighted inner product: let w(x) 0 be a non-negative integrable function on [a, b]. Then we can define the inner product of f and g as (f, g) w = b a w(x)f (x)g(x)dx If we put w(x) = 1, we get the unweighted inner product (f, g) = b a f (x)g(x)dx

86 The weighted norm of f is defined as the square root of the inner product of f with itself: b f w = a w(x)f 2 (x)dx If we put w(x) = 1, then we have the 2-norm of the function which we have seen before: b f 2 = a f 2 (x)dx For example, if f (x) = e x and w(x) = x on [0,1], we have: 1 1 e e x w = (xe x e x )dx = (xe 2x 2 )dx = 4 = e 2 0 0

87 Definition Given a weighted inner product (f, g) w on [a, b], we say that f and g are othogonal if (f, g) w = b a w(x)f (x)g(x)dx = 0 For example, if f (x) = sin x, and g(x) = cos x and the inner product is unweighted, these two functions are orthogonal on [0, π]: π π sin(2x) sin x cos xdx = dx =

88 It can be shown that if we use the families of orthogonal polynomials, we get very good least squares approximations to a given function. There are different ways to generate these families of orthogonal polynomials (Laguerre polynomials, Chebyshev polynomials, Legendre polynomials, etc) and we will explain how to generate such family, the family of Legendre polynomials This can be used to approximate functions on the interval [-1,1]; this is not a major restriction, because [-1,1] can always be changed to any other interval [a, b] using a linear function.

89 Legendre polynomials are defined recursively as: P 0 (x) = 1 P n (x) = 1 n!2 n d n dx n [ (x 2 1) n] The first few Legendre polynomials are: etc. P 1 (x) = x P 2 (x) = 1 2 (3x 2 1) P 3 (x) = 1 2 (5x 3 3x) P 4 (x) = 1 8 (35x 4 30x 2 + 3)

90 It can be shown: 1 All distinct Legendre polynomials are pairwise orthogonal in the unweighted product on [ 1, 1] (P i (x), P j (x)) = 1 1 P i (x)p j (x)dx = 0 2 Every polynomial of degree n can be written as a unique linear combination of Legendre polynomials: Q(x) = c 0 P 0 (x) + c 1 P 1 (x) c n P n (x) where c i are some constants.

91 Question: Given a function f (x) on [ 1, 1] how do we approximate it by the sum of the form c 0 P 0 (x) + c 1 P 1 (x) c n P n (x)? We can calculate the coefficients c i using the following formula: c i = (f, P i) (P i, P i )

92 Example Suppose we want to find the cubic least squares approximation to f (x) = e x on [ 1, 1] We calculate the coefficients c 0, c 1, c 2, and c 3 : c 0 = 1 (ex, P 0 (x)) (P 0 (x), P 0 (x)) = 1 ex 1dx e e 1 1 = 1 1 1dx 2 = Similar calculation using integrals will show that the coefficients are c 1 = , c 2 = , c 3 =

93 So the best cubic least squares approximation to f (x) = e x on [ 1, 1] is: P 0 (x) P 1 (x) P 2 (x) P 3 (x) If we substitute the Legendre polynomials into this formula, we get x x x 3 It can actually be shown that the maximum distance between the graph of e x and this cubic polynomial is e x P = max [ 1,1] ex P(x) =