Chapter 7 Approximations of Functions 7.. Introduction IS A BETTER TITLE FOR THE CHAPTER: INTERPOLATION AND APPROIMATION THEORY Before we can study the solutions of dierential equations such as dy dx = f(y) for x 2 [a b] (7.) we rst need to understand how weapproximate functions on the computer. One common method of approximation, which wehave already discussed, is to approximate y(x) by the discrete set of nodes (or data) (x i y i ). Thus, if we solve an initial value ode with initial condition y(a) =y 0 our solution will be such a set of output data. We often then want to reconstruct a smooth function from this set. Notation: In this chapter we generally let x, rather than t, be the independent variable. The main reason is that we are not considering dynamical systems in this chapter. And it is more common to use x than t in the literature. We will use t whenever we are considering a time evolution problem. SAY BETTER??? In this chapter we are generally working with the output data from some experiment. Usually this data is in the form (x i y i ) and we will simply call this the set of data points, or just points, that we want to interpolate or approximate. The x values are called the abscissas and the y values the ordinates. However, at times the data will be dierent. For example, we might be given (x i y i yi 0), which also includes the slopes at the points. (If our data is actually (ti y i yi) 0 where t is the time, then yi 0 is the velocity at a particular time.) Generally, however, our data is the set of n+ points (x i y i ) i 2N [0 n] where y i 2 R m for all i. In this chapter we normally work with y i 2 R. However, we can easily handle y i 2 R m by ; then we consider x i y (j) i i 2N [0 n] for each j 2N [ m] separately. There are a number ofways to interpolate or approximate a set of points, which is the topic of the present chapter. Let us now discuss interpolation. We can use a polynomial to interpolate the data and we are guaranteed to succeed as long as the degree of the polynomial is at least n. Often we do not attempt to interpolate all the points with one polynomial. Instead we choose a small subset of the points and interpolate it with a polynomial so that the polynomial has low degree. working with each component of y separately. That is, if we let y = ; y () y (2) ::: y (m) T 47
472 Ch. 7. Approximations of Functions We can combine all these subsets together and consider the set of low-degree polynomials to be the interpolation of the data. This is an example of piecewise polynomial interpolation. Another method of interpolation P is to use a set of \simple" functions i (x) and let our interpolating function be (x) = c j i (x). The coecients are chosen so that (x j )=y j. If j (x) =x j then we return to polynomial interpolation. Another common choice of basis functions is trigonometric functions. On the other hand, we mightnotwanttointerpolate the points but only to approximate them. For example, the points might be contaminated by \noise" (such as high-frequency oscillations). We will often want our approximating function to be \reasonably smooth" so that we do not want the approximation to pass through the points but only to be \close" to the points. Thus, we want to be \small", but not usually zero, and max i Z b a (xi ) ; y i 0 (x) dx to be "small". By forcing the integral to be \small" we keep 0 \small" in the L -norm so that does not have high-frequency oscillations. EPLAIN BETTER??? Again, (x) might be a polynomial, a set of low-degree polynomials, or a linear combination of \simple" functions. One reason for interpolating the points is when we need to use them further. Suppose that we want to calculate Lfyg where L is some linear operator and y is the solution of the ode (7.). Suppose, however, that we only have the set of points (x i y i ) which approximate the smooth function y. Then we can discretize L as we have often done in this book, so that Lfyg!L h fyg where y =(y 0 y ::: y n ) T. However, we can also use the interpolating or approximating function (x) and calculate Lfg from Lf i g. Conversely, wemightwant tocalculate Lfyg directly. For example, we mightwant to calculate R b a y(x) dx. Generally, we cannot obtain an exact answer and so we must use numerical methods, which usually involve taking a particular linear combination of values of y(x) for specied values of x. Thus, we have discretized y. Instead, we might be solving a boundary-value problem. In this case we must solve for a discretization of y(x). Again, one common method is to solve for (x i y i ) where y i is our approximation to y(x i ). We can also set and discretize by y(x) = y(x) i= n i= c i i (x) c i i (x) : We then solve the ode using this discretization. There are still a number of methods possible. We can require that satisfy the ode exactly at specied points in [a b]. Thus we solve the exact continuous ode at specied points. This makes the error 0 at these points but does nothing else in the interval. This is called a collocation method. We can also require that the error be minimized over the entire interval. This is called a nite-element method. Most of our discussion is focused on one-dimensional interpolation and approximation theory. However, we briey discuss it in higher dimensions. This is a much more \open" area and often the method depends on the specic problem.
7.2. Polynomial Interpolation 473 7.2. Polynomial Interpolation Suppose we are given the set of points (x i y i ) i 2N [0 n] and we want to interpolate these points by using a polynomial of degree n. As we will see, this polynomial, which wedenote by p n (x), is unique. Also, as we will see, there are a number of ways to calculate the coecients of this polynomial, which can have vastly dierent stability properties and costs. Finally, there are also a number of ways to evaluate this polynomial for a given value of x, which can again have vastly dierent stability properties and costs. These points are the main topics of this section. Throughout this section we assume that these are the points to be interpolated. SAY BETTER??? Notation: To reiterate the point, p n (x) denotes a polynomial of degree n. Only when a polynomial has arbitrary degree will we simply write p(x). 7.2.. Lagrange Polynomials For completeness, we begin with the denition of a polynomial and a polynomial space. Denition 7.. p(x) is a real polynomial of degree n if it can be written as p(x) =a 0 + a x + a 2 x 2 + ::: + a n x n ai i 2N [0 n] R. p(x) is a complex polynomial of degree n if ai i 2N [0 n] where C. (We usually work with real polynomials.) p(x) has exactly degree n if a n 6=0. Sometimes the word \order" is used to describe a polynomial. The order of a polynomial is the number of possible coecients, which is one more than the degree. We denote the set of all polynomials of degree n by n = a 0 + a x + a 2 x 2 + ::: + a n; x n; + a n x n ai 2R (C ) : (7.2) We will need to use a subset of n repeatedly and so we give it special notation. We often want to \normalize" polynomials by making their lead coecient one and we denote the set of all such polynomials by n = a 0 + a x + a 2 x 2 + ::: + a n; x n; + x n ai 2R (C ) : (7.3) Note that n is a subspace of the space of all polynomials. Also, note that n is the set of all polynomials of degree exactly n, which is not a linear space because, for example, the sum of two such polynomials does not have leading coecient one. To begin our discussion of polynomial interpolation we will prove that there exists a unique polynomial p n 2 n that interpolates our data. This proof is quite instructive, particularly because it is constructive. (That is, we will not only prove that such a polynomial exists we will also show how to calculate it.) First, we need a denition and then a lemma. Denition 7.2. Let x i i 2N [0 n] be a set of distinct abscissas, which need not be given in any particular order. The set of Lagrange polynomials for the abscissas f x j g, denoted by `j(x) j 2N [0 n], is dened by (x ; x 0 )(x ; x ) :::(x ; x j; )(x ; x j )(x ; x j+ ) :::(x ; x n ) `j(x) = (x j ; x 0 )(x j ; x ) :::(x j ; x j; )(x j ; x j )(x j ; x j+ ) :::(x j ; x n ) = ny k=0 k6=j x ; x k x j ; x k (7.4)
474 Ch. 7. Approximations of Functions where the notation b 0 b :::b j; b j b j+ :::b n; b n means that the term b j is omitted. Note that each polynomial has degree exactly n. If we need to emphasize the fact that `j(x) is a polynomial of degree n, we will write it as `(n) j (x). The numerators are chosen so that `j(x k )=0 if j 6= k. And the denominators are chosen so that `j(x j ) = for all j. Thus, `j(x k ) = jk for all j k 2 N [0 n]. For example, the four Lagrange polynomials of degree three are shown in Figure 7., where the data points are shown as the dots. The primary reason for using Lagrange polynomials is that it is particularly easy to write any polynomial of degree n as a linear combination of this set of Lagrange polynomials, as we prove in Theorem 7.. First, however, we need a lemma. Figure 7.. The Lagrange polynomials of degree three with the set of abscissas being f 3 4 5 g. The locations of the data points are given by the dots. Lemma 7.. Let x i i 2N [0 n] be a set of distinct abscissas and let `j(x) j 2N [0 n] be the corresponding set of Lagrange polynomials. This set of Lagrange polynomials is a basis for n. Proof: To show that these polynomials are linearly independent, consider the equation c 0`0(x)+c `(x)+ ::: + c n`n(x) =0 for all x 2R : First, c i =0 for each i follows immediately by letting x = x i. And, second, `j 2 n for each j. Thus, we have asetof n+ linearly independent functions in n (which isann+-dimensional space). We now use the Lagrange form to prove the desired theorem. Theorem 7.. Consider the n+ points (x i y i ) i 2N [0 n] where the abscissas are all distinct (but need not be in any order).then there exists a unique polynomial p n 2 n such that p n (x i )=y i for all i. (We will usually just say that the polynomial interpolates the given data.) This polynomial is given by p n (x) = n k=0 y k`k(x) : (7.5) Proof: Obviously, p n, as dened in (7.5), is one such polynomial. Suppose there exists a second such polynomial, say q n 2 P n, and let us write it as q n (x) = k=0 c k`k(x). This can always be done since by Lemma 7. `j(x) j 2N [0 n] spans n. Since q n (x j )=c j we must have c j = y j for all j 2N [0 n] and so q n = p n.
7.2. Polynomial Interpolation 475 There is another way to write this basis of Lagrange polynomials of degree n. If we dene n 2 n+ by then n (x) = `j(x) = ny j=0 (x ; x j ) (7.6) n (x) (x ; x j ) 0 n(x j ) (7.7) as can easily be seem by substituting the denition for n (x) into eq. (7.6). This expression for `j will be needed frequently in the next chapter also, n will be used later in this chapter. Warning: n (x) is a polynomial of degree n+, which is dierent from the convention we are using in this chapter. This is the normal convention for and so we will follow it here. Now suppose we actually wanttoevaluate p n (x) for many values of x. How costly is it? If we write p n as a linear combination of the Lagrange polynomials, eq. (7.5), then the coecients of this linear combination are given immediately. However, the evaluation of each Lagrange polynomial is very costly. The calculation of `j(x) requires setting up n; terms in the numerator and also in the denominator, multiplying together all these terms in the numerator, multiplying together all these terms in the denominator, and then dividing the numerator and denominator. Of course, the denominator need be done only once, since it is the same for each Lagrange polynomial. Thus, the total cost is n 2 & n 2. This is an unacceptable cost for evaluating a polynomial. There are other forms for expressing p n which lead to an O(n) cost for evaluating p n (x). We saythat p n is written in power form if it is written as p n (x) =a 0 + a x + a 2 x 2 + ::: + a n x n : (7.8) p n (x) can be evaluated with a cost of O(n) if we write it in the nested form p n (x) =a 0 + x a + x ; a 2 + x(a 3 + :::x(a n; + xa n ) :::) : (7.9) In nested form p n (x) can be easily evaluated numerically. In Fortran the algorithm is the following. P = A(N) DO 0 I = N-,0,- P = *P + A(I) 0 CONTINUE Algorithm 7.. Horner's method for evaluating p n (x) in power form. On output, p n (x) = P. This algorithm is called Horner's method its cost is n & n, which is quite acceptable. The fundamental diculty with expressing a polynomial in the power form is in calculating the coecients of p n. Suppose we rstwrite p n as a linear combination of the Lagrange polynomials, as in eq. (7.5). If we then calculate the coecients in the power form by multiplying all the terms in eq. (7.4), the cost is O(n2 n ). If, instead, we evaluate the coecients directly by solving the linear system generated by p n (x j )=y j for all j, namely 0 B @ x 0 x 2 0 ::: x n 0 x x 2 ::: x n x 2 x 2 2 ::: x n 2 : : : : : : : x n x 2 n ::: x n n 0 C A B @ a 0 a a 2 : a n C A = 0 B @ y 0 y y 2 : y n C A
476 Ch. 7. Approximations of Functions then our cost is O; n 3. Recall from Example 4. that the matrix is called a Vandermonde matrix and that this calculation is unstable. In addition, the evaluation of p n (x) by Horner's method can be unstable. The diculty is that we might have x. As a simple example, let p 3 (x) =(x ; 00) 3 + so that p 3 (x) =;999 999 + x ; 30 000 + x(;300 + x) : We will have catastrophic cancellation in evaluating p 3 (x) if x 300. This instability can be alleviated if we write p n in a shifted power form, namely p n (x) =a 0 + a (x ; )+a 2 (x ; ) 2 + ::: + a n (x ; ) n (7.0) where we should usually choose to be some sort of average of the values of x we expect to use. Normally, the average value of the abscissas is a good choice. As long as the abscissas are not \too widely separated" and as long as x is not \too far" from, then the evaluation of p n (x) should be stable. Another approach that sometimes works is to write p n as a linear combination of orthogonal polynomials. Example 7.2 gives a case where the evaluation is nearly ten times as accurate for a linear combination of Chebyshev polynomials as for Horner's method. CASE??? The shifted power form improves the stability ofthe calculation. However, we still have the problem of how to calculate the coecients of the polynomial in the rst place. In order to be able to calculate the coecients of the polynomial eciently we will use a dierent form for the polynomial. Finally, we can investigate the conditioning of polynomial interpolation by using eq. (7.5). If we perturb the ordinates y k to y k + y, then the polynomial changes by p n (x) =y k`k(x). Thus, the change is \not large" as long as k`kk is \not large". Note that a change of y might lead to a \small" change in p n for one value of k and a \large" value for another. SAY MORE AND BETTER??? This statement is generally true both analytically and numerically. Note that there is no catastrophic cancellation in the calculation of the Lagrange polynomials unless x is \very close" to an abscissa, in which case the contribution from all of the basis elements, except for one, is small. Of course, we can have catastrophic cancellation in the summing of y k`k(x) for all k. However, this is true of any calculation. Thus, we expect this method to be well-conditioned, as long as we do not have abscissas \very near" others. SAY MORE AND BETTER MENTION WHAT HAPPENS WHEN SOME 'S NEAR OTHERS??? 7.2.2. Divided Dierences A generalization of the shifted power form is Newton's form where we write the polynomial p n (x) as p n (x) =a 0 +[x ; x 0 ] a +[x ; ; x ] a 2 + ::: +[x ; x n;2 ](a n; +[x ; x n; ]a n ) ::: : (7.) Note that Newton's form has good stability properties, similar to the shifted power form. In this case the shifting is quite natural since it involves the abscissas of the points themselves. Also, this polynomial is evaluated quite eciently by Horner's method. P = A(N) DO 0 I = N-,0,- P = ( - (I))*P + A(I)
7.2. Polynomial Interpolation 477 0 CONTINUE Algorithm 7.2. Horner's method for evaluating p n (x) in Newton's form. The result is stored in P. The reason we have introduced Newton's form is that the coecients in eq. (7.) are easily calculated by the procedure of divided dierences. We calculate the coecients of p n (x) by a recursive procedure. To do this it is helpful to write the coecients in a notation which brings out the recursive nature of the calculation. In addition, the coecients depend on the order in which weevaluate the set of abscissas f x i g. In Newton's form we do not require that the abscissas be ordered in any way. However, we do require that all the abscissas are distinct (but we will remove even this requirement in subsection 7.2.4). Example 7.. Suppose the points are ( 8) (3 36) (4 32) (5 0) order so x 0 =, x =3, x 2 =4, and x 3 =5, then Newton's form for a cubic polynomial (cont. on page 478). If wechoose the abscissas in increasing p 3 (x) = 8 + 4(x ; ) ; 6(x ; )(x ; 3) ; 2(x ; )(x ; 3)(x ; 4) : (7.2a) If we choose the abscissas in the order x 0 = 4, x =, x 2 = 5, and x 3 = 3, so that no data point is physically adjacent to either the preceding or following point, then p 3 (x) =32+8(x ; 4) ; 0(x ; 4)(x ; ) ; 2(x ; 4)(x ; )(x ; 5) : (7.2b) Note that the only coecient which is the same in these two representations is the coecient of the highest-order term. This is no accident. In power form p 3 (x) = ;2x 3 +0x 2 so that the coecient ofthe x 3 term is ;2. In Newton's form only the last term, i.e., the term (x ; x 0 )(x ; x )(x ; x 2 ) contains the highest power of x, which is x 3 in our example. Thus the last term in any representation of p 3 (x) using Newton's form must have the coecient ;2. We can rewrite eq. (7.2a) as p 3 (x) =y[] + y[ 3](x ; ) + y[ 3 4](x ; )(x ; 3) + y[ 3 4 5](x ; )(x ; 3)(x ; 4) (7.3a) to indicate which points are needed in the calculation of each coecient in Newton's form. That is, as we will see shortly, only ( 8) is needed to calculate y[], only ( 8) (3 36) is needed to calculate y[ 3], etc. We rewrite eq. (7.2b) as p 3 (x) =y[4] + y[4 ](x ; 4) + y[4 5](x ; 4)(x ; ) + y[4 5 3](x ; 4)(x ; )(x ; 5) (7.3b) to, again, indicate which points are needed in order to calculate a particular coecient. Note that y[ 3 4 5] = y[4 5 3] = ;2 because these are the coecients of the highest power of p 3 (x), i.e., x 3, and this coecient is independent of the ordering of the points.
478 Ch. 7. Approximations of Functions To indicate how the coecients are calculated we use a new notation for Newton's form, namely p n (x) =y[x 0 ]+y[x 0 x ](x ; x 0 )+y[x 0 x x 2 ](x ; x 0 )(x ; x )+ ::: + y[x 0 x x 2 ::: x n ](x ; x 0 )(x ; x ) :::(x ; x n; ) (7.4) or, in compact notation, p n (x) = n j=0 Y j; y[x 0 x ::: x j ] (x ; x k ) : k=0 One important fact about this notation is that y[x 0 x x 2 ::: x n ] is the coecient of the highest power in p n (x), i.e., x n, and this coecient is independent of the ordering of the points. Notation: We use square brackets to denote the coecients so that we do not confuse y(x j ), which is the value of the function y = y(x) at x = x j with y[x j ], which is the coecient of the constant term in p n (x) = a 0 + a (x ; x j )+a 2 (x ; x j )(x ; x k )+ :::. Similarly, y(x j x k ) would be the value of a function of two variables, whereas y[x j x k ] is the coecient of the term x ; x j in p n (x) when the rst two abscissas are x j and x k. Example 7. (continued). [Message:exam:[app: 4pts]cexam] (cont. from page 477 cont. on page 489) We now show how to calculate these coecients for eq. (7.3a). Once we calculate then, it will be clear how to obtain the general procedure. First, we let x = x 0 = so that Second, we let x = x =3 so that y[] = p 3 () = 8 : y[] + y[ 3](2) = p 3 (3) = 36 and y[ 3] = 4. Third, we let x = x 2 =4 so that y[] + y[ 3](3) + y[ 3 4](3) = p 3 (4) = 32 and y[ 3 4] = ;6. Finally, welet x = x 3 =5 so that y[] + y[ 3](4) + y[ 3 4](8) + y[ 3 4 5](8) = p 3 (5) = 0 and y[ 3 4 5] = ;2. We can now obtain a simple recursive procedure. To do so we work with an arbitary ordering of the points, and so write p 3 (x) =y[x 0 ]+y[x 0 x ](x ; x 0 )+y[x 0 x x 2 ](x ; x 0 )(x ; x ) + y[x 0 x x 2 x 3 ](x ; x 0 )(x ; x )(x ; x 2 ) : First, we let x = x 0 so that y[x 0 ]=p 3 (x 0 )=y 0 :
7.2. Polynomial Interpolation 479 We nowdene y[x j ]=y j for j 2N [ 3] since a dierent ordering of the nodes gives this result. (For example, if we order the abscissas as x 3 x 2 x x 0, then we obtain y[x 3 ]=y 3.) Second, we let x = x so that y[x 0 ]+y[x 0 x ](x ; x 0 )=p 3 (x )=y and We dene y[x 0 x ]= y ; y[x 0 ] x ; x 0 = y[x ] ; y[x 0 ] x ; x 0 : y[x j x k ]= y[x k] ; y[x j ] x k ; x j for all j 6= k because a dierent ordering of the abscissas gives ; this result. (Again, if we order the abscissas as x 3 x 2 x x 0, then we obtain y[x 3 x 2 ] = y[x 3 ] ; y[x 2 ] =(x 3 ; x 2 ).) DO WE NEED THIS PARENTHETICAL STATEMENT??? Third, we let x = x 2 so that y[x 0 ]+y[x 0 x ](x 2 ; x 0 )+y[x 0 x x 2 ](x 2 ; x 0 )(x 2 ; x )=p 3 (x 2 )=y 2 and, with a little algebra, y[x 0 x x 2 ]= y 2 ; y 0 ; y[x 0 x ](x 2 ; x 0 ) (x 2 ; x 0 )(x 2 ; x ) = = y 2 ; y 0 ; y ; y 0 (x x ; 2 ; x 0 ) x 0 (x 2 ; x 0 )(x 2 ; x ) y 2 ; y 0 ; y ; y 0 x 2 ; x 0 x 2 ; x x ; x 0 x 2 ; x x 2 ; x 0 y 2 ; y ; y ; y 0 x = 2 ; x x ; x 0 x 2 ; x 0 = y[x x 2 ] ; y[x 0 x ] : x 2 ; x 0 We dene y[x j x k x l ]= y[x k x l ] ; y[x j x k ] x j ; x l for distinct j k l : (7.5) Eq. (7.5) is quite suggestive. We will not derive it here (because the algebra is quite messy) but, in fact, y[x 0 x x 2 x 3 ]= y[x x 2 x 3 ] ; y[x 0 x x 2 ] x 3 ; x 0 (as we will prove shortly in Theorem 7.2). A schematic is shown in Table 7. which makes the simple nature of the recursion obvious.
480 Ch. 7. Approximations of Functions x 0 : y 0 = y[x 0 ] x : y = y[x ] b y[x0 x " ] b y[x x 2 ] " x 2 : y 2 = y[x 2 ] b y[x2 " x " 3 ] x 3 : y 3 = y[x 3 ] b y[x0 x " x 2 ] b y[x x 2 x 3 ] b y[x0 x " x 2 x 3 ] 4:32=32 b 8;32 " ;4 =8 :8 =8 b ;2;8 " 5;4 b = ;0 0;8 " 5; = ;2 " b ;8;(;0) 3;4 = ;2 5:0 =0 b ;8;(;2) " 3; = ;8 b 36;0 " 3;5 = ;8 3:36=36 Table 7.. A schematic of a divided dierence table using four points and the actual data points which generated eq. (7.3b). We have made plausible the fact that there is a simple recursive formula for the coecients in Newton's form of p n (x), eq. (7.4). Now we prove it. Theorem 7.2. Let the n+ points be given by (x i y i ) i 2N [0 n] where all the abscissas are distinct. Let p n 2 n be the unique polynomial which interpolates these points. Then the coecients of p n (x) using eq. (7.4) are calculated by the recursive formula y[x j x j+ ::: x j+k ] = y[x j+ x j+2 ::: x j+k ] ; y[x j x j+ ::: x j+k; ] x j+k ; x j for j 2N [0 n; k] (7.6a) where k = 2 ::: n. The recursion begins with y[x j ]=y j for j 2N [0 n] : (7.6b) Proof: Without loss of generality we can assume that j = 0. (If it is not we simply reorder the.) We prove this theorem by induction. It is obviously points by (x j+i y j+i ) i 2N [0 n; j] true for n = by the same argument as in Example 7.. So we assume it is true for n 2N [ N;] and we prove itfor n = N. Let p N; 2 N; interpolate the points (x i y i ) i 2N [0 N ; ], let q N; 2 N; interpolate the points (x i y i ) i 2N [ N], and dene p N 2 N by p N (x) = x ; x 0 x N ; x 0 q N; (x)+ x N ; x x N ; x 0 p N; (x) : (7.7) We now show that p N interpolates all N+ points. First, p N (x 0 )=p N; (x 0 )=y 0. Next, for
7.2. Polynomial Interpolation 48 i 2N [ N ; ] p N (x i )= x i ; x 0 x N ; x 0 q N; (x i )+ x N ; x i x N ; x 0 p N; (x i ) = x i ; x 0 x N ; x 0 y i + x N ; x i x N ; x 0 y i = y i : Finally, p N (x N )=q N; (x N )=y N. All that remains is to equate the coecients of the term x N on both sides of eq. (7.7). One fact which we want to again emphasize is that in p n (x) the coecient onthe term x n is y[x 0 x ::: x n ] and that this coecient is independent of the order in which we assign the points. We have already demonstrated why this is true in Example 7.. Again, the reason is that y[x 0 x ::: x n ] is the coecient ofthe highest-order term in p n (x), i.e., x n. EPLAIN BETTER??? Note also that this means that it is symmetric in all the elements. That is, a reordering of the abscissas does not change its value. SAY BETTER??? The numerical algorithm for the calculation of the coecients of p n (x) in Newton's form is quite simple. In Fortran it is DO 0 I = 0,N A(I) = Y(I) 0 CONTINUE DO 30 I =,N DO 20 K = N,I,- A(K) = ( A(K) - A(K-) ) / ( (K) - (K-I) ) 20 CONTINUE 30 CONTINUE Algorithm 7.3. The coecients in Newton's form of p n (x). where (I) = x I, Y(I) = y I, and, on output, A(I) = a I. The cost of this algorithm is n 2 & = 2 n 2, which is quite reasonable. We can then use Horner's method, Algorithm 7.2, to evaluate p n (x) for any given x. In subsection 7.2.5 we will show how to calculate derivatives of p n (x). Let x min = min k x k and x max = max k x k. If x 2 [x min x max ] then we say that we are interpolating the data. If x =2 [x min x max ] then we say that we areextrapolating the data. We will show in the next subsection that polynomial interpolation can be stable or unstable, depending on the particular data and on n. (However, as long as n is \small" it is never unstable. CHECK THIS???) We will also show that polynomial extrapolation is always dangerous. There are occasions when we will use extrapolation. However we will make sure that n is \small" and we will usually do a second procedure to improve the accuracy. SAY BETTER??? EAMPLE: ABM So far we have shown how to calculate the interpolating polynomial and how to evaluate it. We next discuss how accurately the polynomial approximates the underlying function. HOW STABLE IS THIS NUMERICALLY??? ANALYTICALLY IT IS THE SAME AS US- ING THE LAGRANGE BASIS??? 7.2.3. The Errors in the Interpolating Polynomial There is no way to determine the error in the interpolating polynomial p n if we are simply given the data, as has been the case so far in this chapter. The reason for this is exactly the same as occurs in achievement andintelligence tests when they ask a multiple choice question such as the following:
482 Ch. 7. Approximations of Functions What number should come next in the sequence 3 5 7 :::? The correct answer, although it is never one of the choices, is any number. The proof is quite simple. The quartic polynomial p 4 (x)=+2(x; ) + 0(x ; )(x ; 2) + 0(x ; )(x ; 2)(x ; 3) + ; 9 (x ; )(x ; 2)(x ; 3)(x ; 4) 24 takes on the values p 4 () =, p 4 (2) = 3, p 4 (3) = 5, p 4 (4) = 7, and p 4 (5) = for any 2 R. Thus, we have found a deterministic sequence p 4 (k) k = 2 3 :::, based on a \simple" function, which has the values 3 5 7 for any real number. Of course, the right answer (according to the test makers) is 9, which is obtained by using the polynomial p (x) = +2(x;). p is certainly a \simpler" function than p 4. However, this extra assumption is never included in the test question and so any answer should be allowed. y Thus, if we are simply given a set of data points, then there is no error in calculating the interpolating polynomial since it passes through all the points (except for round-o errors). In addition, there are an innite numberofcontinuous functions y = y(x) that we can devise to also go through these data points. Thus, the error kp n ; yk is completely indeterminate, where we are free to use any norm. We will generally use the -norm. However, suppose that we start with a \simple" and \smooth" function y = y(x) and we use it to generate our data, so that y k = y(x k ) for k 2 N [0 n]. Then we want to know how accurately p n (x) approximates y(x). Of course, we do not expect p n (x) to be a very good approximation everywhere. Practically,we usually restrict our attention to points z 2 [x min x max ], where x min = min k x k and x max =max k x k. However, theoretically this is not necessary. Theorem 7.3. Let y 2 C n+ (I) for some open interval I R, let x i i 2N [0 n] I, and let y i = y(x i ) for all i. For every z 2 I there exists a = (z) 2 ; minfz x min g maxfz x max g such that y(z) ; p n (z) = n(z) y (n+) () (n +)! : (7.8) Proof: Dene e n y ; p n and note that e n (x j )=0 for all j 2N [0 n]. Thus we can dene a function r 2 C(I) by r(x) = y(x) ; p n(x) n (x) where n (x) = Q n k=0 (x ; x k). Note that r(x) is indeterminate at x = x k since we have 0=0. These are removable singularities and so we dene r(x k ) by lim x!xk r(x). The error in the interpolating polynomial is given by e n (x) =y(x) ; p n (x) = n (x)r(x) and we dene a function closely related to e n (x) ; n (x)r(x) by (x) y(x) ; p n (x) ; r(z) n (x) : (7.9) Note that has at least n+2 zeroes in minfz x min g maxfz x max g since (x j ) = 0 for j 2 N [0 n] and, in addition, (z) =0. By Rolle's theorem (see Appendix I) 0 (x) has at least y The authors dislike multiple \guess" tests intensely.
7.2. Polynomial Interpolation 483 n+ zeroes in ; minfz x min g maxfz x max g. Continuing this argument, 00 (x) has at least n zeroes in ; minfz x min g maxfz x max g, etc. Finally, we can conclude that (n+) (x) has at least one zero, say at = (z) 2 ; minfz xmin g maxfz x max g. Thus, y (n+) () ; p (n+) n () ; r(z) (n+) n () =0: Since p n is a polynomial of degree n we have p (n+) n y (n+) () ; r(z)(n +)!=0 and substituting for r(z) in eq. (7.9) we obtain the desired result. (x) 0 also, (n+) (x) =(n + )! Thus, Several words of caution about this result are necessary. The function n (z) grows rapidly in absolute value if z =2 [x min x max ]. For example, we plot 3 for the abscissas 3 4 5 in Figure 7.2. Note that 3 (0) > 50 even though 3 () = 0. In Figure 7. we have shown the corresponding Lagrange polynomials, which are not nearly as large. Note that the vertical axes are very dierent for these two gures. The rapid growth outside the interval [x min x max ] is certainly evident in this gure it is much more evident forlarger n. Consequently, it is undesirable to use interpolating polynomials to extrapolate the data points since the error can be very large. Occasionally, wedo use extrapolation but we are very careful. SAY BETTER??? EAMPLE??? n Figure 7.2. The behavior of 3 (x) using the same abscissas as in Example 7. and Figures 7.. Note that the vertical axes are very dierent. Even when z 2 [x min x max ], the error kp n ; yk can be large. This can occur because n (z)y (n+) () can grow faster than n! for the \right" choice of the underlying function y and of the abscissas. We will discuss this growth in detail shortly. Now let us return to one of the fundamental principles of scientic computing, namely that a discretized solution should approach the true continuous solution as the discretization goes to zero. Otherwise, scientic computing has no rm ground to stand on. Suppose that the solution to the continuum problem is y 2 C (a b) and that we obtain f y k g as the solution to the discretized problem. We then construct p n (x) from (x k y k ) k 2N [0 n]. We expect p n (x)! y(x) as n! for all x 2 [x min x max ]. However, as the following example shows, this need not occur.
484 Ch. 7. Approximations of Functions Example 7.2. Runge's example The \canonical" example of the growth of kp n ; yk as n! is Runge's example, f(x) = +25x 2 for x 2 [; ]: Suppose that the n+ abscissas are equally spaced in this interval. We can show that f (n+) = O(5 n n!) and also k n k only decays slowly (see Problem 9). Thus, the error grows unboundedly as n!. See Experiment??? where we do this exact problem. SAY BETTER??? That is, for the \wrong" function and/or the \wrong" distribution of abscissas this is an illposed method (IS ILL-POSED THE RIGHT WORD???) to approximate the continuous solution. Note that f(x) = =( + 25x 2 ) might be the \wrong" function, but it is certainly a reasonable function to want to interpolate. Thus, we will need a dierent approach to obtain convergence. 7.2.4. Osculatory Polynomial Interpolation Up until now wehave assumed that the abscissas were all distinct. However, suppose we know more information about the data than just the values at particular abscissas. Suppose we also know the values of some derivatives at particular abscissas. For example, suppose we not only know the values at the abscissas but we also know the values of the rst derivatives at these abscissas. How can we use this additional information to calculate our interpolating polynomial now? Example 7.3. Specifying rst derivatives Suppose that we want to calculate the unique cubic polynomial p 3 which takes on specied values and rst derivatives at two abscissas x A and x B where, for simplicity, we require that x A <x B. That is, p 3 (x A )=y A, p 0 3(x A )=ya 0, p 3(x B )=y B, and p 0 3(x B )=y B. The notation might be slightly confusing. By p 0 3(x) we, of course, mean dp 3 (x)=dx. However, by ya 0 we do notmeanthederivative of y because we have no underlying function. Instead, this symbol merely denotes the desired slope at x A. We knowhow to calculate p 3 if all the abscissas are distinct by using divided dierences. We want to use the same recursive formulas and numerical algorithms as before (of course, with slight modications), and not have to use a completely dierent form to calculate p 3 or to evaluate p 3 (z). Thus, we will calculate p 3 by rst using divided dierences with four distinct abscissas and we will use a limiting procedure to obtain the desired polynomial. Suppose that our data comes from a function y 2 C (I) for some open interval I R where [x A x B ] I. Then our data is ; xa y(x A ) ; x A + h y(x A + h) ; x B y(x B ) ; x B + h y(x B + h) (7.20) and we represent the abscissas by f x 0 x x 2 x 3 g[a b] where x 0 = x A, x = x A +h, x 2 = x B, and x 3 = x B + h. First, however we will need some notation for our values and their derivatives. The set of points (7.20) is suggestive. If we simply let h =0 then ; x A y(x A ) appears twice (and the same for the other point at x B ). We have lost a crucial piece of information at x A.
7.2. Polynomial Interpolation 485 That is, y(x A + h) y(x A )+hy 0 (x A ) and y 0 (x A ) is the data that we lose when h =0. Thus, when h =0 we write our points as (xa y A ) (x A y 0 A) (x B y B ) (x B y 0 B) or, using our standard notation, (x0 y 0 ) (x 0 y 0 0) (x 2 y 2 ) (x 2 y 0 2) : (7.2) Thus, we use repeated abscissas with the corresponding values of the ordinates being y 0, y0, 0 y0 00, etc. Note that this means that the data is discontinuous. When h 6= 0 we use the set of points (7.20) but when h =0 we use the set of data (7.2). When h 6= 0 our polynomial is p 3 (x h) =y[x 0 ]+y[x 0 x ](x ; x 0 )+y[x 0 x x 2 ](x ; x 0 )(x ; x ) + y[x 0 x x 2 x 3 ](x ; x 0 )(x ; x )(x ; x 2 ) where x = x 0 + h and x 3 = x 2 + h. When h =0 our polynomial is p 3 (x) =y[x 0 ]+y[x 0 x 0 ](x ; x 0 )+y[x 0 x 0 x 2 ](x ; x 0 ) 2 + y[x 0 x 0 x 2 x 2 ](x ; x 0 ) 2 (x ; x 2 ) where we expect p 3 (x) =lim h!0 p 3 (x h) for all x. Although the data which is used to calculate p 3 (x h) is dierent than the data which is used to calculate p 3 (x), (as we will see) the coecients of p 3 (x h) approach the coecients of p 3 (x) as h! 0. It is annoying to have discontinuous data, but it makes the analysis, and particularly the numerical algorithm which implements this, much simpler. SAY THIS BETTER??? Using divided dierences and our new notation, we begin with y[x 0 ] = y 0 and y[x 2 ] = y 2. Now, y[x 0 x ]= y(x ) ; y(x 0 )! y x ; 0 (x 0 ) as h! 0 x 0 so that we should set y[x 0 x 0 ]=y 0 0 and, also, y[x 2 x 2 ]=y 0 2 : Next, Thus, and, also, Finally, y[x 0 x x 2 ]= y[x x 2 ] ; y[x 0 x ] x 2 ; x 0! y[x 0 x 2 ] ; y[x 0 x 0 ] x 2 ; x 0 as h! 0 : y[x 0 x 0 x 2 ]= y[x 0 x 2 ] ; y[x 0 x 0 ] x 2 ; x 0 y[x 0 x 2 x 2 ]= y[x 2 x 2 ] ; y[x 0 x 2 ] x 2 ; x 0 : y[x 0 x x 2 x 3 ]= y[x x 2 x 3 ] ; y[x 0 x x 2 ] x 3 ; x 0! y[x 0 x 2 x 2 ] ; y[x 0 x 0 x 2 ] as h! 0 x 2 ; x 0 so we have found the desired coecients. Table 7.2 presents a schematic of these formulas. We do divided dierences normally on y[x i x i+ ::: x i+k ] unless all the elements have the same value. If they do, we use the appropriate derivative. For example, y[x 0 x 0 x 2 ] merely uses divided dierences on y[x 0 x 0 ] and y[x 0 x 2 ]. However, to calculate y[x 0 x 0 ] we simply read o y 0 (x 0 ).
486 Ch. 7. Approximations of Functions x 0 : y 0 = y[x 0 ] A" y[x 0 x 0 ] x 0 : y0 0 A " b y[x 0 x 0 x 2 ] y[x " 0 x 2 ] x 2 : y 2 = y[x 2 ] x 2 : y 0 2 b y[x0 x " 2 x 2 ] " y[x 2 x 2 ] b y[x0 x " 0 x 2 x 2 ] Table 7.2. A schematic for calculating the coecients of p 3 (x) when rst derivatives are also specied. Since we can use this technique for any number of derivatives, we will repeat it here for one other example, namely (x0 y 0 ) (x y ) (x y 0 ) (x y 00 ) : Table 7.3 shows the schematic in this case. We will not prove that it is correct. (We leave that to Problem 0.) It follows immediately from Theorem 7.4, which we will prove shortly. SAY BETTER??? x 0 : y 0 = y[x 0 ] b y[x0 x " ] x : y = y[x ] " y[x x ] b y[x0 x " x ] x : y 0 " y[x x x ] x : y 00 " b y[x0 x " x x ] Table 7.3. Aschematic for calculating the coecients of p 3 (x) when a second derivative isspeci- ed. In the above example we have made one fundamental assumption, namely that y[x 0 ::: x k ] is a continuous function of its arguments. That is, suppose we have n+ distinct points again, but (to be perverse) we denote the rst one by, i.e., = x 0. Then Now suppose we let! x k p n (x) =y[]+y[ x ](x ; )+y[ x x 2 ](x ; )(x ; x )+ ::: + y[ x x 2 ::: x n; x n ](x ; )(x ; x ) :::(x ; x n; ) : for some k 2 N [ n]. Then we expect the coecients to vary in a continuous fashion as long as the data points originated from a \smooth enough" function (since the limits must exist). That is, if two abscissas are the same then we require y 2 C [x min x max ] if three abscissas are the same then we require y 2 C 2 [x min x max ] etc. We will not prove this assumption now. It is done in gory detail in Theorem 8.. However, it is not needed in the theorem below, even though it makes the theorem easier to understand. We now show that this limiting process really works. First, we reiterate that we now are using a special ordering for our data (x k y k ). Notation: We require that x i x i+ for all i 2N [0 n; ]. Also, if x i = x i+ = ::: = x i+j are all the abscissas with the same value, then y i+l = y (l) i for l 2N [0 j].
7.2. Polynomial Interpolation 487 Theorem 7.4. Let the n+ points be given by (x i y i ) i 2N [0 n] where the data is given in the format just described. Let p n 2 n be the unique polynomial which interpolates the data. Then the coecients of p n (x) using eq. (7.4) are calculated by the recursive formula y[x j x j+ ::: x j+k ] = 8 >< >: y[x j+ x j+2 ::: x j+k ] ; y[x j x j+ ::: x j+k; ] x j+k ; x j y j+k k! = y(k) j k! for j 2N [0 n; k] where k =0 2 ::: n. The recursion begins with if x j+k 6= x j if x j+k = x j (7.22a) y[x j ]=y j if j =0 or x j; 6= x j : (7.22b) Proof: Without loss of generalitywe can assume that j =0 since, if it is not, simply let the data points be (x j+i y j+i ) i 2N [0 n; j]. We prove this theorem by induction. The theorem is true for n = since we have already shown in Example 7.3 that y[x 0 x ]= 8 >< >: y[x ] ; y[x 0 ] x ; x 0 if x 6= x 0 y0 0! if x = x 0. Now assume it is true for n 2N [ N ; ] and we will prove itfor n = N. Let us get the case x 0 = x = ::: = x N out of the way rst. In this case p N (x) =y 0 + y0 0! (x ; x 0)+ y00 0 2! (x ; x 0) 2 + ::: + y(n) 0 N! (x ; x 0) N (7.23) so that obviously y[x 0 x ::: x N ]=y (N) =N! =y N =N! Now we assume that x N 6= x 0. Let p N; 2 N; interpolate the data (xi y i ) i 2N [0 N ; ] and q N; 2 N; interpolate the data (x i y i ) i 2N [ N] and dene p N 2 N by p N (x) = x ; x 0 x N ; x 0 q N; (x)+ x N ; x x N ; x 0 p N; (x) : Note: To calculate p N; and q N; we use two sequences of N data points from our sequence of N+ data points, and we have tobecarefulhowweinterpret these sequences. For example, suppose N =4 and our sequence is (x 0 y 0 ) (x 0 y0) 0 (x 0 y0 00 ) (x 3 y 3 ) (x 3 y3) 0. Then p 3 is calculated from the sequence (x 0 y 0 ) (x 0 y0) 0 (x 0 y0 00 ) (x 3 y 3 ), and q 3 is calculated from the sequence (x 0 y 0 ) (x 0 y0) 0 (x 3 y 3 ) (x 3 y3) 0. Because of this diculty, wehave to consider many more cases than in Theorem 7.2. We nowshowthat p N interpolates all the data. Suppose that we have a distinct abscissa x k. That is, either k =0 and x >x 0, k = N and x N; <x N, or x k; <x k <x k+. The proof is immediate in this case by exactly the same argument as in the proof of Theorem 7.2. We have just handled the case when an abscissa is distinct. Thus, we now need only consider abscissas which are not distinct. Let x i = x i+ = ::: = x i+j be all the abscissas with this value when j>0. We will show that p N (x i+l )=y (l) i for l =0 ::: j: (7.24)
488 Ch. 7. Approximations of Functions We have to consider a number of cases. First, we prove eq. (7.24) when l =0. If i =0 then p N (x 0 )=p N; (x 0 )=y 0. If i>0 then p N; (x i )=q N; (x i )=y i. We have just handled the case when l =0. Thus, we now need only consider l>0, so we need to calculate p (l) N (x). It is p (l) N (x) = x ; x 0 q x N ; (l) N; x (x)+ x N ; x p 0 x N ; (l) q(l;) N; (x) ; p(l;) N; N; (x)+l (x) : x 0 x N ; x 0 If i =0 then p (l) N (x l)=p (l) N; (x l)=y l since q (l;) N; (x l)=p (l;) N; (x l) and x = x 0. If i > 0 and i + l < N p (l;) N; (x i+l). Finally, if i + l = N it is again true since p (l) x = x N. it is again true since q (l) N; (x i+l) = p (l) N; (x i+l) and q (l;) N; (x i+l) = The Fortran code which does all this is the following. N (x N) = q (l) N; (x N), q (l;) N; (x N)=p (l;) N; (x N), and DO 0 I = 0,N A(I) = Y(I) 0 CONTINUE DO 30 I =,N ALAST = A(I-) DO 20 K = I,N IF ( (K-I).NE. (K) ) THEN TEMP = ( A(K) - ALAST ) / ( (K) - (K-I) ) ALAST = A(K) A(K) = TEMP ELSE A(K) = A(K)/I ENDIF 20 CONTINUE 30 CONTINUE Algorithm 7.4. The calculation of the coecients in Newton's form of p n (x) when the abscissas can be repeated. This is a generalization of Algorithm 7.3. Note that this algorithm is a generalization of Algorithm 7.3. However, Algorithm 7.3 was written in a particularly simple form so the two do not appear to be as similar as they really are. You should always use Algorithm 7.4 since you never know when you might want to calculate an osculatory polynomial. There is a slight overhead in this algorithm because of the extra temporary variables and the IF test. However, this is very small compared to the versatility this algorithm allows. Warning: We must be very careful when using Algorithm 7.4 with repeated abscissas. For example, suppose that x = x 2 = x 3. Then, analytically the IF test \IF ( (K-I).NE. (K) )" in the algorithm is false when K = 2 and K = 3. However, numerically it might not be unless all three abscissas are calculated by exactly the same formula. Thus, before using this algorithm it is wise to include the lines of code (2) = () (3) = () or DO 0 I = 2,3 (I) = () 0 CONTINUE
7.2. Polynomial Interpolation 489 In this way we are guaranteed that the IF tests are false. CHECK THIS OVER AND MAKE IT CLEARER WHY THERE MIGHT BE A PROBLEM??? 7.2.5. Calculating Derivatives of Polynomials We have now completed our discussion of how to calculate p n (x) in Newton's form. However, one question still remains. Since it is dicult to dierentiate p n (x) in Newton's form, how can we easily calculate derivatives of this polynomial at specied points, i.e., p (j) n (z)? To see this diculty, simply dierentiate p 3 (x) = 8 + 4(x ; ) ; 6(x ; )(x ; 3) ; 2(x ; )(x ; 3)(x ; 4) from Example 7.. Even calculating p 0 3(x) is quite dicult, and the result is no longer in Newton form. It would be easy to calculate derivatives if p n (x) was written in the power form (7.8) or the shifted power form (7.0). It would be P particularly easy if the in the shifted power form was n exactly z. That is, suppose p n (x) = i=0 a j(x ; z) j. Then p (j) n (z) =j! a j for j 2N [0 n]. We can go part way in converting from Newton's form to the shifted power form in the following way. Suppose we change the abscissas from f x 0 x x 2 ::: x n; x n g to f z x 0 x x 2 ::: x n;2 x n; g. Then it is trivial to calculate p n (z) because p n (x) =y[z]+y[z x 0 ](x ; z)+y[z x 0 x ](x ; z)(x ; x 0 ) + y[z x 0 x x 2 ](x ; z)(x ; x 0 )(x ; x )+ ::: + y[z x 0 x ::: x n;2 x n; ](x ; z)(x ; x 0 )(x ; x ) :::(x ; x n;2 ) and so p n (z) = y[z]. Next, suppose we further change the abscissas to f z z x 0 x ::: x n;3 x n;2 g. Then it is trivial to calculate p 0 n(z) because p n (x) =y[z]+y[z z](x ; z)+y[z z x 0 ](x ; z) 2 + y[z z x 0 x ](x ; z) 2 (x ; z 0 )+ ::: + y[z z x 0 ::: x n;3 x n;2 ](x ; z) 2 (x ; x 0 ) :::(x ; x n;3 ) and so p 0 n(z) = y[z z]. If we want to evaluate p 00 n(z) we need only operate once more on the abscissas to obtain f z z z x 0 ::: x n;4 x n;3 g and then p 00 n(z) =2!y[z z z]. Obviously, wecan continue this procedure ad innitum. All that remains is to nd an algorithm which changes the abscissas in this way. An example will make this algorithm clear. Example 7. (continued). [Message:exam:[app: 4pts]cexam] (cont. from page 478) Consider the cubic polynomial p 3 (x) =y[x 0 ]+y[x 0 x ](x ; x 0 )+y[x 0 x x 2 ](x ; x 0 )(x ; x ) which we will write for simplicity as + y[x 0 x x 2 x 3 ](x ; x 0 )(x ; x )(x ; x 2 ) p 3 (x) =a 0 + a (x ; x 0 )+a 2 (x ; x 0 )(x ; x ) + a 3 (x ; x 0 )(x ; x )(x ; x 2 ) : (7.25)
490 Ch. 7. Approximations of Functions We will show the steps needed in transformating the abscissas from f x 0 x x 2 x 3 g to f z x 0 x x 2 g. The procedure is quite simple. Begin with the last term on the right-hand side of eq. (7.25), and rewrite x ; x 2 as (x ; z)+(z ; x 2 ). Transfer z ; x 2 to the third term of p 3 (x) where it modies the coecient a 2. Next, in the third term rewrite x ; x as (x ; z)+(z ; x ) and transfer z ; x to the second term where it modies a. Finally, rewrite x ; x 0 in the second term as (x ; z)+(z ; x 0 ) and transfer z ; x 0 to the rst term where it modies a 0. All these steps are shown explicitly below. p 3 (x) =a 0 + a (x ; x 0 )+a 2 (x ; x 0 )(x ; x ) + a 3 (x ; x 0 )(x ; x ) (x ; z)+(z ; x 2 ) = a 0 + a (x ; x 0 )+ a 2 + a 3 (z ; x 2 ) (x ; x 0 )(x ; x ) + b 3 (x ; z)(x ; x 0 )(x ; x ) where b 3 = a 3 = a 0 + a (x ; x 0 )+b 2 (x ; x 0 ) (x ; z)+(z ; x ) + b 3 (x ; z)(x ; x 0 )(x ; x ) where b 2 = a 2 + b 3 (z ; x 2 ) = a 0 + a + b 2 (z ; x ) (x ; x 0 )+b 2 (x ; z)(x ; x 0 ) + b 3 (x ; z)(x ; x 0 )(x ; x ) = a 0 + b (x ; z)+(z ; x0 ) + b 2 (x ; z)(x ; x 0 ) + b 3 (x ; z)(x ; x 0 )(x ; x ) where b = a + b 2 (z ; x ) = a 0 + b (z ; x 0 ) + b (x ; z)+b 2 (x ; z)(x ; x 0 ) + b 3 (x ; z)(x ; x 0 )(x ; x ) = b 0 + b (x ; z)+b 2 (x ; z)(x ; x 0 ) The recursion relation for the coecients is + b 3 (x ; z)(x ; x 0 )(x ; x ) where b 0 = a 0 + b (z ; x 0 ) b 3 = a 3 b j = a j + b j+ (z ; x j ) for j =2 0 : This procedure is quite similar to Algorithm 7.2. The dierence is that in the algorithm the coecients are not saved. Only b 0 is known when the algorithm nishes. Thus, in general we switch the coecients from to by p n (x) =a 0 + a (x ; x 0 )+a 2 (x ; x 0 )(x ; x )+ ::: + a n (x ; x 0 )(x ; x ) :::(x ; x n; ) p n (x) =b 0 + b (x ; z)+b 2 (x ; z)(x ; x 0 )+ ::: + b n (x ; z)(x ; x 0 ) :::(x ; x n;2 ) b n = a n b j = a j + b j+ (z ; x j ) for j = n ; n; 2 ::: 0 :
7.2. Polynomial Interpolation 49 We will not prove that this recursion relation is true in general, but it is easily done by induction. The algorithm which is used to calculate the new coecients is a slight modication of Algorithm 7.2, namely B(N) = A(N) DO 20 I = N-,0,- B(I) = (Z - (I))*B(I+) + A(I) 20 CONTINUE Algorithm 7.5. Modifying the coecients of p n (x) from the abscissas x 0 x ::: x n; x n (array A) to z x 0 ::: x n;2 x n; (array B). Each time this algorithm is called, z is added at the left end of the set of abscissas and the rightmost abscissa is removed. Note that if we only want to calculate p n (z), then the new array B(I) is not needed, and we can simply use Algorithm 7.2. However, to calculate p (j) n (z) for j 2 N [ n] we must use Algorithm 7.5. To calculate p (j) n (z) we simply apply Algorithm 7.5 j+ times and read o the answer as p (j) n (z) =j! b j : The complete algorithm follows where J 0 is the desired derivative. IF ( J.GT. N ) THEN PJZ = 0. ELSE DO 0 I = 0,N B(I) = A(I) 0 CONTINUE FACTRL =. DO 30 JD = 0,J FACTRL = FACTRL*MA(JD, ) DO 20 I = N-,0,- IF ( I.GE. JD ) THEN B(I) = (Z - (I-JD))*B(I+) + B(I) ENDIF 20 CONTINUE 30 CONTINUE PJZ = FACTRL*B(J) ENDIF Algorithm 7.6. The calculation of p (j) n (z) for j 2 N [0 ). The result is stored in PJZ. CHECK THIS ALGORITHM OVER???. The cost of using Algorithm 7.6 is 2n & n Since it takes j+ calls to Algorithm 7.5 to calculate p (j) n (z), the total cost of evaluating the j-th derivative of the interpolating polynomial is 2n(j +) & n(j +), which is quite acceptable. 7.2.6. Uses of Polynomial Interpolation NEVILLE'S ALGORITHM IS A WAY TOEVALUATE A SET OF DATA POINTS AT ONE POINT REVERSE AND Y TO SOLVE F() = 0??? Exercise 7.2. (a) Show that n is a linear space. (b) Show that the set of all polynomials of degree exactly n is not a linear space unless n =0.
492 Ch. 7. Approximations of Functions 5. (P) 2. Let p n (x) =a 0 + a x + :::a n x n and consider the following algorithm for evaluating p n (x) for a given z. PZ = A(0) ZZ =. DO 0 I =,N ZZ = Z*ZZ PZ = PZ + A(I)*ZZ 0 CONTINUE Algorithm 7.7. An alternative to Horner's method for evaluating p n (z). On output, p n (z) =PZ. It is not commonly used because it is slower. (a) Determine the cost of evaluating p n (z) by Algorithm 7.7 and compare it with the cost of using Horner's method, Algorithm 7.. (b) Determine the stability of this algorithm. For simplicity, assume that f a k g and z can be stored exactly in the computer. Thus, you only need to consider the errors in doing the arithmetical operations. (c) Repeat part (b) for Horner's method. 3. Evaluate the polynomial p(x) = x 5 +2x 4 +3x 2 + x ; at x = 3 by hand. Do this in two ways. First, calculate p(3) directly (that is, evaluate each term separately so 2x 4 is evaluated as 23333, etc.). Second, use Horner's method. Determine how many additions and multiplications are required for each method. 4. Apply the algorithm for derivatives of newton polynomial form to compute p 0 (;2) if p(x)=+(x ; )[2 + (x ; 2)[3 + (x ; 3)[4]]] : This is, use the algorithm to rewrite p(x) as c 0 +(x +2)[c +(x + 2)[c 2 +(x ; )[c 3 ]]]. Let p 0 (x) = (x ; ) 0 and consider the evaluation of this polynomial for x 2 [0 2]. Choose z 2 [0 2] which can be stored exactly in the computer and determine the actual round-o errors (consider both absolute and relative errors) using both Algorithm 7.7 and Horner's methods. Are the actual round-o errors \approximately" equal to those you calculated in Problem 2? Hint: You do not need to consider all z 2 [0 2]. Instead, explain why the interval [0 2] can be divided into a small number of subintervals in which the round-o characteristics p 0(z) can be expected to be \approximately" equal. Then choose a few values from each subinterval and compare \theory and experiment". 6. Prove that f[x 0 x ::: x n ]= n k=0 f(x k ) 0 n(x k ) : Hint: Find the leading coecient of the polynomial expressed in Lagrange form. 7. Use Problem 6 to calculate where x 2 x 3 ::: x n are kept xed. Then calculate lim f[x 0 x ::: x n ] x!x 0 lim x 2!x!x 0 f[x 0 x x 2 ] :