Intro Polynomial Piecewise Cubic Spline Software Summary. Interpolation. Sanzheng Qiao. Department of Computing and Software McMaster University

Transcription

1 Interpolation Sanzheng Qiao Department of Computing and Software McMaster University January, 2014

2 Outline 1 Introduction 2 Polynomial Interpolation 3 Piecewise Polynomial Interpolation 4 Natural Cubic Spline 5 Software Packages

3 Outline 1 Introduction 2 Polynomial Interpolation 3 Piecewise Polynomial Interpolation 4 Natural Cubic Spline 5 Software Packages

4 Introduction Problem setting: Given (x 0, y 0 ),(x 1, y 1 ),...,(x n, y n ), x 0 < x 1 < x n, for example, a set of measurements, construct a function f : f(x i ) = y i, i = 0, 1,..., n Desirable properties of f : smooth: analytic and f (x) not too large (the first and second derivatives are continuous). simple: polynomial of minimum degree, easy to evaluate.

5 Example Measurements of the speed of sound in ocean water speed (ft/sec) depth (ft)

6 Example Interpolation speed (ft/sec) depth (ft)

7 Outline 1 Introduction 2 Polynomial Interpolation 3 Piecewise Polynomial Interpolation 4 Natural Cubic Spline 5 Software Packages

8 Polynomial Interpolation Advantages: easy to evaluate and differentiate Weierstrass Approximation Theorem: If f is any continuous function on the finite closed interval [a,b], then for every ǫ > 0 there exists a polynomial p n (x) of degree n = n(ǫ) such that max f(x) p n(x) < ǫ. x [a,b]

9 Polynomial Interpolation Advantages: easy to evaluate and differentiate Weierstrass Approximation Theorem: If f is any continuous function on the finite closed interval [a,b], then for every ǫ > 0 there exists a polynomial p n (x) of degree n = n(ǫ) such that max f(x) p n(x) < ǫ. x [a,b] Impractical (degree is often too high)

10 A straightforward approach A polynomial of degree n is determined by its n + 1 coefficients.

11 A straightforward approach A polynomial of degree n is determined by its n + 1 coefficients. Given (x 0, y 0 ),...,(x n, y n ) to be interpolated, we construct the linear system (Vandermonde matrix): 1 x 0 x0 n a 0 y 0 1 x 1 x1 n a 1 y x n xn n. a n solve for the coefficients of the polynomial =. y n p n (y)(x) = a 0 + a 1 x + + a n x n

12 Horner s rule Evaluating the polynomial: a 0 x 3 + a 1 x 2 + a 2 x + a 3 Horner s form: ((a 0 x + a 1 )x + a 2 )x + a 3

13 Horner s rule Evaluating the polynomial: a 0 x 3 + a 1 x 2 + a 2 x + a 3 Horner s form: ((a 0 x + a 1 )x + a 2 )x + a 3 v = a(0); for (i = 1:n) v = v*x + a(i); end

14 Horner s rule Evaluating the polynomial: a 0 x 3 + a 1 x 2 + a 2 x + a 3 Horner s form: ((a 0 x + a 1 )x + a 2 )x + a 3 v = a(0); for (i = 1:n) v = v*x + a(i); end The optimal (most efficient and accurate) way of evaluating a 0 x n a n.

15 Vandermonde matrix When x 0,..., x n are distinct, the Vandermonde matrix is nonsingular. Thus the system has a unique solution (coefficients of the interpolating polynomial).

16 Vandermonde matrix When x 0,..., x n are distinct, the Vandermonde matrix is nonsingular. Thus the system has a unique solution (coefficients of the interpolating polynomial). Example. Given three points (28, ) and (30, ), (32, ) we have the system and the solution a 0 a 1 a 2 = p 2 (31) = sin(30 ) a 0 a 1 a 2 =

17 Vandermonde matrix problem: The coefficient (Vandermonde) matrix is often ill-conditioned

18 Vandermonde matrix problem: The coefficient (Vandermonde) matrix is often ill-conditioned question What is the condition number of the Vandermonde matrix constructed by x i = i, i = 0, 1,..., 7?

19 Vandermonde matrix problem: The coefficient (Vandermonde) matrix is often ill-conditioned question What is the condition number of the Vandermonde matrix constructed by x i = i, i = 0, 1,..., 7? Answer:

20 Lagrange form (conceptually simple) Basis polynomials: {l j (x)} (j = 0, 1,..., n) of degree n such that l j (x i ) = { 1, if i = j 0, otherwise construct l j (x) = i j x x i x j x i Thus p n (y)(x) = n l j (x)y j j=0

21 Example Given three points: (28, ), (30, ), (32, ), construct a second degree interpolating polynomial in the Lagrange form: p 2 (x) = (x 30)(x 32) (28 30)(28 32) (x 28)(x 32) + (30 28)(30 32) (x 28)(x 30) + (32 28)(32 30) p 2 (31) = sin(31 )

22 Example Given three points: (28, ), (30, ), (32, ), construct a second degree interpolating polynomial in the Lagrange form: p 2 (x) = (x 30)(x 32) (28 30)(28 32) (x 28)(x 32) + (30 28)(30 32) (x 28)(x 30) + (32 28)(32 30) p 2 (31) = sin(31 ) Expensive to evaluate.

23 Dangers of polynomial interpolation An example. Runge s function (continuous derivatives of all order) 1 y(x) = x 2 on [ 1, 1] equally spaces x 0 = 1, x 1,, x n = 1 1 Equal Spacing (n = 13)

24 Dangers of polynomial interpolation An example. Runge s function (continuous derivatives of all order) 1 y(x) = x 2 on [ 1, 1] equally spaces x 0 = 1, x 1,, x n = 1 1 Equal Spacing (n = 13) It is often best not to use global polynomial interpolation.

25 Outline 1 Introduction 2 Polynomial Interpolation 3 Piecewise Polynomial Interpolation 4 Natural Cubic Spline 5 Software Packages

26 Piecewise Polynomial Interpolation Given the partition α = x 1 < x 2 < < x n = β, interpolate on each [x i, x i+1 ] with a low degree polynomial.

27 Piecewise Polynomial Interpolation Given the partition α = x 1 < x 2 < < x n = β, interpolate on each [x i, x i+1 ] with a low degree polynomial. Linear L i (z) = a i + b i (z x i ), z [x i, x i+1 ] a i = y i, b i = y i+1 y i x i+1 x i, 1 i n 1

28 Algorithm. Piecewise linear interpolation Given vectors x and y with interpolating points, this function returns the piecewise linear interpolation coefficients in the vectors a and b. function [a,b] = pwl(x,y) n = length(x); a = y(1:n-1); b = diff(y)./diff(x);

29 Evaluation Given the piecewise linear interpolation L(z) represented by the coefficient vectors a, b, how do we evaluate this function at z [α,β]? First, we locate [x i, x i+1 ] such that z [x i, x i+1 ]. Then, we evaluate L(z) using L i (z). Search method: binary search, since x i are sorted.

30 Evaluation Given the piecewise linear interpolation L(z) represented by the coefficient vectors a, b, how do we evaluate this function at z [α,β]? First, we locate [x i, x i+1 ] such that z [x i, x i+1 ]. Then, we evaluate L(z) using L i (z). Search method: binary search, since x i are sorted. Observation: If [x i, x i+1 ] is associated with the current z, then it is likely that this subinterval will be the one for the next value.

31 Algorithm. Locate Idea: Use the previous subinterval as a guess. If not, do binary search. Given the vector x of breakpoints and a scalar z between x 1 and x n, this function locates i so that x i z x i+1. The optional g is a guess. function i = Locate(x,z,g) if nargin==3 % try the guess if (x(g)<=z)&(z<=x(g+1)) i = g; return % quick return end end

32 Algorithm. Locate (cont.) n = length(x); if z==x(n) i = n-1; % quick return else % binary search left = 1; right = n; while right > left+1 mid = floor((left + right)/2); if z < x(mid) right = mid; else left = mid; end end i = left; end

33 Algorithm. pwleval Given a piecewise linear interpolation coefficient vectors a and b from pwl and its breakpoints in x, this function returns the values of the interpolation evaluated at the points in z. function v = pwleval(a,b,x,z) m = length(z); v = zeros(m,1); g = 1; for j=1:m i = Locate(x,z(j),g); v(j) = a(i) + b(i)*(z(j) - x(i)); g = i; end

34 Example y = 1 (x 0.3) (x 0.9) Interpolation of humps(x) with PWL, n =

35 Outline 1 Introduction 2 Polynomial Interpolation 3 Piecewise Polynomial Interpolation 4 Natural Cubic Spline 5 Software Packages

36 Problem setting Given (x 1, y 1 ),(x 2, y 2 ),...,(x n, y n ), find s(x): in each subinterval [x i, x i+1 ], s(x) is cubic

37 Problem setting Given (x 1, y 1 ),(x 2, y 2 ),...,(x n, y n ), find s(x): in each subinterval [x i, x i+1 ], s(x) is cubic s(x i ) = y i, i = 1,..., n

38 Problem setting Given (x 1, y 1 ),(x 2, y 2 ),...,(x n, y n ), find s(x): in each subinterval [x i, x i+1 ], s(x) is cubic s(x i ) = y i, i = 1,..., n s (x) and s (x) are continuous at x 2, x 3,..., x n 1

39 Problem setting Given (x 1, y 1 ),(x 2, y 2 ),...,(x n, y n ), find s(x): in each subinterval [x i, x i+1 ], s(x) is cubic s(x i ) = y i, i = 1,..., n s (x) and s (x) are continuous at x 2, x 3,..., x n 1 s (x 1 ) = s (x n ) = 0 The second derivative of s(x) is zero at the end points means that s(x) is linear at the end points.

40 A straightforward approach Suppose a i + b i x + c i x 2 + d i x 3 on [x i, x i+1 ], i = 1,..., n 1. 4(n 1) unknowns to be determined.

41 A straightforward approach Suppose a i + b i x + c i x 2 + d i x 3 on [x i, x i+1 ], i = 1,..., n 1. 4(n 1) unknowns to be determined. Interpolation: a i + b i x i + c i xi 2 + d i xi 3 = y i, i = 1,..., n 1 a i + b i x i+1 + c i xi d ixi+1 3 = y i+1, i = 1,..., n 1

42 A straightforward approach Suppose a i + b i x + c i x 2 + d i x 3 on [x i, x i+1 ], i = 1,..., n 1. 4(n 1) unknowns to be determined. Interpolation: a i + b i x i + c i xi 2 + d i xi 3 = y i, i = 1,..., n 1 a i + b i x i+1 + c i xi d ixi+1 3 = y i+1, i = 1,..., n 1 Continuous first derivative (consider [x i 1, x i ] and [x i, x i+1 ]): b i 1 + 2c i 1 x i + 3d i 1 xi 2 = b i + 2c i x i + 3d i xi 2, i = 2,..., n 1

43 A straightforward approach Suppose a i + b i x + c i x 2 + d i x 3 on [x i, x i+1 ], i = 1,..., n 1. 4(n 1) unknowns to be determined. Interpolation: a i + b i x i + c i xi 2 + d i xi 3 = y i, i = 1,..., n 1 a i + b i x i+1 + c i xi d ixi+1 3 = y i+1, i = 1,..., n 1 Continuous first derivative (consider [x i 1, x i ] and [x i, x i+1 ]): b i 1 + 2c i 1 x i + 3d i 1 xi 2 = b i + 2c i x i + 3d i xi 2, i = 2,..., n 1 Continuous second derivative: 2c i 1 + 6d i 1 x i = 2c i + 6d i x i, i = 2,..., n 1

44 A straightforward approach Suppose a i + b i x + c i x 2 + d i x 3 on [x i, x i+1 ], i = 1,..., n 1. 4(n 1) unknowns to be determined. Interpolation: a i + b i x i + c i xi 2 + d i xi 3 = y i, i = 1,..., n 1 a i + b i x i+1 + c i xi d ixi+1 3 = y i+1, i = 1,..., n 1 Continuous first derivative (consider [x i 1, x i ] and [x i, x i+1 ]): b i 1 + 2c i 1 x i + 3d i 1 xi 2 = b i + 2c i x i + 3d i xi 2, i = 2,..., n 1 Continuous second derivative: 2c i 1 + 6d i 1 x i = 2c i + 6d i x i, i = 2,..., n 1 Two end conditions: 2c 1 + 6d 1 x 1 = 0 and 2c n 1 + 6d n 1 x n = 0

45 A straightforward approach Suppose a i + b i x + c i x 2 + d i x 3 on [x i, x i+1 ], i = 1,..., n 1. 4(n 1) unknowns to be determined. Interpolation: a i + b i x i + c i xi 2 + d i xi 3 = y i, i = 1,..., n 1 a i + b i x i+1 + c i xi d ixi+1 3 = y i+1, i = 1,..., n 1 Continuous first derivative (consider [x i 1, x i ] and [x i, x i+1 ]): b i 1 + 2c i 1 x i + 3d i 1 xi 2 = b i + 2c i x i + 3d i xi 2, i = 2,..., n 1 Continuous second derivative: 2c i 1 + 6d i 1 x i = 2c i + 6d i x i, i = 2,..., n 1 Two end conditions: 2c 1 + 6d 1 x 1 = 0 and 2c n 1 + 6d n 1 x n = 0 Total of 4(n 1) equations, a dense system.

46 A clever approach: Constructing s(x) In the subinterval [x i, x i+1 ], let h i = x i+1 x i and introduce new variables: w = (x x i )/h i, w = 1 w. Note: w(x i ) = 0, w(x i+1 ) = 1 and w(x i ) = 1, w(x i+1 ) = 0, (linear Lagrange polynomials). Thus wy i+1 + wy i is the (linear) Lagrange interpolation on [x i, x i+1 ].

47 A clever approach: Constructing s(x) In the subinterval [x i, x i+1 ], let h i = x i+1 x i and introduce new variables: w = (x x i )/h i, w = 1 w. Note: w(x i ) = 0, w(x i+1 ) = 1 and w(x i ) = 1, w(x i+1 ) = 0, (linear Lagrange polynomials). Thus wy i+1 + wy i is the (linear) Lagrange interpolation on [x i, x i+1 ]. Construct s(x) = wy i+1 + wy i + h 2 i [(w 3 w)σ i+1 + ( w 3 w)σ i ] where σ i to be determined, so that the properties (the first and second derivatives are continuous) are satisfied.

48 Properties of s(x) Using w = 1/h i and w = 1/h i, we can verify 1 s(x i ) = y i, s(x i+1 ) = y i+1, independent of σ, that is, s(x) interpolates (x i, y i ). 2 s (x) = 6wσ i wσ i, linear Lagrange interpolation at the points (x i, 6σ i ) and (x i+1, 6σ i+1 ). Clearly s (x i ) = 6σ i, which implies that s (x) is continuous.

49 Properties of s(x) Using w = 1/h i and w = 1/h i, we can verify 1 s(x i ) = y i, s(x i+1 ) = y i+1, independent of σ, that is, s(x) interpolates (x i, y i ). 2 s (x) = 6wσ i wσ i, linear Lagrange interpolation at the points (x i, 6σ i ) and (x i+1, 6σ i+1 ). Clearly s (x i ) = 6σ i, which implies that s (x) is continuous. Is s (x) continuous?

50 Properties of s(x) (cont.) It remains to determine σ i so that s (x) is continuous.

51 Properties of s(x) (cont.) It remains to determine σ i so that s (x) is continuous. Consider, on [x i, x i+1 ], s (x) = y i+1 y i h i + h i [(3w 2 1)σ i+1 (3 w 2 1)σ i ] Let i = (y i+1 y i )/h i. On [x i, x i+1 ], w(x i ) = 0 and w(x i ) = 1, s +(x i ) = i + h i ( σ i+1 2σ i ).

52 Properties of s(x) (cont.) On [x i 1, x i ], s (x) = y i y i 1 h i 1 + h i 1 [(3w 2 1)σ i (3 w 2 1)σ i 1 ] and w(x i ) = 1, w(x i ) = 0. Thus s (x i) = i 1 + h i 1 (2σ i + σ i 1 ).

53 Making s (x) continuous Setting we get n 2 equations: s +(x i ) = s (x i ), i = 2, 3,..., n 1, h i 1 σ i 1 + 2(h i 1 + h i )σ i + h i σ i+1 = i i 1 for i = 2, 3,..., n 1. Solve for σ 2,...,σ n 1, recalling that σ 1 = σ n = 0 (natural cubic spline).

54 Matrix form diagonal: [2(h 1 + h 2 ),, 2(h n 2 + h n 1 )] supper/subdiagonal: [h 2,, h n 2 ] unknowns: [σ 2,,σ n 1 ] T right-hand side: [ 2 1,, n 1 n 2 ] T

55 Matrix form diagonal: [2(h 1 + h 2 ),, 2(h n 2 + h n 1 )] supper/subdiagonal: [h 2,, h n 2 ] unknowns: [σ 2,,σ n 1 ] T right-hand side: [ 2 1,, n 1 n 2 ] T The matrix is symmetric tridiagonal diagonally dominant ( a i,i > j i a i,j ), when x 1 < x 2 < < x n, postive definite Can apply the Cholesky factorization, working on two vectors with O(n) operations.

56 Modeling a problem Note. Had we taken the straightforward approach to determining the coefficients of the piecewise cubic polynomials, four coefficients for each of n 1 cubic polynomials, we would have ended up with a large (4(n 1) 4(n 1)) and dense system requiring O(n 3 ) operations. Now we have an O(n) method.

57 Evaluating s(x) If s(x) is evaluated many times, arrange s(x) so that s(x) = y i + b i (x x i ) + c i (x x i ) 2 + d i (x x i ) 3 and rearrange it in the Horner s form, for x i x x i+1 and calculate and store b i, c i, d i (instead of σ i ) b i = y i+1 y i h i h i (σ i+1 + 2σ i ) for i = 1, 2,..., n 1 c i = 3σ i d i = σ i+1 σ i h i

58 Algorithm. Natural cubic spline ncspline Given a vector x with breakpoints and vector y with function values, this algorithm computes the coefficients b, c, d of natural spline interpolation. 1 Compute h i and i ; 2 Form the tridiagonal matrix (two arrays) and the right hand side; 3 Solve for σ i ; 4 Compute the coefficients b, c, and d.

59 Outline 1 Introduction 2 Polynomial Interpolation 3 Piecewise Polynomial Interpolation 4 Natural Cubic Spline 5 Software Packages

60 Software packages IMSL csint, csdec, csher, csval MATLAB polyfit, spline, ppval NAG e01aef, e01baf, e01bef, e02bbf, e01bff Octave interp1

61 Summary Polynomial interpolation: General idea and methods, Lagrange interpolation Piecewise polynomial interpolation: Construction of piecewise polynomial (linear and cubic), evaluation of a piecewise function, ncspline, seval

62 References [1 ] George E. Forsyth and Michael A. Malcolm and Cleve B. Moler. Computer Methods for Mathematical Computations. Prentice-Hall, Inc., Ch 4.