Engineering 7: Introduction to computer programming for scientists and engineers Interpolation Recap Polynomial interpolation Spline interpolation
Regression and Interpolation: learning functions from data
Regression and Interpolation: Regression Find a function, from a specified class, that best fits the data. For example, Training Data: x i, y i i=1,,n Interpolation Find a function, from a specified class, that exactly matches the data. For example, find with Often use a squared-error criterion to score a function s performance. Let denote the best-fit function. Let denote such an interpolator. The obtained function (f R or f I ) is often used for prediction: given another value of x (with y unknown), estimate/predict what is the corresponding value of y?
Perspectives: Differences/similarities Regression: Measured data is inaccurate in a random fashion. Data seems to exhibit complex variability, but that variability is not truly indicative of the underlying phenomena that produces the data Choose a model class of functions to represent the phenomena. It should be no more complex than necessary Get approximate fit from this simple class of functions. Interpolation: Measured data is believed to be accurate. 0-0.5-1 -1.5-2 0-0.5-1 -1.5 Regression 2 4 6 8 10 Interpolation 0-0.5-1 -1.5-2 Interpolation 0 2 4 6 8 10 Interpolation Any complex variability is thought to be truly -2-2 0 2 4 6 8 10 indicative of the underlying phenomena that produces the data 0 2 4 6 8 10 Functional representation should pass through all data points; no reason to expect wild variability between points. The smoothness of the interpolating function used depends on assumptions about the phenomena that produces the data. 0-0.5-1 -1.5
Regression or Interpolation? Accurate, fast approximation to f r 0 r e x 3/2 tan 1 x dx Interpolation: it makes sense that the fitting function should pass through the training points over the range r 0, 5. Data, r i, f i i=1,,n is obtained by careful numerical integration, is repeatable, and not subject to any appreciable error. Regression: the experimental aspect hints at errors and some degree of non-repeatability in the training data due to uncontrolled/unknown factors. Rebound velocity, as a function of impact velocity for a composite baseball bat and baseball, r(v). Data, v i, r i i=1,,n is obtained from experimental lab apparatus.
Polynomial Interpolation
Linear interpolation A linear (affine) polynomial has 2 parameters Given 2 data points, (x 1, y 1 ), (x 2, y 2 ) with the x 1 x 2, there exists a unique choice of parameters (a 0, a 1 ) so that p a (xi) = y i for i = 1, 2. 0.4 0.2 (x 2, y 2 ) 0-0.2-0.4 (x 1, y 1 ) -0.6-1.5-1 -0.5 0 0.5
quadratic interpolation A quadratic polynomial has 3 parameters Given 3 data points, (x 1, y 1 ), (x 2, y 2 ), (x 3, y 3 ), with the x i i=1,2,3 all distinct from one another, there exists a unique choice of parameters (a 0, a 1, a 2 ) so that p a (xi) = y i for i = 1, 2, 3. 0.5 (x 2, y 2 ) 0-0.5 (x 1, y 1 ) (x 3, y 3 ) -1 0.4 0.6 0.8 1 1.2
Cubic interpolation A cubic polynomial has 4 parameters Given 4 data points, (x 1, y 1 ), (x 2, y 2 ), (x 3, y 3 ), (x 4, y 4 ), with the x i i=1,2,3,4 all distinct from one another, there exists a unique choice of parameters (a 0, a 1, a 2, a 3 ) so that p a (xi) = y i for i = 1, 2, 3, 4. 1 0.5 (x 1, y 1 ) 0-0.5-1 -1.5 (x 3, y 3 ) (x 4, y 4 ) (x 2, y 2 ) 1 1.5 2 2.5 3 3.5
Polynomial interpolation More generally, an n th degree polynomial has n + 1 parameters Given n + 1 data points, x 1, y 1, x 2, y 2,, (x n+1, y n+1 ), with the x i i=1,2,,(n+1) all distinct from one another, there exists a unique choice of parameters (a 0, a 1, a 2,, a n ) so that p a (xi) = y i for i = 1, 2,, n + 1. 2 1.5 1 0.5 (x 1, y 1 ) (x 2, y 2 ) 0-0.5 (x n+1, y n+1 ) -1 1 2 3 4 5 6
Behavior of Polynomial Interpolation For large n, this polynomial tends to have large oscillations near the end points. The behavior of the interpolating function, between points is unexpected and non-intuitive. For evenly-spaced x, the prediction (ie., interpolating polynomial s value) at the midpoint between x 1 and x 2 becomes highly sensitive to y i= n 2. n 5 7 9 11 13 15 Sens 1.2 1.5 3.3 8.6 24 72 3.8 3.6 3.4 3.2 3 8 6 4 3 rd order poly through 4 points 2.8 0 1 2 3 4.5 4 3.5 3 2.5 0 1 2 3 80 60 40 7 th order poly through 8 points 15 th order poly through 16 points Polynomial interpolating functions using large n are generally thought of as bad predictors 2 0 11 th order poly through 12 points -2 0 1 2 3 20 0-20 0 1 2 3
Spline interpolation: linear and cubic
linear-spline Given: data, x i, y i i=1,,n. The (x i ) are distinct and sorted, so that x i < x i+1. A linear-spline interpolation consists of n 1 linear (affine) functions, defined separately on each interval [x i x i+1 ]. 1 st function connects (x 1, y 1 ) to (x 2, y 2 ) with a straight line, 2 nd function connects (x 2, y 2 ) to (x 3, y 3 ) with a straight line, k th function connects (x k, y k ) to x k+1, y k+1 with a straight line For X in [x k x k+1 ] interpolation function (relating X to Y) is easily expressed as (n-1) th function connects (x n 1, y n 1 ) to (x n, y n ) with a straight line. Viewed as a whole: the interpolation function is piecewise-linear
Linear-spline (or just linear ) interpolation y k 1 k Y yk X xk xk 1 xk y 1.4 1.2 Starting height Run 1 (x k+1, y k+1 ) Slope in this interval 0.8 (X, Y) 0.6 (x k, yk) 0.4 0.2 Data 0 1 2 3 4 5 6 7 8 9
Cubic-spline Given: data, x i, y i i=1,,n. The (xi) are distinct and sorted, so that x i < x i+1. A cubic-spline interpolation consists of cubic polynomial functions, defined on each interval [xi x i+1 ]. 1 st function connects (x 1, y 1 ) to (x 2, y 2 ) with cubic polynomial, 2 nd function connects (x 2, y 2 ) to (x 3, y 3 ) with cubic polynomial, k th function connects (x k, yk) to (x k+1, y k+1 ) with cubic polynomial, (n-1) th function connects (x n 1, y n 1 ) to (x n, yn) with cubic polynomial Constraints @ x 1 @ x k @ x n (2 k n 1) k th function is parametrized by 4 coefficients (to be determined by the constraints)
Cubic spline n 1 cubic polynomial functions, each defined on [xi x i+1 ]. 0.8 0.6 4(n 1) parameters to choose (x n,y n ) k th function connects (x k, yk) to (x k+1, y k+1 ) Constraints @ x 1 1 equation 0.4 0.2 0 f 1 (x 1,y 1 ) (x 2,y 2 ) f 2 f n-1-0.2 @ x k (2 k n 1) -0.4 f k-1 (x k,y k ) -0.6 4(n 2) equations -0.8 f k @ x n 1 equation -1 (x k+1,y k+1 ) -1.2 0 2 4 6 8 10 12 14 16 18 4 n 2 + 2 equations
Two Extra conditions As there are fewer equations that unknowns (coefficients of the cubic functions), the matching conditions (constraints) are not enough to uniquely determine the cubic functions. Two more conditions can be imposed. Four common approaches are Natural 0.8 0.6 0.4 Periodic 0.2 0-0.2 Not-a-knot -0.4-0.6-0.8 Specified end-slope, given values -1-1.2 0 2 4 6 8 10 12 14 16 18
Setting up the equations
Values and derivatives of cubic What are the values and derivative of a cubic
Values and derivatives of f k For f k, the value of x = x k, so
Constraints
Rewrite Constraints
Assembling the equations (n = 6) Two more conditions
Assembling the equations (arbitrary n) Cols= 3*(k-1)+1:3k+3 Rows= 3*(k-1)+1:3k Two more conditions
Incorporating extra conditions
Two Extra conditions As there are fewer equations that unknowns (coefficients of the cubic functions), the matching conditions are not enough to uniquely determine the cubic functions. Two more conditions can be imposed. Four common approaches are Natural 0.8 0.6 0.4 Periodic 0.2 0-0.2 Not-a-knot -0.4-0.6-0.8 Specified end-slope, given values -1-1.2 0 2 4 6 8 10 12 14 16 18
Values and derivatives of f k For f k, the value of x = x k, so
Extra equations for Natural spline Conditions for Natural Spline Expressions
Extra equations for Periodic spline Conditions for Periodic Spline Expressions
Extra equations for Not-a-knot Conditions for Not-a-knot spline Expressions
Extra equations for Specified end-slope Conditions for specified end-slope Expressions
Implementation Need one function to transform x i, y i i=1,,n into the coefficients a 0, a 1, a 2, a 3 i=1,,(n 1). Just build the arrays, and use \ (backslash) Need another function to evaluate the spline at an arbitrary value of x. This function needs the coefficients, a 0, a 1, a 2, a 3 i=1,,(n 1), the x i i=1,,n samples, and the x-values for which the evaluation should take place. The commands spline, ppval implement the ideas put forth here, in a more efficient manner. Only the not-a-knot and specified end-slope conditions are available.