Curve Fitting and Approximation

Transcription

1 Department of Physics Cotton College State University, Panbazar Guwahati , Assam December 6, 2016

2 Outline Curve Fitting 1 Curve Fitting

3 Outline Curve Fitting 1 Curve Fitting

4 Topics in the Syllabus In this Chapter 1 Curve Fitting by Least Square Methods: Fitting a Straight Line Non-Linear Curve Fitting (a) Power Function (b) Polynomial of nth Degree (c) Exponential Function 2 Linear Weighted Least Square Approximation 3 Orthogonal Polynomials, Gram-Schmidt Orthogonalization Process 4 Least-Squares Solution 5 Approximation of Functions-Chebyshev Polynomials

5 Method of Least Squares The method of least square states that A curve is a best fit if the sum of the square of the deviations of the individual points from the curve is minimum. Let us consider a set of tabulated data points (x i, y i ), where i = 1, 2,..., m. Here x i s are the values of independent variables and y i s are the corresponding dependent variables. Let y = f (x) be a function which approximates the given data y i, corresponding to the values of x i s. The deviations between the actual tabulated values of y i s and the values of y i s obtained from the approximation y = f (x) are given by d 1 = y 1 f (x 1 ) d 2 = y 2 f (x 2 ). d m = y m f (x m )

6 Method of Least Squares The sum of the squares of the deviations is given by S = d d d 2 m = m d 2 i. Minimization of S is obtained from the condition δs δa j = 0 (1) where a j s are the coefficients of the approximating function. The values of a j s are obtained by solving the set of equations resulting from Equation (1).

7 Fitting a Straight Line Given a set of data points (x i, y i ), i = 1, 2,..., m, the task is to find a polynomial P 1 (x) of degree 1 (straight line): P 1 (x) = a 0 + a 1 x which minimizes the the sum of the squared deviations : S = m [y i P 1 (x i )] 2 = m [y i a 0 a 1 x i ] 2, where a 0 and a 1 represent the intercept and the slope of the straight line, respectively. To determine the values of a 0, a 1, we have to use the minimization condition: δs δa 0 = 0, δs δa 1 = 0 (2)

8 Fitting a Straight Line δs δa 0 = 2 δs δa 1 = 2 m (y i a 0 a 1 x i ) m (y i a 0 a 1 x i )(x i ) Imposition of the conditions of Equation (2) yields: m a 0 ma 0 + a 1 x i + a 1 m m x i = x 2 i = m y i m x i y i These equations are called Normal Equations.

9 Fitting a Straight Line Let us apply Cramer s rule to solve the above two equations: yi xi xi y i x 2 a 0 = i yi x 2 = i x i xi y i m xi m xi 2 ( x i ) 2 xi x 2 i a 1 = m m xi xi yi xi y i xi x 2 i = m x i y i x i yi m x 2 i ( x i ) 2

10 Example Curve Fitting Fit a straight line to the x and y values in the first two columns of the following Table: x i y i xi 2 x i y i P 1 (x i ) xi = 55 yi = 81.0 x 2 i = 385 xi y i = S 2.34 S = [P 1 (x i ) y i ] 2

11 Example Curve Fitting Let the straight line be given by P 1 (x) = a 0 + a 1 x, where a 0, a 1 are calculated to be yi x 2 a 0 = i x i xi y i m xi 2 ( 385(81) 55(572.4) x i ) 2 = 10(385 (55) 2 = ) a 1 = m x i y i x i yi m xi 2 ( 10(572.4) 55(81) x i ) 2 = 10(385) (55) 2 = Therefore, the least square (or the best) straight line fit is P 1 (x) = x

12 Example Curve Fitting Figure 1: Filled circles denote the tabulated data points, whereas, the solid line shows the straight line fit to the given data.

13 Least Squares Polynomial Fitting Given a set of data points (x i, y i ), i = 1, 2,..., m, the task is to find a polynomial P n (x) of degree n < m: P n (x) = a 0 + a 1 x + + a n x n which minimizes the the sum of the squared deviations : S = m [y i P n (x i )] 2 = m [ yi (a 0 + a 1 x i + a 2 xi a n xi n ) ] 2, For S to be minimum, we must have δs δa j = 0, j = 0, 1, 2,..., n

14 Least Squares Polynomial Fitting δs δa 0 = 2 δs δa 1 = 2. δs δa n = 2 m (y i a 0 a 1 x i a n xi n ) m (x i )(y i a 0 a 1 x i a n xi n ) m (xi n )(y i a 0 a 1 x i a n xi n ) Setting these equations to zero, i.e., δs δa 0 = 0, δs = 0, δs = 0, δa 1 δa n

15 Least Squares Polynomial Fitting we get: ma 0 + a 1 m m a 0 m a 0 x i + a 2 x i + a 1 m m x n i + a 1 m m xi a n xi n = x 2 i + + a n m x n+1 i + + a n m x n+1 i =. x 2n i = m y i m x i y i m xi n y i

16 Least Squares Polynomial Fitting Let us now set m xi k = s k, k = 0, 1,..., 2n and denoting the right hand side entries as b 0, b 1,..., b n, the above equations can be written as: s 0 a 0 + s 1 a s n a n = b 0 s 1 a 0 + s 2 a s n+1 a n = b 1. s n a 0 + s n+1 a s 2n a n = b n This is a system of (n + 1) equations in (n + 1) unknowns a 0, a 1,..., a n. These equations are called Normal Equations.

17 Least Squares Polynomial Fitting This system now can be solved to obtain these (n + 1) unknowns, provided a solution to the system exists. The system has a unique solution if x i s are distinct. The system can be written in the following matrix form: s 0 s 1... s n a 0 b 0 s 1 s 2... s n+1 a 1 b 1... s n s n+1... s 2n. a n =. b n or sa = b s 0 s 1... s n a 0 b 0 s 1 s 2... s n+1 where s =..., a = a 1., b = b 1. s n s n+1... s 2n a n b n

18 Least Squares Polynomial Fitting Let us define 1 x 1 x x n 1 1 x 2 x x n 2 V = x m xm 2... xm n It follows that the normal equations can be written as a 0 y 1 V T a 1 V. = V T y 2. a n y m The matrix V is known as the Vandermonde matrix.

19 Example 1: Linear Polynomial Fit x i y i The linear polynomial is: P 1 (x) = a 0 + a 1 x. The Vandermonde matrix is : 1 x x 2 V = 1 x 3 1 x 4 = x V T V = [ ] , V T = 1.00 [ ]

20 Example 1: Linear Polynomial Fit Therefore, the normal equations are [ ] [ ] [ ] a = a [ ] [ ] a = Therefore, the linear polynomial fit is a 1 P 1 (x) = x

21 Example 2: Quadratic Polynomial Fit x i y i xi The quadratic polynomial is: P 2 (x) = a 0 + a 1 x + a 2 x 2. The Vandermonde matrix is : 1 x 1 x x 2 x2 2 V = 1 x 3 x3 2 1 x 4 x4 2 = x 5 x V T V = , V T =

22 Quadratic Polynomial Fit Therefore, the normal equations are: Therefore, the quadratic polynomial fit is: a 0 a 1 a 2 a 0 a 1 a = = P 2 (x) = x x 2

23 Linearization (For Non-Polynomial Fits)

24 Non-Polynomial Fits Exponential Fit: Let y = ae bx Taking the natural logarithm on both sides log y = log[ae bx ] = log(a) + }{{}}{{}}{{} b x Y a 0 a 1 The problem therefore reduces to finding a least squares straight line through the given data points: (x 1, log y 1 ), (x 2, log y 2 ),..., (x m, log y m ). Then, Power Law Fit: Let a = e a 0 and b = a 1 y = ax b

25 Non-Polynomial Fits Taking the natural logarithm on both sides log y = log[ax b ] = log(a) + }{{}}{{}}{{} b log x }{{} Y a 0 a 1 X The problem therefore reduces to finding a least squares straight line through the given data points: (log x 1, log y 1 ), (log x 2, log y 2 ),..., (log x m, log y m ). Then, a = e a 0 and b = a 1

26 Example 1: Exponential Fit [y = ae bx ] Fit a function of the form y = ae bx to the following data: x i y i The given relation is y = ae bx. Taking natural logarithm on both sides, log y = log a + }{{}}{{}}{{} b }{{} x Y a 0 a 1 X Thus, if we set log y = Y, log a = a 0, b = a 1, x = X, the above relation takes the form Y = a 0 + a 1 X, which is a straight line. We know that a 0 and a 1 are given by a 0 = Yi X 2 i m X 2 X i Xi Y i i ( X i ) 2, a 1 = m Xi Y i X i Yi m Xi 2 ( X i ) 2 Now, we make the following Table:

27 Example 1: Exponential Fit [y = ae bx ] X = x Y = log y X 2 XY X = 30 Y = X 2 = 220 XY = (17.025)(220) (30)( ) a 0 = 5(220) (30 2 ) 5 ( ) (30) (17.025) a 1 = 5(220) (30 2 ) = = Hence a = e a 0 = e = 1.499, and b = a 1 = y = ae bx = 1.499e 0.5x

28 Example 2: Exponential Fit [y = ae bx ] x i y i log(y i ) The linear fit is: log y = log a + }{{}}{{}}{{} b x Y a 0 a 1 1 x x 2 The Vandermonde matrix is: V = 1 x 3 1 x 4 = x

29 Example 2: Exponential Fit [y = ae bx ] V T V = [ ] , V T = Therefore, the normal equations are: [ ] [ ] [ ] a = a [ ] [ ] a = a 1 [ ] Therefore, y = e x

30 Example 1: Power Law Fit [y = ax b ] x i y i log(x i ) log(y i ) The linear fit is: log y = log a + }{{}}{{}}{{} b log x }{{} Y a 0 a 1 X The Vandermonde matrix is: 1 log(x 1 ) log(x 2 ) V = 1 log(x 3 ) 1 log(x 4 ) = log(x 5 )

31 Example 1: Power Law Fit [y = ax b ] V T V = [ ] , V T = Therefore, the normal equations are: [ ] [ ] [ ] a = a [ ] [ ] a = a 1 [ ] Therefore, y = P(x) = x

32 Linear Weighted Least Square Approximation Let P 1 (x) = a 0 + a 1 x be the straight line to be fitted to the given data points (x i, y i ), i = 1, 2,..., m.then, S(a 0, a 1 ) = m W i [y i P 1 (x)] 2 = m W i [y i a 0 a 1 x] 2, where W i s are the prescribed positive numbers and are called weights. A weight is prescribed according to the relative accuracy of a given data point. If all the data points are accurate, we set W i = 1, i. For minima: δs δa 0 δs δa 0 = 2 δs δa 1 = 2 = δs δa 1 = 0, which give m W i [y i a 0 a 1 x i ] = 0 m W i [y i a 0 a 1 x i ]x i = 0

33 Linear Weighted Least Square Approximation Simplifying the above system of equations, we get m a 0 m a 0 W i + a 1 W i x i + a 1 m m W i x i = W i x 2 i = m W i y i m W i x i y i These are the Normal Equations in this case and can be solved to obtain a 0 and a 1.

34 Method of Least Squares Using Orthogonal Polynomials Definition: The set of functions {φ i } n i=0 is called a set of orthogonal functions, with respect to a weight function w(x) if { b 0, if i j w(x)φ i (x)φ j (x)dx = C i, if i = j a Where C i is a positive real number. Furthermore, if C i = 1, i = 0, 1,..., n, then the orthogonal set is called an orthonormal set. Idea: The idea is to find an approximation of f (x) on [a, b] by means of a polynomial of the form P n (x) = n a i φ i (x), i=0 where {φ i } n i=0 is a set of orthogonal polynomials. That is, the basis for generating P n (x) in this case is a set of orthogonal polynomials.

35 Method of Least Squares Using Orthogonal Polynomials Theorem If {φ 0, φ 1,..., φ n } is a set of orthogonal functions on an interval [a, b] with respect to the weight functions w, then the least square approximation to f on [a, b] with respect to w is P n (x) = where, for each j = 0, 1,..., n, n a j φ j (x), j=0 a j = b a w(x)φ j(x)f (x)dx b a w(x)[φ j(x)] 2 dx = 1 C j b a w(x)φ j (x)f (x)dx

36 Method of Least Squares Using Legendre s Polynomials The Legendre Polynomials {φ i } given by φ 0 (x) = 1 φ 1 (x) = x φ 2 (x) = x φ 3 (x) = x x. etc. are orthogonal polynomials on [ 1, 1] with respect to the weight function w(x) = 1.

37 Legendre s Polynomials Question: Find linear and quadratic least squares approximation to f (x) = e x using Legendre polynomials. Answer: Linear Approximations: Let P 1 (x) = a 0 φ 0 (x) + a 1 φ 1 (x), where φ 0 (x) = 1, φ 1 (x) = x. Therefore, C 0 = 1 1 φ2 0 (x)dx = 1 1 dx = [x]1 1 = 2 a 0 = φ 0(x)e x dx = 1 2 [ex ] 1 1 = 1 2 (e 1 e ) C 1 = 1 1 φ 1(x) 2 dx = 1 1 x 2 dx = [ x3 3 ]1 1 = 2 3 a 1 = 1 1 C 1 1 φ 1(x)f (x) = xex dx = 3 2 [ 2 e ] = 3 e. The linear least-squares polynomial is: P 1 (x) = a 0 φ 0 (x) + a 1 φ 1 (x) = 1 2 [e 1 e ] + 3 e x Accuracy Check: P 1 (0.5) = 1 2 [e 1 e ] + 3 e 0.5 = and e 0.5 = So, Relative error= =

38 Legendre s Polynomials Quadratic Approximations: Let P 2 (x) = a 0 φ 0 (x) + a 1 φ 1 (x) + a 2 φ 2 (x), where φ 0 (x) = 1, φ 1 (x) = x, φ 2 = x a 0 = 1 2 (e 1 e ), a 1 = 3 e C 2 = 1 1 φ 2(x) 2 dx = 1 1 (x )2 dx = So a 2 = 1 1 C 2 1 φ 2(x)f (x)dx = ex (x )dx = e 7 e Therefore, quadratic least-squars polynomial is: P 2 (x) = 1 2 (e 1 e ) + 3 e x + (e 7 e )(x ) Accuracy Check: P 2 (0.5) = and e 0.5 = So, Relative error= =

39 Gram-Schmidt Orthogonalization Process Theorem: The set of polynomial functions φ 0, φ 1,..., φ n defined in the following way is orthogonal on [a, b] with respect to the weight function w: φ 0 (x) = 1, φ 1 (x) = x B 1, for each in [a, b], where, And when k 2, B 1 = b a xw(x)[φ 0(x)] 2 dx b a w(x)[φ 0(x)] 2 dx φ k (x) = (x B k )φ k 1 (x) C k φ k 2 (x), for each x in [a, b] where, b a B k = xw(x)[φ k 1(x)] 2 dx b a w(x)[φ k 1(x)] 2 dx and C k = b a xw(x)φ k 1(x)φ k 2 (x)dx b a w(x)[φ k 2(x)] 2 dx

40 Legendre s Polynomial Using GS-Orthogonalization Process The above Theorem provides a recursive procedure for constructing a set of polynomials. The set of Legendre Polynomials, {φ n (x)} is orthogonal on [ 1, 1] with respect to the weight function w(x) = 1. The classical definition of the Legendre polynomials requires that φ n (1) = 1 for each n, and a recursive relation is used to generate the polynomials when n 2. Using the Gram-Schmidth orthogonalization process with φ 0 (x) = 1 gives, B 1 = 1 1 xdx 1 1 dx = 0 and φ 1(x) = x B 1 = x 0 = x Also, B 2 = 1 1 x 3 1 dx 1 1 x 2 dx = 0 and C 1 2 = x 2 dx 1 1 dx = 1 3 So, φ 2 (x) = (x B 2 )φ 1 (x) C 2 φ 0 (x) = (x 0)x = x 2 1 3

41 Legendre s Polynomial Using GS-Orthogonalization Process Now, B 3 = 1 1 xφ 2(x) 2 dx 1 1 φ 2(x) 2 dx = 1 1 x(x )2 1 1 (x )dx = 0 C 3 = 1 1 xφ 2(x)φ 1 (x)dx 1 1 φ 1(x) 2 dx = 1 1 x(x )dx 1 1 x 2 dx = 4 15 Thus, φ 3 (x) = xφ 2 (x) 4 15 φ 1(x) = x x 4 15 x = x x The next two Legendre polynomials are: φ 4 (x) = x x and φ 5(x) = x x x

42 Chebyshev Polynomials The Chebyshev Polynomials {T n (x)} are orthogonal on [ 1, 1] with respect to the weight function w(x) = The set of polynomials defined by 1 1 x 2. T n (x) = cos(n cos 1 x), n 0 on [ 1, 1] are called the Chebyshev polynomials. It may be noted that T 0 (x) = cos 0 = 1 and T 1 (x) = cos[cos 1 x] = x

43 Recurrence Relation Curve Fitting Let us substitute θ = cos 1 x. Then, T n (x) = cos(nθ), 0 θ π T n+1 (x) = cos(n + 1)θ = cos nθ cos θ sin nθ sin θ T n 1 (x) = cos (n 1)θ = cosnθ cos θ + sin nθ sin θ Adding the last two equations, we obtain T n+1 (x) + T n 1 (x) = 2 cos nθ cos θ T n+1 (x) = 2 cos nθ cos θ T n 1 (x) T n+1 (x) = 2x cos(n cos 1 x) T n 1 (x) T n+1 (x) = 2xT n (x) T n 1 (x), n 1 Therefore, T n+1 (x) = 2xT n (x) T n 1 (x), n 1 Using this recursive relation, the Chebyshev polynomials of the successive degress can be generated.

44 Recurrence Relation Curve Fitting Generation of Chebyshev Polynomials The recurrence relation is: T n+1 (x) = 2xT n (x) T n 1 (x), n 1 n = 1 : T 2 (x) = 2xT 1 (x) T 0 (x) = 2x 2 1 n = 2 : T 3 (x) = 2xT 2 (x) T 1 (x) = 4x 3 3x n = 3 : T 4 (x) = 2xT 3 (x) T 2 (x) = 8x 4 8x n = 4 : T 5 (x) = 2xT 4 (x) T 3 (x) = 16x 5 20x 3 + 5x n = 5 : T 6 (x) = 2xT 5 (x) T 4 (x) = 326x 6 48x x 2 1 And so on. It is easy to see that the coefficient of x n in T n (x) is always 2 n 1.

45 Plots of Chebyshev Polynomials

46 The Orthogonal Property 1 1 T m (x)t n (x) dx = 1 x 2 0 if m n π 2 if m = n 0 π if m = n = 0 The least-squares approximating polynomial P n (x) of f (x) using Chebyshev polynomials is given by: where, P n (x) = C 0 T 0 (x) + C 1 T 1 (x) + + C n T n (x) C i = 2 π and C 0 = 1 π f (x)t i (x) 1 x 2, f (x)dx 1 x 2 i = 1, 2,..., n

47 Problems on Chebyshev Polynomials Find a linear least-squares approximation of f (x) = e x using Chebyshev polynomials. Let P 1 (x) = C 0 T 0 (x) + C 1 T 1 (x) = C 0 + C 1 x, where C 0 = 1 π C 1 = 2 π e x dx 1 x xe x 1 x Thus, P 1 (x) = x

48 Internal Assessment-I (Submission on:) Fit a straight line of the form y = a 0 + a 1 x to the data: x i y i Find the values of a 0, a 1 and a 2 : so that y = a 0 + a 1 x + a 2 x 2 is the best fit to the data: x i y i Fit a function of the form y = ax b to the following data: x i y i

49 Internal Assessment-I (Submission on:) Fit an exponential function of the type y = ae bx to the following data: x i y i

50 A Quote Curve Fitting Imagination begets innovations! Imagination is more important than knowledge. Albert Einstein.

51 Contact Me? Curve Fitting For any queries, suggestions: k sukantodeb@gmail.com T (In case of urgency)