Numerical Analysis MTH603

Transcription

1 Numerical Analysis Course Contents Solution of Non Linear Equations Solution of Linear System of Equations Approximation of Eigen Values Interpolation and Polynomial Approximation Numerical Differentiation Numerical Integration Numerical Solution of Ordinary Differential Equations Introduction We begin this chapter with some of the basic concept of representation of numbers on computers and errors introduced during computation. Problem solving using computers and the steps involved are also discussed in brief. Number (s) System (s) In our daily life, we use numbers based on the decimal system. In this system, we use ten symbols,,,9 and the number is called the base of the system. Thus, when a base N is given, we need N different symbols,,,,(n ) to represent an arbitrary number. The number systems commonly used in computers are Base, N Number Binary 8 Octal Decimal 6 Hexadecimal An arbitrary real number, a can be written as m m m a= amn + am N + + an + a+ a N + + a mn In binary system, it has the form, a= am + am + + a + a + a + + a m The decimal number 79 is represented and calculated (79) = While the decimal equivalent of binary number is m m m = = (.955) Electronic computers use binary system whose base is. The two symbols used in this system are and, which are called binary digits or simply bits. The internal representation of any data within a computer is in binary form. However, we prefer data input and output of numerical results in decimal system. Within the computer, the arithmetic is carried out in binary form. Conversion of decimal number 47 into its binary equivalent Sol. Copyright Virtual University of Pakistan

2 47 Remainder 5 Most significant bit (47) = () Binary equivalent of the decimal fraction.765 Sol. Product.765 x x.5.5 x.. x.. x.4.4 x.8.8 x.6.6 x.. x.4 (.765) = (...()) Integer Conversion (59) into binary and then into octal. Sol Copyright Virtual University of Pakistan

3 (59) = () () = = (7) 8 Copyright Virtual University of Pakistan

4 Errors in Computations Numerically, computed solutions are subject to certain errors. It may be fruitful to identify the error sources and their growth while classifying the errors in numerical computation. These are Inherent errors, Local round-off errors Local truncation errors Inherent errors It is that quantity of error which is present in the statement of the problem itself, before finding its solution. It arises due to the simplified assumptions made in the mathematical modeling of a problem. It can also arise when the data is obtained from certain physical measurements of the parameters of the problem. Local round-off errors Every computer has a finite word length and therefore it is possible to store only a fixed number of digits of a given input number. Since computers store information in binary form, storing an exact decimal number in its binary form into the computer memory gives an error. This error is computer dependent. At the end of computation of a particular problem, the final results in the computer, which is obviously in binary form, should be converted into decimal form-a form understandable to the user-before their print out. Therefore, an additional error is committed at this stage too. This error is called local round-off error. (.765) = (.) If a particular computer system has a word length of bits only, then the decimal number.765 is stored in the computer memory in binary form as.. However, it is equivalent to Thus, in storing the number.765, we have committed an error equal to.5, which is the round-off error; inherent with the computer system considered. Thus, we define the error as Error = True value Computed value Absolute error, denoted by Error, While, the relative error is defined as Error Relative error = True value Local truncation error It is generally easier to expand a function into a power series using Taylor series expansion and evaluate it by retaining the first few terms. For example, we may approximate the function f (x) = cos x by the series 4 n x x n x cos x = + + ( ) +! 4! ( n)! If we use only the first three terms to compute cos x for a given x, we get an approximate answer. Here, the error is due to truncating the series. Suppose, we retain the first n terms, the truncation error (TE) is given by Copyright Virtual University of Pakistan

5 n+ x TE ( n + )! The TE is independent of the computer used. If we wish to compute cos x for accurate with five significant digits, the question is, how many terms in the expansion are to be included? In this situation n+ x 5 6 <.5 = 5 (n + )! Taking logarithm on both sides, we get (n+ )log x log[(n+ )!] < log 5 6 log = = 5. or log[(n+ )!] (n+ )log x> 5. We can observe that, the above inequality is satisfied for n = 7. Hence, seven terms in the expansion are required to get the value of cos x, with the prescribed accuracy The truncation error is given by 6 x TE 6! Copyright Virtual University of Pakistan

6 Polynomial An expression of the form ( ) n n n f x = ax + ax + ax an x+ an where n is a positive integer and a, a, a +... a n are real constants, such type of expression is called an nth degree polynomial in x if a Algebraic equation: An equation f(x)= is said to be the algebraic equation in x if it is purely a polynomial in x. For example 5 4 x + x + x + x 6= It is a fifth order polynomial and so this equation is an algebraic equation. x 6= 6 x 7 x= 4 y 4y + y y = polynomial in y 4 t 6t = polynomail in t These all are the examples of the polynomial or algebraic equations. Some facts. Every equation of the form f(x)= has at least one root,it may be real or complex.. Every polynomial of nth degree has n and only n roots.. If f(x) = is an equation of odd degree, then it has at least one real root whose sign is opposite to that of last term. 4.If f(x)= is an equation of even degree whose last term is negative then it has at least one positive and at least one negative root. Transcendental equation An equation is said to be transcendental equation if it has logarithmic, trigonometric and exponential function or combination of all these three. For example x e 5x = it is a transcendental equation as it has an exponential function x e sin x= ln x sin x= x sec x tan x e = These all are the examples of transcendental equation. Root of an equation For an equation f(x) = to find the solution we find such value which satisfy the equation f(x)=,these values are known as the roots of the equation. A value a is known as the root of an equation f(x) = if and only if f (a) =. Copyright Virtual University of Pakistan

7 Properties of an Algebraic equation. Complex roots occur in the pairs. That is,if (a+ib ) is a root of f(x)= then (a-ib ) is also a root of the equation. if x=a is a root of the equation f(x)= a polynomial of nth degree,then (x-a) is a factor of f(x) and by dividing f(x) by (x-a) we get a polynomial of degree n-. Descartes rule of signs This rule shows the relation ship between the signs of coefficients of an equation and its roots. The number of positive roots of an algebraic equation f(x) = with real coefficients can not exceed the number of changes in the signs of the coefficients in the polynomial f(x) =.similarly the number of negative roots of the equation can not exceed the number of changes in the sign of coefficients of f (-x) = Consider the equation x x + 4x 5= here it is an equation of degree three and there are three changes in the signs First +ve to ve second ve to +ve and third +ve to ve so the tree roots will be positive Now f ( x) = x x 4x 5 so there is no change of sign so there will be no negative root of this equation. Intermediate value property If f(x) is a real valued continuous function in the closed interval a x bif f(a) and f(b) have opposite signs once; that is f(x)= has at least one root β such that a β b Simply If f(x)= is a polynomial equation and if f(a) and f(b) are of different signs,then f(x)= must have at least one real root between a and b. Numerical methods for solving either algebraic or transcendental equation are classified into two groups Direct methods Those methods which do not require any information about the initial approximation of root to start the solution are known as direct methods. The examples of direct methods are Graefee root squaring method, Gauss elimination method and Gauss Jordan method. All these methods do not require any type of initial approximation. Iterative methods These methods require an initial approximation to start. Copyright Virtual University of Pakistan

8 Bisection method, Newton raphson method, secant method, jacobie method are all examples of iterative methods. How to get an initial approximation? The initial approximation may be found by two methods either by graphical method or analytical method Graphical method The equation f(x)= can be rewritten as f( x) = f( x) and initial approximation of f(x) may be taken as the abscissa of the point of intersection of graphs of y = f( x) and y = f( x) for example f( x) = x sin x = so this may be written as x = sinx Now we shall draw the graphs of y = x and y = sinx Here both the graphs cut each other at.9 so the initial approximation should be taken as.9 Analytical method This method is based on the intermediate value property in this we locate two values a and b such that f(a) and f(b) have opposite signs then we use the fact that the root lies Copyright Virtual University of Pakistan

9 between both these points,this method can be used for both transcendental and algebraic equations. Consider the equation () = f( x) = x + sinx = f () = 8 + sin( ) = π =.6499 Here f() and f() are of opposite signs making use of intermediate value property we infer that one root lies between and. So in analytical method we must always start with an initial interval (a,b) so that f(a) and f(b) have opposite signs. Bisection method (Bolzano) Suppose you have to locate the root of the equation f(x)= in an interval say ( x, x ),let f ( x) and f ( x ) are of opposite signs such that f( x) f( x ) < Then the graph of the function crossed the x-axis between x and x which exists the existence of at least one root in the interval ( x, x ). x + x The desired root is approximately defined by the mid point x = if f( x ) = then x is the root of the equation otherwise the root lies either between xand xor xand x x + x Now we define the next approximation by x = provided f( x) f( x ) < then root may be found between xand x If provided f( x) f( x ) < then root may be found between xand x by x = Thus at each step we find the desired root to the required accuracy or narrow the range to half the previous interval. This process of halving the intervals is continued in order to get smaller and smaller interval within which the desired root lies. Continuation of this process eventually gives us the required root. NOTE: In numerical analysis all the calculation are carried out in radians mode and the value of pi is supposed to be.4 Example Solve x 9x+ = for the root between x= and x=4 by bisection method Solution: Here we are given the interval (,4) so we need not to carry out intermediate value property to locate initial approximation. Here Copyright Virtual University of Pakistan 4 x + x

10 ( ) 9 f x = x x+ = now f f () 9() = + = + = (4) 4 9(4) = + = + = here f () f () < so root lies between and 4 x = x = 4 x f () 9() = = = + = + = here f () f () < so the root lies between nad x f + = =.5 = + = + = < (.5).5 9(.5) so the root lies between.5 and as f (.5) f () <.5 + x4 = =.75 now similarly x =.875 and x =.975 and the process is continued 5 6 untill the desired accuracy is obtained. n xn f ( xn ) When to stop the process of iteration? Here in the solved example neither accuracy is mentioned and nor number of iteration is mentioned by when you are provided with the number of iteration then you will carry those number of iteration and will stop but second case is accuracy like you are asked to Copyright Virtual University of Pakistan 5

11 find root with an accuracy of then you will check the accuracy by subtracting two consecutive root like.5648 and =. So the accuracy of is achieved so you will stop here. Example: Carry out the five iterations for the function f( x) = xcos( x) ( x+ ) Note: All the calculations should be done in the radians. Solution: f( x) = xcos( x) ( x+ ) f ( ) = ( )cos( ) ( + ) = (.46) =+.8 > f () = ()cos() ( + ) = = < so the root lies between and as f () f ( ) < x = =.5 f (.5) = (.5)cos( ) (.5 + ) =.54.5 =.79 < so root lies between and.5 as f ( ) f (.5).5 x = =.75 f (.75) = (.75)cos(.5) (.75 + ) =.6.65 =.686 < so root lies between and.75 as f ( ) f (.75).75 x4 = =.875 f (.875) = (.875)cos(.75) ( ) = =.9675 > so root lies between.875 and.75 as f (.75) f (.875) x5 = =.85 f (.85) = (.85)cos(.65) (.85 + ) =.88.5 =.597 > so root lies between.85 and.75 as f (.75) f (.85) x5 = =.785 Example : Carry out the first five iterations f( x) = xcosx x + x,. x. Note: All the calculations should be done in the radians. Solution: Copyright Virtual University of Pakistan 6

12 f x x x x x f f ( ) = cos + (.) =. cos. (.) + (.) =.(.6) (.44) +.6 =.548 > (.) =.cos. (.) + (.) =.(.674) (.69) + 9. =. < as f (.) f (.) < so the root lies between both x f. +. = =.5 = + (.5).5cos.5 (.5) (.5) =.5(.5) (.565) +.75 =.9 > as f (.5) f (.) < so the root lies between both.5 +. x = =.75 f (.75) =.75cos.75 (.75) + (.75) =.75(.95) (.656) +.85 =.545 < as f (.5) f (.75) < so the root lies between both x4 = =.65 f (.65) =.65cos.65 (.65) + (.65) =.75(.4) (.599) =.7 < as f (.5) f (.65) < so the root lies between both x f = =.565 = + (.565).565cos.565 (.565) (.56 5) =.565(.9) (.578) + (.565) =.8 > as f (.565) f (.65) < so the root lies between both x6 = =.5975 Copyright Virtual University of Pakistan 7

13 Regula-Falsi method (Method of false position) Here we choose two points x n and xn such that f ( xn) and f( xn ) have opposite signs. Intermediate value property suggests that the graph of the y=f(x) crosses the x-axis between these two points and therefore, a root lies between these two points. Thus to find the real root of f(x)= using Regula-Falsi method,we replace the part of the curve between the points A( xn, f ( xn)) and B( xn, f ( xn )) by a chord in the interval and we take the point of intersection of this chord with x-axis as initial approximation. Now, the equation of the chord joining the points A and B is, y f( xn) x xn = f ( x ) f( x ) x x n n n n Setting y= in the above equation we get = x x n n x xn f xn f( xn) f( xn ) ( ) Hence the first approximation to the root is given by xn xn xn+ = xn f( xn) f( xn) f( xn ) We observe that f ( xn ) and f ( xn+ ) are of opposite signs thus it is possible to apply the above procedure, to determine the line through B and A and so on. Hence for successive approximation to the root above formula is used. Example Use the Regula-Falsi method to compute a real root of the equation x 9x + =, (i) if the root lies between and 4 (ii) if the root lies between and. Comment on the results. Solution Let f (x) = x - 9x + f () = - 9() + =8 8+= 9and f (4) = 4-9(4) + =64 6+=9. Since f () and f (4) are of opposite signs, the root of f (x) = lies between and 4. Taking x =, x = 4 and using Regula-Falsi method, the first approximation is given by x x 4 (9) x = x f( x) = 4 (9) = 4 f( x) f( x) 9 ( 9) 8 58 = 4 = 4.56 = Copyright Virtual University of Pakistan

14 Now f (x) =.476-9(.476) + = = Since f (x) and f (x) are of opposite signs, the root lies between x and x. The second approximation to the root is given as x x x4 = x f( x) =.476 ( 6.644) f( x) f( x) =.476 ( 6.644) =.476 (.445)( 6.644) = =.7989 Therefore f (x4) = (.7989) + = = = Now, since f (x) and f (x4) are of opposite signs, the third approximation is obtained from x4 x x5 = x4 f( x4) =.7989 (.977) =.865 f( x4) f( x) =.7989 (.977) = (.977) = = Now f (x5) =.865-9(.865) + = = (ii) Here f (x) = x - 9x + f () = - 9() + = 8 8 +=- 9 and f () = - 9() + = 7 7+=. Since f () and f () are of opposite signs, the root of f (x) = lies between and. Taking x =, x = and using Regula-Falsi method, the first approximation is given by x x x = x f( x) = () f( x) f( x) + 9 = =.9 f (x ) =.9-9(.9) + = =.7 Since f (x) and f (x) are of opposite signs, the root lies between x and x. The second approximation to the root is given as.9.. x4 =.9 (.7) =.9 (.7) =.9 (.7) =.9 (.5844)(.7) =.9 _.456 =.9456 f( x ) =.7 4 f (x 4) = (.9456) + = =.7 Copyright Virtual University of Pakistan

15 Now, we observe that f (x) and f (x4) are of opposite signs; the third approximation is obtained from x5 =.9456 (.7) =.9456 (.7).7.7 =.9456 (.575)(.7) =.9475 f (x 5) = (.9475) + = =.896 We observe that the value of the root as a third approximation is evidently different in both the cases, while the value of x5, when the interval considered is (, ), is closer to the root. Important observation: The initial interval (x, x) in which the root of the equation lies should be sufficiently small. Example Use Regula-Falsi method to find a real root of the equation lnx cos x = accurate to four decimal places after three successive approximations. Note: here is a transcendental equation all the calculation should be done in the radians mode Sol: f( x) = ln x - cos x we have f()=ln-cos()=-.54=-.54< f()=ln-cos()= =.9 As f()f()< so the root lies between and the first approximation is obtained form x = (.9) = = f( x ) = ln.75 - cos.75 = 8.49 =.44 Now, since f (x) and f (x) are of opposite signs, the second approximation is obtained as (.75)(.44) x4 = =.7 f( x ) = ln.7 - cos.7 = Copyright Virtual University of Pakistan

16 Similarly, we observe that f (x) and f (x4) are of opposite signs, so, the third approximation is given by (.7)(.48) x5 = =. 4 f( x ) = The required real root is. Example: Use method of false position to solve x x e + + cosx 6= x Solution: x x f( x) = e + + cosx 6 x =, x = now x n+ f f n n () e cos 6 () e cos 6 now for n = x x xn xn = f( xn) f( x ) f( x ) = + + = (.54) 6 =.7 = + + = (.46) 6 =.868 x = x f x f( x ) f x f x ( ) = (.868) ( ) (.678) cos(.678) 6 f x f x = (.868) = = e + + =.5457 now for n = x x.678 x = x f( x ) =.678 (.5457) ( ) ( ) (.5457).868 x (.7) =.678 (.5457) = =.88 (.55) Copyright Virtual University of Pakistan 4

17 f(.88) = e + + cos(.88) 6 =.8575 now for n = x x x4 = x f( x) =.88 (.8575) f( x) f( x) (.8575) x =.88 (.8575) = = f (.8) = e + + cos(.8) 6 =.99 now for n = 4 x4 x.8.88 x5 = x4 f( x4) =.8 (.99) f( x4) f( x) x5 =.8 (.99) =.8.96 = f(.89) = e + + cos(.89) 6 =.4 now for n = 5 x5 x x6 = x5 f( x5) =.89 (.4) f( x5) f( x4).4.99 (.) x6 =.89 (.4) =.89. Example: Solve the equation Solution xcos x ( x ) x = Perform only three iterations. f( x) = xcos x ( x ) here x so f f = and x = () = () cos 4 ( ) = 4 cos 4 =.646 () = () cos () ( ) = 6 cos 6 = 4.76 x x here x = x f ( x ) for n = n n n+ n n f( xn) f( xn ) x x x = x f( x ) = (4.76) ( ) ( ) f x f x Copyright Virtual University of Pakistan 5

18 = (4.76) =.6455 = f = = = (.545) (.545) cos (.545) (.545 ) 4.79 cos for n = x x.545 x = x f( x ) =.545 (.46) ( ) ( ) f x f x =.7 f (.7) = (.7) cos (.7) (.7 ) = cos =.9 for n = x x x = x f( x ) =.7 (.) ( ) ( ) f x f x f =.77 (.77) (.77) cos (.77) (.77 ) 4.7 for n = 4 = = x x.77.7 x = x f( x ) =.77 (.) ( ) ( ) f x4 f x = cos =. Copyright Virtual University of Pakistan 6

19 Example Using Regula-Falsi method, find the real root of the following equation correct, to three decimal places: x log x =. Solution: Let f (x) = x logx. f () = log. =.5979, f () = log. =.4. Since f () and f () are of opposite signs, the real root lies betweenx =, x =. The first approximation is obtained from x x x = x f( x) = (.4) f( x) f( x) = = f( x ) = Let f (x) =.797log.797. =.7. Since f (x) and f (x) are of opposite signs, the root of f (x) = lies between x and x. Now, the second approximation is given by x x.797 x4 = x f( x) =.797 (.7) =.74 f( x) f( x) f( x ) =.74 log Thus, the root of the given equation correct to three decimal places is.74 NOTE: Here if TOL is given then we can simply find the value of TOL by subtracting both the consecutive roots and write it in the exponential notation if the required TOL is obtained then we stop. Method of Iteration Method of iterations can be applied to find a real root of the equation f (x) = by rewriting the same in the form. x = φ( x) Let x = x be the initial approximation to the actual root, say, α of the equation.then the first approximation is x = φ( x) and the successive approximation are x = φ( x) x = φ( x), x4 = φ( x),..., xn = φ( xn ) if sequence of the approximate roots, x, x, x,... x n converges to α it is taken as the root of the equation f(x)=. For convergence purpose the initial approximation is to be done carefully.the choice of the x is done according to the theorem. Copyright Virtual University of Pakistan

20 Theorem If α be a root of f(x) = which is equivalent to x = φ( x) I be any interval containing the point x=α and φ '( x) < xε I, then the sequence of approximations x, x, x,... x n will converge to the root α provided that the initial approximation x is chosen in I Example, f (x) = cos x - x + =. It can be Rewritten as x = (cos x+ ) = φ ( x ) φ( x) = (cosx+ ) f (x) = cos x - x + =. f()=cos - () + =.54> f()=cos - () + = =-.464< so root lies between and φ '( x) = (sin x) both φ'() andφ'() < so the method of iterations can be applied let x =.5 x = (cos x + ) = (cos(.5) + ) = x = (cos x+ ) = (cos(.99985) + ) = x = (cos x + ) = (cos(.99965) + ) = So this is the required root correct up to 5 places of decimal. Example Find the real root of the equation x + x = by method of iterations Solution let f ( x) = x + x now f f = + = < () = + = > () hence a real root lies between and Copyright Virtual University of Pakistan

21 here x + x = x ( x+ ) = = = = φ( ) ( x + ) x + x x x hereφ '( x) = / [/( x + ) ] φ'() = / < and φ'() = / < so φ '( x) < for all the values in the int erval let x =.65 x = φ( x) = = = x +.65 x x x x x x x x x x = φ( x ) = = = x = φ( x ) = = = x = φ( x ) = = = x = φ( x ) = = = x = φ( x ) = = = x = φ( x ) = = = x = φ( x7 ) = = = x = φ( x ) = = = x = φ( x ) = = = x = φ( x ) = = = x hence root is Note: In this question the accuracy up to 7 places is acquires or here the TOL is 7 Copyright Virtual University of Pakistan

22 Example Find a real root of the equation cos x = x correct to seven places of decimal. Solution Here it is a transcendental function and all the calculation must be done in the radians mode and value of pi should be.4 f( x) = cosx x+ f () = cos () + = > f ( π / ) = cos(.57) (.57) + = =.797 < so a real root lies between and π / hereφ( x) = (cosx + ) we have φ '( x) = sin x it is clearly less thanas sin is a bounded function and it ' s values lies between and hence iteration method can be applied let x =.5 be the inital approximation then x = φ( x) = [cos(.5) + ] = x = φ( x) = [cos(.65868) + ] =.6486 x = φ( x) = [cos(.6486) + ] = x4 = φ( x) = [cos(.67787) + ] =.6697 x5 = φ( x4) = [cos(.6697) + ] =.6764 x6 = φ( x5) = [cos(.6764) + ] = Copyright Virtual University of Pakistan 4

23 x x x x = φ( x ) = [cos(.67969) + ] =.675 = φ( x ) = [cos(.675) + ] =.674 = φ( x ) = [cos(.674) + ] =.676 = φ( x ) = [cos(.676) + ] = Copyright Virtual University of Pakistan 5

24 Newton -Raphson Method This method is one of the most powerful method and well known methods, used for finding a root of f(x)= the formula many be derived in many ways the simplest way to derive this formula is by using the first two terms in Taylor s series expansion of the form, f ( xn+ ) = f( xn) + ( xn+ xn) f '( xn) setting f ( xn+ ) = gives, f( xn) + ( xn+ xn) f '( xn) = thus on simplification, we get, f( xn ) xn+ = xn for n=,,... f '( x ) Geometrical interpretation n Let the curve f(x)= meet the x-axis at x=α meet the x axis at x=α.it means that α is the original root of the f(x)=.let x be the point near the root α of the equation f(x)= then the equation of the tangent P[ x, f( x )] is y f( x) = f '( x)( x x) f ( x ) This cuts the x-axis at x = x f '( x ) This is the first approximation to the root α.if Px [, f( x )] is the point corresponding to x on the curve then the tangent at P is y f( x ) = f '( x )( x x ) f ( x ) This cuts the x-axis at x = x f '( x ) This is the second approximation to the root α.repeating this process we will get the root α with better approximations quite rapidly. Note:. When f '( x ) very large.i.e. is when the slope is large, then h will be small (as assumed) and hence, the root can be calculated in even less time.. If we choose the initial approximation x close to the root then we get the root of the equation very quickly.. The process will evidently fail if f '( x ) = is in the neighborhood of the root. In such cases the regula falsi method should be used. Copyright Virtual University of Pakistan

25 4. If the initial approximation to the root is not given, choose two say, a and b, such that f(a) and f(b) are of opposite signs. If f(a) < f(b) then take a as the initial approximation. 5. Newton s raphson method is also referred to as the method of tangents. Example Find a real root of the equation x x = using Newton - Raphson method, correct to four decimal places. Solution f(x)=x - x - f()= - - =-< f()= - - = 8 =5> so the root lies betweenand here f '( x) = x and f "( x) = 6x f '() = * = f '( x) = * = here f "() = 6 f "() = 6() = here f ( ) and f "() have the same signs so x = The second approximation is computed using Newton-Raphson method as Copyright Virtual University of Pakistan

26 x x f( x ) = = = f ( x ) f(.54545)= = =.4576 f '( x) = x = (.5454) = (.889) = = x = = f(.596)= = =.4576 f '( x) = x = (.5454) = (.889) = = x = = x =.596 =.579, f( x) = x4 =.579 =.47, f( x4) = x5 =.47+ =.478, f( x5 ) = Hence, the required root is.47 Note! Methods such as bisection method and the false position method of finding roots of a nonlinear equation f(x) = require bracketing of the root by two guesses. Such methods are called bracketing methods. These methods are always convergent since they are based on reducing the interval between the two guesses to zero in on the root. In the Newton-Raphson method, the root is not bracketed. Only one initial guess of the root is needed to get the iterative process started to find the root of an equation. Hence, the method falls in the category of open methods. Newton - Raphson method is based on the principle that if the initial guess of the root of f( x ) = is at xi, then if one draws the tangent to the curve at f( xi ), the point xi+ where the tangent crosses the x-axis is an improved estimate of the root f(x) f(x i ) i, ( i) x f x Copyright Virtual University of Pakistan f(x i+ ) θ X

27 Draw backs of N-R Method Divergence at inflection points: If the selection of a guess or an iterated value turns out to be close to the inflection point of f ( x ), [where f ( x ) = ], the roots start to diverge away from the root. x i + = x i f f ( x i ) '( x ) i Division of zero or near zero: If an iteration, we come across the division by zero or a near-zero number, then we get a large magnitude for the next value, xi+. Root jumping: In some case where the function f (x) is oscillating and has a number of roots, one may choose an initial guess close to a root. However, the guesses may jump and converge to some other root. Oscillations near local maximum and minimum: Results obtained from N-R method may oscillate about the local max or min without converging on a root but converging on the local max or min. Eventually, it may lead to division to a number close to zero and may diverge. Convergence of N-R Method Let us compare the N-R formula Copyright Virtual University of Pakistan 4

28 f ( xn) x = n x + n f ( x ) n with the general iteration formula x = n+ φ( xn), f ( xn) φ ( xn) = xn f ( xn) f ( x) φ ( x) = x f ( x) The iteration method converges if φ ( x) <. Therefore, N-R formula converges, provided f ( x) f ( x) < f ( x) in the interval considered. Newton-Raphson formula therefore converges, provided the initial approximation x is chosen sufficiently close to the root and are continuous and bounded in any small interval containing the root. Definition Let xn = α + εn, xn+ = α + εn+ where α is a root of f (x) =. p If we can prove that ε = n+ Kε n, where K is a constant and ε n is the error involved at the n - th step, while finding the root by an iterative method, then the rate of convergence of the method is p. The N-R method converges quadratically x, n = α + ε n x = α + ε n+ n+ where α is a root of f (x) = and finding the root by N-R formula ε n is the error involved at the n-th step, while f ( α + ε n) α + εn+ = α + εn f ( α + ε n) f( α + εn) εnf ( α + εn) f( α + εn) εn+ = εn = f ( α + εn) f ( α + εn) Using Taylor s expansion, we get Copyright Virtual University of Pakistan 5

29 ε n εn+ = εn[ f ( α) + εnf ( α) + ] f( α) + εnf ( α) + f ( α) + f ( α) εn f ( α) + + Since α is a root, f ( α ) =. Therefore, the above expression simplifies to ε n+ ε n = f ( α) f ( α) + ε f ( α) n εn f ( α) f ( α) = + ε n f ( α) f ( α) εn f ( α) f ( α) = ε n f ( α) f ( α) εn f ( α) f ( α) = ε n f ( α) f ( α) εn f ( α) εn+ = + O( εn) f ( α) On neglecting terms of order ε n and higher powers, we obtain ε = n+ Kε n Where f ( α) K = f ( α) It shows that N-R method has second order convergence or converges quadratically. Example Set up Newton s scheme of iteration for finding the square root of a positive number N. Solution The square root of N can be carried out as a root of the equation x N =. Let f ( x) = x N. By Newton s method, we have f ( xn ) x = n x + n f ( xn ) In this Problem f ( x) = x N, f ( x) = x. Therefore Copyright Virtual University of Pakistan 6

30 xn N N xn+ = xn = xn + xn xn Example Evaluate, by Newton s formula. Solution Since 9 =, 6 = 4, We take x = (+ 4) / =.5. N x = x + =.5 + =.464 x.5 x =.464+ = x =.464+ = Hence =.464. Here in this solution we use the iterative scheme developed in the above example and simply replace the previous value in the next iteration and finally come to the result. Example Find the first three iteration of the equation f ( x) = x.8.sinx in the interval [, π / ]. Solution Copyright Virtual University of Pakistan 7

31 f () =.8.sin() =.8.() =.8 f (.57) =.57.8.sin(.75) =.57.8.(.99999) = =.57 f '( x) =.cos x f '() =.cos() =. =.8 here f () is greater then x = f( x ).8 x = x = = f '( x ).8 now f () =.8.sin() =.8.68 =.7 f '( x) =.cos x f '() =. cos() =.8 =.899 f( x ).7 x = x = =.55 =.9645 f '( x ).899 f (.9645) = sin(.9645) = =. f '( x) =.cos x f '(.9645) =. cos(.9645) =.96 =.8864 f( x ). x = x = f '( x ).8864 = = NOTE: In the above question the function is a transcendental function and we have to perform all the calculation in the radians mode and the value of pi should be taken as.4 Example Consider the equation x 8 f( x) 4xcos x ( x ) = find the root of the equation in the range Solution Copyright Virtual University of Pakistan 8

32 f( x) = 4xcos x ( x ) here f = = () 4() cos () ( ) 4 f (8) = 4(8) cos (8) (8 ) = = = cos6 (6) f '( x) = 4 cos x 8xsin x ( x ) = 4 cos x 8xsin x ( x ) f '(8) = 4 cos6 64sin6 (8 ) = =.595 sin ce f (8) is greater so x = 8 x f( x ) ( ) 8.68 = x = = f '( x ).595 f (.68) = 4(.68) cos (.68) (.68 ) = = 8.5 f '(.68) = 4 cos (.68) 8(.68) sin (.68) (.68 ) = = 6.6 x f( x ) = x = + = f '( x ).79 f (8.8) = 4(8.8) cos (8.8) (8.8 ) = = f '(8.8) = 4 cos (8.8) 8(8.8) sin (8.8) (8.8 ) = = 6.5 x f( x ) = x = + = + = f '( x) 6.5 Example Perform three iteration of the equation ln( x ) + cos( x ) = when. x.use Newton Raphson method to calculate the root. Solution Here ln( x ) + cos( x ) = when. x f( x) = ln( x ) + cos( x ) Copyright Virtual University of Pakistan 9

33 f( x) = ln( x ) + cos( x ) f (.) = ln(. ) + cos(. ) = =.69 f () = ln( ) + cos( ) = +.54 =.54 now f ( x) = ln( x ) + cos( x ) f '( x) = sin( x ) x f '(.) = sin(. ). = = 4.84 f( x ).69 x = x =. =. +. =. f '( x ) 4.84 f (.) = ln(. ) + cos(. ) = =.596 f '(.) = sin(. ). =..5 =.695 f( x ).596 x = x =. =.+.59 =.9 f '( x ).695 f (.9) = ln(.9 ) + cos(.9 ) = =.6 f '(.9) = sin(.9 ).9 = =.86 f( x).6 x = x =.9 =.9+.7 =.976 f '( x ).86 Copyright Virtual University of Pakistan

34 Secant Method The secant method is modified form of Newton- Raphson method.if in Newton-Raphson method; we replace the derivative f '( x n ) by the difference ratio, i.e, f( xn) f( xn ) f '( xn ) = xn xn Where x and x are two approximations of the root we get n n f( xn)( xn xn ) xn+ = xn f( xn) f( xn ) xn f( xn) xnf( xn ) f( xn)( xn xn ) = f( xn) f( xn ) xn f( xn) xn f( xn ) = f( xn) f( xn ) Provided f ( xn) f( xn ) This method requires two starting values x and x ; values at both these points are calculated which gives two points on the curve the next point on the curve are obtained by using the derived formula. We continue this procedure until we get the root of required accuracy. Geometrical Interpretation Geometrically, the secant method corresponds to drawing secants rather than tangents to obtain various approximations to rootα ; To obtain x we find the intersection between the secant through the points ( x, f( x )) And ( x, f( x )) and the x-axis. It is experienced that the secant method converges rather quickly.there is a possibility of the divergence if the two roots lie on the same side of the curve.the order of the convergence of the decant method equal to ( + 5) =.68,which shows that this method has the order of convergence slightly inferior that of Newton-Raphson method, In this method f(x) is not required to change signs between the estimate. Copyright Virtual University of Pakistan

35 y y=f(x) f(x) x O x f(x) α x x F(x) Some more description We choose x, x close to the root a of f (x) = such that f (x) f(x) As a next approximation x is obtained as the point of intersection of y = and the chord passing through the points (x, f(x )), (x f(x )). f( x) f( x) y f( x) = ( x x), x x Putting y = we get xf( x) xf( x) x = x= f( x) f( x) Convergence of Secant Method Here the sequence is generated by the rule xn f( xn) xnf( xn ) xn+ = f( xn) f( xn ) Starting with x and x as { x, x } It will converge to a, that is f (a) = NOTE The Secant method converges faster than linear and slower than Newton s quadratic Copyright Virtual University of Pakistan

36 Example Do three iterations of secant method to find the root of f( x) = x x+ =, Taking x =, x =.5 n =, f x = f = + = ( ) () () f x = f = + = + = xf( x) xf( x) x = f( x ) f( x ) ( ) (.5).5 (.5) ()(.75) (.5)( ) = =..75 ( ) n =, f x xf ( x) xf ( x) x = f( x ) f( x ) ( ) = f(.) =. (.) + = =.48 (.5)(.48).(.75) = = (.75) n =, f x x x ( ) = f(.56) =.56 (.56) + = =.67 4 xf( x) xf( x) = f( x ) f( x ) (.) f(.56).56 f(.) = f(.56) f(.) =.47, f( x ) = and x x = Though X5 is not the true root, yet this is good approximation to the root and convergence is faster than bisection. Example Find the root of cosh sin x x = using the secant method, with accuracy of 4 decimal point.take.4 and.5 as the two starting values Copyright Virtual University of Pakistan

37 f ( x) = cosh xsin x now f ( x) = cosh xsin x = cosh.4sin.4 = =.58 f( x) = cosh xsin x = cosh.5sin.5 = =.8 xf( x) xf( x) x = f( x) f( x) = = = f( x) = cosh xsin x = cosh.5sin.5 = =.8 xf ( x) xf ( x) x = f( x) f( x) = = = f( x) = cosh xsin x =.9 xf( x) xf( x) x4 = f( x) f( x) = = = Comparison: In secant method, we do not check whether the root lies in between two successive approximates Xn-, and Xn. This checking was imposed after each iteration, in Regula Falsi method. Muller s Method In Muller s method, f (x) = is approximated by a second degree polynomial; that is by a quadratic equation that fits through three points in the vicinity of a root. The roots of this quadratic equation are then approximated to the roots of the equation f (x).this method is iterative in nature and does not require the evaluation of derivatives as in Newton- Copyright Virtual University of Pakistan 4

38 Raphson method. This method can also be used to determine both real and complex roots of f (x) =. Suppose, xi, xi, xi be any three distinct approximations to a root Of f (x) =. f ( xi ) = fi, f( xi ) = fi f( xi) = fi. Noting that any three distinct points in the (x, y)-plane uniquely; determine a polynomial of second degree. A general polynomial of second degree is given by Noting that any three distinct points in the (x, y)-plane uniquely; determine a polynomial of second degree. A general polynomial of second degree is given by f ( x) = ax + bx+ c Suppose, it passes through the points ( x, f ),( x, f ),( x, f ) i i i i i i Then the following equations will be satisfied ax + bx + c = f i i i ax + bx + c = f i i i ax + bx + c = f i i i Eliminating a, b, c, we obtain x x f x x f i i i i i i i i i x x f x x f = This can be written as ( x xi )( x xi) ( x xi )( x xi) ( x xi )( x xi ) f = fi + fi + f ( xi xi ) ( xi xi )( xi xi) ( xi xi )( xi xi ) We further define i Copyright Virtual University of Pakistan 5

39 h x xi λ = = h x x hi λi = h i i i i δ = + λ i i Fig: Quadratic polynomial With these substitutions we get a simplified Equation as f = [ λλ ( + ) λ f i i δi ( i ) i i fi i i fi λλ+ + λ λδ + ( λ+ + λ ) λ ] Or Copyright Virtual University of Pakistan 6

40 f = λ ( f λ f λδ + fλ ) δ i i i i i i i i + λ[ f λ f δ i i i i + f ( λ + δ )] δ + f i i i i i To compute λ, set f =, we obtain λi( fi λi fi δi + fi) λ + giλ+ δi fi = Where gi = fi λi fi δi + fi( λi + δi) A direct solution will lead to loss of accuracy and therefore to obtain max accuracy we rewrite as: fiδi gi + + λ ( i fi λi fi δi + fi) = λ λ So that, / gi ± [ gi 4 fiδλ i i( fi λi fi δi + fi)] = λ fi δi Or fiδi λ = / gi ± [ gi 4 fiδλ i i( fi λi fi δi + fi)] Here, the positive sign must be so chosen that the denominator becomes largest in magnitude. We can get a better approximation to the root, by using x = i x + + i hλ i Copyright Virtual University of Pakistan 7

41 Numerical Analysis MTH6 Muller s Method Muller s method is a generalization of the secant method, in the sense that it does not require the derivative of the function. It is an iterative method that requires three starting points,, and. A parabola is constructed that passes through the three points; then the quadratic formula is used to find a root of the quadratic for the next approximation. It has been proved that near a simple root Muller's method converges faster than the secant method and almost as fast as Newton's method. The method can be used to find real or complex zeros of a function and can be programmed to use complex arithmetic. Example Do two iterations of Muller s method to solve Solution x x+ = starting with x =.5, x =, x = f x ( ) (.5) (.5).75 f x ( ) () () f( x ) = f = + = + c = f =.75 = f = + = = f = + = h = x x =.5 h = x x =.5 hf ( h+ h) f + hf a = hh ( h + h ) (.5)( ) (.75) + (.5) = =.5.5 b f f ah = = h x = x + b (.75) = (.5)(.75).75 =.5 = c b 4ac Copyright Virtual University of Pakistan

42 Numerical Analysis MTH6 Take x =, x =., x =.5 h = x x =.6667, h = x x =. c = f = f(.) = x x + =.746 f = x x + =.75 f = x x + = a h f ( h + h ) f + h f.48 hh ( h + h ).7778 = = =.8 f f ah b = =.6 h c x = x b b 4ac.749 =. 5.6 =.475. = x For third iteration take, x =., x =.475, x =.5 Graeffe s Root Squaring Method GRAEFFE S ROOT SQUARING METHOD is particularly attractive for finding all the roots of a polynomial equation. Consider a polynomial of third degree f ( x) = a + ax+ ax + ax f( x) = a + ax+ ax + ax f( x) = a ax+ ax ax 6 4 f( x) f( x) = ax ( a aa) x + ( a aa) x a f( x) f( x) = at ( a aa) t + ( a a a ) t a The roots of this equation are squares or i (i = ), powers of the original roots. Here i = indicates that squaring is done once. The same equation can again be squared and this squaring process is repeated as many times as required. After each squaring, the coefficients become large and overflow is possible as i increase. Suppose, we have squared the given polynomial i times, then we can estimate the value of the roots by evaluating i root of ai, i =,,, n a i Where n is the degree of the given polynomial. Copyright Virtual University of Pakistan

43 Numerical Analysis MTH6 The proper sign of each root can be determined by recalling the original equation. This method fails, when the roots of the given polynomial are repeated. This method has a great advantage over other methods in that it does not require prior information about approximate values etc. of the roots. But it is applicable to polynomials only and it is capable of giving all the roots. Let us see the case of the polynomial equation having real and distinct roots. Consider the following polynomial equation n n n f( x) = x + ax + ax an x+ an ( ) separating the even and odd powers of x and squaring we get n n n 4 n n n 5 ( x + ax + a4x +...) = ( ax + ax + a5x +...) puttig x = y and simplifying we get the new equation n n n n y + by + by by + bn = () b = a + a b = a aa + a n bn = ( ) an if p, p,... pn be the roots of eqthen the roots of the equation are p, p,... pn Example Using Graeffe root squaring method, find all the roots of the equation x x x 6 + 6= Solution Using Graeffe root squaring method, the first three squared polynomials are as under: For i =, the polynomial is x x x (6 ) + ( 7) 6 = x 4x + 49x 6 For i =, the polynomial is Copyright Virtual University of Pakistan

44 Numerical 4 Analysis MTH6 x (96 98) x + (4 8) x 96 = x 98x + 9x 96 For i =, the polynomial is x ( ) x + ( ) x = x 688x x The roots of polynomial are =.8574, =.878, = Similarly the roots of the p; oynomial are =.98, 4 =.947, 4 = Still better estimates of the roots obtained from polynomial () are =.99949, 8 =.994, 8 = The exact values of the roots of the given polynomial are, and. Example Apply Graeffe,s root squaring method to solve the equation x x x = Solution ( ) 8 7 f x = x x + x here three chnges are observed form + ve to ve, ve to + ve, + ve to ve hence according to deccarte rule of signs f ( x) may have three positive roots rewriting eq as x x yy ( + 7) = (8y+ ) ( 7) (8 ) + = x + and squaring on both sides and putting x = y y y y ( 9) yy + = y + putting y = z we get zz ( + 9) = (z+ ) = z z z z z ( 664) (64 zz + = z + squaring again and putting z uu ( + 664) = (64u+ ) = y u + 8u u = 464u + 84u+ 8 8 u 7888u + 648u = ) = u we get Copyright Virtual University of Pakistan 4

45 Numerical 5 Analysis MTH6 if the roots of the eq are p, p p and those of eqiuation 5 are q, q, q /8 /8 /8 /8 /8 /8 /8 /8 8,, p = ( q ) = ( λ ) = (7888) = p = ( q ) = ( λ / λ ) = (648 / 7888) = /8 p = ( q ) = ( λ / λ ) = ( /648) =.64 here f (5) = f () = f () = hence all are the roots of the eqiuation Example Find all the root of the equation Solution 4 x x+ = Copyright Virtual University of Pakistan 5

46 Numerical 6 Analysis MTH6 f x = x x+ 4 ( ) () here there are two changes in the sign so there are two positive roots of the equation and there are no chnge in the roots of the equation f ( x) so two positive real roots and two are complex roots rewriting the above equation as 4 x + = x squaring the equation and putting x we have y + = ( ) 9 y squaring again and putting y 4 u u u u = z = y 4 ( z+) = 8z 4 z + 4z + 6z 77z + = 4 z + 6z + = z(4z 77) () squaring once again and putting z ( u + 6u+ ) = u(4u 77) = u, we get = () if p, p, p, p 4 4 /8 /8 /8 /8 /8 /8 λ 4 /8 /8 /8 λ are the roots of the equation and q, q, q, q are roots of equation then p = ( q ) = ( λ ) = (4) =.897 p λ 654 = ( q ) = [ ] = [ ] =.8999 λ 597 p = ( q ) = [ ] = [ ] =.6984 p λ = ( q ) = [ ] = [ ] = /8 4 /8 /8 4 λ 597 from equation ( ) and () from equation and, we observe the magnitude of the cofficients λ and λ have become 4 cons tan t which implies p and p are the real roots, then p and p are real roots, let the complex roots be ρ e = ξ + iη ( ) ρ λ 4 ± iϕ from equation () it ' s magnitude is given by ρ λ 597 = = = also from equation sum of roots is zero i. eρ + ξ + p = 4 ξ = / ( p + p ) = η = ρ ξ = =.87 hence, the four roots are.897,.76659,.7465 and ±.87i Revision Example Obtain the Newton-Raphson extended formula f( x) [ f( x)] x = x f ( x ) f ( x ) [ f( x )] For finding the root of the equation f(x)= Copyright Virtual University of Pakistan 6

47 Numerical 7 Analysis MTH6 Solution Expanding f (x) by Taylor s series and retaining up to second order term, = f( x) = f( x ) + ( x x ) f ( x ) ( x x ) + f ( x ) Therefore, f( x ) = f( x ) + ( x x ) f ( x ) ( x x) + f ( x ) = This can also be written as [ f( x )] f( x) + ( x x) f ( x) + f ( x ) = [ f ( x )] Thus, the Newton-Raphson extended formula is given by f( x) [ f( x)] x = x f ( x ) f ( x ) [ f ( x )] This is also known as Chebyshev s formula of third order Copyright Virtual University of Pakistan 7

48 Solution of Linear System of Equations and Matrix Inversion The solution of the system of equations gives n unknown values x, x,, xn, which satisfy the system simultaneously. If m > n, the system usually will have an infinite number of solutions. If A then the system will have a unique solution. If A =, Then there exists no solution. Numerical methods for finding the solution of the system of equations are classified as direct and iterative methods. In direct methods, we get the solution of the system after performing all the steps involved in the procedure. The direct methods consist of elimination methods and decomposition methods. Under elimination methods, we consider, Gaussian elimination and Gauss-Jordan elimination methods Crout s reduction also known as Cholesky s reduction is considered under decomposition methods. Under iterative methods, the initial approximate solution is assumed to be known and is improved towards the exact solution in an iterative way. We consider Jacobi, Gauss- Seidel and relaxation methods under iterative methods. Gaussian Elimination Method In this method, the solution to the system of equations is obtained in two stages. i) the given system of equations is reduced to an equivalent upper triangular form using elementary transformations ii) the upper triangular system is solved using back substitution procedure This method is explained by considering a system of n equations in n unknowns in the form as follows ax + ax + + anxn = b ax + ax + + anxn = b.. an x+ anx+ + annxn = bn Stage I: Substituting the value of x from first equation into the rest x+ a x + + a nxn = b a x + + a nxn = b a nx + + a nnxn = b n Copyright Virtual University of Pakistan

49 Now, the last (n ) equations are independent of x, that is, x is eliminated from the last (n ) equations. This procedure is repeated and x is eliminated from rd, 4th,, n-th equations The same procedure is repeated till the given system assumes the upper triangular form: cx + c x + + c nxn = d cx + + cnxn = d cnnxn = d n Stage II: The values of the unknowns are determined by backward substitution. First xn is found from the last equation and then substitution this value of xn in the preceding equation will give the value of xn-. Continuing this way, we can find the values of all other unknowns Example Solve the following system of equations using Gaussian elimination method x+ y z = 5 4x+ 4y z = x+ y z = Solution Stage I (Reduction to upper-triangular form): Divide first equation by and then subtract the resulting equation (multiplied by 4 and ) from the nd and rd equations respectively. Thus, we eliminate x from the nd and rd equations. The resulting new system is given by z 5 x+ y = y z = 7 6y z = 6 Now, we divide the nd equation by and eliminate y from the last equation and the modified system is given by Copyright Virtual University of Pakistan

50 z 5 x+ y = z 7 y + = 5z = 5 Stage II (Backward substitution): From the last equation, we get z = Using this value of z, the second equation gives 7 y = = Putting these values of y and z in the first equation, we get x = 5 + = Thus, the solution of the given system is given by x =, y =, z = Partial and Full Pivoting The Gaussian elimination method fails if any one of the pivot elements becomes zero. In such a situation, we rewrite the equations in a different order to avoid zero pivots. Changing the order of equations is called pivoting. Partial pivoting If the pivot happens to be zero, then the i-th column elements are searched for the numerically largest element. Let the j-th row (j > i) contains this element, then we interchange the i-th equation with the j-th equation and proceed for elimination. This process is continued whenever pivots become zero during elimination. For example, let us examine the solution of the following simple system 5 x+ x = x+ x = Using Gaussian elimination method with and without partial pivoting, assuming that we require the solution accurate to only four decimal places. The solution by Gaussian elimination gives x =, x =. If we use partial pivoting, the system takes the form Copyright Virtual University of Pakistan

51 x + x = 5 x+ x = Using Gaussian elimination method, the solution is found to be x =, x =, which is a meaningful and perfect result. In full pivoting (or complete pivoting), we interchange rows and columns, such that the largest element in the matrix of the variables also get changed. Full pivoting, in fact, is more complicated than the partial pivoting. Partial pivoting is preferred for hand computation. The general form of a system of m linear equations in n unknowns x, x, x,, xn can be represented in matrix form as under: a a a a n x b a a a a n x b = am am am amn xn bm Using matrix notation, the above system can be written in compact form as [ A]( X) = ( B) Note:. This method fails if any of he pivots become zero in such cases, by interchanging the rows we can get the non-zero pivots. Example Solve the system of equations by Gaussian elimination method with partial pivoting. x+ y+ z = 7 x+ y+ 4z = 4 x+ y+ z = 6 Solution x 7 4 y = 4 z 6 To start with, we observe that the pivot element a = ( ). However, a glance at the first column reveals that the numerically largest element is which is in second row. Hence R Copyright Virtual University of Pakistan 4

52 Thus the given equation takes the form after partial pivoting 4 x 4 y = 7 z 6 Stage I (Reduction to upper triangular form): 4 x 8 y = z Stage II (Back substitution): z = y+ = or y = x+ + 4= 8 or x= Example Solve the following system of equations by Gaussian elimination method with partial pivoting x + 4x + x + 8x = 4 4 4x + x + 5x + 4x = 4 4x + 5x + 6.5x + x = 6 4 9x + 4x + 4x + x = 4 Solution In matrix notation, the given system can be written as 4 8 x x = x x4 To start with, we observe that the pivot row, that is, the first row has a zero pivot element (a = ). This row should be interchanged with any row following it, which on becoming a pivot row should not have a zero pivot element. Copyright Virtual University of Pakistan 5

53 While interchanging rows it is better to interchange the first and fourth rows, which is called partial pivoting and get, x x = x x4 4 Stage I (Reduction to upper-triangular form): x x = x x x. 9 9 x = x x x x = x.5.5 x Stage II Back substitution x =., x =., x =., x =. 4 Example Solve the system of equations x+ y z = x 8y+ z = 5 x y+ 9z = 8 using Gauss elimination method. Solution The given system is equivalent to Copyright Virtual University of Pakistan 6

54 x 8 y 5 = 9 z 8 Α X = B the arg umented matrix is [ A B] = now making Α as an upper triangular matrix 6 5 ~ 7 R R, R R now choosin g as the pivot from the sec ond column, 6 5 ~ 7 R R, R R from this we get x+ y z = 6 5 y+ z = 7 69 z = 78 6 now by back substitution z = y = 7 ( z) = 7 () = y= now x = [ y + z] = [ + ] = so the solution is x = y = z = Example Copyright Virtual University of Pakistan 7

55 Using Gauss elimination method, solve the system..5x.96y+.85z =.95.x+ 5.y.89z = x+.5y+.5z = 6.88 Solution The given system is equivalent to x y 8.6 = z 6.88 A X = Β [ A B] = ~ R R, R R choosin g as pivot ~ R R form this, we get.5x.96y+.85z = y 5.49z = z =.685 by backward substitution.685 z = = y = = y.85z+.95 x = = Example Copyright Virtual University of Pakistan 8

56 Solve the system of equation x + x + x + x = 4 x + x + x x = 6 4 x + x x + x = 7 4 x+ x + x x4 = By using Gauss elimination method. Copyright Virtual University of Pakistan 9

57 x + x + x + x = 4 x + x + x x = 6 4 x + x x + x = 7 4 x + x + x x = 4 the given system in matrix form is x x 6 = x 7 x4 A X = B 6 [ A B] = 7 8 ~ R R, R R, R4 R 4 sin ce the element in the sec ond column is zero so int erchanging the rows ~ R 8 4 ~ R R 8 4 now the pivot is, therefore ~ 5 4 R R Copyright Virtual University of Pakistan

58 from this we get x + x + x + x = x 4 x = x x = ( ) x4 = 5 now x = 4 x = ( 8+ x4) = ( 8+ 6) = now x = + x = = now from equation x = x x x = + = 4 so x =, x =, x =, x = 4 Copyright Virtual University of Pakistan

59 Solution of Linear System of Equations and Matrix Inversion Gauss Jordon Elimination Method This method is a variation of Gaussian elimination method. In this method, the elements above and below the diagonal are simultaneously made zero. That is a given system is reduced to an equivalent diagonal form using elementary transformations. Then the solution of the resulting diagonal system is obtained. Sometimes, we normalize the pivot row with respect to the pivot element, before elimination. Partial pivoting is also used whenever the pivot element becomes zero. Example Solve the system of equations using Gauss-Jordan elimination method: x+ y+ z = 8 x+ y+ 4z = 4x+ y+ z = 6 Solution In matrix notation, the given system can be written as x 8 4 y = 4 z 6 x 8 y = 4 (-) R +R and (-4) R+R 5 z 6 Now, we eliminate y from the first and third rows using the second row. Thus, we get 5 x 6 y = 4 z 6 Before, eliminating z from the first and second row, normalize the third row with respect to the pivot element, we get 5 x 6 y = 4 z Using the third row of above Equation, and eliminating z from the first and second rows, we obtain x y = z The solution is x =, y =, z =. Example Solve the system of equation by using the Gauss Jordan elimination method x+ y+ z = x+ y+ z = x+ y+ 5z = 7 Solution Copyright Virtual University of Pakistan

60 x+ y+ z = x+ y+ z = x+ y+ 5z = 7 the given system x y = 5 z 7 the arg umented matrix may be written as / = R 9R R R, R R R R R+ 8 R, R 9R R, R 4 R, R 58R 47 thus the system reduces to reduced echlon form [ A B] so x = y = z = Example Solve the system of equations by Gauss Jordan method x + x + x = x + x x = x x + x = 9 Solution Copyright Virtual University of Pakistan

61 x+ x + x = x+ x x = x x + x = 9 the matrix form of the given system may be written as x x = x 9 the arg umented mairix may be written as = ~ 88 R 9R ~ 9 8 R R, R R R R ~ 9 8, ~ R 8R ~ R 89 R, R R ~ R + 79 R, R + R so the system gives the values of all the three var iables x = x = x = [ A/ B] Example Solve the system of equations by using Gauss Jordan method. x+ y+ z w= x+ y z+ w= 7 x+ y+ z w= 6 x+ y+ z+ w= Solution x+ y+ z w= x+ y z+ w= 7 x+ y+ z w= 6 x+ y+ z+ w= Copyright Virtual University of Pakistan

62 the system may be written as x y 7 = z 6 w the arg umented matrix may be written as 7 = 6 4 R R, R R, R4 R R R, ( R + R ), R + R R+ 5 R, R R, R4 R R R4, R + R4, R + R4 the system maybe written as x y = z w the values of all the ariables are x =, y =, z =, w= [ A/ B] Crout s ReductionMethod 4 Here the coefficient matrix [A] of the system of equations is decomposed into the product of two matrices [L] and [U], where [L] is a lower-triangular matrix and [U] is an upper-triangular matrix with s on its main diagonal. For the purpose of illustration, consider a general matrix in the form L U = A [ ][ ] [ ] Copyright Virtual University of Pakistan 4

63 l u u a a a l l u = a a a l l l a a a The sequence of steps for getting [L] and [U] are given below: Step I: Multiplying all the rows of [L] by the first column of [U], we get l = a, l = a, l = a Thus, we observe that the first column of [L] is same as the first column of [A]. Step II: Multiplying the first row of [L] by the second and third columns of [U], we obtain lu = a, lu = a Or a a u =, u = l l Thus, the first row of [U] is obtained. We continue this process, getting alternately the column of [L] and a row of [U]. Step III: Multiply the second and third rows of [L] by the second column of [U] to get lu + l = a, lu + l = a This gives l = a lu, l = a lu Step IV: Now, multiply the second row of [L] by the third column of [U] which yields lu + lu = a a lu u = l Step V: Lastly, we multiply the third row of [L] by the third column of [U] and get lu + lu + l = a This gives l = a lu lu Thus, the above five steps determine [L] and [U]. This algorithm can be generalized to any linear system of order n. Consider a system of equations ax + a x + a x = b ax + a x + a x = b ax + a x + a x = b In matrix notation as [A](X) = (B). Let [A] = [L] [U], then we get, [ L][ U]( X) = ( B) Substituting [U] (X) = (Z) in Eq. we obtain [L] (Z) = (B) lz = b lz + l z = b lz + l z + l z = b Having computed z, z and z, we can compute x, x, and x from equation [U] (X) = (Z) or from u u x z u x = z x z This method is also known as Cholesky reduction method. This technique is widely used in the numerical solutions of partial differential equations. Copyright Virtual University of Pakistan 5

64 This method is very popular from computer programming point of view, since the storages space reserved for matrix [A] can be used to store the elements of [L] and [U] at the end of computation. This method fails if any a ii = Example Solve the following system of equations by Crout s reduction method 5x x + x = 4 7x + x 5x = 8 x + 7x + 4x = Solution Let the coefficient matrix [A] be written as [L] [U]. Thus, l u u 5 l l u 7 5 = l l l 7 4 Step I: Multiply all the rows of [L] by the first column of [U], we get l = 5, l = 7, l = Step II: Multiply the first row of [L] by the second and third columns of [U], we have lu =, lu = u =, u = 5 5 STEP III: Multiply the nd and rd rows of [L] by the nd column of [U], we get 4 9 lu + l = or l = + = lu + l = 7 or l = 7+ = 5 5 STEP IV: Multiply the nd row of [L] by the rd column of [U] lu + lu = u = u = 9 STEP V: Finally, multiply the rd row of [L] with the rd column of [U], we obtain lu + lu + l = 4 7 l = 9 Thus, the given system of equations takes the form [L][U][X] = (B). x x = x That is, Copyright Virtual University of Pakistan 6

65 z z = z Let [U](X) = (Z), then [ L]( Z ) = (4 8 ) T Or z z = z Which gives utilizing these values of z, the Eq becomes By back substitution method, we obtain x =, x =, x = This is the required solution. Example Solve the following system x y+ z = x+ 4y+ x = 5x+ y+ z = Solution x y+ z = x+ 4y+ x = 5x+ y+ z = the given system is AX = B x A= 4 X y = Β = 5 z let LU = A u u u L = l U = u u l l u u u u lu lu u lu u = lu lu + lu lu + lu + u 5 Copyright Virtual University of Pakistan 7

66 here u =, u = u = lu = l = 5 lu = 5 l = 5 lu + u = 4 u = 4 lu = 4 ( )( ) = lu + u = u = lu = ( )= 7 9 lu + lu = 5 l = [5 lu ] = u 5 5 lu + lu + u u = lu lu = 5 L = 5 U = y let UX = Y where y = y, then LY = B, y y ie. y = y and x y y y = 5 9 z y 5 now eqn () implies y = 4 y + y = y = y + y + y = y = 5 5 Copyright Virtual University of Pakistan 8

67 x y+ z =, y+ 7 z =, z = 5 by back substitution x = 4, y = and z = Example Solve the following system x+ y+ 4z = 4 x+ 4y+ z = x+ y+ 4z = Solution x+ y+ 4z = 4 x+ 4y+ z = x+ y+ 4z = the given system is AX = B 8 x 4 A= 4 X y = Β = 4 z let LU = A u u u L = l U = u u l l u u u u 8 lu lu u lu u = lu lu + lu lu + lu + u 4 here u =, u = u = 8 l u = l = l u = l = Copyright Virtual University of Pakistan 9

68 5 lu + u = 4 u = 4 lu = 4 ( )( ) = lu + u = u = 4 lu = 4 ()= l u + l u = l = [ l u ] = u lu + lu + u u = 4 lu lu = 4 8 L = U = 5 4 y let UX = Y where y = y, then LY = B, y y 4 ie. y = y and x y y = y z y now eqn () implies y = 4 y = y + y = y = y = we also have x+ y+ 8z = 4 y 5z = 4z = by back substitutiton z = 4 7 y = + 5z = + 5( ) = x = 4 y 8z = 4 ( ) 8( ) = Copyright Virtual University of Pakistan

69 Solution of Linear System of Equationsand Matrix Inversion Jacobi s Method This is an iterative method, where initial approximate solution to a given system of equations is assumed and is improved towards the exact solution in an iterative way. In general, when the coefficient matrix of the system of equations is a sparse matrix (many elements are zero), iterative methods have definite advantage over direct methods in respect of economy of computer memory Such sparse matrices arise in computing the numerical solution of partial differential equations Let us consider ax + ax + + anxn = b ax + ax + + anxn = b an x+ anx+ + annxn = b In this method, we assume that the coefficient matrix [A] is strictly diagonally dominant, that is, in each row of [A] the modulus of the diagonal element exceeds the sum of the off-diagonal elements. We also assume that the diagonal element do not vanish. If any diagonal element vanishes, the equations can always be rearranged to satisfy this condition. Now the above system of equations can be written as b a an x = x xn a a a b a an x = x xn a a a b a n an nn ( ) xn = x xn ann ann a nn We shall take this solution vector ( x, x,..., x ) T n as a first approximation to the exact solution of system. For convenience, let us denote the first approximation vector by () () () ( x, x,..., x ) got after taking as an initial starting vector. n Substituting this first approximation in the right-hand side of system, we obtain the second approximation to the given system in the form Copyright Virtual University of Pakistan

70 () b a () a n () x = x xn a a a () b a () an () x = x xn a a a a () bn an () nn ( ) () xn = x xn ann ann a nn This second approximation is substituted into the right-hand side of Equations and obtain the third approximation and so on. This process is repeated and (r+)th approximation is calculated ( r+ ) b a ( r) a n ( r) x = x xn a a a ( r+ ) b a ( r) an ( r) x = x xn a a a a ( r+ ) bn an ( r) nn ( ) ( r) xn = x xn ann ann a nn Briefly, we can rewrite these Equations as n a ( r+ ) bi ij ( r) xi = xj, a a ii j= j i ii r =,,..., i =,,..., n It is also known as method of simultaneous displacements, ( r ) since no element of x + is used in this iteration until every element is computed. i A sufficient condition for convergence of the iterative solution to the exact solution is n aii > aij, i =,,..., n When this condition (diagonal dominance) is true, Jacobi s j= j method converges Example Find the solution to the following system of equations using Jacobi s iterative method for the first five iterations: 8x+ y 4z = 95 7x+ 5y+ z = 4 x+ 8y+ 9z = 7 Solution Copyright Virtual University of Pakistan

71 95 4 x = y+ z y = x z z = x y T Taking the initial starting of solution vector as (,,), approximation as from Eq.,we have the first () x.446 () y =. () z.448 Now, using Eq.,the second approximation is computed from the equations () () () x =.446.5y +.48z () () () y =..46x.5z () () () z =.448.5x.759y Making use of the last two equations we get the second approximation as () x.9976 () y =.9 () z.744 Similar procedure yields the third, fourth and fifth approximations to the required solution and they are tabulated as below; Variables Iteration number r x y z Copyright Virtual University of Pakistan

72 Example Solve the system by jacobi s iterative method 8x y+ z = 4x+ y z = 6x+ y+ z = 5 (Perform only four iterations) Solution Consider the given system as 8x y+ z = 4x+ y z = 6x+ y+ z = 5 the system is diagonally do min ant x= [ + y z] 8 y = [ 4 x+ z] z = [ 5 6 x y] we start with an initial aproximation x = y = z = substituting these first iteration x = [ + () () ] =.5 8 y = [ 4() + ] = z = [ 5 6() () ] = Second iteration x = [ + () ( ) ] = y = [ 4(.5) ] =.5666 z = [ 5 6(.5) () ] = Copyright Virtual University of Pakistan 4

73 third iteration x = [ + (.5666) ( ) ] = y = [ 4(.8958) ] =. z = [ 5 6(.8958) (.5666) ] = fourth iteration x4 = [ + (.) ( ) ] = y4 = [ 4(.5456) ] =.997 z4 = [ 5 6(.5456) (.) ] =.898 Example Solve the system by jacobi s iterative method x+ 4y+ 5z = 54.8 x+ y+ z = 9.66 x+ y z = 7.74 (Perform only four iterations) Solution Consider the given system as x + 4y+ 5z = 54.8 x+ y+ z = 9.66 x+ y z = 7.74 the system is not diagonally do min ant we rearrange the system x+ y z = 7.74 x+ y+ z = 9.66 x+ 4y+ 5z = 54.8 x= [ 7.74 y+ z] y = 9.66 x z z = 54.8 x 4 y 5 [ ] [ ] Copyright Virtual University of Pakistan 5

74 we start with an initial aproximation x = y = z = substituting these first iteration x = [ 7.74 () + () ] =.774 y = [ 9.66 () () ] =.8 z = [ 54.8 () 4() ] =.65 5 Second iteration x = [ (.65) ] =.98 y = [ (.65) ] =.7667 z = [ 54.8 (.774) 4(.8) ] = third iteration x = [ ( ) ] =.8789 y = [ ( ) ] =.958 z = [ 54.8 (.98) 4(.7667) ] = fourth iteration x4 = [ (.77985) ] =.7874 y4 = [ ( ) ] = z4 = [ 54.8 (.8789) 4(.958) ] = Copyright Virtual University of Pakistan 6

75 Solution of Linear System of Equations and Matrix Inversion Gauss Seidel Iteration Method It is another well-known iterative method for solving a system of linear equations of the form ax + ax + + anxn = b ax + ax + + anxn = b an x+ anx+ + annxn = b n In Jacobi s method, the (r + )th approximation to the above system is given by Equations ( r+ ) b a ( r) a n ( r) x = x xn a a a ( r+ ) b a ( r) an ( r) x = x xn a a a a ( r+ ) bn an ( r) nn ( ) ( r) xn = x xn ann ann a nn ( r ) Here we can observe that no element of x + ( ) i replaces x r i entirely for the next cycle of computation. ( r ) In Gauss-Seidel method, the corresponding elements of x + i replaces those of ( r ) x i as soon as they become available. Hence, it is called the method of successive displacements. For illustration consider ax + ax + + anxn = b ax + ax + + anxn = b an x+ anx+ + annxn = b n In Gauss-Seidel iteration, the (r + )th approximation or iteration is computed from: ( r+ ) b a ( r) a n ( r) x = x xn a a a ( r+ ) b a ( r+ ) an ( r) x = x xn a a a a ( r+ ) bn an ( r+ ) nn ( ) ( r+ ) xn = x xn ann ann a nn Thus, the general procedure can be written in the following compact form Copyright Virtual University of Pakistan

76 b a a x x x i n ( r+ ) i ij ( r+ ) ij ( r) i = j j aii j= aii j= i+ aii for all i,,..., = n and r =,,... To describe system in the first equation, we substitute the r-th approximation into the right-hand side and denote the result by ( r x + ). In the second equation, we substitute ( r+ ) ( r) ( r) ( r ) ( x, x,..., x n ) and denote the result by x + ( r+ ) ( r+ ) ( r) ( r) In the third equation, we substitute ( x, x, x4,..., x n ) and denote the result by ( r ) x +, and so on. This process is continued till we arrive at the desired result. For illustration, we consider the following example : Note The difference between jacobi s method and gauss Seidel method is that in jacobi s method the approximation calculated are used in the next iteration for next approximation but in Gauss-seidel method the new approximation calculated is instantly replaced by the previous one. Example Find the solution of the following system of equations using Gauss-Seidel method and perform the first five iterations: 4x x x = x+ 4x x4 = x+ 4x x4 = x x + 4x = 4 Solution The given system of equations can be rewritten as x =.5 +.5x +.5x x =.5 +.5x+.5x 4 x =.5 +.5x+.5x4 x4 =.5 +.5x +.5x Taking x = x = x4 = on the right-hand side of the first equation of the system, we get () x =.5. Taking x = x4 = and the current value of x, we get from the nd equation of the system () x =.5 + (.5)(.5) + =.65 Further, we take x4 = and the current value of x the system we obtain from the third equation of () x = (.5)(.5) =.75 Copyright Virtual University of Pakistan

77 Now, using the current values of x and x the fourth equation of system gives () x 4 =.5 + (.5)(.65) + (.5)(.75) =.5 The Gauss-Seidel iterations for the given set of equations can be written as ( r+ ) ( r) ( r) x =.5 +.5x +.5x ( r+ ) ( r+ ) ( r) x =.5 +.5x +.5x4 ( r+ ) ( r+ ) ( r) x =.5 +.5x +.5x4 ( r+ ) ( r+ ) ( r+ ) x4 =.5 +.5x +.5x Now, by Gauss-Seidel procedure, the nd and subsequent approximations can be obtained and the sequence of the first five approximations are tabulated as below: Variables Iteration number r x x x x Example Solve the system by Gauss-Seidel iterative method 8x y+ z = 4x+ y z = 6x+ y+ z = 5 (Perform only four iterations) Solution Consider the given system as Copyright Virtual University of Pakistan

78 8x y+ z = 4x+ y z = 6x+ y+ z = 5 the system is diagonally do min ant x= [ + y z] 8 y = [ 4 x+ z] z = [ 5 6 x y] we start with an initial aproximation x = y = z = substituting these first iteration x = [ + () () ] =.5 8 y = [ 4(.5) + ] =.999 z = [ 5 6(.5) (.999) ] =.4994 Second iteration x = [ + y z] = [ + (.999) (.4994) ] = y = [ 4x + z] = [ 4(.9986) ] =.774 z = [ 5 6x y] = [ 5 6(.9986) (.774) ] =.947 third iteration x = [ + (.774) (.947) ] = y = [ 4(.668) ] =.9856 z = [ 5 6(.668) (.9856) ] =.9776 Copyright Virtual University of Pakistan 4

79 fourth iteration x4 = [ + (.9856) (.9776) ] =.65 8 y4 = [ 4(.65) ] = z4 = [ 5 6(.65) (.98567) ] =.898 Example Solve the system by suing Gauss-seidel iteration method 8x+ 4y z = x+ y+ z = 4 x+ 7y+ 4z = 5 Solution 8x+ 4y z = x+ y+ z = 4 x+ 7y+ 4z = 5 the given system is diagonally do min ant so we will make it diagonaaly do min ant by iterchanaginhg the equations 8x+ 4y z = x+ 7y+ 4z = 5 x+ y+ z = 4 hence we can apply Gauss Seidel method from the above equations x= [ 4 y+ z ] 8 y = [5 x 4 z ] 7 z = [4 x y ] Copyright Virtual University of Pakistan 5

80 First approximation putting y = z = x = [] = puting x =.4857, z = y = [5 (.4857) 4()] = putting x =.4857, y = z = [ (.94697)] =.7844 Second iteration x = [ 4(.94697) ] = y = [5 (.98965) 4(.7844)] = z = [ ( )] = third iteration x = [ 4( ) ] = y = [5 (.98759) 4(.84868)] = z = [ (.5974)] = fourth iteration x4 = [ 4(.5974) ] = y4 = [5 (.998) 4(.84868)] = z 4 = [4.998 (.5758)] = Example Using Gauss-Seidel iteration method, solve the system of the equation. x y z w= x+ y z w= 5 x y+ z w= 7 x y z+ w= 9 Copyright Virtual University of Pakistan 6

81 (Perform only four iterations) Solution x y z w= x+ y z w= 5 x y+ z w= 7 x y z+ w= 9 it is diagonally do min anat and we may write eqaution as x= [ + y+ z+ w ] y = [5 + x+ z+ w ] z = [7 + x+ y+ w ] w= [ 9 + x+ y+ z ] first approximation putting y = z = w = on RHS of (), we get x =. y = [5 + (.)] =.56 putting x =., y =.56 and w = z = [ ] =.886 putting x =., y =.56 and z =.886 w = [ (.886)] =.68 second iteration x = [ + (.56) ] =.8869 y = [5 + (.8869) ] =.954 z = [ (.68)] = w = [ ( )] =.4765 third iteration x = [+ (.954) ] = Copyright Virtual University of Pakistan 7

82 y = [5 + (.98645) ] = z = [ (.4765)] =.9949 w = [ (.9949)] =.4647 fourth iteration x4 = [ + ( ) ] = y4 = [5 + ( ) ] = z4 = [ (.4647)] = w4 = [ ( )] =.7677 Note When to stop the iterative processes,we stop the iterative process when we get the required accuracy means if your are asked that find the accurate up to four places of decimal then we will simply perform up to that iteration after which we will get the required accuracy. If we calculate the root of the equation and its consecutive values are.89565,.96665,.99565,.99655,.99745,.9978 Here the accuracy up to seven places of decimal is achieved so if you are asked to acquire the accuracy up to six places of decimal then we will stop here. But in the solved examples only some iteration are carried out and accuracy is not considered here. Copyright Virtual University of Pakistan 8

83 Solution of Linear System of Equations and Matrix Inversion Relaxation Method This is also an iterative method and is due to Southwell.To explain the details, consider again the system of equations ax + ax + + anxn = b ax + ax + + anxn = b an x+ anx+ + annxn = b n Let ( ) ( ) ( ) ( ) X p = ( x p, x p,..., x p ) T n ( ) be the solution vector obtained iteratively after p-th iteration. If R p i denotes the residual of the i-th equation of system given above, that is of ax i + ax i + + ax in n = bi defined by ( p) ( p) ( p) ( p) Ri = bi ai x aix ainxn we can improve the solution vector successively by reducing the largest residual to zero at that iteration. This is the basic idea of relaxation method. To achieve the fast convergence of the procedure, we take all terms to one side and then reorder the equations so that the largest negative coefficients in the equations appear on the diagonal. Now, if at any iteration, R i is the largest residual in magnitude, then we give an increment to x i ; aii being the coefficient of xi Ri dxi = aii In other words, we change x i. to ( xi + dxi) to relax R i that is to reduce R i Example to zero. Solve the system of equations 6x x + x = x+ x 8x = 5 x 7x + x = by the relaxation method, starting with the vector (,, ). Solution At first, we transfer all the terms to the right-hand side and reorder the equations, so that the largest coefficients in the equations appear on the diagonal. Copyright Virtual University of Pakistan

84 Thus, we get = 6x+ x x = x+ 7x x = 5 x x + 8x after interchanging the nd and rd equations. Starting with the initial solution vector (,, ), that is taking x = x = x =, we find the residuals R =, R =, R = 5 of which the largest residual in magnitude is R, i.e. the rd equation has more error and needs immediate attention for improvement. Thus, we introduce a change, dxin x which is obtained from the formula R 5 dx = = =.875 a 8 Similarly, we find the new residuals of large magnitude and relax it to zero, and so on. We shall continue this process, until all the residuals are zero or very small. Iteration Residuals Maximum Difference Variables number R R R R i dx x x x i Iteration Residuals Maximum Difference Variables Copyright Virtual University of Pakistan

85 number R R R Ü Ü x x x /8 = /(-6) = /7 = /(-6) = /8 = Iteration Residuals Maximum Difference Variables number R R R Ü Ü x x x /7 = /(-6) = = At this stage, we observe that all the residuals R, R and R are small enough and therefore we may take the corresponding values of xi at this iteration as the solution. Hence, the numerical solution is given by Copyright Virtual University of Pakistan

86 x =.7, x =.99, x =.7, The exact solution is x =., x =., x =. Example Solve by relaxation method, the equation x y z = 6 x y z = 7 x y+ z = 8 Solution The residual r, r, r are given by r = 6 x+ y+ z r = 7+ x y+ z r = 8+ x+ y z The operation table is as follows x y z r r r - L - L - L The relaxation table is as follows x y z r r r L L5=L4+L - - L6=L5+L L7=L6+L Explanation () In L4,the largest residual is 8.to reduce it, To reduce it,we give an increment of 8 8 = =.8 c the resulting residulas are obtained by L + () L, ielinel. 4 5 () In line L5 the largest residual is 9 Copyright Virtual University of Pakistan 4

87 9 9 Increment=.9 b = = The resulting residuals (= L 6 ) = L 5 +. L () In line L 6,the largest residual is Increment = a = The resulting residuals (= L 6 ) = L 5 +. L Exact solution is arrived and it is x=,y=,z= Example Solve the system by relaxation method, the equations 9x y+ z = 7 x+ y z = 5 x y z = 7 Solution The residuals r, r, r are given by 9x y+ z = 9 x+ y z = 5 x y z = 7 here r = 9 9x+ y z r = 5 x y+ z r = 7 x+ y+ z Operation table x y z r r r Relaxation table is x y z r r r Copyright Virtual University of Pakistan 5

88 Then x=.89+.8=.98;y=+.6+.9=.694 And z=+.9+.5=.95 Now substituting the values of x,y,z in (),we get r=9-9(.98)+.649-(.95)=-.46 r=5-.98-(.649)+(.95)=-.754 r=-7-(.98) +(.649) +(.95) =-. Which is agreement with the final residuals. Copyright Virtual University of Pakistan 6

89 Solution of Linear System of Equations and Matrix Inversion Matrix Inversion Consider a system of equations in the form [ A]( X) = ( B) One way of writing its solution is in the form ( X ) = [ A] ( B) Thus, the solution to the system can also be obtained if the inverse of the coefficient matrix [A] is known. That is the product of two square matrices is an identity matrix [ A][ B] = [ I] then, [ B] = [ A] and [ A] = [ B] Every square non-singular matrix will have an inverse. Gauss elimination and Gauss-Jordan methods are popular among many methods available for finding the inverse of a matrix. Gaussian Elimination Method In this method, if A is a given matrix, for which we have to find the inverse; at first, we place an identity matrix, whose order is same as that of A, adjacent to A which we call an augmented matrix. Then the inverse of A is computed in two stages. In the first stage, A is converted into an upper triangular form, using Gaussian elimination method In the second stage, the above upper triangular matrix is reduced to an identity matrix by row transformations. All these operations are also performed on the adjacently placed identity matrix. Finally, when A is transformed into an identity matrix, the adjacent matrix gives the inverse of A. In order to increase the accuracy of the result, it is essential to employ partial pivoting. Example Use the Gaussian elimination method to find the inverse of the matrix A = 4 5 Solution At first, we place an identity matrix of the same order adjacent to the given matrix. Thus, the augmented matrix can be written as Copyright Virtual University of Pakistan

90 4 5 Stage I (Reduction to upper triangular form): Let R, R and R denote the st, nd and rd rows of a matrix. In the st column, 4 is the largest element, thus interchanging R and R to bring the pivot element 4 to the place of a, we have the augmented matrix in the form 4 5 Divide R by 4 to get Perform R R, which gives Perform R R R in the above equation, which yields Now, looking at the second column for the pivot, the max (/4. /4) is /4. Therefore, we interchange R and R in the last equation and get Copyright Virtual University of Pakistan

91 Now, divide R by the pivot a = /4, and obtain Performing R ( 4) R R yields Finally, we divide R by (/), thus getting an upper triangular form Stage II Reduction to an identity matrix (/4)R + R and (-5/)R + R Finally, performing R ( 4) R R we obtain Copyright Virtual University of Pakistan

92 Thus, we have A = 5 Example Find the inverse of the using gauss elimination method. 4 9 Solution We have the argumented system [ A/ I] = 4 9 ~ R ( ) R, R ( ) R 7 7 ~ R 7R 7 x x x now if x x x x x x then the system is the inverse of the given matrix the system is eqivalent to three systems Copyright Virtual University of Pakistan 4

93 x x = x x x = x 7 x = x = x = x = x = x = x = x = x = and the inverse martix is x x = x by back substitution, the three systems of equation may be written as Example Fine the inverse of the matrix Solution 4 using gauss elimination method. Copyright Virtual University of Pakistan 5

94 4 [ Α/Ι ] = 4 5 ~ R R, R R x x x now if x x x is the inverse of the given matrix, then the system() is x x x equivalent to three systems 4 x 5 x = x x 5 x = x x 5 x = 9 x x = x = x = x = x = x = 7 8 x = x = x = Copyright Virtual University of Pakistan 6

95 and the inverse martix is = Gauss - Jordan Method This method is similar to Gaussian elimination method, with the essential difference that the stage I of reducing the given matrix to an upper triangular form is not needed. However, the given matrix can be directly reduced to an identity matrix using elementary row operations. Example Find the inverse of the given matrix by Gauss-Jordan method A = 4 5 Solution Let R, R and R denote the st, nd and rd rows of a matrix. We place the identity matrix adjacent to the given matrix. So the augmented matrix is given by 4 5 Performing R 4 R R, we get 5 5 Now, performing R R R, we obtain 5 4 Copyright Virtual University of Pakistan 7

96 Carrying out further operations R + R R and R+ R R, we arrive at Now, dividing the third row by, we get Further, we perform R + 4 R R, and R + 5R R to get Finally, multiplying R by, we obtain Hence, we have A = 5 Exercise Solve all the above examples solved by Gauss elimination by using gauss Jordan method. Copyright Virtual University of Pakistan 8

97 Eigen Value Problems Let [A] be an n x n square matrix. Suppose, there exists a scalar and a vector X = ( x x x ) T n such that [ A]( X) = λ( X) d ( e ax ax ) = a ( e ) dx d (sin ax) = a (sin ax) dx Then λ is the eigen value and X is the corresponding eigenvector of the matrix [A]. We can also write it as [ A λi]( X) = ( O) This represents a set of n homogeneous equations possessing non-trivial solution, provided A λi = This determinant, on expansion, gives an n-th degree polynomial which is called characteristic polynomial of [A], which has n roots. Corresponding to each root, we can solve these equations in principle, and determine a vector called eigenvector. Finding the roots of the characteristic equation is laborious. Hence, we look for better methods suitable from the point of view of computation. Depending upon the type of matrix [A] and on what one is looking for, various numerical methods are available. Power Method and Jacobi s Method Note! We shall consider only real and real-symmetric matrices and discuss power and Jacobi s methods Power Method To compute the largest eigen value and the corresponding eigenvector of the system [ A]( X) = λ( X) where [A] is a real, symmetric or un-symmetric matrix, the power method is widely used in practice. Procedure Step : Choose the initial vector such that the largest element is unity. Step : The normalized vector Step :The resultant vector is again normalized. () v is pre-multiplied by the matrix [A]. Copyright Virtual University of Pakistan

98 Step 4: This process of iteration is continued and the new normalized vector is repeatedly pre-multiplied by the matrix [A] until the required accuracy is obtained. At this point, the result looks like ( k) ( k ) ( k) u = [ A] v = q v k Here, q k is the desired largest eigen value and ( k ) v is the corresponding eigenvector. Example Find the eigen value of largest modulus, and the associated eigenvector of the matrix by power method [ A] = Solution () υ We choose an initial vector T as (,,). Then, compute first iteration 7 () () u [ A] v 4 5 = = 9 4 Now we normalize the resultant vector to get () 6 () u = 4 7 = qv The second iteration gives, 9 7 () () 6 67 u [ A] v 4 5 = 7 = () = = qv. Continuing this procedure, the third and subsequent iterations are given in the following slides u.4564 [ A] v 4 5 = = () () Copyright Virtual University of Pakistan

99 = = = qv (4) () u = [ A] v = (4) = = qv (5) (4) u = [ A] v = (5) = = qv 5. After rounding-off, the largest eigen value and the corresponding eigenvector as accurate to two decimals are.44 λ =.84 ( X ) =.76. Example Find the first three iterations of the power method of the given matrix Solution () Copyright Virtual University of Pakistan

100 we choose finitial vector as v () = (,,) first iteration () () u = [ A] v = = + = () () by diagonali sin g 8 u.8 = = qv. second iteration () () u = [ A] v = = + = () () by diagonali sin g.8 u = = qv.4.48 t third iteration () () u = [ A] v = = + + = now daigonali sin g now normali sin g Example Find the first three iteration of the power method applied on the following matrices use x = t 4 (,,) Copyright Virtual University of Pakistan 4

101 Solution USE x = st iterations () t 4 (,,) + () () u = [ Α ] x = = + = + now we normalize the resul tan t vector to get u u u u 8 = 8 = 8 = qx () () () () () = [ Α ] x = = + + = = = = [ Α ] x = = = () () Exercise Find the largest eigen value and the corresponding eigen vector by power method after () fourth iteration starting with the initial vector υ = (,,) T Copyright Virtual University of Pakistan 5

102 [ A] = Let λ, λ,, λn be the distinct eigen values of an n x n matrix [A], such that λ > λ > > λn and suppose v, v,, vn are the corresponding eigen vectors Power method is applicable if the above eigen values are real and distinct, and hence, the corresponding eigenvectors are linearly independent. Then, any eigenvector v in the space spanned by the eigenvectors v, v,, vn can be written as their linear combination v= cv + cv+ + cnvn Pre-multiplying by A and substituting Av = λv, Av = λv, Avn = λnvn We get λ λ n Av = λ cv + c v + + cn vn λ λ Again, pre-multiplying by A and simplifying, we obtain λ λ n Av= λ cv + c v + + cn vn λ λ Similarly, we have r r r r λ λ n Av= λ cv + c v + + cn vn λ λ and r r r r λ λn ( λ) = n n λ λ A v cv c v c v Now, the eigen value λ can be computed as the limit of the ratio of the corresponding components of r+ and A v. That is, r A v Copyright Virtual University of Pakistan 6

103 r+ r r+ ( A v) p r r ( Av) p λ λ = = Lt, p =,,, n λ Here, the index p stands for the p-th component in the corresponding vector Sometimes, we may be interested in finding the least eigen value and the corresponding eigenvector. In that case, we proceed as follows. We note that [ A]( X) = λ( X). Pre-multiplying by [ A ], we get [ A ][ A]( X) = [ A ] λ( X) = λ[ A ]( X) Which can be rewritten as [ A ]( X) = ( X) λ which shows that the inverse matrix has a set of eigen values which are the reciprocals of the eigen values of [A]. Thus, for finding the eigen value of the least magnitude of the matrix [A], we have to apply power method to the inverse of [A]. Copyright Virtual University of Pakistan 7

104 Jacobi s Method Definition An n x n matrix [A] is said to be orthogonal if T [ A] [ A] = [ I], T i.e.[ A] = [ A] In order to compute all the eigen values and the corresponding eigenvectors of a real symmetric matrix, Jacobi s method is highly recommended. It is based on an important property from matrix theory, which states that, if [A] is an n x n real symmetric matrix, its eigen values are real, and there exists an orthogonal matrix [S] such that the diagonal matrix D is [ S ][ A][ S] This digitalization can be carried out by applying a series of orthogonal transformations S, S,..., S, n aij Let A be an n x n real symmetric matrix. Suppose be numerically the largest element amongst the offdiagonal elements of A. We construct an orthogonal matrix S defined as s ij sin θ, s = = ji sin θ, sii = cos θ, sjj = cosθ While each of the remaining off-diagonal elements are zero, the remaining diagonal elements are assumed to be unity. Thus, we construct S as under i-th column j-th column cosθ sinθ i-th row S = sinθ cosθ j-th row Where cos θ, sin θ, sinθ and cosϑ are inserted in ( ii, ),( i, j),( ji, ),( j, j) th positions respectively, and elsewhere it is identical with a unit matrix. Now, we compute T D = S AS = S AS Since S is an orthogonal matrix, such that.after the transformation, the elements at the position (i, j), (j, i) get annihilated, that is dij and dji reduce to zero, which is seen as follows: Copyright Virtual University of Pakistan

105 dii dij d ji d jj cosθ sinθ aii aij cosθ sinθ = a sinθ cosθ aij a jj sinθ cosθ aii cos θ sinθ cosθ + ajj sin θ ( ajj aii ) sinθ cosθ + aij cos θ ( aji aii )sinθ cosθ + aij cos θ aii sin θ + ajj cos θ aij sinθcosθ Therefore, d ij = only if, ajj aii aij cos θ + sin θ = That is if aij tan θ = aii ajj Thus, we choose θ such that the above equation is satisfied, thereby, the pair of off-diagonal elements dij and dji reduces to zero.however, though it creates a new pair of zeros, it also introduces non-zero contributions at formerly zero positions. Also, the above equation gives four values of, but to get the least possible rotation, we choose Copyright Virtual University of Pakistan

106 Example Find all the eigen values and the corresponding eigen vectors of the matrix by Jacobi s method A = Solution The given matrix is real and symmetric. The largest off-diagonal element is found to be a = a =. Now, we compute aij a 4 tan θ = = = = aii ajj a a This gives, θ = π 4 Thus, we construct an orthogonal matrix Si as π π cos 4 sin 4 S = = π π sin 4 cos 4 The first rotation gives, D = S AS = = We observe that the elements d and d got annihilated. To make sure that calculations are correct up to this step, we see that the sum of the diagonal elements of D is same as the sum of the diagonal elements of the original matrix A. As a second step, we choose the largest off-diagonal element of D and is found to be d = d =, and compute 4 tan θ = d d d = = This again gives θ = π 4 Thus, we construct the second rotation matrix as Copyright Virtual University of Pakistan

107 S = At the end of the second rotation, we get D = S DS = 5 = This turned out to be a diagonal matrix, so we stop the computation. From here, we notice that the eigen values of the given matrix are 5, and. The eigenvectors are the column vectors of S = SS Therefore S = = Example Find all the eigen values of the matrix by Jacobi s method. A = Solution Here all the off-diagonal elements are of the same order of magnitude. Therefore, we can choose any one of them. Suppose, we choose a as the largest element and compute tan θ = = Which gives, θ = π 4. Then cosθ = sinθ = and we construct an orthogonal matrix S such that Copyright Virtual University of Pakistan

108 S = The first rotation gives D = S AS = = Now, we choose d = As the largest element of D and compute d tan θ = = d d o θ = 7 4. Now we construct another orthogonal matrix S, such that S = At the end of second rotation, we obtain.64.5 D = S DS = Now, the numerically largest off-diagonal element of D is found to be d =.68 and compute..68 tan θ =.65 o θ = 5 4. Thus, the orthogonal matrix is S = At the end of third rotation, we get Copyright Virtual University of Pakistan

109 D = S DS = To reduce D to a diagonal form, some more rotations are required. However, we may take.64,.86 and.979 as eigen values of the given matrix. Example Using Jacobi s method, find the eigenvalues and eigenvectors of the following matrix, / / / / /4 / /4 /5 Solution: The given matrix is real and symmetric. The l argest off diagonal element is found to be a = a = Now we comute aij a tan θ = = = = a ii ajj a a tan θ = = 8.55 Thus we construct an orthogonal matrix S as cos8.55 sin S = sin 8.55 cos = The first rotation gives, D = S AS / / = / / / / /4 / = We see that sum of the diagonal elements of D =.5 Copyright Virtual University of Pakistan 4

110 And the sum of the diagonal elements of A =.5 This means that our question is going right. As a second step we choose the largest of off-diagonal element of D, which is d = d =.4, and comput d (.4) tan θ = = =.77 d d.68. tan (.77) θ = = 8.84 Thus we construct an orthogonal matrix S as S cos8.84 sin = = sin8.84 cos Thus the rotation gives, D = S DS = = We again see that sum of the diagonal elements of D =.5 Also the sum of the diagonal elements of A =.5 This means that our question is going right. Hence the eigenvalues are.48,.66 and.59 and the corresponding eigenvectors are the columns of S.Where S = SS = = Copyright Virtual University of Pakistan 5

111 Copyright Virtual University of Pakistan 6

112 Interpolation Introduction Finite differences play an important role in numerical techniques, where tabulated values of the functions are available. For instance, consider a function y = f( x). As x takes values x, x, x,, x n, Let the corresponding values of y be y, y, y,, y n. That is, for given a table of values, ( xk, yk), k =,,,, n; The process of estimating the value of y, for any intermediate value of x, is called interpolation. The method of computing the value of y, for a given value of x, lying outside the table of values of x is known as extrapolation. If the function f (x) is known, the value of y corresponding to any x can be readily computed to the desired accuracy. For interpolation of a tabulated function, the concept of finite differences is important. The knowledge about various finite difference operators and their symbolic relations are very much needed to establish various interpolation formulae. Finite Difference Operators Forward Differences For a given table of values ( xk, yk), k =,,,..., nwith equally spaced abscissas of a function y = f( x), we define the forward difference operator as follows yi = yi+ yi, i=,,...,( n ) To be explicit, we write y = y y y = y y y = y y n n n These differences are called first differences of the function y and are denoted by the symbol yi Here, is called the first difference operator Similarly, the differences of the first differences are called second differences, defined by y = y y, y = y y Thus, in general yi = yi+ yi Here is called the second difference operator. Thus, continuing, we can define, r-th difference of y, as r r r yi = yi+ yi By defining a difference table as a convenient device for displaying various differences, the above defined differences can be written down systematically by constructing a difference table for values ( xk, yk), k =,,...,6 Forward Difference Table Copyright Virtual University of Pakistan

113 This difference table is called forward difference table or diagonal difference table. Here, each difference is located in its appropriate column, midway between the elements of the previous column. Please note that the subscript remains constant along each diagonal of the table. The first term in the table, that is y is called the leading term, while the differences y, y, y,... are called leading differences Example Construct a forward difference table for the following values of x and y: Solution Copyright Virtual University of Pakistan

114 Example Express y and y in terms of the values of the function y. Solution: Noting that each higher order difference is defined in terms of the lower order difference, we have y = y y = ( y y) ( y y) = y y+ y And y = y y = ( y y) ( y y) = ( y y) ( y y) ( y y) + ( y y) = y y + y y Hence, we observe that the coefficients of the values of y, in the expansion of y, y, are binomial coefficients. Thus, in general, we arrive at the following result: - n n n n n y = yn Cyn + Cyn Cyn + + ( ) y Example Show that the value of yn can be expressed in terms of the leading value y and the leading differences n y, y,, y. Solution The forward difference table will be Copyright Virtual University of Pakistan

115 y y = y or y = y + y y y = y or y = y+ y y y = y or y = y + y Similarly, y y = y or y = y + y y y y or y y y = = + Similarly, we can also write y y = y or y = y + y y y y or y y y = = + y = ( y + y ) + ( y + y ) = y + y + y Similarly, we can symbolically write y Hence, we obtain OR y = y + y = ( y + y ) + ( y + y ) n y y = y + y + y + y = ( + ) y = ( + ) y, = ( + ) y,... y n = ( + ) y = ( + ) n y y = y + C y + C y + C y + + y n n n n n n i yn = Ci y i= Copyright Virtual University of Pakistan 4

116 Backward Differences For a given table of values ( xk, yk), k =,,,..., nof a function y = f (x) with equally spaced abscissas, the first backward differences are usually expressed in terms of the backward difference operator as yi = yi yi i= n, ( n ),, To be explicit, we write OR y = y y y = y y yn = yn yn The differences of these differences are called second differences and they are denoted by y, y,, y n. y = y y That is y = y y yn = yn yn Thus, in general, the second backward differences are yi = yi yi, i= n,( n ),..., While the k-th backward differences are given as k k k yi = yi yi, i= n,( n ),..., k These backward differences can be systematically arranged for a table of values ( x, y ), k =,,...,6 shown below. k k Backward Difference Table Copyright Virtual University of Pakistan

117 From this table, it can be observed that the subscript remains constant along every backward diagonal. Example Show that any value of y can be expressed in terms of y n and its backward differences. Solution: From We get yi = yi yi i= n, ( n ),, y = n y n yn yn = yn yn From We get yi = yi yi i= n n y = y y n n n,,( ),..., From these equations, we obtain y = y y + y n n n n Similarly, we can show that y = y y + y y n n n n n Symbolically, these results can be rewritten as follows: y y y n n n... y n r = ( ) y = ( ) n y = ( ) y r = ( ) y n n n n n r r y = n r y n C y + n C y + n ( ) yn Central Differences In some applications, central difference notation is found to be more convenient to represent the successive differences of a function. Here, we use the symbol δ to represent central difference operator and the subscript of δ y bb for any difference as the average of the subscripts In General, δ y = y y, δ y = y y, = δ yi yi+ ( ) yi ( ) Copyright Virtual University of Pakistan

118 Higher order differences are defined as follows: δ y = δy δy i i+ ( ) i ( ) δ y = δ y δ y n n n i i+ ( ) i ( ) These central differences can be systematically arranged as indicated in the Table Thus, we observe that all the odd differences have a fractional suffix and all the even differences with the same subscript lie horizontally. The following alternative notation may also be adopted to introduce finite difference operators. Let y = f (x) be a functional relation between x and y, which is also denoted by y x. Suppose, we are given consecutive values of x differing by h say x, x + h, x +h, x +h, etc. The corresponding values of y are yx, yx+ h, yx+ h, yx+ h, As before, we can form the differences of these values. Thus y = y y = f( x+ h) f( x) x x+ h x yx = yx+ h yx Similarly, y = y y = f( x) f( x h) x x x h h h y = y y = f x+ f x δ x x+ ( h/) x ( h/) To be explicit, we write Copyright Virtual University of Pakistan

119 y = y y y = y y y = y y n n n OR yi = yi yi i= n, ( n ),, y = y y y = y y y = y y n n n In General, δ y = y y, δ y = y y, δ yi = yi+ ( ) yi ( ) Higher order differences are defined as follows: δ y = δy δy i i+ ( ) i ( ) δ y = δ y δ y n n n i i+ ( ) i ( ) Copyright Virtual University of Pakistan 4

120 Shift operator, E Let y = f (x) be a function of x, and let x takes the consecutive values x, x + h, x + h, etc. We then define an operator having the property E f( x) = f( x+ h) Thus, when E operates on f (x), the result is the next value of the function. Here, E is called the shift operator. If we apply the operator E twice on f (x), we get E f( x) = E[ E f( x)] = E[ f( x+ h)] = f( x+ h) Thus, in general, if we apply the operator E n times on f (x), we get n E f( x) = f( x+ nh) OR n E yx = y x + nh 4 Ey = y, E y = y, E y = y,, E y = y 4 4 The inverse operator E - is defined as E f( x) = f( x h) Similarly n E f( x) = f( x nh) Average Operator, µ ; it is defined as h h µ f( x) = f x+ + f x = yx+ ( h/) + y x ( h/) Differential Operator, D it is defined as d Df ( x) = f ( x) = f ( x) dx d D f( x) = f( x) = f ( x) dx Important Results Using {,, δ, E, µ } Also y = y y = Ey y x x+ h x x x = ( E ) y = E x y = y y = y E y x x x h x x = E ( ) y x Copyright Virtual University of Pakistan

121 And = E = E E δ yx = yx+ ( h/) yx ( h/) = E y E y / / x / / ( ) = E E y δ = E E The definition of µ and E similarly yields / / x x µ yx = yx+ ( h/) + yx ( h/) ( / / ) = E + E yx µ ( / / ) E E = + We know that Eyx = yx+ h = f ( x + h) h Eyx = f ( x) + hf ( x) + f ( x) +! h = f( x) + hdf( x) + D f( x) +! hd h D hd = f ( x) = e y!! Thus hd = log E Example: Prove that hd = log( + ) = log( ) = sinh ( µδ ) Solution: Using the standard relations we have hd = log E = log( + ) = log E = log( ) x Copyright Virtual University of Pakistan

122 Example Prove that Solution From the definition, we have: () ) ) ) 4) 5) ( / / )( / / E E E E ) µδ = + ( E E ) = ( hd hd = e e ) = sinh( hd) hd = sinh µδ δ + δ µ = + / δ E = µ + δ = + δ + ( δ /4) E µδ = + + µδ = µδ = + = + µδ = + ( E + E ) = ( E+ E ) 4 4 δ / / + = + ( E E ) = ( E+ E ) / / / / ( E E )( E E ) ( E E ) () () µ + ( δ /) = ( ) E + E + E E = E / / / / / Copyright Virtual University of Pakistan

123 δ + δ + ( δ /4) = + 4 / / / / ( ) ( ) ( ) / / E E E E + E E E + = + E E E + E E ( / / )( / / ) E + E E E = + (4) (5) = E = µδ = E + E E E = E E = ( + E ) = + ( E ) E = + = + E E / / / / ( )( ) ( ) ( / / )( / / E E E E ) µδ = + ( E E ) = = (+ + ) = ( + ) Copyright Virtual University of Pakistan 4

124 Interpolation Newton s Forward Difference Formula Let y = f (x) be a function which takes values f(x ), f(x + h), f(x +h),, corresponding to various equi-spaced values of x with spacing h, say x, x + h, x + h,. Suppose, we wish to evaluate the function f (x) for a value x + ph, where p is any real number, then for any real number p, we have the operator E such that p E f( x) = f( x+ ph). p p f ( x + ph) = E f( x) = ( + ) f( x) p( p ) p( p )( p ) = + p f( x)!! f( x + ph) = f( x) + p f( x) p( p ) p( p )( p ) + f ( x) + f( x)!! p( p ) ( p n+ ) n + + f( x ) + Error n! This is known as Newton s forward difference formula for interpolation, which gives the value of f(x + ph) in terms of f(x ) and its leading differences. This formula is also known as Newton-Gregory forward difference interpolation formula. Here p=(x-x )/h. An alternate expression is p( p ) p( p )( p ) yx = y + p y + y + y +!! p( p )( p n+ ) n + y + Error n! If we retain (r + ) terms, we obtain a polynomial of degree r agreeing with y at x x, x,, xr. This formula is mainly used for interpolating the values of y near the beginning of a set of tabular values and for extrapolating values of y, a short distance backward from y Example: Evaluate f (5), given the following table of values: Solution: The forward differences are tabulated as Copyright Virtual University of Pakistan

125 We have Newton s forward difference interpolation formula p( p ) p( p )( p ) yx = y + p y + y + y +!! p( p )( p n+ ) n + y + Error n! Here we have x =, y = 46, y =, 4 y = 5, y =, y = Let y 5 be the value of y when x = 5, then x x 5 p = = =.5 h (.5)(.5 ) f(5) = y5 = 46 + (.5)() + ( 5) (.5)(.5 )(.5 ) (.5)(.5 )(.5 )(.5 ) + () + ( ) 6 4 = f (5) = correct to four decimal places. Example Find Newton s forward difference, interpolating polynomial for the following data: Solution; The forward difference table to the given data is Copyright Virtual University of Pakistan

126 Since, rd and 4 th leading differences are zero, we have Newton s forward difference interpolating formula as p( p ) y = y + p y + y In this problem, x =., y =.4, y =.6, y =.4, and x. p = = x. Substituting these values, ( x )(x ) y = f( x) =.4 + ( x )(.6) + (.4) This is the required Newton s interpolating polynomial. Example Estimate the missing figure in the following table: Solution Since we are given four entries in the table, the function y = f (x) can be represented by a polynomial of degree three. In particular, Equivalently, = f( x) Constant f x = x 4 and ( ), 4 f x = ( ) 4 ( E ) f( x ) = Copyright Virtual University of Pakistan

127 Expanding, we have That is, 4 ( E 4E + 6E 4E+ ) f( x ) = f( x ) 4 f( x ) + 6 f( x ) 4 f( x ) + f( x ) = 4 Using the values given in the table, we obtain 4 f( x ) = which gives f (x ), the missing value equal to 4. Example Consider the following table of values x F(x) Find f (.6) using Newton s Forward Difference Formula. Solution x y = f( x) y y y 4 y pp ( ) pp ( )( p ) yx = y+ p y+ y+ y!! pp ( )( p )( p ) 4 pp ( )( p )...( p n+ ) n + y+ + y 4! n! Where x =., y =.4, y =.484, x x.6. p = = = 4.6 y =.5, y =., y =.5 h. y x.6(.6 ).6(.6 )(.6 ).6(.6 )(.6 )(.6 ) =.4+.6(.484) + (.5 ) + (.) + (.5)!! 4!.6(.6)(.4).6(.6)(.4)(.4) = (.) + (.5) 6 4 = =.5 Copyright Virtual University of Pakistan 4

128 Example Find a cubic polynomial in x which takes on the values -,,, 7, 57 and 7, when x =,,,, 4 and 5 respectively. Solution Here, the observations are given at equal intervals of unit width. To determine the required polynomial, we first construct the difference table Difference Table Since the 4 th and higher order differences are zero, the required Newton s interpolation formula p( p ) p( p )( p ) f ( x + ph) = f( x) + p f( x) + f( x) + f( x) 6 Here x x x p = = = x h f( x ) = 6 f( x ) = f( x ) = 6 Substituting these values into the formula, we have xx ( ) xx ( )( x ) f( x) = + 6 x+ () + (6) 6 f( x) = x x + 7x, The required cubic polynomial. Copyright Virtual University of Pakistan 5

129 NEWTON S BACKWARD DIFFERENCE INTERPOLATION FORMULA For interpolating the value of the function y = f (x) near the end of table of values, and to extrapolate value of the function a short distance forward from y n, Newton s backward interpolation formula is used Derivation Let y = f (x) be a function which takes on values f (x n ), f (x n -h), f (x n -h),, f (x ) corresponding to equispaced values x n, x n -h, x n -h,, x. Suppose, we wish to evaluate the function f (x) at (x n + ph), where p is any real number, then we have the shift operator E, such that p p p f ( xn + ph) = E f( xn) = ( E ) f( xn) = ( ) f( xn) Binomial expansion yields, p( p+ ) p( p+ )( p+ ) f( xn + ph) = + p + + +!! p( p+ )( p+ ) ( p+ n ) n + + Error f ( xn ) n! That is p( p+ ) p( p+ )( p+ ) f( xn + ph) = f( xn) + p f( xn) + f( xn) + f( xn) +!! p( p+ )( p+ ) ( p+ n ) n + f( xn ) + Error n! This formula is known as Newton s backward interpolation formula. This formula is also known as Newton s-gregory backward difference interpolation formula. If we retain (r + )terms, we obtain a polynomial of degree r agreeing with f (x) at x n, x n-,, x n-r. Alternatively, this formula can also be written as Here p( p+ ) p( p+ )( p+ ) yx = yn + p yn + yn + yn +!! p( p+ )( p+ ) ( p+ n ) n + yn + Error n! x xn p = h Example For the following table of values, estimate f (7.5). Copyright Virtual University of Pakistan

130 Solution The value to be interpolated is at the end of the table. Hence, it is appropriate to use Newton s backward interpolation formula. Let us first construct the backward difference table for the given data Difference Table Since the 4 th and higher order differences are zero, the required Newton s backward interpolation formula is p( p+ ) yx = yn + p yn + yn! p( p+ )( p+ ) + yn! In this problem, x x n p = = =.5 h yn = 69, yn = 4, yn = 6 (.5)(.5) y7.5 = 5 + (.5)(69) + (4) (.5)(.5)(.5) + (6) 6 = = Example The sales for the last five years is given in the table below. Estimate the sales for the year 979 Copyright Virtual University of Pakistan

131 Solution Newton s backward difference Table In this example, p = =.5 and yn = 5, yn =, 4 yn =, yn = 5 Newton s interpolation formula gives (.5)(.5) (.5)(.5)(.5) y979 = 57 + (.5)5 + () + () 6 (.5)(.5)(.5)(.5) + (5) 4 = Therefore, y 979 = 5.7 Example Consider the following table of values x F(x) Use Newton s Backward Difference Formula to estimate the value of f(.45). Solution x y=f(x) y y y 4 y y Copyright Virtual University of Pakistan

132 x x n p = = =.5, yn =, y n =., y n = -., y n = -.4, h. 5 y n = -.5 As we know that p( p+ ) p( p+ )( p+ ) yx = yn + p yn + yn + yn!! p( p+ )( p+ )( p+ ) 4 p( p+ )( p+ )( p+ )( p+ 4) 5 + yn + yn 4! 5! (.5 )(.5 + ) (.5 )(.5 + )(.5 + ) y x = (.5 )() + (.) + (.)!! (.5 )(.5 + )(.5 + )(.5 + ) (.5 )(.5 + )(.5 + )(.5 + )(.5 + 4) + (.4) +.5 4! 5! ( ) = = Copyright Virtual University of Pakistan 4

133 LAGRANGE S INTERPOLATION FORMULA Newton s interpolation formulae developed earlier can be used only when the values of the independent variable x are equally spaced. Also the differences of y must ultimately become small. If the values of the independent variable are not given at equidistant intervals, then we have the basic formula associated with the name of Lagrange which will be derived now. Let y = f (x) be a function which takes the values, y, y, y n corresponding to x, x, x n. Since there are (n + ) values of y corresponding to (n + ) values of x, we can represent the function f (x) by a polynomial of degree n. Suppose we write this polynomial in the form. n n f ( x) = Ax + Ax + + A n or in the form, y = f( x) = a( x x)( x x) ( x xn ) + a( x x)( x x) ( x xn ) + a( x x)( x x) ( x xn ) + + an( x x)( x x) ( x xn ) Here, the coefficients a are so chosen as to satisfy this equation by the (n + ) pairs k (x i, y i ). Thus we get Therefore, Similarily,we obtain and y = f x = a x x x x x x x x n a ( ) ( )( )( ) ( ) y = ( x x )( x x ) ( x x ) y a = ( x x )( x x ) ( x x ) yi ai = ( xi x)( xi x) ( xi xi )( xi xi+ ) ( xi xn) yn an = ( x x )( x x ) ( x x ) n n n n Substituting the values of a, a,, a we get n ( x x)( x x) ( x xn) ( x x)( x x) ( x xn) y = f( x) = y + y+ ( x x)( x x) ( x xn) ( x x)( x x) ( x xn) ( x x)( x x) ( x xi )( x xi+ ) ( x xn) + yi + ( x x )( x x ) ( x x )( x x ) ( x x ) The Lagrange s formula for interpolation Copyright Virtual University of Pakistan n n i i i i i i+ i n

134 This formula can be used whether the values x, x,, x n are equally spaced or not. Alternatively, this can also be written in compact form as y = f( x) = L ( x) y + L ( x ) y + L( x ) y + + L ( x ) y Where n = Lk( x) yk k = i i i n n n n = Lk( x) f( xk) Copyright Virtual University of Pakistan k = ( x x)( x x) ( x xi )( x xi+ ) ( x xn) Li ( x) = ( x x )( x x ) ( x x )( x x ) ( x x ) i i i i i i+ i n We can easily observe that, L( x ) = and L( x ) =, i j. Thus introducing Kronecker delta notation, if Li( xj) = δij =, if Further, if we introduce the notation i i n i= i = j i j ( x) = ( x x ) = ( x x )( x x ) ( x x ) i That is ( x) is a product of (n + ) factors. Clearly, its derivative ( x) contains a sum of (n + ) terms in each of which one of the factors of will ( x) be absent. We also define, P ( x) = ( x x ) k i k i Which is same as ( x) except that the factor (x x k ) is absent. Then ( x) = P( x) + P( x) + + P ( x) But, when x = x k, all terms in the above sum vanishes except P k (x k ) Hence ( x ) = P ( x ) = ( x x ) ( x x )( x ) ( x x ) k k k k k k xk k+ k n Pk( x) Pk( x) Lk ( x) = = Pk( xk) ( xk) ( x) = ( x x ) ( x ) k Finally, the Lagrange s interpolation polynomial of degree n can be written as n ( x) y( x) = f( x) = f( xk ) ( x x ) ( x ) Example n k k = k k n = L ( x) f( x ) = L ( x) y k k k k k= k= n i j n

135 Find Lagrange s interpolation polynomial fitting the points y() = -, y() =, y(4) =, y(6) =. Hence find y(5). Solution The given data can be arranged as Using Lagrange s interpolation formula, we have ( x )( x 4) x 6) ( x )( x 4) x 6) yx ( ) = f( x) = ( ) + () ( )( 4)( 6) ( )( 4)( 6) ( x )( x )( x 6) ( x )( x )( x 4) + () + () (4 )(4 )(4 6) (6 )(6 )(6 4) On simplification, we get yx ( ) = 5x + 5x 46x+ ( ) ( 7 9 6) = x + x x+ Which is required Lagrange s interpolation polynomial. Now, y(5) = 75. Example Given the following data, evaluate f () using Lagrange s interpolating polynomial. Solution Using Lagrange s formula, ( x x)( x x) ( x x)( x x) ( x x)( x x) f ( x) = f( x) + f( x) + f( x) ( x x)( x x) ( x x)( x x) ( x x)( x x) Therefore, ( )( 5) ( )( 5) ( )( ) f () = () + (4) + () ( )( 5) ( )( 5) (5 )(5 ) = = 6.5 Example Find the interpolating polynomial for the data using Lagrange s formula Copyright Virtual University of Pakistan

136 x -4 F(x) -5-4 Solution: As Lagrange s formula for interpolating polynomial is given by, ( x x)( x x) ( x x)( x x) ( x x)( x x) y( x) = f( x) = f( x) + f( x) + f( x) ( x x)( x x) ( x x)( x x) ( x x)( x x) ( x )( x+ 4) ( )( ) ( ) ( x )( x+ 4) ( )( ) ( ) ( x )( x ) = ( )( ) ( 4 ) = ( x + x 8) ( x + x 4) ( x x+ ) = x x+ x x+ x + x = x + + x = x x+ 5 Which is the required polynomial. Copyright Virtual University of Pakistan 4

137 DIVIDED DIFFERENCES Let us assume that the function y = f (x) is known for several values of x, (x i, y i ), for i=,,..n. The divided differences of orders,,,, n are now defined recursively as: yx [ ] = yx ( ) = y is the zero-th order divided difference The first order divided difference is defined as y y yx [, x] = x x Similarly, the higher order divided differences are defined in terms of lower order divided differences by the relations of the form yx [, x] yx [, x] yx [, x, x] = x x Generally, yx [, x,, x] = yx [, x,, x] yx [, x,, x ] [ ] n n n xn x Standard format of the Divided Differences We can easily verify that the divided difference is a symmetric function of its arguments. That is, y y yx [, x] = yx [, x] = + x x x x Now yx [, x, x] = yx [, x] yx [, x] x x y y y y x x x x x x = Copyright Virtual University of Pakistan

138 Therefore y y y yx [, x, x] = + + ( x x)( x x) ( x x)( x x) ( x x)( x x) This is symmetric form, hence suggests the general result as yx [,, x] OR k y y y ( x x ) ( x x ) ( x x ) ( x x ) ( x x ) ( x x ) k = k k k k k yx [,..., x] = k k k i= i= i j y i ( x x ) Example: Construct the Newton s divided difference table for values of x=,,,4,5,6 and f(x)= -,,5,48,5,9. Solution: i j x F(x) st difference nd difference - 5-/- =6 -(-)= -5/4- =9 5 5-=5 57-/5- = = 87-57/6-4 rd difference 9-6/4-= -9/5- = 5-/ = =87 =5 = NEWTON S DIVIDED DIFFERENCE INTERPOLATION FORMULA Let y = f (x) be a function which takes values y, y,, y n corresponding to x = x i, i =,,, n. We choose an interpolating polynomial, interpolating at x = x i, i =,,, n in the following form y = f( x) = a + a( x x) + a( x x)( x x) + + an( x x)( x x) ( x xn ) Copyright Virtual University of Pakistan

139 Here, the coefficients a k are so chosen as to satisfy above equation by the (n + ) pairs (x i, y i ). Thus, we have yx ( ) = f( x) = y = a yx ( ) = f( x) = y = a + a( x x) y( x) = f( x) = y = a + a( x x) + a( x x)( x x) yn = a + a( xn x) + a( xn x)( xn x) + + an( xn x) ( xn xn ) The coefficients a, a,, a n can be easily obtained from the above system of equations, as they form a lower triangular matrix. The first equation gives a = y( x) = y The second equation gives y y a = = y[ x, x ] x x Third equation yields y y ( x x) y[ x, x] a = ( x x)( x x) Which can be rewritten as y y y y+ ( x x) ( x x) y[ x, x] x x a = ( x x)( x x) That is y y+ y[ x, x]( x x) y[ x, x] y[ x, x] a = = ( x x)( x x) x x Thus, in terms of second order divided differences, we have a = y[ x, x, x] Similarly, we can show that Newton s divided difference interpolation formula can be written as y = f( x) = y + ( x x) y[ x, x] + ( x x)( x x) y[ x, x, x] + + ( x x )( x x ) ( x x ) y[ x, x,..., x ] n Newton s divided differences can also be expressed in terms of forward, backward and central differences. They can be easily derived. Assuming equi-spaced values of abscissa, we have n Copyright Virtual University of Pakistan

140 y y y yx [, x] = = x x h yx [, x] yx [, x] h h y x x = = = x x h! h yx [,, ] By induction, we can in general arrive at the result n y yx [, x,, xn ] = n nh! Similarly, y y y yx [, x] = = x x h y y yx [, x] yx [, x] h h y x x = = = x x h! h yx [,, ] y y In General,we have n yn yx [, x,..., xn ] = n nh! Also, in terms of central differences, we have y y δ y/ yx [, x] = = x x h δ y/ δ y/ yx [, x] yx [, x] y yx [, x, x] h h δ = = = x x h! h In general, we have the following pattern m δ y m yx [, x,..., xm ] = m ( m)! h m δ + ym+ (/ ) m+ = m+ yx [, x,..., x ] (m+ )! h Copyright Virtual University of Pakistan 4

141 Example Find the interpolating polynomial by (i) Lagrange s formula and (ii) Newton s divided difference formula for the following data. Hence show that they x 4 y 5 represent the same interpolating polynomial. X Y st D.D nd D.D rd D.D / -/ 4 5 / /6 Solution The divided difference table for the given data is constructed as follows: i) Lagrange s interpolation formula gives ( x )( x )( x 4) ( x )( x )( x 4) y = f( x) = () + () ( )( )( 4) ( )( )( 4) ( x )( x )( x 4) ( x )( x )( x ) + () + (5) ()( )( 4) 4(4 )(4 ) ( x 7x + 4x 8) x 6x + 8x x 5x + 4x = + 8 5( x x + x) + 4 x x = + x + 4 (ii) Newton s divided difference formula gives y = f( x) = + ( x )() + ( x )( x ) + ( x )( x )( x ) x x = + x + 4 We observe that the interpolating polynomial by both Lagrange s and Newton s divided difference formulae is one and the same. Copyright Virtual University of Pakistan

142 Note! Newton s formula involves less number of arithmetic operations than that of Lagrange s. Example Using Newton s divided difference formula, find the quadratic equation for the following X 4 Y 4 data. Hence find y (). Solution: The divided difference table for the given data is constructed as: X Y st D.D nd D.D - / 4 4 Now, using Newton s divided difference formula, we have y = + ( x )( ) + ( x )( x ) ( = x x+ 4) Hence, y () =. Example Find the equation of a cubic curve which passes through the points (4, -4), (7, 8), (9, 7) and (, 5) using Dividing Difference Formula. Solution The Newton s divided difference table is given by X Y st divided difference nd divided difference rd divided difference Copyright Virtual University of Pakistan

143 Newton ' s Divided Difference formula is y = f( x) = y + ( x x) y[ x, x] + ( x x)( x x) y[ x, x, x] + + ( x x)( x x) ( x xn ) y[ x, x,..., xn] = 4 + ( x 4)(4) + ( x 4)( x 7)(6) + ( x 4)( x 7)( x 9)() = 4 + ( x 4) { 4 + 6x + x 6x+ 6} = 4 + ( x 4)( x 7) = 4+ x 7x 4x + 8 = x 4x 7x 5 Which is the required polynomial. Example A function y = f (x) is given at the sample points x = x, x and x. Show that the Newton s divided difference interpolation formula and the corresponding Lagrange s interpolation formula are identical. Solution For the function y = f (x), we have the data ( xi, yi), i =,,. The interpolation polynomial using Newton divided difference formula is given as y = f( x) = y + ( x x) y[ x, x] + ( x x)( x x) y[ x, x, x] Using the definition of divided differences, we can rewrite the equation in the form ( y y) y y = y( x x) + ( x x)( x x) ( x x) ( x x)( x x) y y + + ( x x)( x x) ( x x)( x x) ( x x) ( x x)( x x) = + y ( x x ) ( x x )( x x ) ( x x) ( x x)( x x) ( x x)( x x) + + y+ y ( x x) ( x x)( x x) ( x x)( x x) On simplification, it reduces to ( x x)( x x) ( x x)( x x) ( x x)( x x) y = y + y+ y ( x x)( x x) ( x x)( x x) ( x x)( x x) which is the Lagrange s form of interpolation polynomial. Hence two forms are identical. Newton s Divided Difference Formula with Error Term Following the basic definition of divided differences, we have for any x Copyright Virtual University of Pakistan

144 yx ( ) = y + ( x x) yxx [, ] yxx [, ] = yx [, x] + ( x x) yxx [,, x] yxx [,, x] = yx [, x, x] + ( x x) yxx [,, x, x] yxx [,,..., xn ] = yx [, x,..., xn] + ( x xn) yxx [,,..., xn] Multiplying the second Equation by (x x ), third by (x x )(x x ) and so on, and the last by (x x )(x x ) (x x n- ) and adding the resulting equations, we obtain y( x) = y + ( x x) y[ x, x] + ( x x)( x x) y[ x, x, x] + + ( x x)( x x) ( x xn ) y[ x, x,..., xn] + ε ( x) Where ε ( x) = ( x x)( x x) ( x xn) y[ x, x,..., xn] Please note that for x = x, x,, x, the error term ε ( x) vanishes n Copyright Virtual University of Pakistan 4

145 Error Term in Interpolation Formulae We know, if y (x) is approximated by a polynomial P n (x) of degree n then the error is given by ε ( x) = yx ( ) Pn ( x), Where ε ( x) = ( x x)( x x) ( x xn) y[ x, x,..., xn] Alternatively it is also expressed as ε ( x) = ( xyxx ) [,,..., xn ] = K ( x) Now, consider a function F (x), such that F( x) = y( x) Pn ( x) K ( x) Determine the constant K in such a way that F(x) vanishes for x = x, x,, x n and also for an arbitrarily chosen point x, which is different from the given (n + ) points. Let I denotes the closed interval spanned by the values x,...,,. xn x Then F (x) vanishes (n + ) times in the interval I. By Rolle s theorem F ( x) vanishes at least (n + ) times in the interval I, F ( x) at least n times, and so on. ( n+ Eventually, we can show that F ) ( x) vanishes at least once in the interval I, say at x = ξ Thus, we obtain ( ) ( ) ( ) n + ( ) n + ( ) n + = y ξ Pn ξ K ( ξ) Since P n (x) is a polynomial of degree n, its (n + )th derivative is zero. Also, from the definition of Π ( x) Therefore we get Hence ( n+ ) ( x) = ( n+ )!. K = ( n+ y ) ( ξ ) ( n + )! ( n+ y ) ( ξ ) ε ( x) = yx ( ) Pn ( x) = ( x) ( n + )! for some in the interval I. Thus the error committed in replacing y (x) by either Newton s divided difference formula or by an identical Lagrange s formula is ( n+ ) y ( ξ ) ε ( x) = ( x) y[ x, x,..., xn ] = ( x) ( n + )! Copyright Virtual University of Pakistan

146 INTERPOLATION IN TWO DIMENSIONS Let u be a polynomial function in two variables, say x and y, in particular quadratic in x and cubic in y, which in general can be written as u = f( x, y) = a + ax+ ay+ ax + axy 4 + ay 5 + ay 6 + ayx 7 + ayx 8 + ayx 9 + ayx + ayx This relation involves many terms. If we have to write a relation involving three or more variables, even low degree polynomials give rise to long expressions. If necessary, we can certainly write, but such complications can be avoided by handling each variable separately. If we let x, a constant, say x = c, the equation simplifies to the form u = = b + b y+ b y + b y x= c Now we adopt the following procedure to interpolate at a point (, m) in a table of two variables, by treating one variable a constant say x = x. The problem reduces to that of a single variable interpolation. Any one of the methods discussed in preceding sections can then be applied to get f (x, m). Then we repeat this procedure for various values of x say x = x, x,, x n keeping y constant. Thus, we get a new table with y constant at the value y = m and with x varying. We can then interpolate at x =. Example Tabulate the values of the function f (x) = x +y -y for x =,,,,4 and y =,,,,4. Using the table of the values, compute f(.5,.5) by numerical double interpolation Solution The values of the function for the given values of the given values of x and y are given in X.. Y the following table Copyright Virtual University of Pakistan

147 Y Numerical Analysis MTH6 f Cont! Using quadratic interpolation in both x and y directions we need to consider three points in x and y directions. To start with we have to treat one variable as constant, say x. Keeping x =.5, y =.5 as near the center of the set, we choose the table of values corresponding to x =,, and y =,, 4. Cont! The region of fit for the construction of our interpolation polynomial is shown in different color in the table X. Y Cont! Thus using Newton s forward difference formula, we have At x= Y f f f Cont! Similarly At x= Y f f f Copyright Virtual University of Pakistan

148 Cont! Similarly At x= Y f f f y y.5 with p = = =.5 h Cont! y y.5 with p = = =.5 h p( p ) f (,.5) = f + p f + f! (.5)(.5) = + (.5)(4) + () = 9.75 Cont! (.5)(.5) f (,.5) = 6 + (.5)(4) + () =.75 (.5)(.5) f (,.5) = + (.5)(4) + () = 7.75 Cont! Therefore we arrive at the following result At x=.5 Y f f f Copyright Virtual University of Pakistan 4

149 Cont! Cont!.5 Now defining p = =.5 (.5)(.5) f (.5,.5) = (.5)() + () = 5 From the functional relation, we also find that f (.5,.5) = (.5) + (.5).5 = 5 And hence no error in interpolation!!! Copyright Virtual University of Pakistan 5

150 DIFFERENTIATION USING DIFFERENCE OPERATORS Introduction Consider a function of single variable y = f (x). If the function is known and simple, we can easily obtain its derivative (s) or can evaluate its definite integral However, if we do not know the function as such or the function is complicated and is given in a tabular form at a set of points x,x,,x n, we use only numerical methods for differentiation or integration of the given function. We shall discuss numerical approximation to derivatives of functions of two or more variables in the lectures to follow when we shall talk about partial differential equations. In what follows, we shall derive and illustrate various formulae for numerical differentiation of a function of a single variable based on finite difference operators and interpolation. Subsequently, we shall develop Newton-Cotes formulae and related trapezoidal rule and Simpson s rule for numerical integration of a function. We assume that the function y = f (x) is given for the values of the independent variable x = x + ph, for p =,,, and so on. To find the derivatives of such a tabular function, we proceed as follows Case I: Using forward difference operator and combining equations and Ef( x) = f( x+ h) = E hd = log E = log( + ) Remember the Differential operator, D is known to represents the property d Df ( x) = f ( x) = f ( x) dx d " D f( x) = f( x) = f ( x) dx This would mean that in terms of : Therefore 4 5 D = + + h 4 5 f( x) f( x) Df ( x) = f ( x) = f ( x) + h 4 5 f( x) f( x) d + = f ( x ) 4 5 dx Copyright Virtual University of Pakistan

151 in other words 4 y y y Dy = y = y + + h 4 Also, we can easily verify 4 D = + + h = + + h 6 y y d y Dy = = y = dx h y y Case II: Using backward difference operator, we have On expansion, we have we can also verify that Hence and hd = log( ) 4 D = h 4 4 D = h = h 6 d y n = Dy n = y n dx 4 yn yn y n = yn h 4 y n = D yn = ( yn yn yn yn h The formulae for Dy and D y are useful to calculate the first and second derivatives at the beginning of the table of values in terms of forward differences; while formulae for y n and y n Copyright Virtual University of Pakistan

152 The formulae for Dy and D y are useful to calculate the first and second derivatives at the beginning of the table of values in terms of forward differences; while formulae for y and y n n Case III: Using central difference operator δ and following the definitions of differential operator D, central difference operator δ and the shift operator E, we have / / hd / hd / δ = E E = e e hd = sinh Therefore, we have hd δ = sinh But, 5 x x sinh x= x x Using the last two equations we get hd δ δ = + δ That is, Therefore D δ δ δ h = + d y = y = Dy dx 5 y y y h δ 4 δ 64 δ = + Also 4 6 D = δ δ δ + h 9 Hence 4 6 y = D y = δ y δ y δ y + h 9 For calculating the second derivative at an interior tabular point, we use the equation Copyright Virtual University of Pakistan

153 4 6 D = δ δ δ + h 9 while for computing the firstderivative at an interior tabular point, we in general use another convenient form for D, which is derived as follows. Multiply the right hand side of d y y 5 Dy y y y dx h δ 4 δ 64 δ = = = + by µ + ( δ 4) which is unity and noting the Binomial expansion + δ = δ δ δ we get µ 4 D = δ + δ h δ δ + δ 4 64 On simplification, we obtain µ 4 D δ δ δ h 6 5 = + Therefore the equation can also be written in another useful form as µ 5 y = D= δ y δ y+ δ y h 6 The last two equations for y and y respectively are known as Stirling s formulae for computing the derivatives of a tabular function. Similar formulae can be derived for computing higher order derivatives of a tabular function. The equation for y can also be written as µ () () 5 () () () 7 y = δ y δ y+ δ y δ y+ h! 5! 7! In order to illustrate the use of formulae derived so far, for computing the derivatives of a tabulated function, we shall consider some examples in the next lecture. Copyright Virtual University of Pakistan 4

154 Copyright Virtual University of Pakistan 5

155 DIFFERENTIATION USING DIFFERENCE OPREATORS: Applications: Remember Using forward difference operator, the shift operator, the backward difference operator and the average difference operator, we obtained the following formulae: 4 y y y Dy = y = y + + h 4 y y d y Dy = = y = dx h y y d yn yn y y n n = Dy n = y n = y n dx h y = D y = ( y + y + y + y + n n h n n n 6 n 4 6 y = D y = δ y δ y δ y + h 9 d y y 5 Dy y y y dx h δ 4 δ 64 δ = = = + Recall from what we mentioned in the last lecture that for calculating the second derivative at an interior tabular point, we use the equation 4 6 D = δ δ δ + h 9 While for computing the first derivative at an interior tabular point, we in general use another convenient form for D, which is derived as follows. Multiply the right hand side of d y y Dy y y 5 y dx h δ 4 δ 64 δ = = = + µ By + ( δ 4) Which is unity and noting the Binomial expansion δ = δ + δ δ We get µ 4 5 D = δ + δ δ δ + δ h Copyright Virtual University of Pakistan

156 f( x) f ( x ) Simplification we get µ 4 5 D = δ δ + δ h 6 Therefore the equation can also be written in another useful form as µ 5 y = D= δ y δ y+ δ y h 6 The last two equations y andy respectively are known as Stirling s formulae for computing the derivatives of a tabular function. Similar formulae can be derived for computing higher order derivatives of a tabular function. The equation for y can also be written as µ () () 5 () () () 7 y = δ y δ y δ y δ y h + +! 5! 7! In order to illustrate the use of formulae derived so far, for computing the derivatives of a tabulated function, we consider the following example : Example: Compute f () and f (.) from the following tabular data. x F(x) Solution Since x = and. appear at and near beginning of the table, it is appropriate to use formulae based on forward differences to find the derivatives. The difference table for the given data is depicted below: x f(x) f ( x) f ( x) f ( x) 4 f ( x) 5 f ( x) Using forward difference formula for i.e ( ), D f x Copyright Virtual University of Pakistan

157 f( x) f( x) D f( x) = f( x) f( x) + h 6 we obtained 5 f () = (.84) (). () + = 6 Also, using the formula for Df ( x ), we have Df ( x) = f ( x) h f ( x) f( x) f( x) Hence, f (.) =.4 + =.. 4 Example Find y (.) and y (.) from the table. x Y(x) Solution: Since x=. occurs at the end of the table, it is appropriate to use backward difference formulae for derivatives. The backward difference table for the given data is shown below: x f(x) y y y y Using backward difference formulae for y ( x) and y ( x), we have Therefore, y y y y n = yn h 4 4 n n n Copyright Virtual University of Pakistan

158 y (.) = = 5(.84) = Also 4 y n = y n + yn + yn h Therefore y (.) = ( (.94) (.) = 5(.585) = Example Given the table of values, estimate, y // (.) x y Solution Since x =. appear at beginning of the table, it is appropriate to use formulae based on forward differences to find the derivatives. The difference table for the given data is depicted below: x y f ( x) f ( x) f ( x) 4 f ( x) 5 f ( x) H=. Using forward difference formula for D f( x ), D f ( x) = f ( x) f ( x) f( x) f( x) h + 6 We obtain Copyright Virtual University of Pakistan 4

159 5 f (.) = (.4) (-.4) (.) + 6 f (.) = (.967)(.4) (.84)(-.4) (.) [ ] f (.) = (.667) + (.4) (.) [ ] f (.) = -. [ 8] =.75 (.) Case IV: Derivation of Two and three point formulae: Retaining only the first term in equation: 4 y y y Dy = y = y + + h 4 we can get another useful form for the first derivative as yi yi+ yi y( xi + h) y( xi) i = = = h h h Similarly, by retaining only the first term in Eqn. 4 d yn yn y y n n = Dy n = y n = y n dx h 4 yi yi yi y( xi) y( xi h) y i = = = h h h Adding the last two equations, we have yx ( i + h) yx ( i h) y i = h These Equations constitute two-point formulae for the first derivative. By retaining only the first term in Equation, 4 y y y d y + Dy = = y = dx h 5 5 y we get, yi yi+ yi+ + yi yx ( i + h) yx ( i + h) + yx ( i) y i = = = h h h Similarly we get yi y( xi) y( xi h) + y( xi h) y i = = h h While retaining only the first term in the expression for y in terms of δ we obtain Copyright Virtual University of Pakistan 5

160 δ y δ y i i+ ( ) δ yi ( ) yi+ yi + yi y i = = = h h h yx ( i h) yx ( i) + yx ( i + h) = h The last three equations constitute three-point formulae for computing the second derivative. We shall see later that these two- and three-point formulae become handy for developing extrapolation methods to numerical differentiation and integration. Copyright Virtual University of Pakistan 6

161 Example ; Write first derivative of f (x) at x =.,.,. where f (x) is given by X F(x) solution; Using the two point equation we have Example : f(.) f(.) f (.) = = = =.5. h.. f(.) f(.) f (.) = = = =.75. h.. f(.4) f(.) f (.) = = = =.5. h.. Find the nd derivative at.,.4,.5 for the function given in the example above. Solution; f(.4) f(.) + f(.).5 f (.) = = =.5 h =.. f(.5) f(.4) + f(.).5 f (.4) = = =.5 h =.. f(.6) f(.5) + f(.4).75 f (.5) = = = DIFFERENTIATION USING INTERPOLATION If the given tabular function y(x) is reasonably well approximated by a polynomial P (x) n of degree n, it is hoped that the result of P n( x) will also satisfactorily approximate the corresponding derivative of y(x). However, even if P (x) and y(x) coincide at the tabular points, their derivatives or slopes n may substantially differ at these points as is illustrated in the Figure below: Copyright Virtual University of Pakistan

162 Y(x) P n (x) Deviation of derivatives O X i X For higher order derivatives, the deviations may be still worst. However, we can estimate the error involved in such an approximation. For non-equidistant tabular pairs (x i, y i ), i =,, n we can fit the data by using either Lagrange s interpolating polynomial or by using Newton s divided difference interpolating polynomial. In view of economy of computation, we prefer the use of the latter polynomial. Thus, recalling the Newton s divided difference interpolating polynomial for fitting this data as P ( x) = y[ x ] + ( x x ) y[ x, x ] + ( x x )( x x ) y[ x, x, x ] n n + + ( x x ) y[ x, x,..., x ] i= i n Assuming that P n (x) is a good approximation to y(x), the polynomial approximation to can be y ( x) obtained by differentiating P (x). Using product rule of differentiation, n the derivative of the products in P n (x) can be seen as follows: Copyright Virtual University of Pakistan

163 n n d ( x x)( x x) ( x xn ) ( x xi ) = dx x x i= i= i Thus, y ( x) is approximated by P ( x) which is given by P n ( x) = y[ x, x] + [( x x) + ( x x)] y[ x, x, x] + + n= ( x x)( x x) ( x xn ) i= x xi yx [, x,, xn ] The error estimate in this approximation can be seen from the following. n We have seen that if y(x) is approximated by P n (x), the error estimate is shown to be ( x) E x y x P x y ( n + )! Its derivative with respect to x can be written as ( n+ ) n( ) = ( ) n( ) = ( ξ ) ( x) ( x) d E x y x P x y y ( n+ )! ( n+ )! dx Its derivative with respect to x can be written as ( x) ( n+ ) ( x) d ( n+ ) E n( x) = y ( x) P n( x) = y ( ξ ) + y ( ξ ) ( n+ )! ( n+ )! dx Since ξ(x) depends on x in an unknown way the derivative ( n+ ) ( n+ ) n( ) = ( ) n( ) = ( ξ ) + ( ξ ) d y ( n+ ) ( ξ ) dx cannot be evaluated. However, for any of the tabular points x = x, (x) vanishes and i the difficult term drops out. Thus, the error term in the last equation at the tabular point x = x i simplifies to ( n+ y ) ( ξ ) E n( xi) = Error = ( xi) ( n + )! for some ξ in the interval I defined by the smallest and largest of x, x, x,, x and n ( x ) = ( x x ) ( x x ) = ( x x ) i i i n i j j= j n The error in the r-th derivative at the tabular points can indeed be expressed analogously. To understand this method better, we consider the following example. Example Copyright Virtual University of Pakistan

164 Find f (.5) and f (.) from the following data using the method based on divided differences: X.5.. Y=f(x) X.7..5 Y=f(x) Solution We first construct divided difference table for the given data as shown below: X Y st D.D nd D.D rd D.D Using divided difference formula n= ( x x)( x x) ( x xn ) i= x xi yx [, x,, xn ] from a quadratic polynomial, we have y ( x) = P ( x) = y[ x, x ] + {( x x ) + ( x x )} y[ x, x, x ] + {( x x)( x x) + ( x x)( x x) + ( x x)( x x)} y[ x, x, x, x] Thus, using first, second and third differences from the table, the above equation yields y (.5) =.45 + [(.5.) + (.5.5)]( 5.75) + [(.5.)(.5.) + (.5.5)(.5.) + (.5.5)(.5.)](5.65) Therefore f (.5) = =.76 Similarly we can show that f (.) =.974 Example Copyright Virtual University of Pakistan 4

165 The following divided difference table is for y=/x.use it to find quadratic polynomial Fit. x Y=/x st divided difference nd divided difference y / (.75) from a Thus, using first, second, third differences from the table, we get rd divided difference Solution: Using divided difference formula n= ( x x)( x x) ( x xn ) i= x xi yx [, x,, xn ] If we find the values for quadratic fit then we use the following formula, y ( x) = P ( x) = yx [, x] + {( x x) + ( x x)} yx [, x, x] + {( x x )( x x ) + ( x x )( x x ) + ( x x )( x x )} y[ x, x, x, x ] y ( x) = P ( x) = {( x.5) + ( x.5)}.6664 ( ) + {( x.5)( x.75) + ( x.5)( x.75) + ( x.5)( x.5)} -4.9 Put x = 7.5 y (.75) = P (.75) = {(.75.5) + (.75.5)} {(.75.5)(.75.75) + (.75.5)(.75.75) + (.75.5)(.75.5)} (-4.9 ) y (.75) = P (.75) = {(.5) + (.5)}.6664 ( ) + {(.5)() + (.5)() + (.5)(.5)} -4.9 ( ) y (.75) = P (.75) = {.75}.6664+{.5} -4.9 y (.75) = P (.75) = y (.75) = P (.75) =.7779 Copyright Virtual University of Pakistan 5

166 Copyright Virtual University of Pakistan 6

167 RICHARDSON S EXTRAPOLATION METHOD To improve the accuracy of the derivative of a function, which is computed by starting with an arbitrarily selected value of h, Richardson s extrapolation method is often employed in practice, in the following manner: Suppose we use two-point formula to compute the derivative of a function, then we have yx ( + h) yx ( h) y ( x) = + ET h = Fh ( ) + ET Where E is the truncation error. Using Taylor s series expansion, we can see that T 4 6 ET = ch + ch + ch + The idea of Richardson s extrapolation is to combine two computed values of y ( x) using the same method but with two different step sizes usually h and h/ to yield a higher order method. Thus, we have 4 y ( x) = F( h) + ch + ch + And 4 h h h y ( x) = F + c + c Here, c are constants, independent of h, and F(h) and F(h/) represent approximate i values of derivatives. Eliminating c from the above pair of equations, we get h 4 F F( h) 4 6 y ( x) = + dh + O( h ) Now assuming that h 4 F F( h) h F = Equation for y (x) above reduces to h 4 6 y ( x) F = + dh + O( h ) Thus, we have obtained a fourth-order accurate differentiation formula by combining two results which are of second-order accurate. Now, repeating the above argument, we have h 4 6 y ( x) F = + dh + O( h ) 4 h dh 6 y ( x) = F + + O( h ) 4 6 Eliminating d from the above pair of equations, we get a better approximation as h y x F = + O h 4 6 ( ) ( ) Copyright Virtual University of Pakistan

168 Which is of sixth-order accurate, where h h 4 F F h F = 4 4 This extrapolation process can be repeated further until the required accuracy is achieved, which is called an extrapolation to the limit. Therefore the equation for F above can be generalized as Where F (h) = F(h). F m h h 4 F m F m m m h =, 4 m =,,, m m m To illustrate this procedure, we consider the following example. Example: Using the Richardson s extrapolation limit, find y (.5) to the function y = -/x, with h =.8,.64,.. Solution To start with, we take, h =.8, then compute F (h) as + yx ( + h) yx ( h) Fh ( ) = = h (.8) =.56 = Similarly, F(h/) = Therefore, using Eq. (7.), we get h h 4F F h = = Which is accurate to O(h 4 ). Halving the step size further, we compute + h F = (.) = And Copyright Virtual University of Pakistan

169 h h 4F F h F = 4 = Again, using Eq., we obtain h h 4 F F h F = 4 = 4.95 The above computation can be summarized in the following table: h F F F Thus, after two steps, it is found that y (.5) = 4.95 while the exact value is y (.5) = = = 4.5 x x=.5 Copyright Virtual University of Pakistan

170 Numerical Differentiation and Integration INTRODUCTION DIFFERENTIATION USING DIFFERENCE OPREATORS DIFFERENTIATION USING INTERPOLATION RICHARDSON S EXTRAPOLATION METHOD NUMERICAL INTEGRATION NEWTON-COTES INTEGRATION FORMULAE THE TRAPEZOIDAL RULE ( COMPOSITE FORM) SIMPSON S RULES (COMPOSITE FORM) ROMBERG S INTEGRATION DOUBLE INTEGRATION Basic Issues in Integration What does an integral represent? AREA = VOLUME= b a db ca Basic definition of an integral:: Sum of Height x Width Objective: Evaluate I = b a f(x)dx g(x, y) dx dy f(x)dx= lim f (x k ) x n b a n k= f(x)dx without doing calculation analytically. When would we want to do this?. Integrand is too complicated to integrate analytically. + cos(+ x) e.5x dx +.5x. Integrand is not precisely defined by an equation,i.e., we are given a set of data (x i,ƒ(x i )), i=,...,n. All methods are applicable to integrands that are functions. Some are applicable to tabulated values. Key concepts:. Integration is a summing process. Thus virtually all numerical approximations can be represented by Copyright Virtual University of Pakistan

171 b I = f(x)dx = a n i= xf(x ) + E i t Where:. Closed & Open forms: x = weights x i = sampling points E t = truncation error Closed forms include the end points a & b in x i. Open forms do not. NUMERICAL INTEGRATION Consider the definite integral I = b x= a f( x) dx Where f (x) is known either explicitly or is given as a table of values corresponding to some values of x, whether equispaced or not. Integration of such functions can be carried out using numerical techniques. Of course, we assume that the function to be integrated is smooth and Riemann integrable in the interval of integration. In the following section, we shall develop Newton-Cotes formulae based on interpolation which form the basis for trapezoidal rule and Simpson s rule of numerical integration. NEWTON-COTES INTERGRATION FORMULAE In this method, as in the case of numerical differentiation, we shall approximate the given tabulated function, by a polynomial P n (x) and then integrate this polynomial. Suppose, we are given the data (x i, y i ), i = ()n, at equispaced points with spacing h = x i+ x i, we can represent the polynomial by any standard interpolation polynomial. Suppose, we use Lagrangian approximation, then we have f ( x) = Lk( x) y( xk) With associated error given by ( x) ( n+ ) Ex ( ) = y ( ξ ) ( n + )! Where And ( x) Lk ( x) = ( x x ) ( x ) k k ( ) = ( )( ) ( ) x x x x x x x n Copyright Virtual University of Pakistan

172 Then, we obtain an equivalent integration formula to the definite integral in the form b a f ( xdx ) cyx k ( k) n k = Where c k are the weighting coefficients given by b ck = Lk( x) dx a Which are also called Cotes numbers. Let the equispaced nodes are defined by b a x = a, xn = b, h =, and xk = x + kh n So that x x = ( k )h etc. Now, we shall change the variable x to p such that, k x = x + ph, then we can rewrite equations. ( x) Lk ( x) = ( x x ) ( x ) k k ( ) = ( )( ) ( ) x x x x x x x n As = n+ ( x) h p( p ) ( p n) ( x x )( x x ) ( x x )( x x ) ( x x ) ( )( ) ( )( ) ( ) k k+ n And Lk ( x) = xk x xk x xk xk xk xk+ xk xn ( ph)( p ) h ( p k+ ) h( p k ) h ( p n) h = ( kh)( k ) h ()( h)( ) h ( k n) h Or ( n k) p( p ) ( p k+ )( p k ) ( p n) Lk ( x) = ( ) k!( n k)! Also, noting that dx = h dp. The limits of the integral in Equation b ck = Lk( x) dx a change from to n and equation reduces to p( p ) n k ( ) ( h) n ck = ( p k+ )( p k ) k!( n k)! ( p n) dp The error in approximating the integral can be obtained from p( p ) n+ h n ( n+ ) En = ( p n) y ( n + )! ( ξ ) dp Where x < ξ < x. For illustration, consider the cases for n =, ; For which we get n Copyright Virtual University of Pakistan

173 And h h c = h ( p ) dp =, c = h pdp = h h E = y ( ξ ) p( p ) dp= y ( ξ ) Thus, the integration formula is found to be x h h f( x) dx= cy + cy+ Error = ( y + y) y ( ξ ) x This equation represents the Trapezoidal rule in the interval [x, x ] with error term. Geometrically, it represents an area between the curve y = f (x), the x-axis and the ordinates erected at x = x ( = a) and x = x as shown in the figure. Y y = f(x) (x, y ) (x, y ) (x, y ) y y y y y n- y n O x = a x x x x n- x n = b X This area is approximated by the trapezium formed by replacing the curve with its secant line drawn between the end points (x, y ) and (x, y ). For n =, We have h h c = ( )( ) p p dp= 4 c = h p( p ) dp= h h h c = ( ) p p dp= Copyright Virtual University of Pakistan 4

174 and the error term is given by 5 h ( iv) E = y ( ξ ) 9 Thus, for n =, the integration takes the form x f( x) dx= x y + x y + x y + Error x 5 h h ( iv) = ( y + 4 y+ y) y ( ξ ) 9 This is known as Simpson s / rule. Geometrically, this equation represents the area between the curve y = f (x), the x-axis and the ordinates at x = x and x after replacing the arc of the curve between (x, y ) and (x, y ) by an arc of a quadratic polynomial as in the figure Y y = f(x) (x, y ) (x, y ) y y y O x = a x x x x n- x n = b X Thus Simpson s / rule is based on fitting three points with a quadratic. Similarly, for n =, the integration is found to be x 5 ( iv) f( x) dx= h( y + y+ y + y) h y ( ξ ) x 8 8 This is known as Simpson s /8 rule, which is based on fitting four points by a cubic. Still higher order Newton-Cotes integration formulae can be derived for large values of n. But for all practical purposes, Simpson s / rule is found to be sufficiently accurate. Copyright Virtual University of Pakistan 5

175 Copyright Virtual University of Pakistan 6

176 The Trapezoidal Rule (Composite Form) The Newton-Cotes formula is based on approximating y = f (x) between (x, y ) and (x, y )by a straight line, thus forming a trapezium, is called trapezoidal rule. In order to evaluate the definite integral b I = f( x) dx a we divide the interval [a, b] into n sub-intervals, each of size h = (b a)/n and denote the sub-intervals by [x, x ], [x, x ],, [x, x ], such that x = a and x = b and x = x n- n n k + k h, k =,,, n. Thus, we can write the above definite integral as a sum. Therefore, xn x x xn I = f ( x) dx = f ( x) dx + f ( x) dx + + f ( x) dx x x x xn The area under the curve in each sub-interval is approximated by a trapezium. The integral I, which represents an area between the curve y = f (x), the x-axis and the ordinates at x = x and x = x n is obtained by adding all the trapezoidal areas in each subinterval. Now, using the trapezoidal rule into equation: We get x x h h f( x) dx= cy + cy+ Error = ( y + y) y ( ξ ) x n h h h h I = f( x) dx= ( y + y) y ( ξ) + ( y+ y) y ( ξ) x h h + + ( yn + yn) y ( ξn) Where x < ξ < x, for k =,,, n. k- k Thus, we arrive at the result x n h f ( xdx ) = ( y x + y+ y + + y + y) + E Where the error term E is given by n n n n h En = [ y ( ξ) + y ( ξ) + + y ( ξn)] Equation represents the trapezoidal rule over [x, x ], which is also called the composite n form of the trapezoidal rule. The error term given by Equation: h En = [ y ( ξ) + y ( ξ) + + y ( ξn)] is called the global error. Copyright Virtual University of Pakistan

177 However, if we assume that y ( x) is continuous over [x, x ] then there exists some ξ n in [x, x n ] such that x n = x + nh and h x x En = ny = h y n [ ( ξ )] ( ξ ) Then the global error can be conveniently written as O(h ). Simpson s Rules (Composite Forms) In deriving equation., x x h h f x dx x y x y x y y y y y ξ 9 5 ( iv) ( ) = Error = ( ) ( ) The Simpson s / rule, we have used two sub-intervals of equal width. In order to get a composite formula, we shall divide the interval of integration [a, b] Into an even number of sub- intervals say N, each of width (b a)/n, thereby we have x = a, x,, x = b and x =x +kh, k =,, (N ). N k Thus, the definite integral I can be written as b x x4 xn ( ) ( ) ( ) ( ) a x x xn I = f x dx = f x dx + f x dx + + f x dx Applying Simpson s / rule as in equation x h h f x dx x y x y x y y y y y ξ x 9 5 ( iv) ( ) = Error = ( ) ( ) to each of the integrals on the right-hand side of the above equation, we obtain h I = [( y + 4 y+ y) + ( y + 4 y+ y4) + N 5 ( iv) + ( yn + 4 yn + yn)] h y ( ξ ) 9 That is x N h f( x) dx= [ y + 4( y+ y+ + yn ) + ( y + y4 + + yn ) + yn] + Error term x This formula is called composite Simpson s / rule. The error term E, which is also called global error, is given by N 5 ( iv) xn x 4 ( iv) E = h y ( ξ ) = h y ( ξ ) 9 8 for some ξ in [x, x N ]. Thus, in Simpson s / rule, the global error is of O(h 4 ). Similarly in deriving composite Simpson s /8 rule, we divide the interval of integration into n sub-intervals, where n is divisible by, and applying the integration formula Copyright Virtual University of Pakistan

178 x 5 ( iv) f( x) dx= h( y + y+ y + y) h y ( ξ ) x 8 8 to each of the integral given below xn x x6 xn f ( xdx ) = f( xdx ) + f( xdx ) + + f( xdx ) x x x xn We obtain the composite form of Simpson s /8 rule as b f( x) dx= h[ y( a) + y+ y + y+ y4 + y5 + y6 + a 8 + yn + yn + yn + y( b)] With the global error E given by xn x 4 ( iv) E = h y ( ξ ) 8 It may be noted that the global error in Simpson s / and /8 rules are of the same order. However, if we consider the magnitudes of the error terms, we notice that Simpson s / rule is superior to Simpson s /8 rule. For illustration, we consider few examples. Example π Find the approximate value of y = sin xdx using (i) Trapezoidal rule (ii) Simpson s / rule by dividing the range of integration into six equal parts. Calculate the percentage error from its true value in both the cases. Solution We shall at first divide the range of integration (, π ) into six equal parts so that each part is of width π 6 and write down the table of values: X π/6 π/ π/ π/ 5π/6 π Y=sinx Applying trapezoidal rule, we have π h sin xdx = [ y + y6 + ( y+ y + y+ y4 + y5)] Here, h, the width of the interval is π /6. Therefore, π π.45 y = sin xdx= [ + + (.7)] =.7 = Applying Simpson s / rule 5 x h h ( iv) f( x) dx= xy + xy+ xy + Error = ( y + 4 y+ y) y ( ξ ) x 9 We have π h sin xdx = [ y + y6 + 4( y+ y+ y5) + ( y + y4)] π.45 = [ + + (4 ) + ()(.7)] =.464 = But the actual value of the integral is Copyright Virtual University of Pakistan

179 π sin xdx = [ cos x ] = Hence, in the case of trapezoidal rule The percentage of error.954 = = While in the case of Simpson s rule the percentage error is.8 =.4 (sign ignored) π Copyright Virtual University of Pakistan 4

180 Example : 5 From the following data, estimate the value of log xdx using Simpson s / rule.also, obtain the value of h, so that the value of the integral will be accurate up to five decimal places. X Y=logx Solution We have from the data, n =,,, 8, and h =.5. Now using Simpson s / rule, 5 h log xdx = [ y + y8 + 4( y+ y+ y5 + y7) + ( y + y4 + y6)].5 = [( +.694) + 4(4.787) + (.78)].5 = ( ) = The error in Simpson s rule is given by xn x 4 ( iv) E = h y ( ξ ) 8 (ignoring the sign) ( iv) 6 Since y = log x, y =, y =, y =, y = 4 x x x x ( iv) Max y ( x) = 6, x 5 Max ( iv) y x x 5 ( ) =.96 Therefore, the error bounds are given by 4 4 (.96)(4) h (6)(4) h < E < 8 8 If the result is to be accurate up to five decimal places, then 4 4h 5 < 8 That is, h 4 <.75 or h <.9. It may be noted that the actual value of integrals is 5 5 log xdx = [ x log x x ] = 5log 5 4 Example : Evaluate the integral I = dx by using + x Copyright Virtual University of Pakistan

181 (i) Trapezoidal rule (ii) Simpson s / rule by taking h = ¼. Hence compute the approximate value of π. Solution At first, we shall tabulate the function as X ¼ ½ ¾ y = / +x Using trapezoidal rule, and taking h = ¼ h I = [ y + y4 + 4( y+ y + y) ] = [.5 + (.) ] = using Simpson s / rule, and taking h = ¼, we have h I = [ y + y4 + 4( y+ y) + y] = [.5 + 4(.5) +.6] =.7854 But the closed form solution to the given integral is dx π + = + x 4 tan x Equating the last two equations, we get π =.46. x / Example: Compute the integral I = e π dx using Simpson s / rule, Taking h =.5. Solution At the outset, we shall construct the table of the function as required. X exp( / ) y = π x Using Simpson s. rule, we have h = [ y + y 8 + 4( y + y + y 5 + y 7) + ( y + y 4 + y 6)].5 = [ ( ) + ( )].5 = ( ) =.687 Example : A missile is launched from a ground station. The acceleration during first 8 seconds of flight, as recorded, is given in the following table: Copyright Virtual University of Pakistan

182 ts () am ( / s ) Compute the velocity of the missile when t = 8 s, using Simpson s / rule. Solution: Since acceleration is defined as the rate of change of velocity, we have 8 dv v= a dt Or a dt = Using Simpson s /-rule, we have h v= [( y + y8) + 4( y+ y+ y7) + ( y + y4 + y6)] = [( ) + 4( ) + ( )] = 86. m/ s Therefore, the required velocity is given by y =.86 km/s. Copyright Virtual University of Pakistan

183 Simpson s /8 rule b Consider the definite integral I = f( x) dx x= a x h h f( x) dx= cy + cy+ Error = ( y + y) y ( ξ ) x Then, if n =, the integration takes the form x f( x) dx= x y + x y + x y + Error x 5 h h ( iv) = ( y + 4 y+ y) y ( ξ ) 9 Thus Simpson s / rule is based on fitting three points with a quadratic. Similarly, for n =, the integration is found to be x 5 ( iv) f( x) dx= h( y + y+ y + y) h y ( ξ ) x 8 8 This is known as Simpson s /8 rule, which is based on fitting four points by a cubic. Still higher order Newton-Cotes integration formulae can be derived for large values of n. TRAPEZOIDAL RULE x n h f ( xdx ) = ( y + y+ y + + yn + yn) + En x x n h f ( xdx ) = ( y + y+ y + + yn + yn) + En x SIMPSON S / RULE 5 x h h ( iv) I = f( x) dx= ( y + 4 y+ y) y ( ξ ) x 9 x N h f( x) dx= [ y + 4( y+ y+ + yn ) + ( y + y4 + + yn ) + yn] + Errorterm x xn x 4 ( iv) E = h y ( ξ ) 8 Simpson s /8 rule is b f ( xdx ) = hya [ ( ) + y+ y + y+ y4 + y5 + y6 + + yn + yn + yn + yb ( )] a 8 With the global error E given by xn x E = 8 h y ξ ROMBERG S INTEGRATION 4 ( iv) ( ) We have observed that the trapezoidal rule of integration of a definite integral is of O(h ), while that of Simpson s / and /8 rules are of fourth-order accurate. Copyright Virtual University of Pakistan

184 We can improve the accuracy of trapezoidal and Simpson s rules using Richardson s extrapolation procedure which is also called Romberg s integration method. For example, the error in trapezoidal rule of a definite integral b I = f( x) dx a can be written in the form 4 6 I = IT + ch + ch + ch + By applying Richardson s extrapolation procedure to trapezoidal rule, we obtain the following general formula m h h 4 IT( m ) I m T( m ) m h I Tm m = m 4 Where m =,,, with I (h) = I (h). T T For illustration, we consider the following example. Example: Using Romberg s integration method, find the value of trapezoidal rule, for the tabular values.8 yxdx ( ) starting with x y = f(x) Solution: x =, x =.8, n.8. h=, xi = x + ih N Let I T denote the integration by Trapezoidal rule, then for h N =, h=.8, IT = ( y + y) =.4(.54+.7) =.86 h N =, h=.4, IT = ( y + y+ y) =.[.54+ (.5) +.7] =.794 h N = 4, h=., IT = [ y + ( y+ y + y) + y4] =.[.54+ ( ) +.7] =.778 Similarly for Copyright Virtual University of Pakistan

185 N = 8, h=., IT =.7684 Now, using Romberg s formula, we have h 4(.794).86 IT = =.767 h 4 (.778).767 IT = 4 I =.777 h 4 (.767).777 = 4 =.767 T Thus, after three steps, it is found that the value of the tabulated integral is.767. DOUBLE INTEGRATION To evaluate numerically a double integral of the form I = ( x, y) dx dy over a rectangular region bounded by the lines x = a, x =b, y = c, y = d we shall employ either trapezoidal rule or Simpson s rule, repeatedly With respect to one variable at a time. Noting that, both the integrations are just a linear combination of values of the given function at different values of the independent variable, we divide the interval [a, b] into N equal sub-intervals of size h, such that h = (b a)/n; and the interval (c, d) into M equal sub-intervals of size k, so that k = (d c)/m. Thus, we have xi = x + ih, x = a, xn = b, for i =,,..., N yi = y + ik, y = c, ym = d, for i =,,..., M Thus, we can generate a table of values of the integrand, and the above procedure of integration is illustrated by considering a couple of examples. Example Evaluate the double integral dxdy I = x + y by using trapezoidal rule, with h = k =.5. Solution Copyright Virtual University of Pakistan

186 Taking x =,.5,.5,.75,. and y =,.5,.5,.75,., the following table is generated using the integrand f( x, y) = x + y x y Keeping one variable say x fixed and varying the variable y, the application of trapezoidal rule to each row in the above table gives.5 f(, y) dy = [.5 + ( ) +.] =.46.5 f(.5, y) dy = [ ( ) +.77] =.68.5 f(.5, y) dy = [.4 + ( )] =.69.5 f(.75, y) dy = [.66 + ( ) +.667] =.5 and.5 f(, y) dy = [.+ ( ) +.5] =.879 Therefore Copyright Virtual University of Pakistan 4

187 dxdy h I = = { f(, y) + [ f(.5, y) + f(.5, y) + f(.75, y)] + f(, y) } x+ y By use of the last equations we get the required result as.5 I = [.46 + ( ) ] =.47 Example :Evaluate π / π / sin( x + ydxdy ) by numerical double integration. Solution Taking x = y = π/4, π /8, π /, we can generate the following table of the integrand f ( xy, ) = sin( x+ y) x y π/8 π/4 π/8 π/ π/ π/ π/ π/ Keeping one variable as say x fixed and y as variable, and applying trapezoidal rule to each row of the above table, we get π / π f(, y) dx= [. + ( ) +.] = π π, ydx π [.686 ( ).96].46 = = 8 6 Similarly we get Copyright Virtual University of Pakistan 5

188 π π f, y dx=.4778, 4 π π f, y dx=.46 8 and π π f, y dx=.469 Using these results, we finally obtain π π π π π π π sin( x y) dxdy f(, y), y f, y f, y f, y + = π = [ ( ) +.469] 6 =.86 Copyright Virtual University of Pakistan 6

189 Ordinary Differential Equations Introduction Taylor Series Euler Method Runge-Kutta Method Predictor Corrector Method Introduction Many problems in science and engineering when formulated mathematically are readily expressed in terms of ordinary differential equations (ODE) with initial and boundary condition. Example The trajectory of a ballistic missile, the motion of an artificial satellite in its orbit, are governed by ordinary differential equations. Theories concerning electrical networks, bending of beams, stability of aircraft, etc., are modeled by differential equations. To be more precise, the rate of change of any quantity with respect to another can be modeled by an ODE Closed form solutions may not be possible to obtain, for every modeled problem, while numerical methods exist, to solve them using computers. In general, a linear or non-linear ordinary differential equation can be written as n n d y dy d y = f t, y,,, n n dt dt dt Here we shall focus on a system of first order dy differential equations of the form f (, t y) dt = with the initial condition y (t ) = y, which is called an initial value problem (IVP). It is justified, in view of the fact that any higher order ODE can be reduced to a system of first order differential equations by substitution. For example, consider a second order differential equation of the form y = f(, t y, y ) Introducing the substitution p = y, the above equation reduces to a system of two first order differential equations, such as Theorem y = p, p = f( t, y, p) Let f (t, y) be real and continuous in the strip R, defined by [ ] t t, T, y Then for any t [ t, T] and for any y, y, there exists a constant L, satisfying the inequality f (, t y) f(, t y) L y y so that f y (, t y) L, for every t, y R Here, L is called Lipschitz constant. Copyright Virtual University of Pakistan

190 If the above conditions are satisfied, then for any y, the IVP has a unique solution y ( t ), for t [ t T ], In fact, we assume the existence and uniqueness of the solution to the above IVP The function may be linear or non-linear. We also assume that the function f (t, y) is sufficiently differentiable with respect to either t or y. TAYLOR S SERIES METHOD Consider an initial value problem described by dy f (, t y), y( t) y dt = = Here, we assume that f (t, y) is sufficiently differentiable with respect to x and y. If y (t) is the exact solution, we can expand y (t) by Taylor s series about the point t = t and obtain 4 ( t t) ( t t) ( t t) IV yt ( ) = yt ( ) + ( t t) y ( t) + y ( t) + y ( t) + y ( t) +!! 4! Since, the solution is not known, the derivatives in the above expansion are not known explicitly. However, f is assumed to be sufficiently differentiable and therefore, the derivatives can be obtained directly from the given differential equation. Noting that f is an implicit function of y, we have y = f(, t y) f f dy y = + = fx + ffy x y dx Similarly y = fxx + ffxy + f ( fxy + ff yy ) + f y ( fx + ff y ) = fxx + ffxy + f fyy + fy ( fx + ffy ) IV y = fxxx + ffxxx + f fxyy + fy( fxx + ffxy + f fyy) + ( fx + ffy )( ffxy + ffyy ) + fy ( fx + ffy) Continuing in this manner, we can express any derivative of y in terms of f (t, y) and its partial derivatives. Example Using Taylor s series method, find the solution of the initial value problem dy t y, y() dt = + = at t =., with h =. and compare the result with the closed form solution Solution Let us compute the first few derivatives from the given differential equation as follows: Copyright Virtual University of Pakistan

191 y = t+ y, y = + y, y = y, IV V IV y = y, y = y Prescribing the initial condition, that is, at t =, y y (t ) =, we have y =, y =, IV V y = y = y = Now, using Taylor s series method, we have ( t t) ( t t) yt () = y + ( t t) y + y + y ( t t) IV ( t t) V + y + y + 4 Substituting the above values of the derivatives, and the initial condition, we obtain... y (.) = + (.)() + () + () + () = = =.44 Therefore y(.) = y =.44. Taking y =. at t =., the values of the derivatives are y =.+. =. y = +. =. IV V y = y = y =. Substituting the value of y and its derivatives into Taylor s series expansion we get, after retaining terms up to fifth derivative only ( t t) y(.) = y+ ( t t) y + y 4 5 ( t t) ( t t) IV ( t t) V + y + y + y 6 4 y (.) = = To obtain the closed form solution, we rewrite the given IVP as dy y t dt = or ( d ye t ) = te t On integration, we get Copyright Virtual University of Pakistan

192 y = e ( te + e ) + ce t t t t t = ce t Using the initial condition, we get Therefore, the closed form solution is y = t + e t When t =., the closed form solution becomes y (.) =. + (.48) =.48.4 Example Using Taylor s Series method taking algorithm of order, solve the initial value problem / y = y ; y() = with h=.5 at x=.5 Solution: Copyright Virtual University of Pakistan 4

193 Let us com pute the first three derivatives y = y y = y y = y / // / /// // The initial condition is t = y =, we have y y y / // / /// // ( ) = = = y = ( ) ( ) ( ) = y = = N ow, Taylor ' s series m ethosd a lg rithm is ( t t ) ( t t ) / // /// y t = y + t t y + y + y! / // /// OR y( t) = y + hy + h y + h y h =.5! y y y y / ( ) ( )( ) ( ) ( ) ( ) // / ( ) (.5) (.5) ( ) ().5 = ! = = =.5 Taking y y y y =.5, Now = y =.5 =.7786 = y = /// // ( ) = y =.7786 =.7786 ( ) ( ) ( t t ) ( t t ) / // /// y t = y + t t y + y + y! / // /// OR y( t) = y + hy + h y + h y h =.5 6 = = (.5)(.7786) (.5) (.7786) (.5) (.7786) =.969 Copyright Virtual University of Pakistan 5

194 EULER METHOD Euler method is one of the oldest numerical methods used for integrating the ordinary differential equations. Though this method is not used in practice, its understanding will help us to gain insight into nature of predictor-corrector method Consider the differential equation of first order with the initial condition y(t ) = y. dy f (, t y) dt = The integral of this equation is a curve in, ty-plane. Here, we find successively y, y,, y ; where y is the value of y at t =t = t +mh, m m m m =,, and h being small. Here we use a property that in a small interval, a curve is nearly a straight line. Thus at (t, y ), we approximate the curve by a tangent at that point. Therefore, dy y y = = f ( t, y) dt ( t, y) t t That is y = y + ( t t) f( t, y) Hence, the value of y corresponding to t = t is given by y = y + ( t t) f( t, y) Similarly approximating the solution curve in the next interval (t, t ) by a line through (t, y ) having its slope f(t, y ), we obtain y = y+ hf( t, y) Thus, we obtain in general, the solution of the given differential equation in the form of a recurrence relation y = m y + + m hf( tm, ym) Geometrically, this method has a very simple meaning. The desired function curve is approximated by a polygon train, where the direction of each part is determined by the value of the function f (t, y) at its starting point. Example dy y t Given = with the initial condition y = at t =. Using Euler method, find y dt y + t approximately at x =., in five steps. Solution Since the number of steps are five, we shall proceed in steps of (.)/5 =.. Therefore, taking step size h =., we shall compute the value of y at t =.,.4,.6,.8 and. Thus y = y + hf( t, y), where y =, t = Copyright Virtual University of Pakistan

195 Therefore y = +. =. +.. y = y+ hf( t+ y) =. +. = y = y + hf( t, y) =.9 +. = Similarly, y4 = y+ hf( t, y) = = y5 = y4 + hf( t4, y4) = = Hence the value of y corresponding to t =. is.9 Example Solve the differential equation / y = x+ y ; y() = in the interval [,.5] using Euler s method by taking h=. Soution: x x. x. x. x.4 4 x.5 5 Since h =. so we calculate values at x =,.,.,.,.4,.5 y() = here x = y = y = y + hf( x, y ) y y = + (.)( + ) = +. =. =. x =. y = y + hf( x, y ) y =. + (.)(.+.) =.+. =. Copyright Virtual University of Pakistan

196 now y =. x =. y = y + hf( x, y) y =. + (.)(. +.) =. +.4 =.6 now y =.6 x =. y4 = y+ hf( x, y) y4 =.6 + (.)(.6 +.) = =.58 now y4 =.6 x4 =.4 y5 = y4 + hf( x4, y4) y5 =.58 + (.)( ) =.7 now y5 =.7 x5 =.5 y6 = y5 + hf( x5, y5) y6 =.7 + (.)(.7 +.5) =.94 Hencethevalueof y corresponding tot =.5is.94 MODIFIED EULER S METHOD The modified Euler s method gives greater improvement in accuracy over the original Euler s method. Here, the core idea is that we use a line through (t, y ) whose slope is () () the average of the slopes at (t,y ) and (t, y ) Where y = y + hf (t, y ) is the value of y at t = t as obtained in Euler s method, which approximates the curve in the interval (t, t ) Copyright Virtual University of Pakistan

197 y ( t, y ) L B L L ( t, y ) A ( t, y ) L t (Modified Euler s Method) Geometrically, from Figure, if L is the tangent at (t, y ), L is the line through (t, () () - () y ) of slope f(t, y ) and L is the line through (t, y ) but with a slope equal to the average of f(t,y ) and f(t, y () ),. the line L through (t, y ) and parallel to L is used to approximate the curve in the interval (t, t ). Thus, the ordinate of the point B will give the value of y. Now, the equation of the line AL is given by () f( t, y) + f( t, y ) y = y + ( t t ) () f( t, y) f( t, y ) y h + = + Similarly proceeding, we arrive at the recurrence relation () f( tm, ym) + f( tm, ym ) + + ym+ = ym + h Copyright Virtual University of Pakistan 4

198 This is the modified Euler s method. Example Using modified Euler s method, obtain the solution of the differential equation dy t y f(, t y) dt = + = with the initial condition y = at t = for the range t.6 in steps of. Solution At first, we use Euler s method to get () y = y + hf( t, y) = +.( + ) =. Then, we use modified Euler s method to find () f ( t, y) + f( t, y ) y(.) = y = y + h ( ) +.+. =. +. =.95 Similarly proceeding, we have from Euler s method () y = y+ hf( t, y) =.95 +.(. +.95) =.49 Using modified Euler s method, we get () f ( t, y) + f( t, y ) y = y+ h Finally, = =.55 (. +.95) + ( ) () ( ) y = y + hf( t, y ) = =.849 Modified Euler s method gives () f ( t, y) + f( t, y ) y(.6) = y = y + h = ( ) + ( ) =.889 Hence, the solution to the given problem is given by Copyright Virtual University of Pakistan 5

199 t..4.6 y Copyright Virtual University of Pakistan 6

200 RUNGE KUTTA METHOD These are computationally, most efficient methods in terms of accuracy. They were developed by two German mathematicians, Runge and Kutta. They are distinguished by their orders in the sense that they agree with Taylor s series solution up to terms of h r, where r is the order of the method. These methods do not demand prior computation of higher derivatives of y(t) as in TSM. Fourth-order Runge-Kutta methods are widely used for finding the numerical solutions of linear or non-linear ordinary differential equations, the development of which is complicated algebraically. Therefore, we convey the basic idea of these methods by developing the second-order Runge-Kutta method which we shall refer hereafter as R-K method. Please recall that the modified Euler s method: which can be viewed as y = n y + + n h (average of slopes) This, in fact, is the basic idea of R-K method. Here, we find the slopes not only at t but also at several other interior points, and take the n weighted average of these slopes and add to y n to get y n+. Now, we shall derive the second order R-K method in the following slides. Consider the IVP dy f (, t y), y( tn) yn dt = = We also define k = hf( tn, yn), k = hf( tn + αh, yn + βk) and take the weighted average of k and k and add to y to get y n n+ We seek a formula of the form y = n y + n Wk + + Wk Where αβ,,w and W are constants to be determined so that the above equation agree with the Taylor s series expansion as high an order as possible Thus, using Taylor s series expansion, we have h h yt ( n+ ) = yt ( n) + hy ( tn) + y ( tn) + y ( tn) + 6 Rewriting the derivatives of y in terms of f of the above equation, we get h y = n y + n hf( tn, yn) + + ( ft ffy) + h 4 + ftt + ffty + f f yy + f y ( ft + ff y ) + O( h ) 6 Here, all derivatives are evaluated at (t, y ). n n Next, we shall rewrite the given equation after inserting the expressions or k and k as y = n y + n Whf ( tn, yn) + Whf( t + n αh, y + + n β k) Now using Taylor s series expansion of two variables, we obtain Copyright Virtual University of Pakistan

201 y y Whf( t, y ) W h f( t, y ) ( hf k f ) n+ = n + n n + n n + α t + β y α h β k f n α h β k f ty f yy O h ( ) Here again, all derivatives are computed at (t, y ). On inserting the expression for k, n n the above equation becomes yn+ = yn + ( W+ W) hf + Wh ( αhft + βhff y) α h β h + ftt + αβh ffty + f fyy + O( h ) On rearranging in the increasing powers of h, we get y = n y + n ( W + W) hf + Wh ( α f + + t β ffy) α β f ff y 4 + Wh ftt + αβ ffty + + Oh ( ) Now, equating coefficients of h and h in the two equations, we obtain ft + ff y W+ W =, W( α ft + β ffy) = Implying W+ W =, Wα = Wβ = Thus, we have three equations in four unknowns and so, we can chose one value arbitrarily. Solving we get W = W, α =, β = W W Where W is arbitrary and various values can be assigned to it. We now consider two cases, which are popular Case I If we choose W = /, then W = / and α = β = / y = n y + + n ( k k ) + k = hf(, t y), k = hf t+ h, y+ k Case II: If we consider W = ½, then W = ½ and α = β =. Then y n+ k+ k = yn + Copyright Virtual University of Pakistan

202 k = hf(, t y), k = hf( t+ h, y+ k ) In fact, we can recognize that this equation is the modified Euler s method and is therefore a special case of a nd order Runge-Kutta method. These equations are known as nd order R K Methods, since they agree with Taylor s series solution up to the term h. Defining the local truncation error, TE, as the difference between the exact solution y(t n+ ) at t = t n+ and the numerical solution y n+, obtained using the second order R K method, we have TE = y( tn+ ) yn+ Now, substituting W =,,, α W = α β = α into the above equation, we get h yn+ = yn + hftt + ( ft + ffy) t= t n α ( ftt ffty f fyy ) t= tn 4 h Finally we obtain α TE = h ( ftt + ffty + f f yy ) + f y ( ft + ff y ) The expression can further be simplified to α TE = h ( y fyy ) + fyy Therefore, the expression for local truncation error is given by α α TE = h y + f y y Please verify that the magnitude of the TE in case I is less than that of case II Following similar procedure, Runge-Kutta formulae of any order can be obtained. However, their derivations becomes exceedingly lengthy and complicated. Amongst them, the most popular and commonly used in practice is the R-K method of fourth-order, which agrees with the Taylor series method up to terms of O (h 4 ). This well-known fourth-order R-K method is described in the following steps. y = n y + + n ( k k k k4 ) Copyright Virtual University of Pakistan

203 k = hf( tn, yn) h k k = hf tn +, yn + where h k k = hf tn +, yn + k4 = hf( tn + h, yn + k) Please note that the second-order Runge-Kutta method described above requires the evaluation of the function twice for each complete step of integration. Similarly, fourth-order Runge-Kutta method requires the evaluation of the function four times. The discussion on optimal order R-K method is very interesting, but will be discussed some other time. Copyright Virtual University of Pakistan 4

204 Example Use the following second order Runge-Kutta method described by y = n y + + n ( k k ) + where k = hf( xn, yn) and k hf = xn + h, yn + k and find the numerical solution of the initial value problem described as dy y + x =, y() = dx y x at x =.4 and taking h =.. Solution Here y+ x f( x, y) =, h=., x =, y = y x + We calculate k = hf( x, y) =. =. k = hf [ x +., y + (.5)(.)].+. = hf (.,.) =. =... Now, using the given R-K method, we get y(.) = y = + (.4 +.) =.4 Now, taking x =., y =.4, we calculate.4 +. k = hf( x, y) =. = k hf = x+ h, y+ k = hf(.5,.6554) =. = Again using the given R-K method, we obtain y(.4) = y =.4 + [ (.769) +.7] = Example Solve the following differential equation dy t y dt = + with the initial condition y() =, using fourth- order Runge-Kutta method from t = to t =.4 taking h =. Copyright Virtual University of Pakistan

205 Solution The fourth-order Runge-Kutta method is described as y = n y + + n ( k k k k4 ) () Where k = hf( tn, yn) h k k = hf tn +, yn + h k k = hf tn +, yn + k4 = hf( tn + h, yn + k) In this problem f( t, y) = t+ y, h=., t =, y =. As a first step, we calculate k = hf( t, y ) =.() =. k = hf( t +.5, y +.5) = hf (.5,.5) =.[.5 +.5] =. k = hf( t +.5, y +.55) =.( ) =.5 k4 =.(.+.5) =.5 Now, we compute from y = y + ( k+ k + k+ k4 ) 6 = + ( ) 6 =.4 Therefore y(.) = y =. In the second step, we have to find y = y(.) We compute Copyright Virtual University of Pakistan

206 k = hf( t, y) =.(.+.4) =.4 h k k = hf t+, y+ =.[.5 + ( )] =.8 h k k = hf t+, y+ =.[.5 + ( )] =.68 k4 = hf( t+ h, y+ k) =.[. + ( )] = From Equation (), we see that y =.4 + [.4 + (.8) 6 + (.68) ] =.48 Similarly we calculate, k = hf( t, y) =.[. +.48] =.448 h k k = hf t +, y + =.[.5 + ( )] = h k k = hf t+, y+ =.[. + ( )] =.5747 k4 = hf( t + h, y + k) =.[. + ( )] = Using equation (), we compute y(.) = y = y + ( k+ k + k+ k4) = Finally, we calculate k = hf( t, y) =.[.+.997] =.6997 h k k = hf t+, y+ =.[.5 + ( )] = h k k = hf t+, y+ =.[.5 + ( )] =.8444 k4 = hf( t+ h, y+ k) =.[.4 + ( )] = Using them in equation (), we get y(.4) = y4 = y+ ( k+ k + k+ k4) 6 =.586 which is the required result. Example Apply Runge-Kutta Method of order four to solve the initial value problem at x=. / / y = xy ; y() = ; h=. Copyright Virtual University of Pakistan

207 Solution: The Fourth-Order Runge-Kutta Method is described as :- y = y + + k k k k (, ) ( ) n n 4 where, k = hf t y n h k k = hf tn +, yn + h k k = hf tn +, yn + k = hf t + h y + k (, ) 4 n n n ( Here, we are taking x as t ) First Iteration:- Copyright Virtual University of Pakistan 4

208 t =, y = k = hf( t, y) =. =. / h k k = hf tn +, yn + ( ) ( k ) = / =.6767 h k k = hf tn +, yn + ( ) ( k ) = / =.6856 (, ) k = hf t + h y + k 4 n n ( ) ( k ) = = / / / ( ) y = y + k + k + + k + k y = n n 4 y = Second Iteration:- Copyright Virtual University of Pakistan 5

209 t =., y = k = hf( t, y) = = / h k k = hf tn +, yn + ( ) ( ) = / =.9669 h k k = hf tn +, yn + / ( ) ( k ) = / =.8564 (, ) k = hf t + h y + k 4 n n ( ) ( k ) = = ( ) y = y + k + k + k + k y y n+ n 4 = = / / So, at x =., we get: y(.) = y = RUNGE KUTTA METHOD FOR SOLVING A SYSTEM OF EQUATIONS The fourth-order Runge-Kutta method can be extended to numerically solve the higherorder ordinary differential equations- linear or non-linear For illustration, let us consider a second order ordinary differential equation of the form d y dy = f t, y, dt dt Copyright Virtual University of Pakistan 6

210 Using the substitution dy p dt = this equation can be reduced to two first-order simultaneous differential equations, as given by dy dt dp = p = f(, t y, p), = f(, t y, p) dt Now, we can directly write down the Runge-Kutta fourth-order formulae for solving the system. Let the initial conditions of the above system be given by yx ( n) = yn, y ( xn) = px ( n) = pn Then we define k = hf( tn, yn, pn), l = hf( tn, yn, pn) h k l h k l k = hf tn +, yn +, pn +, l = hf tn +, yn +, pn + h k l h k l k = hf tn +, yn +, pn +, l = hf tn +, yn +, pn + k4 = hf( tn + h, yn + k, pn + l), l4 = hf( tn + h, yn + k, p ) n + l Now, using the initial conditions y n, p n and 4 th -order R-K formula, we compute yn+ = yn + ( k+ k + k+ k4 ) 6 pn+ = pn + ( l+ l + l+ l4 ) 6 This method can be extended on similar lines to solve system of n first order differential equations. Example Solve the following equation y (.)( y ) y + y = Using 4 th order Runge-Kutta method for x =., with the initial values y() =, y ()= Solution: dy Let p f (,, ) x y p dx = = dp Then (.)( y) p y f( x, y, p) dx = = Thus, the given equation reduced to two first-order equations. In the present problem, we are given x =, y =, p =y = o Taking h =., we compute Copyright Virtual University of Pakistan 7

211 k = hf ( x, y, p ) =.(.) =. l = hf ( x, y, p ) =.(. ) =. h k l k = hf x +, y +, p + = hf (.,.,.) =. h k l l = hf x +, y +, p + = hf (.,.,.) =.(.) =. h k l k = hf x +, y +, p + = hf (.,.99,.) =.(.) =. h k l l = hf x +, y +, p + = hf(.,.99,.) =.[(.)(.99)(.).99] =.98 k = hf ( x + h, y + k, p + l ) 4 = hf (.,.98,.98) =.96 l = hf ( x + h, y + k, p + l ) = hf (.,.98,.98) 4 =.[(.)(.964)(.98).98] =.966 Now, y (.) = y is given by y(.) = y = y + [ k+ k + k+ k4] 6 = + [. + (.) + (.) + (.96)] 6 =.995 =.98 y (.) = p = p + [ l+ l + l+ l4] 6 = + [. + (.) + (.98) + (.966)] 6 =.9869( =.987) Therefore, the required solution is y(.) =.98, y (.) =.987 Copyright Virtual University of Pakistan 8

212 PREDICTOR CORRECTOR METHOD The methods presented so far are called single-step methods, where we have seen that the computation of y at t n+ that is y n+ requires the knowledge of y n only. In predictor-corrector methods which we will discuss now, is also known as multi-step methods. To compute the value of y at t n+, we must know the solution y at t n, t n-, t n-, etc. Thus, a predictor formula is used to predict the value of y at t n+ and then a corrector formula is used to improve the value of y n+. Let us consider an IVP dy f (, t y), y( tn) yn dt = = Using simple Euler s and modified Euler s method, we can write down a simple predictor-corrector pair (P C) as () P: yn+ = yn + hf( tn, y ) n () h () C: yn+ = yn + f( tn, yn) + f( tn+, yn+ ) Here, y n+ () is the first corrected value of yn+. The corrector formula may be used iteratively as defined below: ( r) h ( r ) yn+ = yn + f( tn, yn) + f( tn+, yn+ ), r =,, The iteration is terminated when two successive iterates agree to the desired accuracy In this pair, to extrapolate the value of y, we have approximated the solution curve in n+ the interval (t n, t n+ ) by a straight line passing through (t n, y n ) and (t n+, y n+ ). The accuracy of the predictor formula can be improved by considering a quadratic curve through the equally spaced points (t n-, y n- ), (t n, y n ), (t n+, y n+ ) Suppose we fit a quadratic curve of the form y = a+ b( t tn ) + c( t tn)( t tn ) where a, b, c are constants to be determined As the curve passes through (t, y ) and (t, y ) and satisfies n- n- n n dy = f ( tn, yn) dt ( tn, yn) We obtain yn = a, yn = a+ bh= yn + bh Therefore yn yn b = h and Copyright Virtual University of Pakistan

213 dy = f( tn, yn) = { b+ c[( t tn ) + ( t tn)]} ( t n, y n ) dt ( tn, yn) Which give f ( tn, yn) = b+ c( tn tn ) = b+ ch f( tn, yn) ( yn yn ) or c = h h Substituting these values of a, b and c into the quadratic equation, we get yn+ = yn + ( yn yn ) + [ hf( tn, yn) ( yn yn )] That is yn+ = yn + hf( tn, yn) Thus, instead of considering the P-C pair, we may consider the P-C pair given by P: yn+ = yn + hf( tn, yn) h C: yn+ = yn + [ f( tn, yn) + f( tn+, yn+ )] The essential difference between them is, the one given above is more accurate However, this one can not be used to predict y for a given IVP, because its use n+ require the knowledge of past two points. In such a situation, a R-K method is generally used to start the predictor method. Milne s Method It is also a multi-step method where we assume that the solution to the given IVP is known at the past four equally spaced point t, t, t and t. To derive Milne s predictor-corrector pair, let us consider a typical IVP dy f (, t y), y( t) y dt = = On integration between the limits t and t, we get 4 t4 dy t4 dt = f (, t y ) dt t dt t t4 y y f (, t y ) dt 4 = t But we know from Newton s forward difference formula ss ( ) ss ( )( s ) f(, t y) = f + s f + f + f + 6 t t where s=, t = t + sh h t4 ss ( ) ss ( )( s ) y4 = y + t f + s f + f + f 6 ss ( ( s )( s ) 4 + f + dt 4 Now, by changing the variable of integration (from t to s), the limits of integration also changes (from to 4), and thus the above expression becomes Copyright Virtual University of Pakistan

214 4 ss ( ) ss ( )( s ) y4 = y + h 4 f + s f + f + f 6 ss ( ( s )( s ) 4 + f + ds 4 which simplifies to y4 y h = + 4f + 8 f + f + f + f 9 Substituting the differences f = f f, f = f f+ f, It can be further simplified to 4h 8 4 y4 = y + ( f f + f) + h f 9 Alternatively, it can also be written as 4h 8 4 y4 = y + (y y + y ) + h y 9 This is known as Milne s predictor formula. Similarly, integrating the original over the interval t to t or s = to and repeating the above steps, we get h 4 y = y + ( y + 4 y + y ) h y 9 which is known as Milne s corrector formula. In general, Milne s predictor-corrector pair can be written as 4h P: yn+ = yn + (y n y n + y n) h C: yn+ = yn + ( y n + 4 y n + y n+ ) From these equations, we observe that the magnitude of the truncation error in 4 / corrector formula is /9h y while the truncation error in predictor formula is 4 / 8/ 9h y Thus: TE in, c-formula is less than the TE in p-formula. In order to apply this P C method to solve numerically any initial value problem, we first predict the value of y by means of predictor formula, where derivatives are n+ computed using the given differential equation itself. Using the predicted value y, we calculate the derivative y from the given n+ n+ differential equation and then we use the corrector formula of the pair to have the corrected value of y This in turn may be used to obtain improved value of y by n+ n+ using corrector again. This in turn may be used to obtain improved value of y n+ by using the corrector again. This cycle is repeated until we achieve the required accuracy. Example Copyright Virtual University of Pakistan

215 dy Find y (.) if y ( t ) is the solution of ( t y ) dt = + y () =, y (.5) =.66, y (.) =.595 and y(.5) = Use Milne s P-C method. Solution Taking t =., t =.5, t =., t =.5 y, y, y and y, are given, we have to compute y 4, the solution of the given differential equation corresponding to t =. The Milne s P C pair is given as 4h P: yn+ = yn + (y n y n + y n) h C: yn+ = yn + ( y n + 4 y n + y n+ ) From the given differential equation, y = ( t+ y)/ We have t+ y y = = =.568 t + y y = = =.975 t + y y = = =.4 Now, using predictor formula, we compute 4 h y4 = y + ( y y + y ) 4(.5) = + [ (.568) (.4) ] = 6.87 Using this predicted value, we shall compute the improved value of y from corrector 4 formula h y4 = y + ( y + 4 y + y 4) Using the available predicted value y and the initial values, we compute 4 t4 + y y 4 = = = t + y y = = =.4 and y =.975 Thus, the first corrected value of y is given by 4 Copyright Virtual University of Pakistan 4

216 ().5 y 4 = [ (.4) ] = Suppose, we apply the corrector formula again, then we have () h () y4 = y + ( 4 ( 4 ) y + y + y = (.4) + + = Finally, y (.) = y = Example dy Tabulate the solution of = t+ y, y() = in the interval [,.4] with h =., dt using Milne s P-C method. Solution Milne s P-C method demand the solution at first four points t, t, t and t. As it is not a self starting method, we shall use R-K method of fourth order to get the required solution and then switch over to Milne s P C method. Thus, taking t =, t =., t =., t =. we get the corresponding y values using R K method of 4 th order; that is y =, y =., y =.48 and y =.997 (Reference Lecture 8) Now we compute y = t+ y =.+. =. y = t + y = =.448 y = t+ y = =.6997 Using Milne s predictor formula 4h P: y4 = y + (y y + y ) 4(.5) = + [ (.) (.6997) ] =.586 Before using corrector formula, we compute y ( ) 4 = t4 + y4 predicted value = =.986 Finally, using Milne s corrector formula, we compute Copyright Virtual University of Pakistan 5

217 h C: y4 = y + ( y y + y ). =.48 + ( ) =.586 The required solution is: t....4 y Example Using Milne s Predictor-Corrector Formula find f(.4) from Ordinary Differential Equation / y = x y ; y() = ; h=. with the help of following table. X... Y Solution: Here, x =, x =., x =., x =., x =.4 4 y ' = x y =..997 =.897 y ' = x y =..875 =.675 y ' = x y =..786 =.486 Now, using Predictor Formula 4h y4 = y + y' y' + y' 4*. y4 = + (-.945) y = ( ) Copyright Virtual University of Pakistan 6

218 Using the predicted value, we shall now compute the corrected value as; h y = y + y + y + y ( ' 4 ' ') 4 4 Now, y ' = x y = = Putting the values,int o the Corrector Formula; h y4 = y + ( y' + 4 y' + y4' ). y4 = * y = ( ( )- ) y 4 = Ans. Copyright Virtual University of Pakistan 7

219 Copyright Virtual University of Pakistan 8

220 Adam-Moulton Method It is another predictor-corrector method, where we use the fact that the solution to the given initial value problem is known at past four equally spaced points t n, t n-, t n-, t n-. The task is to compute the value of y at t n+. Let us consider the differential equation dy f (, t y) dt = Integrating between the limits t to t, we have n n+ That is tn+ tn+ tn dy dt = dt tn f (, t y ) dt = tn+ n+ n (, ) tn y y f t y dt To carry out integration, we proceed as follows. We employ Newton s backward interpolation formula, so that Where ss ( + ) ss ( + )( s+ ) f(, t y) = fn + s fn + fn + fn + 6 t t s = n h After substitution, we obtain tn+ ss ( + ) yn+ = yn + [ fn + s fn + fn tn ss ( + )( s+ ) ss ( + )( s+ )( s+ ) 4 + fn+ fn + dt 6 4 Now by changing the variable of integration (from t to s), the limits of integration also changes (from to ), and thus the above expression becomes ss ( + ) yn+ = yn + h [ fn + s fn + fn ss ( + )( s+ ) ss ( + )( s+ )( s+ ) 4 + fn+ fn + ds 6 4 Actual integration reduces the above expression to yn+ = yn + h fn + fn + fn + fn + fn 8 7 Now substituting the differences such as fn = fn fn Copyright Virtual University of Pakistan

221 Equation simplifies to fn = fn fn + fn fn = fn fn + fn fn h 5 y = y + (55 f 59 f + 7 f 9 f ) + h f n+ n n n n n n Alternatively, it can be written as h 5 4 yn+ = yn + [ 55y n 59y n + 7y n 9y n ] + h y 4 7 This is known as Adam s predictor formula. 4 The truncation error is (5/ 7) h y n. To obtain corrector formula, we use Newton s backward interpolation formula about f instead of f. n+ n We obtain ss ( + ) yn+ = yn + h [ fn+ + s fn+ + fn+ ss ( + )( s+ ) ss ( + )( s+ )( s+ ) 4 + fn+ + fn+ + ds 6 4 Carrying out the integration and repeating the steps, we get the corrector formula as h 4 ( ) 9 yn+ = yn + y n+ + y n y n + y n + h y n Here, the truncation error is ( 9 ) h y n +. The truncation error in Adam s predictor is approximately thirteen times more than that in the corrector, but with opposite sign. In general, Adam-Moulton predictor-corrector pair can be written as h P: yn+ = yn + ( 55y n 59y n + 7y n 9y n ) 4 h C: yn+ = yn + ( 9y n+ + 9y n 5y n + y n ) 4 Example Using Adam-Moulton predictor-corrector method, find the solution of the initial value problem dy y t, y() dt = = at t =., taking h =.. Compare it with the analytical solution. Solution In order to use Adam s P-C method, we require the solution of the given differential equation at the past four equally spaced points, for which we use R-K method of 4 th order which is self starting. Copyright Virtual University of Pakistan

222 Thus taking t =, y =, h =., we compute k =., k =.8, k =.98, k =.596, 4 and get y = y + ( k+ k + k+ k4 ) = Taking t =., y =.859, h =., we compute k =.57, k =.49, k =.564, k =.684, and get 4 y = y+ ( k+ k + k+ k4 ) = Now, we take t =.4, y =.468, h =., and compute k =.66, k =.697, k =.76, k = to get y = y(.6) = y + ( k+ k + k+ k4) = Thus, we have at our disposal y = y() = y = y(, ) =.859 y = (.4) =.468 y = y(.6) =.7779 Now, we use Adam s P-C pair to calculate y (.8) and y (.) as follows: h P: yn+ = yn + ( 55y n 59y n + 7y n 9y n ) 4 h C: yn+ = yn + ( 9y n+ + 9y n 5y n + y n ) 4 p h Thus y4 = y+ ( 55y 59y + 7 y 9y ) () 4 From the given differential equation, we have Therefore y = y t. Copyright Virtual University of Pakistan

223 y = y t =. y = y t =.7859 y = y t =.8 y = y t =.7779 Hence, from Eq. (), we get p. y(.8) = y4 = ( ) =.45 4 Now to obtain the corrector value of y at t =.8, we use ( ) c c h 4 (.8) y = y = y + y + y y + y () 4 p But 9y 4 = 9( y4 t4) = 9[.45 (.8) ] =.759 Therefore c. y4 = y (.8) = ( ) =.44 () 4 Proceeding similarly, we get ( ) p p h 5 = (.) = y y y y y y y 4 Noting that y 4 = y4 t4 =.74, We calculate p. y 5 =.44 + ( ) = Now, the corrector formula for computing y is given by 5 ( ) c c h 5 (.) y = y = y + y + y y + y (4) 4 But p 9y = 9 y t =.56 ( ) Then finally we get. y5 = y(.) = =.89 (5) The analytical solution can be seen in the following steps. dy y t dt = After finding integrating factor and solving, we get d ye e t t = dt ( ) Copyright Virtual University of Pakistan 4

224 Integrating, we get t ye = t e t dt = t t t t d( e ) = t e + te + c That is c = t y t t Now using the initial condition, y() =, we get c =. Therefore, the analytical solution is given by t y = t + t+ e From which we get y(.) = 5 e=.87 Example Using Adam-Moulton Predictor-Corrector Formula find f(.4) from Ordinary Differential Equation / y = + xy ; y() = ; h=. with the help of following table. X... Y Solution: Here e Copyright Virtual University of Pakistan 5

225 h =. f( x, y) = + xy ( )( ) ( )( ) ( )( ) ( )( ) y ' = + x y = + = y ' = + x y = +..7 =.4 y ' = + x y = =.84 y ' = + x y = =.994 Now, Using Adam's P-C Pair Formula:- h yn+ = yn + 55 y' n 59 y' n + 7 y' n 9 y ' 4 ( ) n Putting the values; h y4 = y + ( 55 y' 59 y' + 7 y' 9 y' ) 4. y4 = y = ( ( ) ( ) ( ) ( )) Copyright Virtual University of Pakistan 6

226 Computing y ' for the Corrector Formula; ( )( ) y' = + x y = y ' = Now Applying the Corrector Formula; h yn+ = yn + ( 9 y' n+ + 9 y' n 5 y' n + y' n ) 4 h y4 = y+ ( 9 y' 4+ 9 y' 5 y' + y' ) 4. y4 = ( ) + 9(.994) 5(.84) ( ) y 4 = Convergence and Stability Considerations The numerical solution of a differential equation can be shown to converge to its exact solution, if the step size h is very small.the numerical solution of a differential equation is said to be stable if the error do not grow exponentially as we compute from one step to another. Stability consideration are very important in finding the numerical solutions of the differential equations either by single-step methods or by using multi-step methods. However, theoretical analysis of stability and convergence of R -K methods and P C methods are highly involved and obtain numerically stable solution using 4 th order R K method to the simple problem y = Ay gives us stability condition as -.78<Ah In practice, to get numerically stable solutions to similar problems, we choose the value of h much smaller than the value given by the above condition and also check for consistency of the result. Another topic of interest which is not considered, namely the stiff system of differential equations that arises in many chemical engineering systems, such as chemical reactors, where the rate constants for the reactions involved are widely different. Most of the realistic stiff DE do not have analytical solutions and therefore only numerical solutions can be obtained. However, to get numerically stable solutions, a very small step size h is required, to use either R-K methods or P C methods. More computer time is required Copyright Virtual University of Pakistan 7

227 Lecture # 4 Examples of Differential Equations Recall EULER METHOD We considered the differential equation of first order with the initial condition y(t) = y. dy f (, t y) dt = We obtained the solution of the given differential equation in the form of a recurrence relation y = m y + + m hf( tm, ym) In fact Euler s method constructs wi ~ y(ti ) for each i =,,, N- by deleting the remainder term. Thus the Euler s Method is w = α, wi+ = wi + hf( ti, wi) for each i =,,..., N Euler s algorithm Let us try to approximate the solution of the given IVP at (N+) equally spaced numbers in the interval [a,b] y = f(, t y), a t b, y( a) = α INPUT endpoints a, b; integer N, initial condition (alpha) OUTPUT Step Set approximate w to y at the (N+) values of t h=(b-a) / N t = a w = (alpha) OUTPUT (t, w) Step For i =,, N do Step, 4. Step Set w = w + h f (t, w); (compute wi ). t = a + i h (compute ti ) Step 4 OUTPUT (t, w) Step 5 STOP Example Use Euler s method to approximate the solution of IVP y = y t^ +, < t <, y ( ) =.5 with N =. Solution Here, h =., ti =.i, w=.5 and wi+ = wi + h (wi ti^ + ) = wi+.[wi -.4i +] =. wi -.8i^ +. for i =,,,9. The exact solution is y ( t )= (t+)^ -.5 e t > alg5(); This is Euler's Method. Input the function F(t,y) in terms of t and y

228 For example: y-t^+ > y-t^+ Input left and right endpoints separated by blank > Input the initial condition >.5 Input a positive integer for the number of subintervals > Choice of output method:. Output to screen. Output to text file Please enter or > Output t w > alg5(); This is Euler's Method. Input the function F (t,y) in terms of t and y For example: y-*t^+4 > y-*t^+4 Input left and right hand points separated by a blank > Input the initial condition >.5 Input a positive integer for the number of subintervals > Choice of output method:. Output to screen. Output to text file Please enter or > Output t w Recall Runge-Kutta (Order Four) METHOD The fourth-order R-K method was described as y = n y + + n ( k k k k4 ) where

229 k = hf( tn, yn) h k k = hf tn +, yn + h k k = hf tn +, yn + k = hf( t + h, y + k ) 4 n n Example Solve the following differential equation dy = t+ y with the initial condition y() =, using fourth- order Runge-Kutta method from t = dt to t =.4 taking h =. Solution The fourth-order Runge-Kutta method is described as y = n y + + n ( k k k k4 )...() where k = hf( tn, yn) h k k = hf tn +, yn + h k k = hf tn +, yn + k4 = hf( tn + h, yn + k) In this problem, f( t, y) = t+ y, h=., t =, y =. As a first step, we calculate k = hf( t, y) =.() =. k = hf( t +.5, y +.5) = hf (.5,.5) =.[.5 +.5] =. k = hf( t +.5, y +.55) =.( ) =.5 k =.(.+.5) =.5 4 Now, we compute from

230 y = y + ( k+ k + k+ k4 ) 6 = + ( ) 6 =.4 Therefore y(.) = y=. In the second step, we have to find y = y(.) We compute k = hf( t, y) =.(.+.4) =.4 h k k = hf t+, y+ =.[.5 + ( )] =.8 h k k = hf t+, y+ =.[.5 + ( )] =.68 k4 = hf( t+ h, y+ k) =.[. + ( )] = From Equation (), we see that y =.4 + [.4 + (.8) 6 + (.68) ] =.48 Similarly we calculate, k = hf( t, y) =.[. +.48] =.448 h k k = hf t +, y + =.[.5 + ( )] = h k k = hf t+, y+ =.[. + ( )] =.5747 k4 = hf( t + h, y + k) =.[. + ( )] = Using equation (), we compute y(.) = y = y + ( k+ k + k+ k4) = Finally, we calculate k = hf( t, y) =.[.+.997] =.6997 h k k = hf t+, y+ =.[.5 + ( )] = h k k = hf t+, y+ =.[.5 + ( )] =.8444 k4 = hf( t+ h, y+ k) =.[.4 + ( )] = Using them in equation (), we get

231 y(.4) = y4 = y + ( k+ k + k+ k4 ) 6 =.586 which is the required result Runge-Kutta Order Four w = α k = hf( ti, wi) h k k = hf ti +, wi + h k k = hf ti +, wi + k4 = hf( ti + h, wi + k) w = i w + + i ( k k k k4 )...() RK4 algorithm Let us try to approximate the solution of the given IVP at (N+) equally spaced numbers in the interval [a,b] y = f(, t y), a t b, y( a) = α INPUT endpoints a, b; integer N, initial condition (alpha) OUTPUT Step Set approximate w to y at the (N+) values of t h=(b-a) / N t = a w = (alpha) OUTPUT (t, w) Step For i =,, N do Step - 5. Step Set K = hf(, t w) h K K = hf t+, w+ h K K = hf t+, w+ K4 = hf( t+ h, w+ K) w = i w + + i ( k k k k4 ) Step Set w = w + h f (t, w); (compute wi ). t = a + i h (compute ti ) Step 4 OUTPUT (t, w)

232 Step 5 STOP > alg5(); This is the Runge-Kutta Order Four Method. Input the function F(t,y) in terms of t and y For example: y-t^+ > y-t^+ Input left and right endpoints separated by blank > Input the initial condition >.5 Input a positive integer for the number of subintervals > Choice of output method:. Output to screen. Output to text file Please enter or > Output t w

233 Lecture 4 Examples of Numerical Differentiation The simplest formula for differentiation is h f x = f x + h f x f h () ( ) [ ( ) ( ) ] ( ξ ), Example Let f(x)= In x and x =.8. Then quotient f (.8 + h ) f (.8), h >, h f with error hf ( ξ ) h h =, where.8 < ξ <.8 + h. ξ (.8) is used to approximate (.8) Let us see the results for h =.,., and.. h f(.8 + h) f(.8 + h) f(.8) h h (.8)

234 Since f ( x) = / x, The exact value of f (.8) is.555 and the error bounds are a appropriate. The following two three point formulas become especially useful if the nodes are equally spaced, that is, when x = x + h and x = x + h, h () f ( x) = [ f( x) + 4 f( x + h) f( x + h) ] + f ( ξ), h where ξ lies between x and x + h, and h () f ( x) = [ f( x + h) f( x h) ] f ( ξ), h 6 where ξ lies between (x h) and (x + h). Given in Table below are values for f (x) = xe x. x f (x) Since x f ( x) = ( x+ ) e, f (.) = Approximating f (.) using the various three-and five-point formulas produces the following results. Three point formulas: h () f ( x) = [ f( x) + 4 f( x + h) f( x + h) ] + f ( ξ), h h () f ( x) = [ f( x + h) f( x h) ] f ( ξ), h 6 Using three point formulas we get h=.: [ f(.) + 4 f(.) f(.) ]. =.,

235 h=.: [ f(.) + 4 f(.9) f(.8) ]. =.5455, h=.: [ f(.) f(.9) ]. =.879, h=. : [ f(.) f(.8) ].4 =.446. Five point formula Using the five point formula with h =. (the only formula applicable): f ( x) = [ f( x h) 8 f( x h) + 8 f( x + h) f( x + h) ] h = [ f(.8) 8 f(.9) + 8 f(.) f(.) ]. = The errors in the formulas are approximately.5,., 6.6,.47, 4 and.69, respectively. Clearly, the five-point formula gives the superior result. Consider approximating for f (x) = sin x, using the values in table [ the true value is cos (.9) =.66.] x sin x x sin x Using the formula

236 f (.9) f (.9 + h) f(.9 h) h with different values of h gives the approximations in table given below: h Approximation to f (.9) Error

237 Examples of Numerical Integration EXAMPLE The Trapezoidal rule for a function f on the interval [, ] is h f ( xdx ) = [ f( x ) + f( x)] f( x) dx f() + f(), while Simpson s rule for f on [, ] is h f ( xdx ) = [ f( x) + 4 f( x) + f( x)]. That is f( x) dx [ f() + 4 f() + f()]. f (x) x x4 /(x + ) + x sin x ex Exact value Trapezoidal Simpson s Use close and open formulas listed below to approximate π /4 sin xdx = Some of the common closed Newton-Cotes formulas with their error terms are as follows: n = : Trapezoidal rule x h h f( x) dx= [ f( x) + f( x)] f ( ξ ), x Where x < ξ < x. n = : Simpson s rule

238 x x 5 h h (4) f ( xdx ) = [ f( x) + 4 f( x) + f( x)] f ( ξ), x < ξ < x. 9 n = : Simpson s rule 5 x h h (4) f( x) dx= [ f( x) + f( x) + f( x) + f( x)] f ( ξ ), x 8 8 x < ξ < x. n = 4: 7 x h 8h (6) f( x) dx= [7 f( x) + f( x) + f( x) + f( x) + 7 f( x4)] f ( ξ x Where x < ξ < x. 4 n = : Midpoint rule h x f ( x) dx = hf ( x ) + f ( ξ ), where x < ξ < x. x n 4 Closed formulas Error Open formulas Error Composite Numerical Integration

239 EXAMPLE π Consider approximating sin xdx with an absolute error less than., using the Composite Simpson s rule. The Composite Simpson s rule gives ( n /) / 4 π n h π h sin xdx = sin x j + 4 sin x j sin µ. j= j= 8 Since the absolute error is required to be less than., the inequality πh πh π sin µ = < n is used to determine n and h. Computing these calculations gives n greater than or equal to 8. If n =, then the formula becomes π 9 j ( j ) sin xdx π sin π 4 sin π.6. + = 6 j= j= To be assured of this degree of accuracy using the Composite Trapezoidal rule requires that πh πh π sin µ = <. n or that n 6. Since this is many more calculations than are needed for the Composite Simpson s rule, it is clear that it would be undesirable to use the Composite Trapezoidal rule on this problem. For comparison purposes, the Composite Trapezoidal rule with n = and gives π 9 π jπ sin xdx sin sin sin + + π 4 j= 9 π jπ = sin = j= The exact answer is ; so Simpson s rule with n = gave an answer well within the required error bound, whereas the Trapezoidal rule with n = clearly did not. An Example of Industrial applications: A company advertises that every roll of toilet paper has at least 5 sheets. The probability that there are 5 or more sheets in the toilet paper is given by.88( y 5.) Py ( 5) =.55 e dy 5 Approximating the above integral as 7.88( y 5.) Py ( 5).55 e dy = 5 a)use single segment Trapezoidal rule to find the probability that there are 5 or more sheets. b)find the true error, Et for part (a). C)Find the absolute relative true error for part (a).

240 f ( a) + f( b) I ( b a) where a = 5 b = 7.88( y 5.) f( y) =.55e =.57.88(7 5.) f(7) =.55e.88(5 5.) f(5) =.55e = I = (7 5) =.57 b) The exact value of the above integral cannot be found. We assume the value obtained by adaptive numerical integration using Maple as the exact value for calculating the true error and relative true error. 7.88( y 5.) Py ( 5) =.55 e dy 5 =.9777 so the true error is = =.4656 The absolute relative true error,, would then be t True Error t = True Value =.9777 = 44.8% Improper Integrals EXAMPLE To approximate the values of the improper integral x e dx, x we will use the Composite Simpson s rule with h =.5. Since the fourth Taylor polynomial for ex about x = is 4 x x x P x x ( ). 4 = We have P4 ( x) dx = lim + x x x x x x M = Table below lists the approximate values of M

241 x e P ( ) 4 x when < x Gx ( ) = 4 when x =, x G(x) Applying the Composite Simpson s rule to G using these data gives Gxdx ( ).5 [ + 4(.7) + (.4) + 4(.66) ] =.769 Hence x e dx =.954. x This result is accurate within the accuracy of the Composite Simpson s rule approximation for the function G. Since G (4) ( x ) < on [, ], the error is bounded by (.5) 4 () =.7. 8 EXAMPLE To approximate the value of the improper integral sin. x I = x dx

242 we make the change of variable t = x - to obtain I = t sin t dt. The fourth Taylor polynomial, P4(t), for sin t about is P4 () t = t t, 6 sin t t+ 5 So we have 6 t I = dt + t t dt t 6 sin t t 6 t 7 dt t t = + + t sin t t+ 6 t = dt t Applying the Composite Simpson s rule with n = 8 to the remaining integral gives I = =.6566, which is accurate to within 8 4..

243 Maple is a comprehensive computer system for advanced mathematics. An Introduction to MAPLE It includes facilities for interactive algebra, calculus, discrete mathematics, graphics, numerical computation etc. It provides a unique environment for rapid development of mathematical programs using its vast library of built-in functions and operations. Syntax :As with any computer language, Maple has its own syntax. We try to explain some of the symbols used in Maple Symbol Description Examples Sample Output ; End-of-line. Tells Maple to process the line and show the output. hello; hello : End-of-line. Tells Maple to process the line and hide the output. hello: := Assignment. Lets you assign values to variables. a := ; a; a := +, - Addition, subtraction. + ; - ; 4 - *, / Multiplication, division *4; 6/; 7/; 6 4 7/ ^, sqrt Power, square root ^; sqrt(); ^(/); evalf,. Floating-point (decimal) evaluation evalf(7/); 7./;..

244 I,Pi Imaginary unit, Pi. + *I; (*I)^; evalf(pi); +I %, %% Recall the last output, recall the secondto-last output, etc. %; %%%; Some syntactical Tips: Maple is case sensitive. foo, Foo, and FOO are three different things. x*y gives the product of x and y, xy is one variable To get the constant e use exp(). Using the % operator can give confusing results. It always returns the last output from the Kernel, which may have nothing to do with where the cursor is (or which worksheet is active). If Maple doesn't recognize something, it assumes it as a variable; e.g. typing i^ will give you i,while we may be wanted -. Spaces are optional. Greek letters may be entered by spelling their name. For example, alpha is always displayed as α, and Gamma is displayed as Γ (note upper-case). Built-in Data Capabilities Maple can handle arbitrary-precision floating point numbers. In other words, Maple can store as many digits for a number as you like, up to the physical limits of your computer's memory. To control this, use the Digits variable. sqrt(.);.4456 Digits := : sqrt(.); Maple sets Digits to be by default. You can also temporarily get precision results by calling evalf with a second argument. evalf(sqrt(), 5); Large integers are handled automatically Using symbolic computation The main feature of Maple is symbolic computation.

245 In other words, Maple does algebra. Example Output Comments (x + y)^; (x + y) A basic expression. k := x*y + y^; k := xy + y k is now an alias for the expression. Note that k is simply another name for the expression - they are not equal in the mathematical sense. p := k /(x+y); You can now use k to refer to the expression. Maple immediately substitutes the value of k. k := 'k'; k You can unassign a variable by assigning it to its own name in single quotes.

246 simplify(p); y The simplify command does algebraic simplification. p := x^ - 8*x +5; p := x - 8x + 5 Maple doesn't mind if you re-use names. The old value is lost. solve(p=,x);,6 Use the solve command to solve equations. Note the use of the = sign. Here, it is used in a mathematical sense. Maple will try different values for x until it finds all of them that make the mathematical statement x - 8x + 5 = true. dpdx := diff(p,x); dpdx := x - 8 The diff command differentiates an expression with respect to a variable. int(p,x); The int command integrates an expression. Note that the constant of integration is left off. Basic Plotting Maple can produce graphs very easily. Here are some examples, showcasing the basic capabilities.

247 plot( x^, x=-..); plot( x^, x=-.., y=-..); A basic plot. A plot with vertical axis control. plot([x, x^, x^], x=-..); Plot multiple expressions by enclosing them in brackets.

248 plotd(4-x^-y^, x=-.., y=-..); A basic -d plot. smartplotd(x^-y^);