Numerical Analysis by Dr. Anita Pal Assistant Professor Department of Mathematics National Institute of Technology Durgapur Durgapur-713209 email: anita.buie@gmail.com 1
. Chapter 4 Solution of Non-linear Equation Module No. 1 Newton s Method to Solve Transcendental Equation
... Finding roots of algebraic and transcendental equations is a very important task. These equations occur in many applications of science and engineering. A function f(x) is called algebraic if each term of f(x) contains only the arithmetic operations between real numbers and x with rational power. On the other hand, a transcendental function includes at least one non-algebraic function, i.e. an exponential function, a logarithmic function, trigonometric functions, etc. An equation f(x) = 0 is called algebraic or transcendental according as f(x) is algebraic or transcendental. The equations x 9 120x 2 +12 = 0 and x 15 +23x 10 9x 8 +30x = 0 are the examples of algebraic equations and the equations logx+xe x = 0, 3sinx 9x 2 +2x = 0 are the examples of transcendental equations. Lot of numerical methods are available to solve the equation f(x) = 0. But, each method has some advantages and disadvantages over another method. Mainly, the following points are considered to compare the methods: rate of convergence, domain of applicability, number of evaluation of functions, precomputation step, etc. The commonly used methods to solve an algebraic and transcendental equations are bisection, regula-falsi, secant, fixed point iteration, Newton-Raphson, etc. In this module, only Newton-Raphson method is discussed. It is a very interesting method and rate of convergence of this method is high compare to other methods. 1.1 Newton-Raphson method This method is also known as method of tangent. The Newton-Raphson method is an iteration method and so it needs an initial or starting value. Let f(x) = 0 be the given equation and x 0 be the initial guess, i.e. the initial approximate root. Let x 1 = x 0 +h be an exact root of the equation f(x) = 0, where h is a correction of the root, i.e. the amount of error. Generally, it is assumed that h is small. Therefore, f(x 1 ) = 0. Now, by Taylor s series, the equation f(x 1 ) = f(x 0 +h) = 0 is expanded as f(x 0 )+hf (x 0 )+ h2 2! f (x 0 )+ = 0. 1
...Newton s Method to Solve Transcendental Equation Since h is small, so the second and higher power terms of h are neglected and then the above equation reduces to f(x 0 )+hf (x 0 ) = 0 or, h = f(x 0) f (x 0 ). Note that this is an approximate value of h. Using this h, the value of x 1 is x 1 = x 0 +h = x 0 f(x 0) f (x 0 ). (1.1) It is obvious that x 1 is a better approximation of x than x 0. Since x 1 is not an exact root of the equation f(x) = 0, therefore another iteration is to be performed to find the next better root. For this purpose, the value of x 0 is replaced by x 1 in equation (1.1) to get second approximate root x 2. That is, In this way, the (n+1)th iterated value is given by x 2 = x 1 f(x 1) f (x 1 ). (1.2) x n+1 = x n f(x n) f (x n ). (1.3) The above formula generates a sequence of numbers x 1,x 2,...,x n,... The terms of this sequence go to the exact root ξ. The method will terminate when x n+1 x n ε, where ε is a pre-assigned very small positive number called the error tolerance. Note 1.1 This method is also used to find a complex root of the equation f(x) = 0. But, for this case, the initial root is taken as a complex number. Geometrical interpretation The geometrical interpretation of Newton-Raphson method is shown in Figure 1.1. Here, a tangent is drawn at the point (x 0,f(x 0 )) to the curve y = f(x). Let the tangent cuts the x-axis at the point (x 1,0). Again, a tangent is drawn at the point (x 1,f(x 1 )). Suppose this tangent cuts the x-axis at the point (x 2,0). This process is repeated until the nth iterated root x n coincides with the exact root ξ, for large n. For this reason this method is known as method of tangents. 2
... f(x) O x ξx 2 x 1 x 0 Figure 1.1: Geometrical interpretation of Newton-Raphson method. The choice of initial guess x 0 is a very serious task. If the initial guess is close to the root then the method converges rapidly. But, if the initial guess is not much close to the root or if it is wrong, then the method may generates an endless cycle. Also, if the initial guess is not close to the exact root, the method may generates a divergent sequence of approximate roots. Thus, to choose the initial guess the following rule is suggested. Let a root of the equation f(x) = 0 be in the interval a,b]. If f(a) f (x) > 0 then x 0 = a be taken as the initial guess of the equation f(x) = 0 and if f(b) f (x) > 0, then x 0 = b be taken as the initial guess. 1.1.1 Convergence of Newton-Raphson method Suppose a root of the equation f(x) = 0 lies in the interval a,b]. The Newton-Raphson iteration formula (1.3) is x i+1 = x i f(x i) f (x i ) = φ(x i) (say). (1.4) If ξ is a root of the equation f(x) = 0, therefore ξ = φ(ξ). (1.5) 3
...Newton s Method to Solve Transcendental Equation get Subtracting (1.4) from (1.5), ξ x i+1 = φ(ξ) φ(x i ) = (ξ x i )φ (ξ i ) (by MVT) (where ξ i lies between ξ and x i ) Now, substituting i = 0,1,2,...,n to the above equation and multiplying them we (ξ x n+1 ) = (ξ x 0 )φ (ξ 0 )φ (ξ 1 ) φ (ξ n ) or ξ x n+1 = ξ x 0 φ (ξ 0 ) φ (ξ 1 ) φ (ξ n ) (1.6) Let φ (x) l for all x a,b]. Then from the equation (1.6) Now, if l < 1 then ξ x n+1. ξ x n+1 l n+1 ξ x 0. Therefore, lim x n+1 = ξ. n Hence, the sequence {x n } converges to ξ for all x a,b], if φ (x) < 1 or { d x f(x) } dx f < 1 or f(x) f (x) < f (x) 2 (1.7) (x) within the interval a, b]. Thus, the Newton-Raphson method converges if the initial guess x 0 is chosen sufficiently close to the root and the functions f(x), f (x) and f (x) are continuous and bounded within a, b]. This is the sufficient condition for the convergence of the Newton- Raphson method. The rate of convergent of Newton-Raphson method is calculated in the following theorem. Theorem 1.1 The rate of convergence of Newton-Raphson method is quadratic. Proof. The Newton-Raphson iteration formula is x n+1 = x n f(x n) f (x n ). (1.8) Let ξ and x n be an exact root and the nth approximate root of the equation f(x) = 0. Let ε n be the error occurs at the nth iteration. Then x n = ε n +ξ and f(ξ) = 0. 4
... Therefore, from the equation (1.8) That is, ε n+1 +ξ = ε n +ξ f(ε n +ξ) f (ε n +ξ) ε n+1 = ε n f(ξ)+ε nf (ξ)+(ε 2 n/2)f (ξ)+ f (ξ)+ε n f (ξ)+ + ] f (ξ) ε n + ε2 n f (ξ) 2 f (ξ) = ε n + ] since f(ξ) = 0] f f (ξ) 1+ε (ξ) n f (ξ) = ε n ε n + ε2 n 2 = 1 f (ξ) 2 ε2 n f (ξ) +O(ε3 n). f (ξ) f (ξ) + ] f ] (ξ) 1 1+ε n f (ξ) + by Taylor s series expansion] Neglecting the third and higher powers of ε n, the above expression reduces to ε n+1 = Cε 2 n, where C = f (ξ) 2f (ξ) a constant number. (1.9) Since the power of ε n is 2, therefore the rate of convergence of Newton-Raphson method is quadratic. Example 1.1 Using Newton-Raphson method find a root of the equation x 3 2sinx 2 = 0 correct up to five decimal places. Solution. Let f(x) = x 3 2sinx 2. One root lies between 1 and 2. Let x 0 = 1 be the initial guess. The iteration scheme is x n+1 = x n f(x n) f (x n ) = x n x3 n 2sinx n 2 3x 2 n 2cosx n. The sequence {x n } for different values of n is shown below. 5
...Newton s Method to Solve Transcendental Equation n x n x n+1 0 1.000000 2.397806 1 2.397806 1.840550 2 1.840550 1.624820 3 1.624820 1.588385 4 1.588385 1.587366 5 1.587366 1.587365 Therefore, one root of the given equation is 1.58736 correct up to five decimal places. Example 1.2 Find an iteration scheme to find the kth root of a number a and hence find the cube root of 2. Solution. Let x be the kth root of a. Therefore, x = a 1/k or x k a = 0. Let f(x) = x k a. Then the Newton-Raphson iteration scheme is x n+1 = x n f(x n) f (x n ) = x n xk n a = kxk n xk n +a = 1 k Second part: Here, a = 2 and k = 3. kx k 1 n (k 1)x n + a x k 1 n Then the above iteration scheme reduces to x n+1 = 1 3 2x n + 2 ] x 2 = 2 n 3 All calculations are shown in the following table. kx k 1 n ]. ( x 3 n +1 x 2 n ). n x n x n+1 0 1.00000 1.33333 1 1.33333 1.26389 2 1.26389 1.25993 3 1.25993 1.25992 Thus, the value of 3 2 is 1.2599, correct up to four decimal places. 6
... Example 1.3 Suppose 2x n+1 = 5x3 n +1 is an iteration scheme to find a root of the 9 equation f(x) = 0. Find the function f(x). Solution. Let l be a root obtained from the given iteration scheme 2x n+1 = 5x3 n +1. 9 Then, lim x n = l. n ] Now, lim 18x n+1 = 5 lim n n x3 n +1. That is, 18l = (5l 3 +1), or 5l 3 18l +1 = 0. Therefore, the required equation is 5x 3 18x+1 = 0, and hence f(x) = 5x 2 18x+1. Example 1.4 Discuss the Newton-Raphson method to find a root of the equation x 15 1 = 0 starting with x 0 = 0.5. Solution. It is obvious that the real roots of the given equation are ±1. Here f(x) = x 15 1. Therefore, x n+1 = x n x15 n 1 15x 14 n = 14x15 n +1 15x 14. Let the initial guess be x 0 = 0.5. Then x 1 = 14 (0.5)15 +1 15 (0.5) 14 = 1092.7333. This is far away from the root 1. This is because 0.5 is not close enough to the exact root x = 1. But, the initial guess x 0 = 0.9 gives the first approximate root as x 1 = 1.131416 and it is close to the root 1. This example shows the importance of initial guess in Newton-Raphson method. The Newton-Raphson method may also be used to find the complex root. This is illustrated in the following example. n 7
...Newton s Method to Solve Transcendental Equation Example 1.5 Find a complex root of the equation z 3 +3z 2 +3z +2 = 0. An initial guess may be taken as 0.5+0.5i. Solution. Letz 0 = 0.5+0.5i = (0.5,0.5) betheinitial guessandf(z) = z 3 +3z 2 +3z+2. Then f (z) = 3z 2 +6z +3. The Newton-Raphson iteration scheme is z n+1 = z n f(z n) f (z n ). The values of z n and z n+1 at each iteration are tabulated below: n z n z n+1 0 ( 0.50000000, 0.50000000) ( 0.10666668, 0.41333333) 1 ( 0.10666668, 0.41333333) ( 0.62715298, 0.53778100) 2 ( 0.62715298, 0.53778100) ( 0.47841841, 1.0874815) 3 ( 0.47841841, 1.0874815) ( 0.50884020, 0.90368903) 4 ( 0.50884020, 0.90368903) ( 0.50117314, 0.86686337) 5 ( 0.50117314, 0.86686337) ( 0.50000149, 0.86602378) 6 ( 0.50000149, 0.86602378) ( 0.49999994, 0.86602539) 7 ( 0.49999994, 0.86602539) ( 0.49999994, 0.86602539) Thus one complex root is ( 0.49999994, 0.86602539), i.e. 0.49999994+ 0.86602539 i correct up to eight decimal places. 1.2 Newton-Raphson method for multiple root Using Newton-Raphson method, one can determined the multiple root of the equation f(x) = 0. But, the following modified formula x n+1 = x n k f(x n) f (x n ) (1.10) gives a more faster convergent scheme, where k is the multiplicity of the root. The term in the formula 1 k f (x n ) is the slope of the straight line passing through point (x n,f(x n )) and intersecting the x-axis at the point (x n+1,0). Let ξ be a root of the equation f(x) = 0 with multiplicity k. Then ξ is also a root of the equation f (x) = 0 with multiplicity (k 1). In general, ξ is a root of the equation f p (x) = 0 with multiplicity (k p), p < k. If the equation f(x) = 0 has a 8
... root with multiplicity k and if the initial guess is very close to the exact root ξ, then the expressions x 0 k f(x 0) f (x 0 ), x 0 (k 1) f (x 0 ) f (x 0 ), x 0 (k 2) f (x 0 ) f (x 0 ),...,x 0 fk 1 (x 0 ) f k (x 0 ) must have the same value. Theorem 1.2 The rate of convergence of the formula (1.10) is quadratic. Proof. Let ξ be a multiple root of the equation f(x) = 0 of multiplicity k. Therefore, f(ξ) = f (ξ) = f (ξ) = = f k 1 (ξ) = 0 and f k (ξ) 0. Let x n and ε n be the nth approximate root and the error at this step. Then ε n = x n ξ. Now, from the iteration scheme (1.10), we have ε n+1 = ε n k f(ε n +ξ) f (ε n +ξ) = ε n k f(ξ)+ε nf (ξ)+ + ε k 1 n (k 1)! fk 1 (ξ)+ εk n k! f k (ξ)+ εk+1 n (k+1)! fk+1 (ξ)+ f (ξ)+ε n f (ξ)+ + εk 2 n (k 2)! fk 1 (ξ)+ εk 1 n (k 1)! fk (ξ)+ εk n k! f k+1 (ξ)+ ε k n k! f k (ξ)+ εk+1 n = ε n k ε k 1 n (k 1)! fk (ξ)+ εk n k! f k+1 (ξ)+ εn = ε n k εn = ε n k k + k + = ε n ε n + ε2 n k +1 = ε 2 n 1 k(k +1) (k+1)! fk+1 (ξ)+ ε 2 n f k+1 (ξ) k(k +1) f k + (ξ) ε 2 n f k+1 (ξ) k(k +1) f k + (ξ) f k+1 (ξ) f k (ξ) f k+1 (ξ) f k (ξ) ε2 n k ] +O(ε 3 n ). f k+1 (ξ) f k (ξ) ] 1+ ε n k ] 1 ε n k f k+1 (ξ) f k (ξ) f k+1 (ξ) f k (ξ) ] + ] 1 + ] + 1 f k+1 (ξ) Let C = k(k +1) f k (ξ). Neglecting cube and higher order terms of ε n, the above equation becomes ε n+1 = Cε 2 n. Thus, the rate of convergence of the scheme (1.10) is quadratic. 9
...Newton s Method to Solve Transcendental Equation Example 1.6 Find the multiple root with multiplicity 3 of the equation x 4 x 3 3x 2 + 5x 2 = 0. Solution. Let the initial guess be x 0 = 0.5. Also, let f(x) = x 4 x 3 3x 2 +5x 2. f (x) = 4x 3 3x 2 6x+5, f (x) = 12x 2 6x 6, f (x) = 24x 6. The first iterated values are x 1 = x 0 3 f(x 0) f (x 0 ) = 0.5 3f(0.5) f (0.5) = 1.035714 x 1 = x 0 2 f (x 0 ) f (x 0 ) = (0.5) 0.5 2f f = 1.083333 and (0.5) x 1 = x 0 f (x 0 ) f (x 0 ) = 0.5 f (0.5) f (0.5) = 1.5. The first two values of x 1 are closed to 1. It indicates that the equation may have a double root near 1. Let x 1 = 1.035714. Then x 2 = x 1 3 f(x 1) f = 1.035714 3f(1.035714) (x 1 ) f (1.035714) = 1.000139 x 2 = x 1 2 f (x 1 ) f (x 1 ) = 1.035714 (1.035714) 2f f (1.035714) = 1.000277 x 2 = x 1 f (x 1 ) f (x 1 ) = 1.000277 f (1.000277) f (1.000277) = 1.000812. Here it is seen that the three values of x 2 are very close to 1. So the equation has a multiple root near 1 of multiplicity 3. Let x 2 = 1.000139. The third iterated values are x 3 = x 2 3 f(x 2) f (x 2 ) = 1.000000, x 3 = x 2 2 f (x 2 ) f = 1.000000 and (x 2 ) x 3 = x 2 f (x 2 ) f (x 2 ) = 1.000000. All the values of x 3 are same and hence one root of the equation is 1.000000 correct up to six decimal places, with multiplicity 3. 1.3 Modification on Newton-Raphson method After development of Newton-Raphson method, some modifications have been made on this method. One of them is discussed below. 10
... Note that in the Newton-Raphson method the derivative of the function f(x) is evaluated at each iteration. That is, to find x n+1, the value of f (x n ) is required for n = 0,1,2,... Therefore, at each iteration two functions are evaluated at the point x n, n = 0,1,2,... So, a separate method is required to find derivatives. Thus, in each iteration of this method more calculations are needed. But, the following proposed method can reduced the computational effort: x n+1 = x n f(x n) f (x 0 ). (1.11) In this method, the derivative of f(x) is calculated only at the initial guess x 0 and obviously it reduces the computation time at each iteration. But, the rate of convergence of this method reduced to 1. This is, proved in the following theorem. Theorem 1.3 The rate of convergence of the modified Newton-Raphson method (1.11) is linear. Solution. Let ξ be an exact root of the equation f(x) = 0 and x n be the approximate root at the nth iteration. Then f(ξ) = 0. Let ε n bethe error occurs at the nth iteration. Then ε n = x n ξ. Now, from the formula (1.11), we have ε n+1 = ε n f(ε n +ξ) f = ε n f(ξ)+ε nf (ξ)+ (x 0 ) f (x 0 ) ( ) = ε n 1 f (ξ) f +O(ε 2 (x 0 ) n). Neglecting square and higher power terms of ε n, the above equation reduces to ( ) ε n+1 = ε n 1 f (ξ) f. (x 0 ) Let C = 1 f (ξ) f (x 0 ), which is free from ε n. Using this notation the above error equation becomes ε n+1 = Cε n. (1.12) This shows that the rate of convergence of the formula (1.11) is linear. 11
...Newton s Method to Solve Transcendental Equation Example 1.7 Find a root of the equation x 3 3x 2 + 1 = 0 using modified Newton- Raphson formula (1.11) and Newton-Raphson method correct up to four decimal places. Solution. Let f(x) = x 3 3x 2 + 1. One root of this equation lies between 0 and 1. Let the initial guess be x 0 = 0.5. Now, f (x) = 3x 2 6x and hence f (x 0 ) = 2.25. The iteration scheme for the formula (1.11) is x n+1 = x n f(x n) f (x 0 ) = x n x3 n 3x 2 n +1 2.25 = x3 n 3x 2 n +2.25x n +1. 2.25 All the approximate roots are calculated in the following table. n x n x n+1 0 0.50000 0.66667 1 0.66667 0.65021 2 0.65021 0.65313 3 0.65313 0.65263 4 0.65263 0.65272 5 0.65272 0.65270 Therefore, 0.6527 is a root of the given equation correct up to four decimal places. By Newton-Raphson method The iteration scheme for Newton-Raphson method is x n+1 = x n f(x n) f (x n ) = x n x3 n 3x 2 n +1 3x 2 n 6x n = 2x3 n 3x 2 n 1 3x 2 n 6x n. Let x 0 = 0.5. The successive iterations are shown below. n x n x n+1 0 0.50000 0.66667 1 0.66667 0.65278 2 0.65278 0.65270 3 0.65270 0.65270 12
... Therefore, 0.6527 is a root correct up to four decimal places. In this example, Newton-Raphson method takes less number of iterations whereas as expected the modified formula (1.11) needs more iterations. 13