Root Finding Convergence Analysis

Transcription

1 Root Finding Convergence Analysis Justin Ross & Matthew Kwitowski November 5, 2012 There are many different ways to calculate the root of a function. Some methods are direct and can be done by simply solving for x, while other methods require multiple steps and the results contain some error. In this paper, we will focus on four methods that require multiple steps and we will compare how effective each method is. 1

2 Contents 1 Introduction 3 2 Root Finding Methods The Bisection Method Fixed-Point Iteration Newton s Method Halley s Method Results 7 4 Conclusion 10 5 Bibliography 11 6 Appendix Bisection Method Matlab Code Fixed Point Iteration Matlab Code Newton s Method Matlab Code Halley s Method Matlab Code Maple Code

3 1 Introduction Our goal is to examine the convergence of various root-finding methods. Specifically, we will examine how x + ln(x) = 0, x > 0 converges using the Bisection Method, Fixed Point Iteration, Newton s Method, and Halley s Method. We need to ensure eight digits of accuracy so that: x c r < where r is the real root and x c is the estimated root. We will compare the rate of convergence for each method as well as compare the time and number of steps required to compute the approximate root. We will also discuss when it is appropriate to use each of the four methods. In order to know if we have reached our desired accuracy, we must first figure out the real root of the function. Using Maple 11 s built in function fsolve, we find the approximate root of x+ln(x) = 0 to be We will assume this to be the most accurate root and compare our findings to it. 2 Root Finding Methods 2.1 The Bisection Method The Bisection Method is a slow, but certain, way to find a root of a function if the root exists. We can find the general area of the root by bracketing it on the interval [a, b] R so that there exists a pair f(a), f(b) where f(a)f(b) < 0. In Sauer s Numerical Analysis, Theorem 1.2 states that if f is a continuous function on [a, b], satisfying f(a)f(b) < 0 then f has a root between a and b. This means there exists a number r R such that a < r < b and f(r) = 0. After each step in the bisection method, we can refine our guess to be the midpoint between the interval [a, b] around the root. We can check the sign of the function evaluated at this midpoint and then refine our bracket around the root further. This gives us a new interval that contains r. We continue this process until we are satisfied with our accuracy or we have reached a maximum number of iterations. Sauer also states that after n steps of the Bisection Method the error term is given by x c r < b a. (1) 2n+1 We can use equation (1) to determine the number of iterations required for a given accuracy and initial guesses a, b. Letting ɛ x c r, we see that a b 2 n+1 < ɛ 2 n > a b 2ɛ a b log( ɛ n > ) 1 (2) log(2) 3

4 From equation (2), we find that the number of iterations, n, depends on both our initial interval length and desired accuracy. 2.2 Fixed-Point Iteration The fixed-point method is a linearly convergent method. We define a fixed point of a function g if g(r) = r. Using the properties of exponents and logarithms we can rewrite the given problem as and define our fixed-point equation as x + ln(x) = 0 ln(x) = x x = e x g(x) = 1 e x. (3) The Fixed Point iterative method is given by x 0 = initial guess x i+1 = g(x i ) for i = 0, 1, 2... If g is a continuous function and the {x i } converge to a number r then r is a fixed point. So if g is continuous and the {x i } converge then g(r) = g( lim i + x i) = lim g(x i) = i + lim x i+1 = r. i + Sauer also states in theorem 1.6 that if S = g (r) < 1 then the fixed-point iterations converge linearly with rate S to the fixed point r for initial guesses sufficiently close to r. S = g (r) = 1 e r = (4) e We will be able to test this against our resulting error ratio e i+1 e i where e i = r x i in our convergence analysis. The fixed point iterative method forms a cobweb diagram geometrically. Figure 1 shows the cobweb diagram for g(x) = 1 e and the first few steps with an initial guess x x 0 = 0.4 With this initial guess, we find that we get closer to the fixed point after each step and spiral inward closer to the root. 4

5 Figure 1: The fixed point is the intersection of g(x) and the diagonal line. 2.3 Newton s Method Newton s Method requires just one initial guess. It is also a specific form of the fixed point method. The general method is x 0 = initial guess x i+1 = x i f(x i) f for i = 0, 1, 2... (x i ) It is important to note that the first derivative is required to compute the root using Newton s Method. This can slow the overall computation time if derivative is difficult to compute. The time required to calculate the root will be reduced if the derivative is known. Using Newton s Method we expect quadratic convergence when the initial guess is sufficiently close to the real root. At the very least, we expect faster convergence than the linearly convergent methods. Since f (x) = 1+x x, the formula for Newton s Method is given by x i+1 = x i x i + ln(x i ) x i + x 2. i 5

6 2.4 Halley s Method Halley s Method is a root finding method that converges cubically. In order to use Halley s method, the function must have a continuous second derivative. The general method is x 0 = initial guess x i+1 = x i 2f(x i )f (x i ) 2[f (x i )] 2 f(x i )f (2), for i = 0, 1, 2... (x i ) Similarly to Newton s Method and Fixed Point Iteration, Halley s Method involves starting with an initial guess close to the root and then substituting it into the equation to get the next guess, and then repeating until you are satisfied with the degree of accuracy. For the function f(x) = x+ln(x), the formula for Halleys method is 2(x i + ln(x i ))(1 + 1 x x i+1 = x i + ln(x i ) i ) 2(1 + 1 x i ) 2 + (x i + ln(x i ))( 1 ). x 2 i 6

7 3 Results From the Bisection method and from equation (2) we expect the number of steps, n, required to approximate the root to be n log( 1 ) log(2) We chose to bracket the root around [0.1, 1.1] as our initial guess. Using Matlab, we performed a convergence analysis and found that 27 steps were required to converge to the root, which was expected. The midpoint is the current step s best approximation to the root. Notice the initial approximation to the root (the first midpoint between [a, b] is already accurate to one decimal place. Step Left Guess Midpoint Right Guess Error Figure 2: Convergence table for the Bisection Method 7

8 Using the fixed-point equation g(x) = e x and the initial guess of 0.4, we performed a convergence analysis for our function f(x) = x + ln(x). We see that this takes 31 steps to converge at the rate S = This confirms what we found in equation (3). This implies a linear rate of convergence. We also note that each x i root approximation is a point given by the cobweb diagram in Figure (1) as it spirals closer to the root. i x i g(x i ) e i = x i -r e i /e i Figure 3: Convergence table for the Fixed-Point Method with fixed point g(x) = e x Another fixed point iteration is given as g 2 (x) = x2 log(x) x+1. We found that using an initial guess of 0.4 g 2 (x) converged in 64 steps as opposed to the 31 steps from g(x). The convergence analysis for g 2 (x) showed an error ratio S = This shows that g 2 (x) converges slower than g(x) which is also confirmed by the number of steps taken to converge. Other derivations of g(x) from f(x) = x + ln(x) may exist that give faster or slower convergence. In fact, Newton s Method is a specific form of fixed point iteration. 8

9 Newton s Method converges with much fewer steps than the Bisection Method and the Fixed Point Method. We found that it converged to our root quadratically in just 4 steps as seen in Figure 4. Figure 4: Convergence table for Newton s Method While Newton s Method takes considerably fewer steps than the Fixed Point method and the Bisection Method, we found that, on average, our fixed point method for g(x) took 0.02 seconds to converge. However, our Newton s Method took 0.03 seconds. We believe this is because the method takes time to calculate derivative f (x). So while Newton s Method converges in fewer steps, more time may be taken to determine the derivative of the function. When the derivative is known, Newton s Method converged in just seconds where the bisection method converged in seconds. Halley s Method converges cubically, requiring only three steps as seen in Figure 5. We find that since both the first and second derivative of f(x) is required that, while this method converges in very few steps, it takes, on average, seconds. This is longer than each of the other methods. However, when the derivatives are known, Halley s Method takes seconds which is slightly longer than when the derivatives were known for Newton s Method. This is most likely due to the added complexity of the iterative method with Halley s Method. Each step of Halley s Method requires more addition and multiplication computations to calculate the next x i+1. Figure 5: Convergence table for Halley s Method 9

10 4 Conclusion As we have seen, each method has its advantages and disadvantages. The Bisection Method requires previous knowledge of where the root is to bracket the root around an interval [a, b]. While, it takes fewer steps compute the root than the Fixed Point Iteration, our initial approximate root was also closer to the real root. With the Bisection Method we are also guaranteed convergence as long as the root exists within our interval and the function is continuous on the interval. Figure 6: Summary comparison of the number of steps and time required to compute the root for f(x) = x + ln(x) for each method. The Fixed Point method can be difficult to compute if the function g(x) is difficult to find. We saw that different functions g(x) of our given problem f(x) = x + ln(x) also converged at different rates. With the Fixed Point Iteration, not only is convergence not guaranteed, but it might also not be very fast. Newton s method was shown to be the best method for finding the root of f(x) = x + ln(x). It took more time to converge than the Bisection Method and Fixed Point Method when the derivative needed to be calculated, but when derivative is known it converged to the root in less time than each other method and with only one more step than the cubically converging Halley s Method. While Halley s method converged at the fastest rate with the least amount of steps, we found that the complexity behind the iterative method added more time to calculate the root. Even when the first and second derivative was known, it took more time to compute than Newton s Method as we see in figure 6. 10

11 5 Bibliography References [1] T. Sauer, Numerical Analysis: Second Edition, Pearson Education, 2012, pp [2] P. Acklam, A small paper on Halley s method, 23 December 2002, pjacklam/notes/halley/halley.pdf, 26 October

12 6 Appendix 6.1 Bisection Method Matlab Code 1 %B i s e c t i o n Method 2 3 function [ Z ] = b i s e c t i o n ( f, a, b, t o l, maxiter ) 4 5 %INPUTS 6 % I n l i n e f u n c t i o n f 7 % a w i l l g i v e us a n e g a t i v e r e s u l t when in f 8 % b w i l l g i v e us a p o s i t i v e r e s u l t when in f 9 % t o l i s the e r r o r we can be w i t h i n ( f o r 8 d i g i t accuracy ) %OUTPUTS 12 % Z i s a matric containng each guess and e r r o r %Real root o f t he s u p p l i e d f u n c t i o n 15 %f = x + l n ( x ) = 0 16 realroot = ; % Z c o n t a i n s the x a x i s v a l u e s to check f o r r o o t s 19 % l e n g t h i s max number o f i t e r a t i o n s 20 Z = zeros ( 1, 4 ) ; %I n i t i a l l e f t guess s t a r t i n g p o i n t 23 Z ( 1, 1 ) = a ; %F i r s t Midpoint v a l u e to check 26 Z ( 1, 2 ) = ( a+b ) / 2 ; %F i r s t r i g h t guess s t a r t i n g p o i n t 29 Z ( 1, 3 ) = b ; %I n i t i a l e r r o r 32 Z ( 1, 4 ) = abs (Z ( 1, 2 ) realroot ) ; %I f s t a r t i n g p o i n t are the same s i g n then break program 35 i f sign ( f (Z ( 1, 1 ) ) ) sign ( f (Z ( 1, 3 ) ) ) >= 0 36 error ( f ( a ) and f ( b ) are both the same s i g n ) 37 end %k i s the counter 40 k = 1 ; %End loop i f midpoint guess i s w i t h i n the t o l e r a n c e or we have reached the 43 %maximum number o f i t e r a t i o n s 12

13 44 while (Z( k, 4 ) > t o l && k < maxiter ) %I f the midpoint has the same s i g n as the l e f t p o i n t then t h a t 47 %midpoint becomes the new l e f t p o i n t 48 i f sign ( f (Z( k, 2 ) ) ) == sign ( f (Z( k, 1 ) ) ) 49 Z( k+1,1) = Z( k, 2 ) ; %S e t s l e f t p o i n t to the midpoint 50 Z( k+1,3) = Z( k, 3 ) ; %S e t s the r i g h t p o i n t to the o l d r i g h t p o i n t 51 else 52 Z( k+1,3) = Z( k, 2 ) ; %S e t s the r i g h t p o i n t to the midpoint 53 Z( k+1,1) = Z( k, 1 ) ; %S e t s the l e f t p o i n t to the o l d l e f t p o i n t 54 end %New Midpoint to check on x a x i s 57 Z( k+1,2) = (Z( k+1,1) + Z( k+1,3) ) / 2 ; %Error i s the d i s t a n c e from the c urrent midpoint guess and the r e a l r o ot 60 Z( k+1,4) = abs ( realroot Z( k+1,2) ) ; %Error r a t i o 63 %Z( k +1,5) = Z( k +1,4)/Z( k, 4 ) ; k = k+1; %I n c r e a s e counter by 1 66 end %Final Midpoint guess 69 xc = Z( k, 2 ) ; 13

14 6.2 Fixed Point Iteration Matlab Code 1 %Fixed Point I t e r a t i o n S o l v e r 2 3 function Z = f i x e d P o i n t ( g, a, t o l, maxiter ) 4 5 %INPUTS 6 % I n l i n e f u n c t i o n g 7 % a i s the i n i t i a l guess 8 % t o l i s the t o l e r a n c e we can be w i t h i n ( f o r 8 d i g accuracy 9 % maxiter i s t he maximum numbre o f i t e r a t i o n s a l l o w e d %OUTPUTS 12 %Z i s a matrix where : 13 %Z( x, 1 ) i s the x a x i s guess 14 %Z( x, 2 ) i s the r e s u l t i n g f u n c t i o n v a l u e 15 %Z( x, 3 ) i s the e r r o r d i s t a n c e between the guess and the r e a l root 16 %Z( x, 4 ) i s the e r r o r r a t i o %Real root o f t he s u p p l i e d f u n c t i o n 19 %f = x + l n ( x ) = 0 20 %g = ( x x l o g ( x ) ) / ( x+1) 21 %h = 1/ eˆx 22 realroot = ; %TEST ROOT FOR x + cos ( x ) s i n ( x ) 25 %realroot = ; %I n i t i a l i z e Z 28 Z = zeros ( 1, 4 ) ; %S t a r t the Counter 31 k = 1 ; %The i n i t i a l guess 34 Z ( 1, 1 ) = a ; %The i n i t i a l e v a l u a t i o n at our guess 37 Z ( 1, 2 ) = g (Z ( 1, 1 ) ) ; %I n i t i a l e r r o r 40 Z ( 1, 3 ) = abs (Z ( 1, 1 ) realroot ) ; %There i s no i n i t i a l e r r o r r a t i o while (Z( k, 3 ) > t o l && Z( k, 3 ) = 0 ) %Needs work when e r r o r = i f isnan (Z( k, 1 ) ) isnan (Z( k, 2 ) ) isnan (Z( k, 3 ) ) 14

15 47 error ( Fixed Point does not converge ) 48 end %Break i f max number o f i t e r a t i o n s reached 51 i f k > maxiter 52 break 53 end k = k + 1 ; %Increment the count %Z( k, 1 ) = 1 ; %Update t he count p l a c e h o l d e r 58 Z( k, 1 ) = g (Z( k 1,1) ) ; %Create next guess 59 Z( k, 2 ) = g (Z( k, 1 ) ) ; %Evaluate the newest guess 60 Z( k, 3 ) = abs (Z( k, 1 ) realroot ) ; %Determine the e r r o r 61 Z( k, 4 ) = Z( k, 3 ) /Z( k 1,3) ; %Determine the e r r o r r a t i o %I f Error r a t i o i s g r e a t e r than 1 i t w i l l not converge 64 i f Z( k, 3 ) > Z( k 1,3) 2 && k > 2 65 error ( Fixed Point does not converge ) %ends program 66 end 67 end 15

16 6.3 Newton s Method Matlab Code 1 %Newtons Method 2 3 function Z = newtons ( f, a, t o l, maxiter, var ) 4 5 %INPUTS 6 % NOT INLINE f u n c t i o n f 7 % a i s the i n i t i a l guess 8 % t o l i s the t o l e r a n c e we can be w i t h i n ( f o r 8 d i g accuracy 9 % maxiter i s t he maximum number o f i t e r a t i o n s a l l o w e d 10 % var i s the v a r i a b l e s u p p l i e d in the i n l i n e f u n c t i o n %OUPUTS 13 % Z i s a matrix c o n s i s t i n g o f : 14 %Z( x, 1 ) i s the x a x i s guess 15 %Z( x, 2 ) i s the r e s u l t i n g f u n c t i o n v a l u e 16 %Z( x, 3 ) i s the e r r o r r a t i o %Make the v a r i a b l e used a symbol 19 syms ( var, r e a l ) ; 20 t i c 21 f D i f f = d i f f ( f ) ; %Take f i r s t d e r i v a t i v e o f f 22 f D i f f = i n l i n e ( f D i f f ) ; %Make f i r s t d e r i v a t i v e an i n l i n e f u n c t i o n 23 f I n l i n e = i n l i n e ( f ) ; %Make s u p p l i e d f u n c t i o n i n l i n e %Real root o f t he s u p p l i e d f u n c t i o n 26 %f = x + l n ( x ) = 0 27 %D e r i v a t i v e o f f = 1 + 1/ x 28 realroot = ; %TEST ROOT FOR x + cos ( x ) s i n ( x ) 31 %realroot = ; %I n i t i a l i z e Z 34 Z = zeros ( 1, 3 ) ; %S t a r t the Counter 37 k = 1 ; %The i n i t i a l guess 40 Z ( 1, 1 ) = a ; %I n i t i a l e r r o r 43 Z ( 1, 2 ) = abs (Z ( 1, 1 ) realroot ) ; %No I n i t i t a l e r r o r r a t i o 46 16

17 47 while (Z( k, 2 ) > t o l && k < maxiter && Z( k, 2 ) = 0) %Create the next guess 50 Z( k+1,1) = Z( k, 1 ) f I n l i n e (Z( k, 1 ) ) / f D i f f (Z( k, 1 ) ) ; %Deterimne e r r o r o f the newly c r e a t e d guess 53 Z( k+1,2) = abs (Z( k+1,1) realroot ) ; %Determine the e r r o r r a t i o 56 Z( k+1,3) = Z( k+1,2) /Z( k, 2 ) ˆ 2 ; %Determine the e r r o r r a t i o %Increase the counter 59 k = k+1; end 17

18 6.4 Halley s Method Matlab Code 1 %Halley s Method 2 3 function Z = h a l l e y ( f, a, t o l, maxiter, var ) 4 5 %INPUTS 6 % NOT INLINE f u n c t i o n f 7 % a i s the i n i t i a l guess 8 % t o l i s the t o l e r a n c e we can be w i t h i n ( f o r 8 d i g accuracy 9 % maxiter i s t he maximum number o f i t e r a t i o n s a l l o w e d 10 % var i s the v a r i a b l e s u p p l i e d in the i n l i n e f u n c t i o n %Make the v a r i a b l e used a symbol 13 syms ( var, r e a l ) ; f i r s t D = d i f f ( f ) ; %F i r s t d e r i v a t i v e o f f 16 secondd = d i f f ( f i r s t D ) ; %Second d e r i v a t i v e o f f f i r s t D = i n l i n e ( f i r s t D ) ; %F i r s t D e r i v a t i v e o f s u p p l i e d function, i n l i n e 19 secondd = i n l i n e ( secondd ) ; %Second D e r i v a t i v e o f s u p p l i e d function, i n l i n e 20 f = i n l i n e ( f ) ; %The s u p p l i e d f u n c t i o n covnerted to i n l i n e %Real root o f t he s u p p l i e d f u n c t i o n 23 %f = x + l n ( x ) = 0 24 %D e r i v a t i v e o f f = 1 + 1/ x 25 %Second D e r i v a t i v e o f f = 1/x ˆ2 26 realroot = ; %I n i t i a l i z e Z 29 Z = zeros ( 1, 3 ) ; %S t a r t the Counter 32 k = 1 ; %The i n i t i a l guess 35 Z ( 1, 1 ) = a ; %I n i t i a l e r r o r 38 Z ( 1, 2 ) = abs (Z ( 1, 1 ) realroot ) ; %No I n i t i t a l e r r o r r a t i o while (Z( k, 2 ) > t o l && k < maxiter && Z( k, 2 ) = 0) %Create the next guess 18

19 45 Z( k+1,1) = Z( k, 1 ) [ 2 f (Z( k, 1 ) ) f i r s t D (Z( k, 1 ) ) ] / [ ( 2 ( f i r s t D (Z( k, 1 ) ) ) ˆ2) f (Z( k, 1 ) ) secondd (Z( k, 1 ) ) ] ; %Deterimne e r r o r o f the newly c r e a t e d guess 48 Z( k+1,2) = abs (Z( k+1,1) realroot ) ; %Determine the e r r o r r a t i o 51 Z( k+1,3) = Z( k+1,2) /Z( k, 2 ) ˆ 3 ; %Determine the e r r o r r a t i o %Increase the counter 54 k = k+1; end 19

20 6.5 Maple Code func := x + ln(x) fsolve(func) =