Some Derivatives of Newton s Method

Some Derivatives of Newton s Method J. B. Thoo Mathematics Department Yuba College 088 N. Beale Road Marysville, CA 9590-7699 Submitted January 9, 00 Revised February 5, 00 Revised August 3, 00 Corrected August 6, 00 Abstract Every first-year calculus student learns Newton s method as part of a repertoire of numerical root-finding methods. We present two other numerical root-finding methods that we derive using Newton s method: a simplified Newton s method, and Halley s method. These two methods are not new, yet they seldom find their way into a first calculus course even though they would not be out of place in such a course. KEYWORDS: Newton s method, Halley s method, numerical rootfinding, dynamical systems, fixed point, orbit, iteration function. Introduction Every first-year calculus student learns Newton s method (N.M.) as part of a repertoire of numerical root-finding methods; see [], for example. Of the variety of root-finding methods introduced, N.M. is a particularly important one for several reasons: it is a marvelous application of the differential calculus; it is Written while a visiting lecturer at the University of California, Davis, CA 9566-8633.

widely applicable; it is relatively fast, being quadratically convergent, while not being computationally too expensive. And it even leads to beautiful pictures that have found their way into art galleries, coffee-table books, and everyday homes [7]. We present two other numerical root-finding methods that we derive using N.M. (hence, the title, Some derivatives of Newton s method ): a simplified Newton s method (S.N.M.), and Halley s method (H.M.). These two methods are not new, yet they seldom find their way into a first calculus course even though they would not be out of place in such a course. We hope that this note will encourage the classroom presentation of these other two methods. This note is organized as follows. In Section, we introduce the methods and give a geometrical interpretation for each method. In Section 3, we compare the methods by using each of them to approximate. In Section 4, we take a dynamical systems approach to the methods, and use this point of view to make a second comparison of the methods, as well as to discuss briefly a couple of ways in which the methods may fail. The methods The basic problem is to find a root, α, of a given equation, f(x) =0, () without having to solve the equation. At the heart of the three numerical rootfinding methods we discuss is the following: make an initial guess, x 0,forα, and then use a recurrence formula to obtain a sequence, x 0,x,x,...,thatconverges to α. We assume throughout that α is a simple root of (), that is, f (α) 0.. Newton s method (N.M.) Go west, young man! was the advice given many of our nation s early settlers. When it comes to solving equations, an appropriate cry may be, Linearize, young student! And linearization is, indeed, the basis for Newton s method This means that an answer that is correct to one decimal place at one step should be correct to two decimal places at the next step, four decimal places at the following step, and so on [6, p. 40].

x x x 0 Figure : Newton s method. (N.M.): x n+ = x n f(x n) f, n =0,,,.... () (x n ) To derive the recurrence formula (), we first linearize () about x = x 0, that is to say, we replace f in () by its st-degree Taylor polynomial centered at x = x 0.Thisgives f(x 0 )+f (x 0 )(x x 0 )=0. (3) The linear equation (3) can be solved easily, and we suppose that x is the solution: x = x 0 f(x 0) f (x 0 ). Geometrically, x is the x-intercept of the line that is tangent to the graph of y = f(x) atthepoint(x 0,f(x 0 )). We repeat the process by linearizing () about x = x next. Then x is the solution of the new linearized equation, x = x f(x ) f (x ), if and only if x is the x-intercept of the line that is tangent to the graph of y = f(x) atthepoint(x,f(x )). This is depicted in Figure. A continuation of this process yields N.M. (). 3

x x x 0 Figure : A simplified Newton s method.. A simplified Newton s method (S.N.M.) The simplified Newton s method (S.N.M.) given in [0] is similar to N.M. except that we keep f (x 0 )fixedin(): x n+ = x n f(x n) f, n =0,,,.... (4) (x 0 ) To derive the recurrence formula (4), we begin as we did for N.M. by linearizing f in () about x = x 0.Thenx is the solution of the linearized equation (3), x = x 0 f(x 0) f (x 0 ), if and only if x is the x-intercept of the line that is tangent to the graph of y = f(x) atthepoint(x 0,f(x 0 )). Unlike in the derivation of N.M., however, we do not repeat the process by linearizing f about x = x next; instead, we translate the first tangent line in a parallel fashion so that it intersects the graph of y = f(x) atthepoint (x,f(x )), and let x be the x-intercept of the translated tangent line: x = x f(x ) f (x 0 ). This is depicted in Figure. Next, we translate the first tangent line in a parallel fashion so that it intersects the graph of y = f(x) atthepoint(x,f(x )), and 4

let x 3 be the x-intercept of the new translated tangent line: x 3 = x f(x ) f (x 0 ). A continuation of this process of parallel translations of the first tangent line leads to the S.N.M. (4)..3 Halley s method (H.M.) Another numerical root-finding method is Halley s method (H.M.) : f(x n )f (x n ) x n+ = x n f (x n ) f(x n )f, n =0,,,.... (5) (x n ) To derive the recurrence formula (5), we first replace f in () by its nddegree Taylor polynomial centered at x = x 0.Thisgives f(x 0 )+f (x 0 )(x x 0 )+ f (x 0 ) (x x 0 ) =0. (6) Next, we let x be a solution of (6), factorize x x 0 from the last two terms on the left-hand side, and isolate the outside factor of x x 0. This yields f(x 0 ) x x 0 = f (x 0 )+ f (x 0 ) (x x 0 ). (7) Then, we use N.M. () to approximate the factor x x 0 on the right-hand side of (7) to obtain, after some algebra, f(x 0 )f (x 0 ) x = x 0 f (x 0 ) f(x 0 )f (x 0 ). A repetition of this process successively with x,x,... leads to H.M. (5). (Frame [4] was the first to derive H.M. using a nd-degree Taylor polynomial. We can also get H.M. by applying N.M. to the function F = f/ f,afactthat was first pointed out by Bateman [].) Surprisingly, H.M. also sports a geometrical interpretation, namely, each approximation x n+ is the x-intercept of the hyperbola that is osculatory (ndorder-tangent) to the graph of y = f(x) atthepoint(x n,f(x n )); see Figure 3. (Salehov [8] was apparently the first to suggest this.) To see this, consider the hyperbola h(x) = (x x n)+c a(x x n )+b, Yes, this is the Halley of Halley s comet fame. See [9] for some historical background. 5

x x x x 0 0.8 0.9. x (a) Osculating hyperbolas. (b) A blow-up of the boxed region in Fig. 3(a). Figure 3: Halley s method. and solve the system of equations h(x n )=f(x n ), h (x n )=f (x n ), h (x n )=f (x n ) for the coefficients a, b, c. Then, the fact that x n+ = x n c if h(x n+ )=0 leads to H.M. (5). Details of this derivation can be found in [9]. 3 Comparing the methods A rigorous treatment of convergence and the speed of convergence of numerical root-finding methods is beyond the scope of this note; on these matters, the interested reader may consult a numerical analysis textbook, for example, [6, ]. Instead, we illustrate with a numerical example 3 using each method to approximate =.44 3 56... what is suggested by the geometry that is depicted in Figures 3: H.M. beats N.M. which, in turn, beats the S.N.M. (This is, in fact, generally the case.) 3 Numerical results are obtained using Maple V Release 4. 6

x 0 = N.M. S.N.M. H.M. x.500 000 000.500 000 000.48 57 49 x.46 666 667.437 500 000.44 3 97 x 3.44 5 686.40 898 438.44 3 56 x 4.44 3 56.46 60 345.44 3 56 x 5.44 3 56.44 78 84.44 3 56 x 6.44 3 56.44 380.44 3 56 Table : Approximating =.44 3 56... Toward this end, we let f(x) =x andx 0 =. We find that for N.M. x n+ = x n + x n, n =0,,,..., S.N.M. x n+ = x n +x 0 x n + x 0, n =0,,,..., H.M. x n+ = x3 n +6x n 3x, n =0,,,.... n + Table shows that H.M. approximates to eight decimal places (d.p.) in x 3, and N.M. in x 4 ; the S.N.M. does not approximate to eight d.p. even by x 6. (In fact, we must wait until x 6 =.44 3 563 before the S.N.M. approximates toeightd.p.) 4 A dynamical systems approach In this section, we take a dynamical systems approach to N.M., the S.N.M., and H.M. From this point of view, we provide a graphical analysis of the convergence of the methods in particular examples, as well as a look at a couple of ways in which the methods may fail. 4. Preliminaries The orbit of a point x 0 under iteration of a function φ is the sequence of points given by the recurrence formula x n+ = φ(x n ), n =0,,,.... 7

y y y = φ(x) y = φ(x) y = x x x x 0 α x y = x x x x0 α x (a) An attracting orbit. (b) A repelling orbit. Figure 4: The orbit of x 0 under iteration of φ, whereα is a fixed point of φ. Here, φ is called an iteration function (I.F.). Apointα is called a fixed point of φ if φ(α) =α. Afixedpointα is said to be attracting ifthereisanopenintervali containing α for which the orbit of x converges to α for all x I; andα is said to be repelling if the orbit of x tends away from α for all x I. See Figures 4(a) (b). An important basic result is that a fixed point α is attracting if φ (α) <, and it is repelling if φ (α) >. If φ n (α) =α where, for example, φ (α) = (φ φ)(α) then we call α a periodic point of period n; the least positive n for which φ n (α) =α is called the prime period of α. If, for some m>0, φ i (α) is periodic for i m, thenwecallα an eventually periodic point. For example, consider the function φ(x) = x 3. Then, φ(0) = 0 and φ (0) = 0 implies that zero is an attracting fixed point of φ. Notethatφ has periodic orbits of period two at ±. See Figure 5. 4 (For a thorough introduction to dynamical systems, see [3].) 4 Orbits are computed and plotted using Mary Ann Software PSMathGraphsII. 8

Figure 5: An attracting orbit and a periodic orbit of φ(x) = x 3. 4. Connection with N.M., the S.N.M., and H.M. Given a function f, define another function, N f,by N f (x) =x f(x) f (x), (8) that we call the Newton I.F. for f. Observe that the orbit of a point x 0 under iteration of N f is precisely the sequence of points we would obtain by applying N.M. () to an initial guess x 0. Further, α is a simple root of f(x) =0ifand only if α is a fixed point of N f. This shows us another way to study numerical root-finding methods like N.M., namely, as I.F. in dynamical systems. For the S.N.M. and H.M., we find that the simplified Newton I.F. and the Halley I.F. are, respectively, SN f (x; x 0 )=x f(x) f (x 0 ), (9) H f (x) =x f(x)f (x) f (x) f(x)f (x). (0) Note that SN f depends on a parameter, x 0, the initial guess for a root of f(x) =0. To illustrate, we apply (8) (0) to the example of approximating in 9

Section 3. First, recall that f(x) =x andx 0 =. Then, N f ( ) =, N f ( ) = 0, SN f ( ; ) =, SN f ( ; ) = H f ( ) =, H f ( ) = 0 <, implies that is an attracting fixed point in each case. Figures 6(a) (d) show the orbit of converging to the fixed point for the three I.F., which corresponds to the initial guess x 0 = converging to, the positive root of x = 0, using N.M., the S.N.M., and H.M. 4.3 A second comparison Garrett s Web page [5] provides an interactive animation of N.M. applied to the function f(x) =x 3 3x +x +0.4. The animation [5] demonstrates, using tangent lines, that the close approach of the graph to the x-axis in the trough [local minimum] causes bizarre and unpredicatable [sic] deflections from the true root. Here, we undertake a graphical analysis of the Newton I.F. for the function f. Note that the equation f(x) = 0 has exactly one (simple) real root, which is approximately 0.59 704 853. Figure 7(a) shows the graph of y = f(x) (heavy line) which has a local maximum at 3/3 andalocalminimumat+ 3/3 and the graph of y = N f (x) which has vertical asymptotes at the critical points of f. Figure 7(b) shows the orbits of 0.5 and under iteration of N f. We see that the orbit of 0.5 converges to the root of f(x) = 0. On the other hand, the orbit of appears erratic which stems from the fact that the orbit jumps back and forth between the branches of the graph of y = N f (x) on either side of the vertical asymptote + 3/3 and appears not to converge at all. Figure 7(c) shows, however, that eventually the orbit of does converge to the root of f(x) =0. What is the orbit of under iteration of SN f (x;)=x 3 3x +3x +0.4, H f (x) = 5x5 45x 4 +35x 3 x +8x 4 30x 4 0x 3 + 65x? 96x +6 We consider H f first. WenotethatH f has three fixed points, one each at the simple zero and the two critical points of f. In addition, H f has two vertical 0

α (a) N f (x) =(x +)/(x). α (b) SN f (x;)=( x +4x +)/4. α (c) H f (x) =(x 3 +6x)/(3x +). (d) A blow-up of [.35,.5] [.35,.5] in Fig. 6(c). Figure 6: The orbit of under iteration of N f, SN f ( ;), and H f, where f(x) =x, converging to the fixed point α =.

(a) The graphs of y = f(x) (heavy line) and y = N f (x). (b) The orbits of 0.5 and. (c) The orbit of after 90 iterations. Figure 7: A graphical analysis of N.M. applied to f(x) =x 3 3x +x +0.4 by examining its Newton I.F., N f (x) =(0x 3 5x )/(5x 30x + 0).

asymptotes, at about.53 80 69 and.63 467 004. As a result, Figure 8 shows that the orbit of under iteration of H f looks to be even more complicated than that under iteration of N f. Figure 8(c) shows that eventually, however, the orbit of converges to the root of f(x) =0. Next, we consider SN f (x; ), which has exactly one fixed point at the root of f(x) = 0. Figure 9(a) shows, however, that the orbit of does not converge to the fixed point; rather, the orbit escapes to infinity. In fact, Figure 9(b) shows that the orbit of tends monotonely toward + 3/3, the critical point of the local minimum of f, but then continues past that value. Hence, we see that the S.N.M. fails to approximate a root of f(x) =0inthiscase. 4.4 Failure of the method We discuss a few ways in which N.M. and the S.N.M. may fail. For the functions that we would usually consider, H.M. is globally convergent []. 4.4. Escape to infinity As a first example, consider the equation arctan(x ) = 0 that has exactly one root, namely,. Let f(x) = arctan(x ). Then, the Newton I.F. for f is N f (x) =x ( + (x ) )arctan(x ), and a straightforward calculation shows that N f () = 0 so that is an attracting fixed point of N f. Figure 0(a) shows that converges to under iteration of N f, so an initial guess of x 0 = converges to using N.M. On the other hand, Figure 0(b) shows that.5 does not converge under iteration of N f,soan initial guess of x 0 =.5 does not converge using N.M. (it escapes to infinity). 4.4. Periodic points As a second example, consider the function f given by x if x 0, f(x) = x if x<0. 3

(a) The orbit of after 00 iterations. (b) The orbit of after 90 iterations. (c) A blow-up of [, ] [, ] in Fig. 8(a) (b). Figure 8: A graphical analysis of H.M. applied to f(x) =x 3 3x +x +0.4 by examining its Halley I.F. In (c), we see the orbit of jumping among three branches of the graph of y = H f (x). 4

(a) The orbit of under iteration of SN f (x;). (b) A blow-up of [.3,.8] [.3,.8] in Fig. 9(a). Figure 9: A graphical analysis of the S.N.M. applied to f(x) =x 3 3x +x+0.4 by examining its simplified Newton I.F. (a) An attracting orbit of N f. (b) A repelling orbit of N f. Figure 0: In (a), the orbit of converges to under iteration of N f ; in (b), the orbit of.5 does not converge under iteration of N f. 5

(a) A periodic orbit of N f. (b)aperiodicorbitofsn f ( ;). Figure : Every point x 0 is a periodic point of period two for N f SN f ( ; x 0 )inthisexample. as well as Then, zero is a root of f(x) =0aswellasafixedpointof N f (x) = x, x x if x 0, SN f (x;)= x + x if x<0. In both cases, we find that is a periodic point of period two, so that does not converge to the root zero of f(x) = 0 using either N.M. or the S.N.M.; see Figure. Indeed, in this particular example, every point x 0 is a periodic point of period two for N f as well as for SN f ( ; x 0 ). Acknowledgements We thank the editor and an anonymous referee for their suggestions on content and style, which vastly improved the manuscripts. References [] H. Bateman, Halley s methods for solving equations, Amer. Math. Monthly 45, Jan 938, pp. 7. [] M. Davies and B. Dawson, On the global convergence of Halley s iteration formula, Numer. Math. 4, 975, pp. 33 35. 6

[3] Robert L. Devaney, A First Course in Chaotic Dynamical Systems: Theory and Experiment, Addison-Wesley, Reading, 99. [4] J. S. Frame, A variation of Newton s method, Amer. Math. Monthly 5, Jan 944, pp. 36 38. [5] Paul Garrett, Pathological example for Newton s method (applet), Web page <http://www.math.umn.edu/~garrett/qy/badnewton.html>, email <garrett@math.umn.edu>. [6] R.J.Hosking,S.Joe,D.C.Joyce,andJ.C.Turner,First Steps in Numerical Analysis, nd ed., Arnold, London, 996. [7] H.-O. Peitgen and P. H. Richter, The Beauty of Fractals: Images of Complex Dynamical Systems, Springer-Verlag, Berlin, 986. [8] G. S. Salehov, On the convergence of the process of tangent hyperbolas (Russian), Dokl. Akad. Nauk SSSR 8, 95, pp. 55 58. [9] T. R. Scavo and J. B. Thoo, On the geometry of Halley s method, Amer. Math. Monthly 0, No. 5, May 995, pp. 47 46. [0] Ivar Stakgold, Green s Functions and Boundary Value Problems, Wiley Interscience Ser. in Pure and Appl. Math., John Wiley & Sons, New York, 979. [] J. F. Traub, Iterative Methods for the Solution of Equations, Prentice-Hall, New Jersey, 964. [] Dale Varberg, Edwin J. Purcell, and Steven E. Rigdon, Calculus, 8th. ed., Prentice-Hall, Upper Saddle River, 000. 7