MATHEMATICAL EXPOSITION

Transcription

1 MATHEMATICAL EXPOSITION An Improved Newton's Method by John H. Mathews California State University, Fullerton, CA (oif John earned his Ph.D. in mathematics from Michigan State University under the direction of Peter Lappan. He has been a member of the mathematics faculty at California State University Fullerton,for nineteen years, and keeps active in 'the areas of complex analysis and numerical analysis. Introduction Newton's method is used to locate roots of the equation f(x) = 0. The Newton- Raphson iteration formula is : g(x) = x - f(x)/f' (x). () Given a starting value po, the sequence {pk} is computed using : pk+, = g(pk) for k=0,,... (provided that f (pk) =A 0). (2) If the value po is chosen close enough to the root p, then the sequence generated in (2) will converge to the root p. Sometimes the speed at which {pk) converges is fast (quadratic) and at other times it is slow (linear). To distinguish these two cases we make the following definitions. DEFINITION. Let {pk} be a sequence that converges to p, and set ek = p - pk for k = 0,,.... If there exists a constant A =A 0 such that then {pk} is said to converge quadratically to p. then {pk} is said to converge linearly to p. lim n-~ Iek 2 I ek, l I IekI =A, If there exists a constant A =A 0 such that The mathematical characteristic for determining which case occurs is the "multiplicity" of the root p. DEFINITION 2. If f(x) can be factored as f(x) = (x-p)mh(x), where M is a positive integer, (5) and h(x) is continuous at x=p and h(p) 0 0, then we say that f(x) has a root of order M at X=P. A root of order M= is called a simple root, and if M > it is called a multiple root. The next result is well known and can be found in the references. The AMATYC Review 9 Spring 989 Volume 0, Number 2

2 THEOREM l. [Order of convergence for Newton-Raphson iteration] Let the sequence {Pk} generated by (2) converge to the root p. If p is a simple root, the convergence is quadratic and I ek+, I 2 ~If~P)II I ek I z for k sufficiently large. (6) If p is a root of order M, convergence is linear and I ek+t " MM I Pk I for k sufficiently large. (7) There are two common ways to use Theorem I and gain quadratic convergence at multiple roots. We shall call these methods A and B (see Mathews, 987, p. 72 and Ralston & Rabinowitz, 978, pp ). Method A Suppose that p is a root of order M > l. Then the accelerated Newton-Raphson formula is : g(x) = x - Mf(x)/f' (x). (8) Let the starting value po be close to P, and compute the sequence {pk} iteratively ; Pk+ = Pk - Mf(pk)/ f (Pk) for k=0,,.... (9) Then the sequence generated by (9) will converge quadratically to p. On the other hand, if f(x) = (x-p)mh(x) then one can show that the function u(x) _ f(x)/ f'(x) has a simple root at x=p. Using u(x) in place of f(x) in formula () yields Method B. Method B Suppose that p is a root of order M >. Then the modified Newton-Raphson formula is : g(x) = x - f x)f'(x) [f'(x)] - f(x)f"(x) (0) Let the starting value po be close to p, and compute the sequence {Pk} iteratively ; Pk+I = Pk - f(pk)f'(pk), [f'(pk)] - f(pk)f (Pk) Then the sequence generated by () converges quadratically to p. Limitations of Methods A and B Method A has the disadvantage that the order M of the root must be known a priori. Determining M is often laborious because some type of mathematical analysis must be used. It is usually found by looking at the values of the higher derivatives of f(x). That is, f(x) = 0 has a root of order M at x = p if and only if f(p)=0, f' (P)=0,..., e m- '' (P)=0 and l+.u' (P) -A-0. (l2) 0

3 Dodes (978, pp. 8-82) has observed that in practical problems it is unlikely that we will know the multiplicity. However, a constant M should be used in (8) to speed up convergence, and it should be chosen small enough so that pk+, does not shoot to the wrong side of p. Rice (983, pp ) has suggested a way to empirically find M. If p* is a good approximation to p and p, and p2 somewhat distant from p* then M can be determined by the calculation : _ M In(f(p,)/f(p2)) ln((p,-p*)/(p2-p*)) Method B has a disadvantage, it involves three functions f(x), f'(x) and f'(x). Again, the laborious task of finding the formula for f"(x) could detract from using Method B. Furthermore, Ralston and Rabinowitz (978, pp ) have observed that u(x) will have poles at points where the zeros of f'(x) are not roots of f(x). Hence, u(x) may not be a continuous function. The New Method C The adaptive Newton-Raphson method incorporates a linear search method with formula (8). The following values are computed : Pi= po - Jf(Po)/f(Po) for j =,2,...,M. (3) Our task is to determine the value M to use in formula (3), because it is not known a priori. First, we take the derivative of f(x) in formula (5), and obtain : V(x) = (x-p)mh'(x) + M(x-p)"'- ' h(x). (4) When (5) and (4) are substituted into formula () we get : g(x) This enables us to rewrite (3) as : = x - M(x - P) +. (x-p)h'(x)/[m h(x)] Pi = Po -M (Po - P) I + (po-p)h'(po)/[m h(po)] (5) We shall assume that the starting value po is close enough to p so that + (po-p)h'(po)/[m h(po)] = + E, where E -'= 0. (6) The iterates p; in (5) satisfy the following : p; = po -, (po - p) ( + E) for j=,2,.... (7) If we subtract p from both sides of (7) then the result after simplification is : Pi - P = [( - J/M) - JE/M ][Po - P]. (8) Since je/ M -0, the iterates p; get closer to p as j goes from to M, which is manifest by the inequalities : IPo- PI > IPi - PI >...>IPj-PI >--->IPm-PI - (9) The values p; are shown in Figure. Notice that ifthe iteration (5) was continued for M+ and M+2 then I f(per+,) I and I f(pm+2) I should be larger than If(pm) I. This is proven by The AMATYC Review Spring 989 Volume 0, Number 2

4 using the derivatives in (2) and the Taylor polynomial approximation of degree M for f(x) expanded about x=p: P 4 P3 Figure. The values p,,p2,p3 and pa obtained by using formula (5) near a root p of order M=2. Notice that I f(p3) I > f(p2) I. If pi is closer to p than pi then (9) and (20) imply that I f(p;) I < I f(pi), hence we have : If(Po)I > If(Pi)I >...> If(Pj)I >...> If(PM)I (2 ) Therefore, the way to computationally determine M is to successfully compute the values p; using formula (3) for j =,2,...,M+ until we arrive at I f(pm) I < I f(pm") I. The Adaptive Newton-Raphson Algorithm Start with po, then we determine the next approximation p, as follows : Dx = f(po)/f(po). (22) p i =po -Dxand yi=f(p,). p2=po - 2Dxandy2=f(p2). j=3 WHILE I Y2 < Iyi DO P : = P2 Yi : = Y2 p 2 : =PO - j*dx Y2 = f(p2) j :=j+l ENDWHILE 2

5 Observe that (22) involves a linear search in either the interval ( --,po) when p, < po or in the interval (po,-) when po < p,. In the algorithm, the value Dx = f(po)/ f'(po ) is stored so that unnecessary computations are avoided. After the point (p,,f(p,)) has been found, it should replace (po,f(po)) and the process is repeated. EXAMPLE. Use the value po =.3 and compare Methods A,B and C for finding the double root p= of the equation x3-3x+ 2=0. Solution : The function is f(x)= x3-3x+ 2 and f'(x) = 3x 2-3 and f"(x) = 6x. The table gives the iterates. For method C, all the values in the linear search are included. k Newton-Raphson Iteration Using Formula (2) pk Method A Using M = 2 Formula (9) Pk. Observe in the example that the standard Newton-Raphson method converges linearly and that methods A and B converge quadratically. The reader can use formulas (2) to verify that M=2 is the order of the root p=. The formula for g(x) in Method B is : g(x) = PROGRAM Adaptive. Newton. Raphson ; FUNCTION F( X : REAL) : REAL ; F:=X*X*X - 3*X + 2; END ; FUNCTION F(X : REAL) : REAL ; Fl :=3*X*X - 3 ; END ; Method B Using Formula () pk x - [x3-3x+2[3x2-3 [3x'-3] - - [x- - 3x + 2][6x] Notice that all the iterates in Method A occur in Method C, hence the subsequence [p2k] in Method C converges quadratically. Only a few extra function evaluations were required by Method C while searching for the best value M. However, this time is well spent when Method C is compared to Method A. A Computer Program for Method C CONST Delta= ; Epsilon= ; Max=200 ; VAR Cond,J,K : INTEGER ; Df,Dp,YO,Error, PO,P,P2,Y,Y2 : REAL ; Method C Using Formula (22) pk The AMATYC Review Spring 989 Volume 0, Number 2

6 WRITE ('ENTER the initial approximation PO READLN(PO) ; K:=O ; Cond:=0 ; YO:=F(PO) ; P :=P0+ ; WRITELN('P(',K :2,') =',PO,' F(P(',K :2,')) YO) ; WHILE (K < Max) AND (Cond =0) DO Df:=F(PO) ; IF Df = 0 THEN Dp :=P-P0 ; Pl :=PO ; Cond:= ; END ELSE Dp:=YO/Df, P :=PO-Dp ; END ; Y :=F(P) ; WRITELN('P(',K+ :2,') =',Pl,' F(P(',K+ :2,'))=',YI) ; P2:=P0-2* Dp ; Y2:=F(P2) ; J :=3 ; WHILE (ABS(Y2) < ABS(Y)) AND (J < ) DO P l :=P2 ; Y l :=Y2 ; P2 :=PO-J*Dp ; Y2:=F(P2) ; J:=J+ ; K :=K+ ; WRITELN('P(',K+ :2,')=',P,' F(P(',K+ :2,'))Y) ; END Error:=ABS(Dp) ; IF (Error<Delta) OR (ABS(Y)<Epsilon) THEN Cond:= ; P0:=P ; Y0 :=Y ; K:=K+I ; END ; END. References Dodes,. A. (978). Numerical analt,sis for computer science. New York : North-Holland. Mathews, J. H. (987). Numerical methods for computer science, engineering and mathematics. Englewood Cliffs, NJ : Prentice-Hall. Ralston, A. & Rabinowitz, P. (978). A first course in numerical analti,sis (2nd ed.). New York : McGraw-Hill. Rice, J. R. (983). Numerical methods, sqfiware and anali^sis : IMSL reference edition. New York : McGraw-Hill. 4

7 Numerical Methods Using Matlab, 4 th Edition, 2004 John H. Mathews and Kurtis K. Fink ISBN: Prentice-Hall Inc. Upper Saddle River, New Jersey, USA