NEWTON's Method in Comparison with the Fixed Point Iteration

Transcription

1 NEWTON's Method in Comparison with the Fixed Point Iteration Univ.-Prof. Dr.-Ing. habil. Josef BETTEN RWTH Aachen University Mathematical Models in Materials Science and Continuum Mechanics Augustinerbach 4-20 D A a c h e n, Germany <betten@mmw.rwth-aachen.de Abstract This worksheet is concerned with finding numerical solutions of non-linear equations in a single unknown. Using MAPLE 2 NEWTON's method has been compared with the fixed-point iteration. Some examples have been discussed in more detail. Keywords: NEWTON's method; zero form and fixed point form; BANACH's fixed-point theorem; convergence order Convergence Order A sequence with a high order of convergence converges more rapidly than a sequence with a lower order. In this worksheet we will see that NEWTON's method is quadratically convergent, while the fixed point iteration converges linearly to a fixed point. Some examples illustrade the convergence of both iterations in the following. Before several examples are discussed in more detail, let us list some definitions in the following. A value x = p is called a fixed point for a given function g(x) if g(p) = p. In finding the solution x = p for f(x) = 0 one can define functions g(x) with a fixed point at x = p in several ways, for example, as g(x) = x - f(x) or as g(x) = x - h(x)*f(x), where h(x) is a continuous function not equal to zero within an interval [a, b] considered. The iteration process is expressed by restart: x[n+]:=g(x[n]); # n = 0,,2,... x n + := g( x n ) with a selected starting value for n = 0 in the neighbourhood of the expected fixed point x = p. An unique solution f(x) = 0 exists, if BANACH's fixed-point theorem is fulfilled: Let g(x) be a continuous function in [a, b]. Assume, in addition, that g'(x) exists on (a, b) and that a constant L = [0, ) exists with restart:

2 abs(diff(g(x),x))<=l; d dx g( x) L for all x in [a,b]. Then, for any selected initial value in [a, b] the sequence defined by x[n+]:=g(x[n]); # n = 0,,2,... x n + := g( x n ) converges to the unique fixed-point x = p in [a, b]. The constant L is known as LIPPSCHITZ constant. Based upon the mean value theorem we arrive from the above assumption at abs(g(x)-g(xi))<=l*abs(x-xi); g( x ) g( ξ ) L x ξ for all x and xi in [a, b]. The BANACH fixed-point theorem is sometimes called the contraction mapping principle. A sequence converges to p of order alpha, if restart: Limit(abs(x[n+]-p)/abs(x[n]-p)^alpha,n=infinity)=epsilon; x n + + p lim = ε n x n + p α with an asymptotic error constant epsilon. Another definition of the convergence order is given by: abs(x[n+]-p)<=c*abs(x[n]-p)^alpha; x n + + p C x n + where C is a constant. The fixed-point iteration converges linearly (alpha = ) with a constant C = (0, ). This can be shown, for example, as follows: g(x)=g(x[n])+(x-x[n])*diff(g(x),x)[x=x[n]]+ (x-x[n])^2*diff(g(x),x$2)[x=xi]/2; # TAYLOR d g( x ) = g( x n ) + ( x x n ) + dx g( x ) 2 ( x x ) 2 d 2 n g( x ) dx 2 x = x n x = ξ g(x):=g(x[n])+(x-x[n])*`g'`(x[n])+(x-x[n])^2*`g''`(xi)/2; g( x ) := g( x n ) + ( x x n ) g' ( x n ) + 2 ( x x n ) 2 g'' ( ξ ) where xi in the remainder term lies between x and x[n]. For x = p we arrive at g(p):=g(x[n])+(p-x[n])*`g'`(x[n])+(p-x[n])^2*`g''`(xi)/2; g( p ):= g( x n ) + ( x n + p ) g' ( x n ) + 2 ( x n + p ) 2 g'' ( ξ ) The 3rd term on the right hand side may be neglected, because x[n] is an approximation near to the fixed-point, (p - x[n])^2 << (p - x[n]), hence: g(p):=g(x[n])+(p-x[n])*`g'`(x[n]); g( p ):= g( x n ) + ( x n + p ) g' ( x n ) Corresponding with the fixed-point iteration g(p) = p and g(x[n]) = x[n+] we arrive from p α 2

3 this equation at: abs(x[n+]-p)=abs(`g'`(x[n]))*abs(x[n]-p); x n + + p = g' ( x n ) x n + p where abs(`g'`(x[n]))<=l; g' ( x n ) L abs(x[n+]-p)<=l*abs(x[n]-p); x n + + p L x n + p Thus, the fixed-point iteration converges linearly to the fixed-point p. In contrast, NEWTON's method is quadratically convergent. This can be shown, for example, as follows. G(x)=G(p)+(x-p)*Diff(G(x),x)[x=p]+ (x-p)^2*diff(g(x),x$2)[x=xi]/2; G( x ) = G( p ) + ( x p) + d G( x ) dx 2 ( x p) 2 d 2 G( x ) dx 2 x= p x = ξ G(x):=G(p)+(x-p)*`G'`(p)+(x-p)^2*`G''`(xi)/2; G( x ) := G( p ) + ( x p ) G' ( p) + 2 ( x p) 2 G'' ( ξ ) where xi lies between x and p. For x = x[n] we have: G(x[n])=G(p)+(x[n]-p)*`G'`(p)+ (x[n]-p)^2*`g''`(zeta)/2; G( x n ) = G( p ) + ( x n p ) G' ( p) + 2 ( x n p ) 2 G'' ( ζ ) where zeta lies between x[n] and p. Because of G(x[n]) = x[n+] and G(p) = p we get: x[n+]-p=(x[n]-p)*`g'`(p)+(x[n]-p)^2*`g''`(zeta)/2; x n + p = ( x n p ) G' ( p) + 2 ( x n p ) 2 G'' ( ζ ) The first term on the right hand side must be equal to zero, if the iteration x[n+] = G(x[n]) should converge quadratically to the fixed-point p : `G'`(p):=0; G' ( p) := 0 x[n+]-p=`g''`(zeta)*(x[n]-p)^2; x n + p = ( x n p) 2 G'' ( ζ ) NEWTON's method has the convergence order alpha = G'(p) = 0. Hence: restart: G(x):=x-h(x)*f(x); # iteration function G( x ) := x h( x ) f( x ) `G'`(x):=diff(G(x),x); d G' ( x ) := dx h( x ) f( x ) h( x ) d dx f( x ) `G'`(x):=-`h'`(x)*f(x)-h(x)*`f '`(x); G' ( x ):= h' ( x ) f( x ) h( x ) f '( x ) 3

4 `G'`(p):=subs(x=p,%); G' ( p ) := h' ( p ) f( p ) h( p ) f '( p) At the fixed-point p we have G'(p) = 0 and f(p) = 0. Hence: h(p):=/`f '`(p); h(x):=/`f '`(x); G(x):=x-f(x)/`f '`(x); h( p) h( x ) := f '( p) := f '( x ) f( x ) G( x ) := x f '( x ) Because of x[n+] = G(x[n] we find NEWTON's iteration method: x[n+]:=x[n]-f(x[n])/`f '`(x[n]); f( x n ) x n + := x n f '( x n ) NEWTON's iteration method has the disadvantage that it cannot be continued, if f '(x[n]) is equal to zero for any step x[n]. However, the method is most effective, if f '(x[n]) is bounded away from zero near the fixed-point p. Note, in cases when there is a point of inflection or a horizontal tangent to the function f(x) in the vicinity of the fixed-point, the sequence x[n+] = G(x[n]) need not converge to the fixed-point. Thus, before applying NEWTON 's method, one should investigate the behaviour of the derivatives f '(x) and f ''(x) in the neighbourhood of the expected fixed-point. Instead of the analytical derivation of NEWTON's method one can find the approximations x[n] to the fixed-point p by using tangents to the graph of the given function f(x) with f(p) = 0. Beginning with x[0] we obtain the first approximation x[] as the x-intersept of the tangent line to the graph of f(x) at (x[0], f(x[0])). The next approximation x[2] is the x-intercept of the tangent line to the graph of f(x) at (x[], f(x[])) and so on. Following this procedure we arrive at NEWTON's iteration characterized by the iteration function G(x) defined before. The derivative of this function is given by: restart: Diff(G(x),x)= simplify(-((diff(f(x),x))^2-f(x)*diff(f(x),x$2))/ Diff(f(x),x)^2); d f ( x ) d 2 G( x ) = dx f( x ) 2 dx 2 d dx f( x ) `G'`(x):=f(x)*`f ''`(x)/(`f '`(x))^2; f( x ) f ''( x ) G' ( x ) := f '( x ) 2 At the fixed-point the given function has a zero, f(p) = 0. Hence: 4

5 `G'`(p):=Diff(G(x),x)[x=p]=0; d G' ( p ) := G( x ) = 0 dx x= p Résumé: NEWTON's method converges optimal ( G'(p) = 0 ) to the fixed-point p unless f '(x) = 0 for some x[n]. The derivative G'(p) = 0 implies quadratic convergence. Examples The first example is concerned with the root-finding problem f(x) = x - cos(x) = 0 by using the iteration functions g(x) und G(x) characterized by linear and quadratic convergence, respectively. restart: f(x):=x-cos(x); f( x ):= x cos( x ) p:=fsolve(f(x)=0,x); # MAPLE solution by the command "fsolve" p := `f '`(x):=diff(f(x),x); f '( x ) := + sin( x ) `f ''`(x):=diff(f(x),x$2); f ''( x ):= cos( x ) alias(h=heaviside,th=thickness,co=color): p[]:=plot({f(x),+sin(x),cos(x)},x=0..pi/2,-..2, th=3,co=black): p[2]:=plot({2*h(x-.57),-h(x-.57)},x= ,co=black, title="f(x), f'(x), f''(x), Fixed Point p = 0.739"): p[3]:=plot({2,-},x=0..pi/2,co=black): p[4]:=plot(.674*h(x-0.74),x= , linestyle=4,co=black): p[5]:=plot([[0.74,.674]],style=point, symbol=circle,symbolsize=30,co=black): p[6]:=plots[textplot]({[.5,,`f(x)`],[0.2,.4,`f'(x)`], [0.2,0.8,`f''(x)`]},co=black): plots[display](seq(p[k],k=..6)); 5

6 In this Figure we see that the first derivative f '(x) is not equal to zero in the entire range considered, which is an essential condition for the application of NEWTON's method. The iteration functions are given by g(x):=x-f(x); G(x):=x-f(x)/diff(f(x),x); g( x ) := cos( x ) G( x ):= x x cos( x ) + sin( x ) alias(h=heaviside,sc=scaling,th=thickness,co=color): p[]:=plot({x,g(x),g(x)},x=0..,0.., sc=constrained,th=3,co=black): p[2]:=plot(h(x-),x= ,co=black): p[3]:=plot(,x=0..,co=black, title="iteration Functions g(x) and G(x)"): p[4]:=plot([[0.739,0.739]],style=point, symbol=circle,symbolsize=30,co=black): p[5]:=plots[textplot]({[0.5,0.8,`g(x)`], [0.6,0.92,`g(x)`]},co=black): p[6]:=plot(0.739*h(x-0.739),x= , linestyle=4,co=black): plots[display](seq(p[k],k=..6)); 6

7 Both operators, G and g, mapp the interval x = [0, ] to itself. The iteration function G(x) has a horizontal tangent in x = p because of G'(p) = 0, id est: quadratic convergence. Now, let's discuss the absolute derivatives of the iteration functions. abs(`g'`(x))=abs(diff(g(x),x)); g' ( x ) = sin( x ) abs(`g'`(x))=abs(diff(g(x),x)); ( x cos( x )) cos( x ) G' ( x ) = 2 ( + sin( x )) p[]:=plot({rhs(%%),rhs(%)},x=0..,0.., sc=constrained,th=3,co=black): p[2]:=plot(0.67*h(x-0.74),x= ,linestyle=4,co=black, title="absolute Derivatives G'(x) and g'(x) "): p[3]:=plot([[0.74,0.67]],style=point, symbol=circle,symbolsize=30,co=black): p[4]:=plot(h(x-),x= ,co=black): p[5]:=plot(,x=0..,co=black): p[6]:=plots[textplot]({[0.6,0.9,` G'(x) `], [0.85,0.9,` g'(x) `]},co=black): plots[display](seq(p[k],k=..6)); 7

8 This Figure illustrades that both derivatives exist on (0, ) with g'(x) < L and G'(x) < K for all x = [0, ], where K < and L <. Considering the last two Figures, we establish that both iteration functions are compatible with BANACH's fixed-point theorem. The iterations generated by g(x) and G(x) are given as follows: x[0]:=0.7; x[]:=evalf(subs(x=0.7,g(x))); x 0 := 0.7 x := Fixed Point Iteration: for i from 2 to 25 do x[i]:=evalf(subs(x=%,g(x))) od; x 2 := x 3 := x 4 := x 5 := x 6 := x 7 := x 8 := x 9 := x 0 := x := x 2 := x 3 := x 4 := x 5 := x 6 := x 7 :=

9 x 8 := x 9 := x 20 := x 2 := x 22 := x 23 := x 24 := x 25 := The 25th iteration x[25] is nearly identical to the MAPLE solution p = NEWTON's Method: X[0]:=0.7; X[]:=evalf(subs(x=%,G(x))); X 0 := 0.7 X := for i from 2 to 5 do X[i]:=evalf(subs(x=%,G(x))) od; X 2 := X 3 := X 4 := X 5 := The 3rd iteration is already identical to the MAPLE solution. In contrast, the fixed-point method needs about 25 iterations. The NEWTON method converges quadratically, while the fixed point sequence is linear convergent. Similar to the first example, the next one is concerned with solving the problem f(x) = 0, where restart: f(x):=x-(sin(x)+cos(x))/2; f( x ) := x sin( x ) cos( x ) 2 2 p:=fsolve(f(x)=0,x); # fixed point by MAPLE p := `f '`(x):=diff(f(x),x); `f ''`(x):=diff(f(x),x$2); f '( x ) := cos( x ) + sin( x ) 2 2 f ''( x ) := sin( x ) + cos( x ) 2 2 alias(h=heaviside,th=thickness,co=color): 9

10 p[]:=plot({f(x),diff(f(x),x),diff(f(x),x$2)}, x=0..pi/2,th=3,co=black,scaling=constrained): p[2]:=plot({.57*h(x-.57),-0.5*h(x-.57)}, x= ,co=black): p[3]:=plot(0.943*h(x-0.705),x= , linestyle=4,co=black): p[4]:=plot({-0.5,pi/2},x=0..pi/2,co=black, title="f(x), f'(x), f''(x), Fixed Point p = "): p[5]:=plot([[0.705,0.943]],style=point, symbol=circle,symbolsize=30,co=black): p[6]:=plots[textplot]({[0.85,0.3,`f(x)`],[0.85,.25,`f'(x)`], [0.85,0.6,`f''(x)`]},co=black): plots[display](seq(p[k],k=..6)); In this Figure we see that the first derivative f '(x) is not equal to zero in the entire range considered, which is an essential condition for the application of NEWTON 's method. The iteration functions are given by g(x):=x-f(x); G(x):=x-f(x)/diff(f(x),x); g( x ) := sin( x ) + cos( x ) 2 2 G( x ):= x x 2 sin( x ) 2 cos( x ) cos( x ) + sin( x ) 2 2 alias(h=heaviside,sc=scaling,th=thickness,co=color): p[]:=plot({x,g(x),g(x)},x=0..,0..,th=3, sc=constrained,co=black): p[2]:=plot({,h(x-)},x=0...00,co=black, title="iteration Functions g(x) and G(x)"): p[3]:=plot(0.7048*h(x ),x= , 0

11 linestyle=4,co=black): p[4]:=plot([[0.7048,0.7048]],style=point, symbol=circle,symbolsize=30,co=black): p[5]:=plots[textplot]({[0.5,0.52,`g(x)`], [0.5,0.8,`G(x)`]},co=black): plots[display](seq(p[k],k=..5)); Both operators, g and G, mapp the interval x = [0, ] to itself. The iteration function G(x) has a horizontal tangent in the fixed-point p = because of G'(p) = 0, id est: quadratic convergence. In contrast, the iteration function g(x) has a horizontal tangent in x = Pi/4 = , id est: in the neighbourhood of the fixed-point. Now let's discuss the absolute derivatives of the iteration functions. abs(`g'`(x))=abs(diff(g(x),x)); g' ( x ) = cos( x ) sin( x ) 2 2 abs(`g'`(x))=abs(diff(g(x),x)); G' ( x ) = x 2 sin( x ) 2 cos( x ) 2 sin( x ) + 2 cos( x ) cos( x ) + sin( x ) 2 2 p[]:=plot({abs(diff(g(x),x)),abs(diff(g(x),x))},x=0..,0.., scaling=constrained,th=3,co=black): p[2]:=plot({,h(x-)},x=0...00,co=black, title="absolute Derivatives g'(x) and G'(x) "): p[3]:=plots[textplot]({[0.2,0.3,` g'(x) `], [0.2,0.9,` G'(x) `]},co=black): plots[display](seq(p[k],k=..3)); 2

12 This Figure illustrades that both derivatives exist on (0, ) with g'(x) < L and G'(x) < K for all x = [0, ], where K < and L <. Considering the last two Figures, we establish that both iteration functions are compatible with BANACH 's fixed-point theorem. The iterations generated by g(x) and G(x) are given as follows: x[0]:=0.5; x[]:=evalf(subs(x=0.5,g(x))); x 0 := 0.5 x := Fixed Point Iteration: for i from 2 to 0 do x[i]:=evalf(subs(x=%,g(x))) od; x 2 := x 3 := x 4 := x 5 := x 6 := x 7 := x 8 := x 9 := x 0 := The 9th iteration x[9] is identical to the MAPLE solution p = based upon the command fsolve. NEWTON 's Method: X[0]:=0.5; X[]:=evalf(subs(x=0.5,G(x))); X 0 := 0.5 X := for i from 2 to 6 do X[i]:=evalf(subs(x=%,G(x))) od; 2

13 X 2 := X 3 := X 4 := X 5 := X 6 := The 5th iteration X[5] is already identical to the MAPLE solution. In contrast, the fixed-point method needs 9 iterations. The next example is concerned with the zero form f(x) = x - exp(x^2-2) = 0. restart: f(x):=x-exp(x^2-2); f( x ) := x e ( x2 2) p:=fsolve(f(x)=0,x); # fixed-point by MAPLE p := `f '`(x):=diff(f(x),x); f '( x ):= 2 x e ( x2 2 ) `f ''`(x):=diff(f(x),x$2); f ''( x ) := 2 e ( x2 2) 4 x 2 e ( x2 2 ) alias(h=heaviside,sc=scaling,th=thickness,co=color): p[]:=plot({f(x),diff(f(x),x),diff(f(x),x$2)}, x=0..0.5,-..,th=3,co=black): p[2]:=plot({h(x-0.5),-h(x-0.5)},x= ,co=black): p[3]:=plot(0.96*h(x-0.4),x= ,linestyle=4,co=black): p[4]:=plot({-,},x=0..0.5,co=black, title="f(x), f'(x), f''(x)"): p[5]:=plot([[0.4,0.96]],style=point,symbol=circle, symbolsize=30,ytickmarks=4,co=black): p[6]:=plots[textplot]({[0.3,0.3,`f(x)`],[0.3,0.8,`f'(x)`], [0.3,-0.5,`f''(x)`]},co=black): plots[display](seq(p[k],k=..6)); 3

14 In this Figure we see that the first derivative f'(x) is not equal to zero in the entire range considered, which is a necessary condition for convergence of the NEWTON method. The iteration functions are given by g(x):=x-f(x); G(x):=x-f(x)/diff(f(x),x); g( x ) := e ( x2 2 ) G( x ) := x x e ( x2 2) 2 x e ( x2 2) p[]:=plot({x,g(x),g(x)},x=0..0.5, sc=constrained,th=3,co=black): p[2]:=plot({0.5,0.5*h(x-0.5)},x= ,co=black, title="iteration Functions g(x) and G(x)"): p[3]:=plot(0.4*h(x-0.4),x= , linestyle=4,co=black): p[4]:=plot([[0.4,0.4]],style=point,symbol=circle, symbolsize=30,co=black): p[5]:=plots[textplot]({[0.3,0.8,`g(x)`], [0.3,0.,`G(x)`]},co=black): plots[display](seq(p[k],k=..5)); 4

15 Both operators, g and G, mapp the interval x = [0, 0.5] to itself. The iteration function G(x) has a horizontal tangent in the fixed-point p = because of G'(p) = 0, id est quadratic convergence. In contrast, the iteration function g(x) has a horizontal tangent in x = 0. Now let's discuss the absolute derivatives of the iteration functions. abs(`g'`(x))=abs(diff(g(x),x)); g' ( x ) = 2 e ( 2 + R( x2) ) abs(`g'`(x))=abs(diff(g(x),x)); x G' ( x ) = ( x e ( x2 2) )( 2 e ( x2 2) 4 x 2 e ( x2 2 ) ) ( 2 x e ( x2 2) ) p[]:=plot({abs(diff(g(x),x)),abs(diff(g(x),x))}, x=0..0.5,th=3,co=black, title="absolute Derivatives g'(x) and G'(x) "): p[2]:=plot({0.25,0.25*h(x-0.5)},x= ,co=black): p[3]:=plots[textplot]({[0.25,0.,` g'(x) `], [0.33,0.05,` G'(x) `]},co=black): plots[display](seq(p[k],k=..3)); 2 5

16 This Figure illustrades that both derivatives exist on (0, 0.5) with g'(x) < L and G'(x) < K for all x = [0, 0.5], where K < and L <. Considering the last two Figures, we find that both iteration functions are compatible with BANACH's fixed-point theorm. The iterations generated by g(x) and G(x) are listed in the following: x[0]:=0.; x[]:=evalf(subs(x=0.,g(x))); x 0 := 0. x := Fixed Point Iteration: for i from 2 to 8 do x[i]:=evalf(subs(x=%,g(x))) od; x 2 := x 3 := x 4 := x 5 := x 6 := x 7 := x 8 := The 7th iteration x[7] is identical to the MAPLE solution p = based upon the command fsolve. NEWTON's Method: X[0]:=0.; X[]:=evalf(subs(x=0.,G(x))); X 0 := 0. X := for i from 2 to 5 do X[i]:=evalf(subs(x=%,G(x))) od; X 2 := X 3 :=

17 X 4 := X 5 := The 4th iteration X[4] is already identical to the MAPLE solution. In contrast, the fixed-point method needs 7 iterations. Another example is concerned with the root-finding problem f(x) = 0, where restart: f(x):=+cosh(x)*cos(x); f( x ):= + cosh( x ) cos( x ) p:=fsolve(f(x)=0,x); # fixed-point by MAPLE p := `f '`(x):=diff(f(x),x); f '( x ):= sinh( x ) cos( x ) cosh( x ) sin( x ) `f ''`(x):=diff(f(x),x$2); f ''( x ) := 2 sinh( x ) sin( x ) alias(h=heaviside,sc=scaling,th=thickness,co=color): p[]:=plot({f(x),diff(f(x),x),diff(f(x),x$2)}, x=.5..2,-7...5,th=3,co=black): p[2]:=plot({-7,-7*h(x-2),.5,.5*h(x-2)},x= ,co=black, title="f(x), f'(x), f''(x), Fixed-Point p = "): p[3]:=plot(-4.7*h(x-.88),x= ,linestyle=4,co=black): p[4]:=plot([[.88,-4.7]],style=point,symbol=circle, symbolsize=30,co=black): p[5]:=plots[textplot]({[2.03,-0.57,`f(x)`],[2.03,-5,`f'(x)`], [2.03,-6.6,`f''(x)`]},co=black): plots[display](seq(p[k],k=..5)); In this Figure we see that the first derivative f '(x) is not equal to zero in the vicinity of the fixed-point, which is a necessary condition for the application of NEWTON 's method. Its iteration function is given by G(xi):=xi-f(xi)/diff(f(xi),xi); G(x):=x-f(x)/diff(f(x),x); 7

18 G( ξ ):= ξ d dξ f( ξ ) + cosh( x ) cos( x ) G( x ) := x sinh( x ) cos( x ) cosh( x ) sin( x ) x[n+]:=g(x[n]); x n + := G( x n ) x[0]:=2; x[]:=evalf(subs(x=2,g(x))); x 0 := 2 x := for i from 2 to 5 do x[i]:=evalf(subs(x=%,g(x))) od; x 2 := x 3 := x 4 := x 5 := The 4th iteration x[4], beginning with a starting point x[0] = 2, is already identical to the MAPLE solution. Selecting the starting point x[0] =.2, then the 6th iteration x[6] is identical to the fixed point. However, the starting point x[0] = does not lead to convergence. Improving the convergence or obtaining convergence, necessary in cases of small derivatives f '(x) <<, one can extend the classical NEWTON method in the following way: restart: X[n+]:=G(X[n]); G(X):=X-lambda*h(X)*f(X); X n + := G( X n ) G( X ):= X λ h( X ) f( X) `G'(X)`:=diff(G(X),X); G'(X) := λ d dx h( X ) f( X) λ h( X) d dx f( X) lambda[lagrange]:=solve(diff(g(x),x)=0,lambda); λ LAGRANGE := d + dx h( X ) f( X ) h( X) d dx f( X) G(X):=subs(lambda=%,G(X)); G( X ) := X Assuming h(x) = -exp(-x), we arrive at h(x):=-exp(-x); f( ξ ) h( X ) f( X) d + dx h( X ) f( X ) h( X) d dx f( X) h( x ) := e ( x ) 8

19 G(x):=subs({X=x,h(X)=h(x)},G(X)); e ( x ) f( x ) G( x ) := x + d dx ( x ) e( ) f( x ) e ( x ) d dx f( x ) G(xi):=xi-f(xi)/(`f '`(xi)-f(xi)); G( ξ ) := ξ f( ξ ) f '( ξ ) f( ξ ) f(x):=+cosh(x)*cos(x); f( x ):= + cosh( x ) cos( x ) G(x):=x-f(x)/(diff(f(x),x)-f(x)); G( x ) := x + cosh( x ) cos( x ) sinh( x ) cos( x ) cosh( x ) sin( x ) cosh( x ) cos( x ) x[0]:=2; x[]:=evalf(subs(x=2,g(x))); x 0 := 2 x := for i from 2 to 5 do x[i]:=evalf(subs(x=%,g(x))) od; x 2 := x 3 := x 4 := x 5 := We see, the iteration has been improved from 4 to 3 iterations. With a starting point x[0] =, the 4th iteration leads to the fixed-point instead of divergence by using the classical NEWTON method. The fixed-point iteration x[n+]:=g(x[n])=x[n]-h(x[n])*f(x[n]); x n + := g( x n ) = x n h( x n ) f( x n ) does not fulfill BANACH 's theorem for h(x) =. A compatible function is given by the function h(x) = -exp(-x) introduced before. Thus, we arrive at the following iteration function: g(x):=x+exp(-x)*f(x); g( x ) := x + e ( x ) ( + cosh( x ) cos( x ) ) x[0]:=2; x[]:=evalf(subs(x=%,g(x))); x 0 := 2 x := for i from 2 to 22 do x[i]:=evalf(subs(x=%,g(x))) od; x 2 := x 3 := x 4 :=

20 x 5 := x 6 := x 7 := x 8 := x 9 := x 0 := x := x 2 := x 3 := x 4 := x 5 := x 6 := x 7 := x 8 := x 9 := x 20 := x 2 := x 22 := The 20th iteration x[20] leads to the fixed-point. In contrast to only 4 or 3 iterations based upon NEWTON 's classical or extended method, respectively. The following two Figures should illustrade that both iteration functions, g(x) and G(x), are compatible with BANACH 's fixed-point theorem. alias(h=heaviside,sc=scaling,th=thickness,co=color): p[]:=plot({x,g(x),g(x)},x=.5..2,.5..2, sc=constrained,th=3,co=black): p[2]:=plot(2*h(x-2),x= ,co=black): p[3]:=plot(2,x=.5..2,co=black, title="g(x) in comparison with g(x)"): p[4]:=plot([[.875,.875]],style=point, symbol=circle,symbolsize=30,co=black): p[5]:=plots[textplot]({[.57,.9,`g(x)`], [.57,.74,`g(x)`]},co=black): p[6]:=plot(.875*h(x-.875),x= , linestyle=4,co=black): plots[display](seq(p[k],k=..6)); 20

21 Both operators, G and g, mapp the interval x = [.5, 2] to itself. The function G(x) has a horizontal tangent in the fixed-point p = within the interval considered. This means quadratic convergence of the extended NEWTON method. Corresponding to BANACH 's theorem the absolute derivatives g'(x) and G'(x) should be less than one as shown in the next Figure. abs(`g'`(x))=abs(diff(g(x),x)); g' ( x ) = + e ( x ) ( + cosh( x ) cos( x ) ) e ( x ) ( sinh( x ) cos( x ) cosh( x ) sin( x ) ) abs(`g'`(x))=abs(diff(g(x),x)); G' ( x ) = sinh( x ) ( ) sinh( x ) cos( x ) cosh( x ) sin( x ) cos x cosh( x ) sin( x ) cosh( x ) cos( x ) ( + cosh( x ) cos( x )) ( 2 sinh( x ) sin( x ) sinh( x ) cos( x ) + cosh( x ) sin( x )) + 2 ( sinh( x ) cos( x ) cosh( x ) sin( x ) cosh( x ) cos( x )) p[]:=plot({abs(diff(g(x),x)),abs(diff(g(x),x))}, x=.5..2,0..0.5,th=3,co=black): p[2]:=plot(0.5*h(x-2),x= ,co=black): p[3]:=plot(0.5,x=.5..2,co=black, title="absolute Derivatives g'(x) and G'(x) "): p[4]:=plot(0.3655*h(x-.875),x= , linestyle=4,co=black): p[5]:=plot([[.875,0.3655]],style=point,symbol=circle, symbolsize=30,co=black): p[6]:=plots[textplot]({[.6,0.32,` g'(x) `], [.6,0.5,` G'(x) `]},co=black): plots[display](seq(p[k],k=..6)); 2

22 The last two Figures illustrade that both iterations, extended NEWTON and fixed-point, are compatible with BANACH 's theorem. Both operators, g and G, mapp the interval x = [.5, 2] to itself. In addition, both derivatives exist on (.5, 2) with g'(x) < L and G'(x) < K for all x = [.5, 2], where K < L <. The number L is the LIPPSCHITZ constant. NEWTON 's method converges quadratically because of G'(p) = 0. Another example illustrades as before that the extended NEWTON method is most effective in cases when the first derivative f '(x[n]) is equal to zero or very small for any step x[n] if the classical NEWTON method does not work. restart: f(x):=x-2*sin(x); f( x ) := x 2 sin( x ) p:=fsolve(f(x)=0,x,..2); # fixed-point immediately found by MAPLE command "fsolve" p := `f '`(x):=diff(f(x),x); f '( x ) := 2 cos( x ) `f '`(P):=evalf(subs(x=p,%)); f '( P) := `f ''`(x):=diff(f(x),x$2); f ''( x ):= 2 sin( x ) `f ''`(P):=evalf(subs(x=p,%)); f ''( P) := alias(h=heaviside,th=thickness,sc=scaling,co=color): p[]:=plot({f(x),diff(f(x),x),diff(f(x),x$2)}, x=0..pi,-..pi,sc=constrained,th=3,co=black): p[2]:=plot({-,pi,pi*h(x-pi),-h(x-pi)}, x=0...00*pi,co=black, title="f(x), f'(x), f''(x), Fixed-Point p"): p[3]:=plot(.638*h(x-.8955),x= , linestyle=4,co=black): p[4]:=plot([[.8955,.638]],style=point, symbol=circle,symbolsize=30,co=black): 22

23 p[5]:=plots[textplot]({[2.8,.5,`f(x)`], [2.0,2.5,`f'(x)`],[0.5,.5,`f''(x)`]},co=black): plots[display](seq(p[k],k=..5)); In the vicinity of the fixed-point the first derivative f '(x) is not very small so that the classical NEWTON method can work, if the starting-point x[0] is close enough to the expected fixed-point. However, in order to test the extended NEWTON formular, the starting-point should be selected, for instance, at x = close to the zero of f '(x): X[ZERO]:=fsolve(diff(f(x)=0,x)); X ZERO := Classical NEWTON Method G(xi):=xi-f(xi)/`f '`(xi); G( ξ ) := ξ f( ξ ) f '( ξ ) G(x):=x-f(x)/diff(f(x),x); x 2 sin( x ) G( x ) := x 2 cos( x ) x[0]:=; x[]:=evalf(subs(x=,g(x))); # starting-point x 0 := x := for i from 2 to 7 do x[i]:=evalf(subs(x=%,g(x))) od; x 2 := x 3 := x 4 :=

24 x 5 := x 6 := x 7 := In the vicinity of the selected starting-point x[0] = the first derivative f '(x) is very small. Thus, the classical NEWTON method does not converge. With the same starting-point we will obtain convergence by applying the extended NEWTON formular: restart: f(x):=x-2*sin(x); f( x ) := x 2 sin( x ) p:=fsolve(f(x)=0,x,..2); p := G(xi):=xi-f(xi)/(`f '`(xi)-f(xi)); G( ξ ) := ξ G(x):=x-f(x)/(diff(f(x),x)-f(x)); G( x ) := x f( ξ ) f '( ξ ) f( ξ ) x 2 sin( x ) 2 cos( x) x + 2 sin( x ) x[0]:=; x[]:=evalf(subs(x=,g(x))); # starting-point x 0 := x := for i from 2 to 7 do x[i]:=evalf(subs(x=%,g(x))) od; x 2 := x 3 := x 4 := x 5 := x 6 := x 7 := We see, the 5th iteration x[5] leads already to convergence although the first derivative f '(x) is very small in the neighbourhood of the selected starting-point. The extended NEWTON method is most effective in cases of small derivatives f '(x). This worksheet is concerned with finding numerical solutions of non-linear equations in a single unknown. A generalization to systems of non-linear equations has been discussed in more detail, for instance, by BETTEN, J. in: Finite Elemente für Ingenieure 2, zweite Auflage, 2004, Springer-Verlag, Berlin / Heidelberg / New York. 24