Outline. Math Numerical Analysis. Intermediate Value Theorem. Lecture Notes Zeros and Roots. Joseph M. Mahaffy,

Similar documents
Math Numerical Analysis

Numerical Analysis and Computing

Math Numerical Analysis

Chapter 3: Root Finding. September 26, 2005

PART I Lecture Notes on Numerical Solution of Root Finding Problems MATH 435

ROOT FINDING REVIEW MICHELLE FENG

THE SECANT METHOD. q(x) = a 0 + a 1 x. with

Numerical Methods in Informatics

Solution of Algebric & Transcendental Equations

1. Method 1: bisection. The bisection methods starts from two points a 0 and b 0 such that

Midterm Review. Igor Yanovsky (Math 151A TA)

SOLUTION OF ALGEBRAIC AND TRANSCENDENTAL EQUATIONS BISECTION METHOD

Introductory Numerical Analysis

3.1 Introduction. Solve non-linear real equation f(x) = 0 for real root or zero x. E.g. x x 1.5 =0, tan x x =0.

CHAPTER 10 Zeros of Functions

Determining the Roots of Non-Linear Equations Part I

Unit 2: Solving Scalar Equations. Notes prepared by: Amos Ron, Yunpeng Li, Mark Cowlishaw, Steve Wright Instructor: Steve Wright

Solution of Nonlinear Equations

Numerical Methods. Root Finding

Chapter 2 Solutions of Equations of One Variable

Zeros of Functions. Chapter 10

Nonlinear Equations. Chapter The Bisection Method

Lecture Notes to Accompany. Scientific Computing An Introductory Survey. by Michael T. Heath. Chapter 5. Nonlinear Equations

Scientific Computing: An Introductory Survey

5 Finding roots of equations

MATH 350: Introduction to Computational Mathematics

15 Nonlinear Equations and Zero-Finders

Math Introduction to Numerical Methods - Winter 2011 Homework 2 Assigned: Friday, January 14, Due: Thursday, January 27,

STOP, a i+ 1 is the desired root. )f(a i) > 0. Else If f(a i+ 1. Set a i+1 = a i+ 1 and b i+1 = b Else Set a i+1 = a i and b i+1 = a i+ 1

MATH 3795 Lecture 12. Numerical Solution of Nonlinear Equations.

Chapter 4. Solution of Non-linear Equation. Module No. 1. Newton s Method to Solve Transcendental Equation

MATH 350: Introduction to Computational Mathematics

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton s Method Secant Method Conclusion. Nonlinear Systems

Root Finding: Close Methods. Bisection and False Position Dr. Marco A. Arocha Aug, 2014

CLASS NOTES Models, Algorithms and Data: Introduction to computing 2018

Bisection and False Position Dr. Marco A. Arocha Aug, 2014

Solutions of Equations in One Variable. Newton s Method

Variable. Peter W. White Fall 2018 / Numerical Analysis. Department of Mathematics Tarleton State University

Numerical Methods Dr. Sanjeev Kumar Department of Mathematics Indian Institute of Technology Roorkee Lecture No 7 Regula Falsi and Secant Methods

Solving Non-Linear Equations (Root Finding)

Lecture 7. Root finding I. 1 Introduction. 2 Graphical solution

Lecture 8. Root finding II

Root Finding Convergence Analysis

Outline. Scientific Computing: An Introductory Survey. Nonlinear Equations. Nonlinear Equations. Examples: Nonlinear Equations

CS 450 Numerical Analysis. Chapter 5: Nonlinear Equations

Chapter 1. Root Finding Methods. 1.1 Bisection method

Computational Methods CMSC/AMSC/MAPL 460. Solving nonlinear equations and zero finding. Finding zeroes of functions

Queens College, CUNY, Department of Computer Science Numerical Methods CSCI 361 / 761 Spring 2018 Instructor: Dr. Sateesh Mane.

Roots of Equations. ITCS 4133/5133: Introduction to Numerical Methods 1 Roots of Equations

Lecture 39: Root Finding via Newton s Method

Math 471. Numerical methods Root-finding algorithms for nonlinear equations

Numerical Methods I Solving Nonlinear Equations

MA 8019: Numerical Analysis I Solution of Nonlinear Equations

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton s Method Secant Method Conclusion. Nonlinear Systems

x 2 x n r n J(x + t(x x ))(x x )dt. For warming-up we start with methods for solving a single equation of one variable.

Figure 1: Graph of y = x cos(x)

Caculus 221. Possible questions for Exam II. March 19, 2002

Numerical Solution of f(x) = 0

Simple Iteration, cont d

Homework and Computer Problems for Math*2130 (W17).

Line Search Methods. Shefali Kulkarni-Thaker

CHAPTER-II ROOTS OF EQUATIONS

Intro to Scientific Computing: How long does it take to find a needle in a haystack?

Section 1.4 Tangents and Velocity

Extended Introduction to Computer Science CS1001.py. Lecture 8 part A: Finding Zeroes of Real Functions: Newton Raphson Iteration

Numerical Methods Lecture 3

Numerical Methods. King Saud University

Numerical Methods 5633

1 What is numerical analysis and scientific computing?

Halley s Method: A Cubically Converging. Method of Root Approximation

Notes for Numerical Analysis Math 5465 by S. Adjerid Virginia Polytechnic Institute and State University. (A Rough Draft)

Numerical Methods in Physics and Astrophysics

Goals for This Lecture:

1.1: The bisection method. September 2017

Iterative Methods to Solve Systems of Nonlinear Algebraic Equations

If you need the source code from the sample solutions, copy and paste from the PDF should work. If you re having trouble, send me an .

Numerical Analysis: Solving Nonlinear Equations

Lecture 44. Better and successive approximations x2, x3,, xn to the root are obtained from

CLASS NOTES Computational Methods for Engineering Applications I Spring 2015

Math 117: Calculus & Functions II

Bisection Method. and compute f (p 1 ). repeat with p 2 = a 2+b 2

Calculus I. George Voutsadakis 1. LSSU Math 151. Lake Superior State University. 1 Mathematics and Computer Science

Solving nonlinear equations

Outline. Formic Acid

Calculus for the Life Sciences

Numerical Methods in Physics and Astrophysics

Lecture 5. September 4, 2018 Math/CS 471: Introduction to Scientific Computing University of New Mexico

X. Numerical Methods

Outline. Math Numerical Analysis. Errors. Lecture Notes Linear Algebra: Part B. Joseph M. Mahaffy,

Lesson 59 Rolle s Theorem and the Mean Value Theorem

Introduction to Numerical Analysis

Jim Lambers MAT 460/560 Fall Semester Practice Final Exam

Finding the Roots of f(x) = 0. Gerald W. Recktenwald Department of Mechanical Engineering Portland State University

Finding the Roots of f(x) = 0

Infinite series, improper integrals, and Taylor series

Project on Newton-Raphson Method

Taylor and Maclaurin Series. Approximating functions using Polynomials.

1 Question related to polynomials

The Steepest Descent Algorithm for Unconstrained Optimization

A Review of Bracketing Methods for Finding Zeros of Nonlinear Functions

Transcription:

Outline Math 541 - Numerical Analysis Lecture Notes Zeros and Roots Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Department of Mathematics and Statistics Dynamical Systems Group Computational Sciences Research Center San Diego State University San Diego, CA 918-77 http://jmahaffy.sdsu.edu Spring 18 1 MatLab 3 MatLab - 4 Modifying Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (1/45) Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (/45) Intermediate Value Theorem Suppose f is continuous on the interval (a, b ) and f(a ) f(b ) < This means the function changes sign at least once in the interval The Intermediate Value Theorem guarantees the existence of c (a, b ) such that f(c) = (could be more than one) The 1 of 3 The approximates the root (f(c) = ) of a continuous function that changes sign at least once for x [a, b ] Thus, f(a ) f(b ) < Iteratively find the midpoint If f(m k ) =, we re done m k = a k + b k f(x) f(x) Check if f(m k ) f(b k ) < or f(m k ) f(a k ) < a c b x a b c x If f(m k ) f(b k ) <, then c [m k, b k ] and we take a k+1 = m k and b k+1 = b k Otherwise f(m k ) f(a k ) <, and c [a k, m k ], so we take b k+1 = m k and a k+1 = a k Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (3/45) Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (4/45)

The of 3 The for solving f(c) = from the previous slide: Constructs a sequence of intervals containing the root c: (a, b ) (a 1, b 1 ) (a n 1, b n 1 ) (a n, b n ) c After k steps b k a k = 1 b k 1 a k 1 = ( ) k 1 b a At step k, the midpoint m k = a k+b k is an estimate for the root c with ( ) k+1 1 m k d k c m k + d k, d k = b a The 3 of 3 Convergence is slow: At each step we gain one binary digit in accuracy Since 1 1 3.3, it takes on average 3.3 iterations to gain one decimal digit of accuracy Note: The rate of convergence is completely independent of the function f We are only using the sign of f at the endpoints of the interval(s) to make decisions on how to update By making more effective use of the values of f we can attain significantly faster convergence Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (5/45) Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (6/45) : 1 of 5 The bisection method applied to ( x ) f(x) = sin(x) = with (a, b ) = (1.5,.), and (f(a ), f(b )) = (.435,.97) gives: k a k b k m k f(m k ) 1.5. 1.75 84 1 1.75. 1.875.75 1.875. 1.9375.5 3 1.875 1.9375 1.96.358 4 1.96 1.9375 1.919.156 5 1.919 1.9375 1.997.54 6 1.997 1.9375 1.9336. 7 1.9336 1.9375 1.9355.4 8 1.9336 1.9355 1.9346.11 9 1.9336 1.9346 1.9341.4 Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (7/45) : of 5 This MatLab code can easily be modified to any function 1 function root = bisection(a,b,tol) %BISECTION METHOD - Modify function below, then can 3 % find its roots in [a,b] to tolerance tol 4 f = @(x) (x/).ˆ - sin(x); 5 while (abs(b-a) tol) 6 m = (a+b)/; 7 if (f(m) == ) 8 break; 9 elseif(f(b)*f(m) < ) 1 a = m; 11 else 1 b = m; 13 end 14 end 15 root = m; Demonstrate in class Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (8/45)

: 3 of 5 : 4 of 5 This MatLab code ( Slides) graphically shows example 1 % WARNING: This example ASSUMES that f(a)<<f(b)... x = 1.5:.1:; 3 f = inline('(x/).ˆ-sin(x)','x'); 4 a = 1.5; 5 b =.; 6 for k = :9 7 plot(x,f(x),'k-','linewidth',) 8 axis([1.45.5 -.5.15]) 9 grid on 1 hold on 11 plot([a b],f([a b]),'ko','linewidth',5) 1 plot([1.45.5],[ ],'r:') 13 hold off 14 m = (a+b)/; 15 if( f(m) < ) 16 a = m; 17 else 18 b = m; 19 end pause 1 print('-depsc',['bisec' intstr(k) '.eps'],'-f1'); end The next Slides show the output Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (9/45) Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (1/45) : 5 of 5 Stopping Criteria - - - When do we stop? We can (1) keep going until successive iterates are close: -. -.3 -. -.3 -. -.3 m k m k 1 < ɛ -.4 -.5 1.5 1.6 1.7 1.8 1.9 -.4 -.5 1.5 1.6 1.7 1.8 1.9 -.4 -.5 1.5 1.6 1.7 1.8 1.9 or () close in relative terms - -. - -. - -. m k m k 1 m k < ɛ -.3 -.4 -.5 1.5 1.6 1.7 1.8 1.9 -.3 -.4 -.5 1.5 1.6 1.7 1.8 1.9 -.3 -.4 -.5 1.5 1.6 1.7 1.8 1.9 or (3) the function value is small enough f(m k ) < ɛ - - - -. -. -. -.3 -.3 -.3 -.4 -.4 -.4 -.5 -.5 -.5 1.5 1.6 1.7 1.8 1.9 1.5 1.6 1.7 1.8 1.9 1.5 1.6 1.7 1.8 1.9 Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (11/45) No choice is perfect. In general, where no additional information about f is known, the second criterion is the preferred one (since it comes the closest to testing the relative error). Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (1/45)

Cauchy Sequence Suppose an algorithm generates a sequence of approximations, c n, which approaches a limit, c, or How quickly does c n c? Definition () lim n c n = c If a sequence c 1, c,..., c n converges to a value c and if there exist real numbers λ > and α 1 such that c n+1 c lim n c n c α = λ Definition (Cauchy Sequence) Consider a sequence c 1, c,..., c n of real numbers. This sequence is called a Cauchy sequence, if for every ε >, there is a positive integer N such that for all natural numbers m, n > N, x m x n < ε. From the properties of real numbers (completeness), a Cauchy sequence, {c n } converges to a unique real number c. then we say that α is the rate of convergence of the sequence. Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (13/45) Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (14/45) - Cauchy A Numerical algorithm produces a sequence of approximations, {c n }, which is hopefully converging to a limit, c, which is NOT known. How can the sequence {c n } be used to find the rate of convergence? Definition (Rate of Cauchy Convergence) If a sequence c 1, c,..., c n converges and if there exist real numbers λ > and α 1 such that lim n c n+1 c n c n c n 1 α = λ then we say that α is the rate of convergence of the sequence. Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (15/45) Numerical Suppose a Numerical algorithm produces a sequence of approximations, {c n }. The rate of Cauchy convergence, α, for the sequence c 1, c,..., c n is derived from the existence of real numbers λ > and α 1 with lim n c n+1 c n c n c n 1 α = λ. By taking logarithms of the expression above, we have ln c n+1 c n = α ln c n c n 1 + ln(λ). Let Y n = ln c n+1 c n and X n = ln c n c n 1, which are readily computed from the sequence, then the rate of Cauchy convergence, α, is approximated by the slope of the best fitting line through (X n, Y n ). Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (16/45)

- MatLab Let c be a root of f, so f(c ) = Let m n = a n+b n be the midpoint between the endpoints of the interval [a n, b n ], which comes from n iterations of the Bisection Method and where c [a n, b n ] Earlier we showed that at most m n c 1 n+1 b a Form the sequence of midpoints with c n = m n, then from the worst case scenario c n c 1 n+1 b a or c n+1 c c n c 1 It follows that for α = 1, so the rate of convergence is linear Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (17/45) Start at x, then follow tangent lines of f(x) to their zeroes. Iterate these zeroes converging to {x n } n= x with f(x ) =. (x 1,f(x 1 )) x x (x,f(x )) x 1 x Tangent Line f(x) Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (18/45) MatLab MatLab for Root Finding 1 of Recall: we are looking for x so that f(x ) =. The graphic from previous slide seems to show rapid convergence to the zero of f(x) The graph shows the use of properties of f(x) Does this sequence always converge to x? Assuming convergence, how rapidly does this sequence converge? The Method employs techniques from Calculus Technique is called What are properties of? If f C [a, b], and we know x [a, b] (possibly by the intermediate value theorem), then we can formally Taylor expand around a point x close to the root: = f(x ) = f(x)+(x x)f (x)+ (x x ) f (ξ(x)), ξ(x) [x, x ]. If we are close to the root, then x x is small, which means that x x x x, hence we make the approximation: f(x) + (x x)f (x), x x f(x) f (x). Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (19/45) Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (/45)

MatLab MatLab for Root Finding of Two Steps of Newton for f(x) = x 3 + 4x 1 = for root finding is based on the approximation x x f(x) f (x) which is valid when x is close to x. 6 4 6 4 is an iterative scheme, where given an x n 1, we compute x n = x n 1 f(x n 1) f (x n 1 ) If x is sufficiently close to a root, x, of f(x), then iterations x n give improved approximations of x, as n 4 6.9 1 1.1 1. 1.3 1.4 1.5 1.6 Start with p = 1 p 1 = p p3 + 4p 1 3p + 8p p = p 1 p3 1 + 4p 1 1 3p 1 + 8p 1 4 6.9 1 1.1 1. 1.3 1.4 1.5 1.6 = 1.45454545454545 = 1.368941695 Geometrically, x n is the intersection of the tangent of the function at x n 1 and the x-axis. p = 1.365313 From MAPLE Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (1/45) Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (/45) MatLab Newton Code MatLab Finding a Starting Point for MatLab 1 function p = newton(p,tol,nmax) %NEWTON'S METHOD: Enter f(x), f'(x), x, tol, Nmax 3 f = @(x) xˆ3 + 4*xˆ - 1; 4 fp = @(x) 3*xˆ + 8*x; 5 p = p - f(p)/fp(p); 6 i = 1; 7 while (abs(p - p) tol) 8 p = p; 9 p = p - f(p)/fp(p); 1 i = i + 1; 11 if (i Nmax) 1 fprintf('fail after %d iterations\n',nmax); 13 break 14 end 15 end 16 end Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (3/45) Recall our initial argument that when x x is small, then x x x x, and we can neglect the second order term in the Taylor expansion. In order for Newton s method to converge we need a good starting point! Theorem Let f(x) C [a, b]. If x [a, b] such that f(x ) = and f (x ), then there exists a δ > such that Newton s method generates a sequence {x n } n=1 converging to x for any initial approximation x [x δ, x + δ]. The theorem is interesting, but quite useless for practical purposes. In practice: Pick a starting value x, iterate a few steps. Either the iterates converge quickly to the root, or it will be clear that convergence is unlikely. Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (4/45)

MatLab MatLab 1 of 3 Suppose x is a root of f(x) and consider the Taylor expansion of f(x ) about x n = f(x ) = f(x n ) + f (x n )(x x n ) + f (ξ n ) (x x n ), where ξ n (x, x n ) Dividing by f (x n ) gives but x n+1 = x n f(x n) f (x n ), so f(x n ) f (x n ) + (x x n ) = f (ξ n ) f (x n ) (x x n ) (x x n+1 ) = f (ξ n ) f (x n ) (x x n ) of 3 Taking absolute values x x n+1 x x n = f (ξ n ) f (x n ), so by our definition for rate of convergence, has quadratic convergence provided 1 f (x), for all x I, where I = [x r, x + r] for some r x x f (x) is continuous for all x I 3 x is sufficiently close to x Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (5/45) Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (6/45) MatLab 3 of 3 MatLab MatLab Newton 1 Sufficiently close means If We can ignore higher order terms of the Taylor expansion f (x n ) f (x n ) < C f (x ) f (x ) for some C < C f (x ) f (x ) x x n < 1 for all n M = sup x I f (x) f (x), we have convergence for an initial point x provided M x x < 1 1 function z = newtoneger(p,tol,nmax) %NEWTON'S METHOD: Enter f(x), f'(x), x, tol, Nmax 3 f = @(x) xˆ3 + 4*xˆ - 1; 4 fp = @(x) 3*xˆ + 8*x; 5 p = p - f(p)/fp(p); 6 z = [p]; i = 1; 7 while (abs(p - p) tol) 8 p = p; 9 p = p - f(p)/fp(p); 1 z = [z,p]; 11 i = i + 1; 1 if (i Nmax) 13 fprintf('fail after %d iterations\n',nmax); 14 break 15 end 16 end 17 end Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (7/45) Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (8/45)

MatLab MatLab Newton The previous code generates Newton iterates to solve with the first five iterates being: f(x) = x 3 + 4x 1 = z = [1.454545454545455, 1.36894169519, 1.365366116, 1.365313435367, 1.36531341497] Quadratic convergence suggests examining: Q n = z n+1 z n z n z n 1. Substituting the sequence {z n } into the fraction above gives: Q =.49949, Q 3 =.4969, Q 4 =.495. MatLab MatLab Newton 3 1 xlab = '$\ln z {n}- z {n-1} $'; % X-label ylab = '$\ln z {n+1}-z {n } $'; % Y-label 3 mytitle = ''; % Title 4 z = newtoneger(1,1e-7,); 5 %z = secanteger(1,,1e-9,); 6 N = length(z); 7 xx = log(abs(z(:n-1)-z(1:n-))); 8 yy = log(abs(z(3:n)-z(:n-1))); 9 p = polyfit(xx,yy,1) 1 x1 = min(xx); x = max(xx); 11 y1 = p(1)*x1 + p(); 1 y = p(1)*x + p(); 13 plot(xx,yy,'bo','markersize',7); 14 hold on 15 plot([x1,x],[y1,y],'r-','linewidth',1.5); 16 grid Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (9/45) Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (3/45) MatLab MatLab Newton 4 The previous program plots Y n = ln z n+1 z n vs. X n = ln z n z n 1 and finds the slope, which is the rate of convergence. -5-1 -15 - -5-1 -1-8 -6-4 - The program gives the best fitting line: Y n =.17 X n.69493. : MatLab solves f(x) = very efficiently Converges quadratically to the solution Roughly doubles the digits with each iteration when close Simple algorithm: Zero crossing of tangent line Problems: x n+1 = x n f(x n) f (x n ) Difficult to determine the range of initial conditions for which Newton s method converges Algorithm often fails to converge Problems if f (x) = Computing the derivative can be expensive If the zero of f(x) isn t simple, then convergence is linear Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (31/45) Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (3/45)

MatLab - MatLab - Main weakness of is computing the derivative Computing the derivative can be difficult Derivative often needs many more arithmetic operations One solution is to Approximate the Derivative By definition, f (x n 1 ) = f(x) f(x n 1 ) lim x x n 1 x x n 1 Take x = x n, then an approximation is f (x n 1 ) f(x n ) f(x n 1 ) x n x n 1 The approximation f (x n 1 ) f(x n ) f(x n 1 ) x n x n 1 is inserted into Newton s method to give The is an iterative scheme, where given an x n and x n 1, we compute x n = x n 1 f(x n 1)(x n x n 1 ) f(x n ) f(x n 1 ) Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (33/45) Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (34/45) MatLab Secant Code MatLab - Convergence MatLab - 1 function xn = secant(x,x1,tol,nmax) %SECANT METHOD: Enter f(x), x, x1, tol, Nmax 3 f = @(x) xˆ3 + 4*xˆ - 1; 4 xn = x1 - f(x1)*(x1-x)/(f(x1)-f(x)); 5 i = 1; 6 while (abs(xn - x1) tol) 7 x = x1; 8 x1 = xn; 9 xn = x1 - f(x1)*(x1-x)/(f(x1)-f(x)); 1 i = i + 1; 11 if (i Nmax) 1 fprintf('fail after %d iterations\n',nmax); 13 break 14 end 15 end 16 end The Secant method satisfies: f (x n 1 ) f(x n ) f(x n 1 ) x n x n 1 Algorithm requires two initial starting points New iterate is the zero crossing of the secant line Do NOT need a derivative, only function evaluations Order of Convergence is superlinear Order of Convergence has been shown to be the golden ratio, φ = 1+ 5 1.6 Faster than Bisection method, but slower than Newton s method Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (35/45) Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (36/45)

MatLab - MatLab Secant 1 1 function z = secanteger(x,x1,tol,nmax) %SECANT METHOD: Enter f(x), x, x1, tol, Nmax 3 f = @(x) xˆ3 + 4*xˆ - 1; 4 xn = x1 - f(x1)*(x1-x)/(f(x1)-f(x)); 5 z=[xn]; i = 1; 6 while (abs(xn - x1) tol) 7 x = x1; 8 x1 = xn; 9 xn = x1 - f(x1)*(x1-x)/(f(x1)-f(x)); 1 z=[z,xn]; 11 i = i + 1; 1 if (i Nmax) 13 fprintf('fail after %d iterations\n',nmax); 14 break 15 end 16 end 17 end Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (37/45) MatLab - MatLab Secant The previous code generates Secant iterates to solve with the first seven iterates being: f(x) = x 3 + 4x 1 = z = [1.6315789473684, 1.3388783887839, 1.366616394719345, 1.365119631857, 1.3653111859, 1.3653134146 1.36531341497] Superlinear convergence with α = 1+ 5 suggests examining: S n = z n+1 z n z n z n 1 α. Substituting the sequence {z n } into the fraction above gives: S = 1.815, S 3 =.468, S 4 =.7465, S 5 =.5799, S 6 =.6871, which is roughly constant. Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (38/45) MatLab - MatLab Secant 3 Root Finding Methods Modifying The program with the Secant method on Slide 3 plots Y n = ln z n+1 z n vs. X n = ln z n z n 1 and finds the slope, which is the rate of convergence. -5-1 -15 - -5-3 -35 - -18-16 -14-1 -1-8 -6-4 - The program gives the best fitting line: Y n = 1.6444 X n.574. Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (39/45) The Bisection method Very stable Algorithm - Good technique to find starting point for Newton s method Costs only one function evaluation, so rapid iterations Linear convergence, so slow (3.3 iterations/digit) The Secant method Hard to find starting points (Unknown basin of attraction) Costs only two function evaluations, so rapid iterations Superlinear convergence, α 1.6, which is pretty fast The Newton s method Hard to find starting points (Unknown basin of attraction) Finding and evaluating derivative requires more machine work at each iteration Quadratic convergence is very fast doubling the digits at each iteration Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (4/45)

Modifying Modifying Modifying Return to : f(x) = x 3 + 4x 1 We know the root is between x = 1 and x = 1.5. (Use for Bisection and Secant methods) n Bisection Secant Newton 1 1.5 1.338983584745 1.45454545454545 1.375 1.3635684991687 1.3689416951 3 1.315 1.365516874565 1.36536611 4 1.34375 1.365999568865 1.36531343536 5 1.359375 1.36531341391 1.3653134149 6 1.3671875 1.3653134149 7 1.363815 8 1.36534375 9 1.36457815 1 1.3647469375 11 1.3649934375 1 1.36511346875 The local nature of Newton s method means that we are stuck with problems of finding the basin of attraction for the root, p, where f(p ) = The Bisection method is a stable routine often used to narrow the search of p, so that Newton s method converges Another major problem is that Newton s method breaks when f (p ) = (division by zero). The good news is that this problem can be fixed! We need a short discussion on the multiplicity of zeroes. Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (41/45) Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (4/45) Modifying Multiplicity of Zeroes 1 of Modifying Multiplicity of Zeroes of Definition (Multiplicity of a Root) A solution p of f(x) = is a zero of multiplicity m of f if for x p we can write f(x) = (x p ) m q(x), lim q(x) x p Basically, q(x) is the part of f(x) which does not contribute to the zero of f(x) If m = 1 then we say that f(x) has a simple zero. Theorem f C 1 [a, b] has a simple zero at p in (a, b) if and only if f(p ) =, but f (p ). Theorem (Multiplicity and Derivatives) The function f C m [a, b] has a zero of multiplicity m at p in (a, b) if and only if = f(p ) = f (p ) = f (m 1) (p ), but f (m) (p ). We know that Newton s method runs into trouble when we have a zero of multiplicity higher than 1 Newton s method only converges linearly in these cases Halley s method is a modification that converges cubically, in general, and quadratically for higher order roots Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (43/45) Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (44/45)

Modifying Halley s Method for Zeroes of Higher Multiplicity Halley s Method (for Zeroes of Multiplicity ) x n+1 = x n f(x n )f (x n ) [f (x n )] f(x n )f (x n ) Drawbacks: We have to compute f (x) more expensive and possibly another source of numerical and/or measurement errors. We have to compute a more complicated expression in each iteration more expensive. Roundoff errors in the denominator both f (x) and f(x) approach zero, so we are computing the difference between two small numbers; a serious cancellation risk. Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Lecture Notes Zeros and Roots (45/45)