Roots of Equations. ITCS 4133/5133: Introduction to Numerical Methods 1 Roots of Equations

Transcription

1 Roots of Equations Direct Search, Bisection Methods Regula Falsi, Secant Methods Newton-Raphson Method Zeros of Polynomials (Horner s, Muller s methods) EigenValue Analysis ITCS 4133/5133: Introduction to Numerical Methods 1 Roots of Equations

2 Motivation Solution to many scientific and engineering problems Equations may be non-linear Equations expressible in the form F (x) = 0 Values of x that satisfy the above equation are termed roots Example:Quadratic Equation ax 2 + bx + c = 0 with roots x 1 x 2 = b + b 2 4ac 2a = b b 2 4ac 2a ITCS 4133/5133: Introduction to Numerical Methods 2 Roots of Equations

3 Example Functions ITCS 4133/5133: Introduction to Numerical Methods 3 Roots of Equations

4 Direct Search Method Trial and Error Approach: Evaluate f(x) over some range of x at multiple points Specify a range within which the root is assumed to occur Subdivide range into smaller, uniformly spaced intervals Search through all subintervals to locate the root. ITCS 4133/5133: Introduction to Numerical Methods 4 Roots of Equations

5 Direct Search:Searhing an interval Evaluate at end points of interval, and check if f(x) has opposite polarity. ITCS 4133/5133: Introduction to Numerical Methods 5 Roots of Equations

6 Direct Search:Disadvantages Computationally expensive (linear search) Intervals have to be very small, for high precision If intervals are too large, roots can be missed. Multiple roots cannot be handled: f(x) = x 2 2x + 1 ITCS 4133/5133: Introduction to Numerical Methods 6 Roots of Equations

7 Bisection Method Assumes a single root within the interval of interest. Similar to binary search: evaluates function at middle of interval and moves to interval containing the root. Requires fewer calculations than direct search method. ITCS 4133/5133: Introduction to Numerical Methods 7 Roots of Equations

8 Bisection Method:Algorithm ITCS 4133/5133: Introduction to Numerical Methods 8 Roots of Equations

9 Bisection Method:Example ITCS 4133/5133: Introduction to Numerical Methods 9 Roots of Equations

10 Error Analysis Error measures can be an absolute value, ɛ d = x m,i+1 x m,i or, a percent error ɛ r = x m,i+1 x m,i x m,i True Accuracy is known only if true solution (root x t ) is known ɛ r = x t x m,i x t 100 ITCS 4133/5133: Introduction to Numerical Methods 10 Roots of Equations

11 Regula Falsi,Secant Methods Bisection makes no use of shape of function to determine root. Use a straightline approximation to f(x)(in contrast to the midpoint) to find the approximate root location. Start with 2 points, (a, f(a)), (b, f(b)) that satisfies (f(a).f(b) < 0. ITCS 4133/5133: Introduction to Numerical Methods 11 Roots of Equations

12 Regula Falsi Method Next approximation: where the straightline approximation crosses the x-axis: f(b) f(b) f(a) = (b s) (b a) b a s = b f(b) f(a) f(b) ITCS 4133/5133: Introduction to Numerical Methods 12 Roots of Equations

13 Regular Falsi: Algorithm ITCS 4133/5133: Introduction to Numerical Methods 13 Roots of Equations

14 Regular Falsi: Example(Cube Root of 2) ITCS 4133/5133: Introduction to Numerical Methods 14 Roots of Equations

15 Secant Method Similar to Regula Falsi method, with a slight modification. New estimate of x i+1 of the root, based on f(x i ) and f(x i 1 ) Especially useful when its difficult to obtain derivatives analytically. and x 2 = x 1 x 1 x 0 y 1 y 0 y 1 x i+1 = x i x i x i 1 y i y i 1 y i ITCS 4133/5133: Introduction to Numerical Methods 15 Roots of Equations

16 Secant Method (contd) Solving for x i+1 gives f(x i 1 ) = f(x i) x i+1 x i 1 x i+1 x i x i+1 = x i f(x i)[x i 1 x i ] f(x i 1 f(x i ) = x i f(x i) f(x i 1 f(x i ) x i 1 x i = x i f(x i) f (x i ) which is similar in form to the Newton s method. Secant method doesnt require testing for interval selection Generally converges faster, but not guaranteed. ITCS 4133/5133: Introduction to Numerical Methods 16 Roots of Equations

17 Secant Method:Algorithm ITCS 4133/5133: Introduction to Numerical Methods 17 Roots of Equations

18 Secant Method:Example(Square Root of 2) ITCS 4133/5133: Introduction to Numerical Methods 18 Roots of Equations

19 Newton-Raphson Method Motivation: Rate of convergence of bisection method is slow. Newton s iteration method uses the linear portion of the Taylor series where x = x 1 x 0, df dx when x 1 is the root of the function. f(x 1 ) = f(x 0 ) + df dx x is the derivative w.r.t x. f(x 1 ) = 0 = f(x 0 ) + df dx (x 1 x 0 ) x 1 = x 0 f(x 0) df/dx = x 0 f(x 0) f (x 0 ) (derivative f evaluated at x 0 ) ITCS 4133/5133: Introduction to Numerical Methods 19 Roots of Equations

20 Newton-Raphson Method (contd) Iteratively, Accuracy x i+1 = x i f(x i) f (x i ) If f(x) is linear, first iteration produces exact solution If f(x) is non-linear, accuracy depends on importance of the truncated non-linear terms of the Taylor series Graphical Interpretation Approximate f (x i ) over [x i, x i+1 ] as f (x i ) = f(x i 0) x i+1 x i from which the iteration is derived. ITCS 4133/5133: Introduction to Numerical Methods 20 Roots of Equations

21 Newton-Raphson Method: Algorithm ITCS 4133/5133: Introduction to Numerical Methods 21 Roots of Equations

22 Newton-Raphson Method: Example ITCS 4133/5133: Introduction to Numerical Methods 22 Roots of Equations

23 Newton-Raphson Method:Non-Convergence f (x) approaches zero, exception (division by zero). Solution oscillates between two different solutions f(x i )/f (x i ) = f(x i+1 /f (x i+1 ) ITCS 4133/5133: Introduction to Numerical Methods 23 Roots of Equations

24 Solving Non-Linear Equations Addresses root finding to two or more variables Approach: Solve for the variables and a Jacobi style iteration technique Example Solve for x and y, x 3 3x 2 + xy = 0 4x 2 4xy 2 + 3y 2 = 0 x = (3x 2 xy) 1/3 y = 4x2 + 3y 2 4x Use initial estimates for x and y to begin the iteration using most recent values of x and y. ITCS 4133/5133: Introduction to Numerical Methods 24 Roots of Equations 1/2

25 Muller s Method Extension of the Secant Method; uses a quadratic approximation. Starts with 3 initial approximations x 0, x 1, x 2 and consider the next approximation x 4 as the intersection of the parabola connecting (x 0, f(x 0 )), (x 1, f(x 1 )), (x 2, f(x 2 )) with the x-axis In general, less sensitive to starting values than Newton s method. ITCS 4133/5133: Introduction to Numerical Methods 25 Roots of Equations

26 Muller s Method(contd) Start with P (x) = a(x x 2 ) 2 + b(x x 2 ) + c As the quadratic passes through (x 0, f(x 0 )), (x 1, f(x 1 ) and (x 2, f(x 2 ) f(x 0 ) = a(x 0 x 2 ) 2 + b(x 0 x 2 ) + c f(x 1 ) = a(x 1 x 2 ) 2 + b(x 1 x 2 ) + c f(x 2 ) = a b.0 + c = c We can solve for a, b, c a = (x 1 x 2 )[f(x 0 ) f(x 2 )] (x 0 x 2 )[f(x 1 ) f(x 2 )] (x 0 x 2 )(x 1 x 2 )(x 0 x 1 ) b = (x 0 x 2 ) 2 [f(x 1 ) f(x 2 )] (x 1 x 2 ) 2 [f(x 0 ) f(x 2 )] (x 0 x 2 )(x 1 x 2 )(x 0 x 1 ) c = f(x 2 ) ITCS 4133/5133: Introduction to Numerical Methods 26 Roots of Equations

27 Muller s Method (contd.) We can solve the quadratic P (x) for the next estimate, x 3. resulting in P (x) = P (x 3 x 2 ) = a(x 3 x 2 ) 2 + b(x 3 x 2 ) + c = 0 x 3 x 2 = b ± b 2 4ac 2a The above form can cause roundoff errors, when b 2 use x 3 x 2 = 2c b ± b 2 4ac 4ac. Instead Muller s method chooses the root, that agrees with the sign of b to make the denominator large, and hence closest root to x 2. ITCS 4133/5133: Introduction to Numerical Methods 27 Roots of Equations

28 Eigenvalue Analysis Eigen values, denoted by λ, are values for the matrix system having a non-zero solution vector X. A and I are n n I is the identity matrix. [A λi]x = 0 Applications: Analysis of sediment deposits in river beds, analyzing flutter in airplane wings. Solve A λi = 0 resulting in an nth order polynomial, to be solved for the λs. ITCS 4133/5133: Introduction to Numerical Methods 28 Roots of Equations

29 Eigenvalue Analysis(contd.) For a 3 3 system, A λi = a 11 λ a 12 a 13 a 21 a 22 λ a 23 a 31 a 32 a 33 λ = 0 resulting in λ 3 + b 2 λ 2 + b 1 λ + b 0 = 0 ITCS 4133/5133: Introduction to Numerical Methods 29 Roots of Equations

30 Analyis: Root Finding Methods Need to understand errors Need to understancd convergence rates Convergence Rate If e n+1 approaches K e n p as =, then the method is of order p Typically convergence rates are valid only close to the root. Can study convergence rates experimentally or theoretically. ITCS 4133/5133: Introduction to Numerical Methods 30 Roots of Equations

31 Convergence Rate: Fixed Point Iteration x n+1 = g(x n ) If x = r is a solution of f(x) = 0, then f(r) = 0, r = g(r). Thus, x n+1 r = g(x n ) g(r) = g(x n) g(r) (x n r) (x n r) We can use the mean value theorm (assuming g(x), g (x) are continuous) x n+1 r = g (ξ n ) (x n r) where ξ n (x n, r). Similarly, the error can be defined as e i+1 = g (ξ n ) e i ITCS 4133/5133: Introduction to Numerical Methods 31 Roots of Equations

32 Convergence Rate: Fixed Point Iteration e i+1 = g (ξ n ) e i Suppose g (x) < K < 1 for all x (r h, r + h), then if x 0 is in this interval, then the iterates converge, as e n+1 < K e n < K 2 e n 1 < K 3 e n 2 <... < K n+1 e 0 which implies fixed point iteration is of order 1. ITCS 4133/5133: Introduction to Numerical Methods 32 Roots of Equations

33 Convergence Rate: Newton s Method x n+1 = x n f(x n) f (x n ) = g(x n) Has the same form as the fixed point iteration. Thus, successive iterates converge if g (x) < 1 Assume a simple root at x = r, g (x) = 1 f (x) f (x) f(x) f (x) = f(x) f (x) [f (x)] 2 [f (x)] 2 Thus, if f(x) f (x) [f (x)] 2 < 1 in an interval around the root r, the method will converge for any x 0. ITCS 4133/5133: Introduction to Numerical Methods 33 Roots of Equations

34 Convergence Rate: Newton s Method If r is a root of f(x) = 0, then r = g(r), x n+1 = g(x n ), so x n+1 r = g(x n ) g(r) Expanding g(x n ) using Taylor series, expanding about (x n r), with ξ (x n, r) thus g(x n ) = g(r) + g (r) (x n r) + g (ξ) 2 (x n r) 2 g (r) = f(r) f (r) [f (r)] 2 = 0 g(x n ) = g(r) + g (ξ) 2 (x n r) 2 ITCS 4133/5133: Introduction to Numerical Methods 34 Roots of Equations

35 Convergence Rate: Newton s Method If x n r = e n, we have g(x n ) = g(r) + g (ξ) 2 (x n r) 2 e n+1 = x n+1 r = g(x n ) g(r) = g (ξ) 2 e n 2 Thus, each error in the limit is proportional to the second power of the previous error, resulting convergence of order 2, or quadratic convergence. ITCS 4133/5133: Introduction to Numerical Methods 35 Roots of Equations

36 Convergence Rate: Bisection Method Method based on intermediate value theorem: root exists in the interval [a, b], if f(a) and f(b) are of opposite sign. Error at stage k is at most (b a)/2 k Error estimates(on the average) are reduced by half at each iteration Bisection is linearly convergent. e n+1 = 0.5 e n ITCS 4133/5133: Introduction to Numerical Methods 36 Roots of Equations

37 Analysis: Regula Falsi, Secant Methods Rate of Convergence: Must investigate for large values of p. x k x x k 1 x p It can be shown that errors are reduced as follows: e n+1 = g (ξ i, ξ 2 ) (e n )e n 1 2 Convergence rate can be shown to be about ITCS 4133/5133: Introduction to Numerical Methods 37 Roots of Equations