Root Finding Methods

Transcription

1 Root Finding Methods Let's take a simple problem ax 2 +bx+c=0 Can calculate roots using the quadratic equation or just by factoring Easy and does not need any numerical methods How about solving for the roots of e -x x = 0? How do we solve this? Can't do analytically Could try graphically, however this requires human observation Good for one problem, not so good for multiple problems Numerical analysis normally requires multiple problems Could do some sort of trail an error method... This is good but where to start? How to ensure stability? What are the errors? What is so special about root solving methods? Where is it in reality?

2 Root Finding Methods Roots come about when we have a balance Kirchoff's Laws Electrical Engineering Newton's Laws of Motion Mechanical, Civil Engineering (Dynamics) Energy balance problems ( KE, PE) Mechanical, Civil Engineering Force balance Mechanical, Civil Engineering (Statics) Mass balance Chemical Engineering Heat balance Chemical Engineering Etc. Roots come from basic laws we should remember from Physics what are they? Yes?

3 Conservation Laws Noether's theorem -- Symmetry implies conservation (breaking symmetry is a phase change) Conservation of Energy Historically promoted by engineers over the objections of physicists First Law of Thermodynamics Chemical Engineering Voltage Balance Electrical Engineering Invariant under time translation Conservation of linear momentum (or just momentum) Force balance -- Civil and Mechanical Engineering No external force is required Invariant under translation Conservation of angular momentum Angular analogy to linear momentum Civil and Mechanical Engineering No external torque is required Invariant under rotations

4 Conservation Laws Symmetry implies conservation (breaking symmetry is a phase change) Conservation of electric charge Current balance Electrical Engineering Charge in a node must eventually leave the a node Conservation of mass Mass balance equation -- Chemical Engineering Mass into a system must equal mass out of a system plus any accumulated mass in the system Conservation of baryon number Baryon number balance Nuclear Engineering In a nuclear reaction the sum of baryon numbers going into a reaction must be equal to the sum of baryon numbers of the final products Similar laws are conservation of leptons and conservation of strangeness Conservation fails in certain instances if the equivalence of energy and mass is not considered, but it never fails in reality...just in our understanding of what is going on.

5 Example of Equations Example is a simple circuit used in electrical engineering Kirchoff's Law that the sum of voltages in a closed loop is zero This idea is applicable to physics as well All based off conservation of energy Example circuit: V L =L di dt V C = q C V R =Ri Charge the capacitor with the voltage Switch to the RLC circuit and let's get an equation that deals with roots V L +V C +V R =0 L di dt + q C +Ri=0 i= dq dt Therefore L d2 q dq +R 2 dt dt + q C =0

6 Example of Equations General solution is very standard, let q = e dt Substitute into the previous equation Ld 2 e dt Rd e dt edt C =0 Ld 2 Rd 1 =0 This isacharacteristic equationwith dbeing the eigenvalues C Eigenvalues are fundamental to the system They are the roots of this problem as well Here seems simple, but beyond the second-order equation it could be difficult Lets assume a solution and from there produce an answer to this problem Assumeasolutionof i Thenq=c 1 e i c 2 e i whichisthe general solution

7 Example of Equations Newton's Law of Motion Let us assume we have a 1 kg mass that is subject to a force f(t) Therefore f t =t e t 1kg dv dt =t e t v t = e t whichwe integrate over t t If alpha is known this is not really a numerical problem, however one may wish to know what values of alpha will produce certain results Can't solve explicitly for alpha Therefore in those cases alpha is implicit We rewrite the equationas f = e t t v t Solvefor f =0whichgives usaroot type problem

8 Example of Equations Chemical type problem (Van der Wall forces) Normally we learn the gas law PV = nrt, however that is for an extremely ideal situation, so it is preferable to make some modifications to achieve a more realistic equation... P a 2 v b =RT v where v= V n f v = P a 2 v b RT v solvefor f v =0whichgivesus aroot type problem R is the universal gas constant (re: chemistry class), a is a correction to the intermolecular force, and b is a correction for the real volume of a gas 2 b=n a 3 R 3 0

9 Bracketing methods So how can we solve for roots without using a graphical method or by trail and error (at least by hand)? We use the method most people use when they are playing the game of guessing a number between 1 and 100 In order to do this however we would need to bracket our roots as in the game above (1 to 100) Bracketing the roots have a number of advantages, but some disadvantages as well. Pros: Convergence Cons: Slow, not good for multiple roots tightly packed

10 Bisection Method Bisection Method Two guess, x l and x h that should be tested by see if f(x l )f(x h ) < 0 Estimate a new root by using the equation x new = x l+ x h 2 Repeat the estimate of a new root until you reach a tolerance that is acceptable using the following conditions f(x l )f(x new ) < 0 then x h =x new f(x l )f(x new ) > 0 then x l =x new f(x l )f(x new ) = 0 then x new = root (here 0 is within your tolerance)

11 Bisection Method Error Bisection Method error Δ x=x u x l ϵ 0 =Δ x zeroth approximation ϵ 1 = Δ x 2 first approximation ϵ 2 = Δ x 2 2 second approximation... ϵ n = Δ x 2 n n th approximation ln (ϵ n )=ln(δ x) nln (2) n= ln (Δ x) ln (ϵ n ) ln (2) n= ln (Δ x) ϵ n ln (2) n=log 2( Δ x ϵ n )

12 Bisection Method Error Can use the error in the bisection method to determine the tolerance we want The bisection method is Easy Converges Slow in most cases (which isn't an issue if you only intend to use it once)

13 False Position (regula falsi) Method False position method improves the bisection method which is inefficient in many cases However not perfect; occasionally bisection method is better (which my graph here illustrates) Use knowledge of magnitudes of function to improve guess See the figure; using similar figures we have f (x l ) x r x l = f (x u ) x r x u

14 False Position (regula falsi) Method Rearranging the terms above we get x r =x u f (x u )(x l x u ) f (x l ) f ( x u ) Still need to be able to chose the right points though, so first you must be assure that your lower and upper points are on opposite sides of the function f ( x l ) f (x u )<0 Then you need to chose whether to keep the lower or upper point f (x l ) f ( x r )>0 f ( x l ) f (x r )=0 f ( x l ) f (x r )<0 then x l =x r then x r =root then x u =x r

15 False Position (regula falsi) Method In the case of a sticking situation of the root not moving efficiently enough you could use the bisection method once to get it moving more efficiently The idea of using two or more methods to get to a root efficiently is used many times in root finding methods

16 Open methods Bracketing the roots have a number of advantages, but some disadvantages as well. Uses two or more bounding points Pros: Convergence Cons: Slow, not good for multiple roots tightly packed Disadvantages of the bracketing method can be huge if multiple roots need to be calculated in a numerical problem, so in order to speed up the process another method is available called an open method Uses only one point and hopefully moves towards the root Pros: Fast Cons: May not converge

17 Fixed-Point Iteration Method Fixed-point iteration Method Open Method (only one starting point) f (x)=0 x i +1 =g (x i ) where g(x i )is part of f (x) Example would be f ( x)=e x x=0 x i +1 =e x i

18 Fixed-Point Iteration Method Basically the approach of two lines intersecting as being the same as the two functions of those lines subtracted and set equal to zero Requires an initial guess that must be close to the true root (however most open methods are like that...) Not very useful in general Useful for developing other root finding methods

19 Newton-Raphson (Newton) Method Newton Method Open Method Use iterative method that comes from Taylor series f ( x i+1 )= f ( x i )+( x i+1 x i ) f ' ( x i ) So if we set this equation equal to zero (root equation), therefore x i+1 = x i f ( x i) f ' ( x i ) This is a very popular method Pros: Fast Cons: Need derivative; need guess close to true root; may not converge

20 Newton-Raphson (Newton) Method 2-D Newton Method (non-linear) Can be extended to many variables by just taking the Taylor series with multiple variables f ( x i+1, y i+1 )= f ( x i, y i )+ x f ( x i, y i )( x i+1 x i )+ y f ( x i, y i )( y i+1 y i ) However remember that we to solve a two or more variable equation we will need two or more equations v i+1 =v i + v i x ( x i+1 x i )+ v i y ( y i+1 y i ) w i+1 =w i + w i x ( x i+1 x i )+ w i y ( y i+1 y i ) So if we set these equations equal to zero and rewrite these equations we get v i w i y v i x i+1 = x i J w i y w i v i x w i y i+1 = y i J v i x J is a Jacobian matrix (which we are taking the determinant of above)

21 Jacobian The Jacobian matrix (should have had this in Calculus, but for review) is =[ J v i x w i x v i ] y w i y

22 Secant Method Secant Method Open Method that uses two initial guesses like with bracketing methods Does not require a derivative as the Newton method does

23 Secant Method Secant Method Open Method that uses two initial guesses like with bracketing methods Difference between false-position method is replacement of new starting values False-position method must stay bracketed so initial guesses must bracket the root Secant method is not bracketed so initial guesses do not need to bracket the root Basically secant method uses the backward finite divided difference to replace Newton's method, might not be the best way to go... Start with the equation y-y 1 = m(x-x 1 ) and look for x intercept (which is x i+1 ) x i+1 =x i f (x i )(x i 1 x i ) f ( x i 1 ) f (x i ) Advantage of this method is it does not require a derivative like the Newton method Maybe slower than Newton method though Has all the problems that all open methods have

24 Multiple Roots Multiple roots Bracketing method not effective to get multiple roots Any derivative method (Newton and Secant fall under this category) will have trouble with multiple roots as well because the derivative goes to zero at the root as well A way out of this is that the function itself will go to zero faster than the derivative therefore a test could be made Method to modify Newton method is to define a new function which is a ratio of the original function to its derivative. This new function's roots will be the same as the original function's roots. u(x)= f (x) f ' ( x) x i+1 = x i u( x i) u' ( x i ) f ( x i ) f ' ( x i ) x i+1 = x i f ' ( x i ) 2 f ( x i ) f ' ' ( x i )

25 Brent Method Brent Method A method that combines two speedy open methods with one bracketing method to ensure a convergent direction Secant method Inverse quadratic interpolation method Parabola (three points) instead of two (which would be a linear interpolation which is like what the secant method does) Inverse because there is the possibility of the parabola not intersecting the x-axis; so it is inversed x=f(y) Need a discussion of quadratic interpolation that is outside the time requirements of this class (sorry) Bisection method

26 Root Finding Methods Bairstow's Method, Bernoulli's Method, Bisection, Brent's Method Crout's Method Durand-Kerner method Graeffe's Method Halley's Irrational Formula, Halley's Method, Halley's Rational Formula, Horner's Method (~Newton's Method), Householder's Method Jenkins-Traub Method Laguerre's Method, Larkin's Method, Lehmer-Schur Method, Lin's Method Muller's Method Ridder's Method Schröder's Method, Secant Method, Splitting Circle Method Etc. Note the number of methods! Clearly an important subject

27 Root Finding Methods Householder's Method is an iteration method like Halley's method has similar problems to Newton-Raphson Methods. Jenkins-Traub Method is a complicated black-box method used by IMSL. Standard (most likely combines methods). Laguerre's Method converges from any starting point. For polynomials...requires first and second derivative. Larkin's Method is an iteration of the Horner's Method. Lehmer-Schur Method is a root finding method that generalizes bracketing for the complex plane. Lin's Method is a root method for quartic equations (x 4 type equations). Muller's Method is a generalization of the secant method using three points. Sort of like the inverse quadratic interpolation method.

28 Root Finding Methods Ridder's Method is a variation on the false position method using an exponential. Schröder's Method is a powerful method for robustly getting roots. Can be used for multiple roots that are in the same neighborhood. Splitting circle method is used for finding roots of a polynomial with a high degree of precision. Newton's Method and Horner's Method (with divided differences) are related

29 Root Finding Methods Inverse Quadratic Interpolation uses three points (as opposed to two) Bailey's, Lambert's or Hutton's Methods used for problematic roots like x 10-1 Bairstow's Method (helps for quadratics factors of the complex conjugate) finds roots of a polynomial using only real arithmetic Bernoulli's Method for polynomial roots Bisection we did in class Brent's Method (Van Wijngaarden-Deker-Brent) combines secant, bisection (bracketing), and inverse quadratic interpolation needs three points and will apparently always converge... Crout's Method solves N 2 equations (good for multiple roots, equations) Durand-Kerner Method (Weierstrass) finds roots of a polynomial using complex number arithmetic (a generalization of Bairstow's method)

30 Root Finding Methods Graeffe's Method was used in the 19 th and 20 th century, but fell out of favor as it exceeded floating point arithmetic limits on computers. Malajovich and Zubelli re-design this method to work more efficiently. Note: uses Vieta's formulae (green book) Halley's Irrational Formula uses third-order Taylor series expansion (related to Laguerre's method) Halley's Method (Halley's Rational Formula, Tangent Hyperbolas method)