Newton-Raphson. Relies on the Taylor expansion f(x + δ) = f(x) + δ f (x) + ½ δ 2 f (x) +..

Transcription

1 2008 Lecture 7 starts here Newton-Raphson When the derivative of f(x) is known, and when f(x) is well behaved, the celebrated (and ancient) Newton- Raphson method gives the fastest convergence of all ( quadratic, i.e. m = 2, such that ε n = ε n 12 ) Relies on the Taylor expansion f(x + δ) = f(x) + δ f (x) + ½ δ 2 f (x) +.. If ith iteration, x i, is close to the root, then for the next iteration, try x i+1 = x i + δ with δ = f (x i ) / f (x i ) 131

2 Geometric representation Newton-Raphson method: picture from Recipes Convergence is quadratic, with m=2 Extrapolate the local derivative to find the next estimate of the root 132

3 Newton-Raphson Convergence is rapid, and the method is very useful for polishing a root (i.e. refining an estimate that is nearly correct) Clearly, a few iterations usually yields an accurate result in the limit of small δ, because terms of order ½ δ 2 f (x) or higher are much smaller than δ f(x) f(x + δ) = f(x) + δ f (x) + ½ δ 2 f (x) +.. Exception: when f (x) is very small (or zero) 133

4 Newton-Raphson Cases where f (x) is small (i.e. where there is a local extremum) can send the solution shooting off into outer space. unless the method is modified to keep the solution within a known bracket (as in routine RTSAFE ) 134

5 Summary of 1-D root finding Preferred methods: If f is known analytically, safe Newton-Raphson is the preferred method If f is not known analytically, Brent s method is the method of choice Could use safe Newton-Raphson with the derivative computed numerically, but the latter takes up as much time as is saved by the use of N-R Newton-Raphson is also useful for rapid polishing (very close to the solution) and for multidimensional root finding. 135

6 Roots of polynomial functions General polynomial equation 0 = P(x) Σa k x k has an analytic solution for cases up to and including the quartic No general analytic solution for quintics and higher ( the equation that couldn t be solved ) 136

7 Roots of polynomial functions General features: Polynomial of order N will have N roots, which can be real or complex and may or may not be distinct If the a k are all real, the roots may be real or complex, but any complex roots will occur in pairs of complex conjugates, a ± bi Polynomials can be ill-conditioned: a small change in the coefficients can cause a large change in the roots 137

8 Roots of polynomial functions General features: Degenerate roots, or nearly equal roots, present the biggest numerical problems Example: P(x) (x c) 2 = 0 Brackets don t exist since P(x) is never negative Derivative f (x) = 0 at the solution x = c Newton-Raphson is potentially unstable 138

9 Deflation Every time a root, x k, is found, the order of the polynomial can be decreased by one, and the equation becomes 0 = P(x) = (x x k ) Q(x) where Q(x) can be obtained by synthetic division (see 5.3 in Recipes) This procedure, known as deflation, allows the roots to be obtained one-by-one 139

10 Laguerre s method for obtaining a single root Mathematical background P(x) = (x x 1 ) (x x 2 ) (x x n ) ln P(x) = ln x x 1 + ln x x ln x x n G d ln P(x) /dx = (x x 1 ) 1 + (x x 2 ) (x x n ) 1 H d ln P(x) /dx 2 = (x x 1 ) 2 + (x x 2 ) (x x n ) 2 140

11 Laguerre s method for obtaining a single root Now make a drastic set of assumptions We suppose x 1 is located at distance a from our current guess, x, and that all the other roots lie at a distance b G = (x x 1 ) 1 + (x x 2 ) (x x n ) 1 = a 1 + (n 1) b 1 H = (x x 1 ) 2 + (x x 2 ) (x x n ) 2 = a 2 + (n 1) b 2 We can then eliminate b from the above equations to obtain a quadratic equation for a

12 Laguerre s method for obtaining a single root We then obtain a = n G ± (n 1)(nH G 2 ) This, of course, is not the exact value of (x x 1 ), because of the approximation made in obtaining it, but it is useful for obtaining the next estimate of x 1, which we take as (x a) 142

13 Overview Find roots one at a time, using Laguerre s method iteratively until a is sufficiently small After each root is found, reduce the order of the polynomial by deflation. 143

14 Multidimensional non-linear systems of equations In general, a horrible problem, for reasons shown schematically in Recipes, Fig

15 Multidimensional non-linear systems of equations The only general algorithm is Newton-Raphson Generalization to multidimensional case f(x + δ) = f(x) + δ f (x) + (1/2) δ 2 f (x) +.. becomes f(x + δ) = f(x) + J(x) δ + O(δ 2 ) where J is the Jacobian matrix, with J ik = f i / x k 145

16 Multidimensional non-linear systems of equations The next iteration on x is therefore x' = x + δ = x J 1 f(x) where δ = J 1 f(x) is obtained by the solution of the linear set of equations J δ = f (which we know how to do! ) As in 1-D, this only works with a smooth function or a good first guess 146

17 Newton-Raphson with a modified step size Can improve the robustness of multidimensional N-R by considering the quantity: Q ½ f 2 = ½ Σ f i f i which tends to zero at the root Any good step should make Q smaller 147

18 Newton-Raphson with a modified step size Consider now the gradient of Q Q/ x j = (½ f 2 )/ x j = Σ f i f i / x j Q = f T J Our step δ = J 1 f is in the right direction to reduce Q, because Q. δ = f T (J J 1 ) f = 2Q < 0 148

19 Newton-Raphson with a modified step size So if adding δ to x doesn t decrease Q, a sufficiently small step in the same direction must Strategy: if the solution starts to overshoot as indicated by monitoring of Q ½ f 2 use a smaller step λδ in the same direction (where λ is a positive scalar < 1) See Recipes 9.7 for the details of how to choose λ This is implemented in the function NEWT, which will successfully find a root for almost any reasonable initial guess. 149

20 Example application: interstellar chemistry More than one hundred different molecules have been detected in the interstellar gas They are formed and destroyed by a complex network of reactions involving bimolecular reactions and unimolecular processes such as photodissociation 150

21 Example application: interstellar chemistry Consider the reaction A + B C + D: This destroys A and B (and creates C and D) at a rate k r n(a)n(b) per unit volume, where n(x) is the density of species X (number per unit volume) and k r is the rate coefficient (units: cm 3 s 1 ) for the reaction in question Molecule A might also be destroyed by photodissociation, A + hν E + F This destroys A (and creates E and F) at a rate ζ p n(a) per unit volume, where n(x) is the density of species X (number per unit volume) and ζ p is the photodissociation rate (units: s 1 ) for this process 151

22 Rate equations for interstellar chemistry The density of the ith molecule therefore obeys the rate equation: Processes that create ith molecule Reactions that destroy ith molecule n i / t = Σ Σ k ijk n j n k + Σ ζ ik n k n i Σ Σ k mik n k n i Σ ζ mi j k k m k m Sum of rate coefficients for all reactions of j and k that produce i Total rate at which k undergoes unimolecular processes to produce i 152

23 Equilibrium solution In equilibrium, the molecular densities will obey f(n) =0 where n i is the density of molecule number i, and f i n i / t is a quadratic function of the n i A multidimensional root-finding problem! and one for which the Jacobian is easily computed (quadratic function) f i / n k = Σ k ijk n j + ζ ik n i Σ k mik (i k) j f i / n i = Σ k mik n k Σ ζ mi (diagonal elements) m Newton s method generally works well m m 153

24 Complication The equilbrium equations, as written, are singular, since they don t uniquely specify the total density of molecules We need to apply some constraints of the form Σ n i N ic = n(e c ) N ic = number of atoms of element C contained in molecule i n(e c ) = (fixed) density of C nuclei 154

25 Non-equilibrium solution In astrochemistry, the timescale for reaching equilibrium can sometimes be long compared to the ages of molecular clouds Therefore, it is also interesting to integrate the equations n i / t = Σ Σ k ijk n j n k + Σ ζ ik n k n i Σ Σ k mik n k (a coupled set of ODE s, to be considered later) n i Σ ζ mi j k k m k m In the limit of large t, this tends to the equilibrium solution an alternative method even if we are uninterested in the time evolution 155