C O M P U L S O R Y R E A D I N G S 1

Transcription

1 C O M P U L S O R Y R E A D I N G S 1 1 According to the author of the module, the compulsory readings do not infringe known copyright.

2 COMPILED LIST OF COMPULSORY READINGS 1. Wikipedia: Numerical Methods/Errors Introduction 2. Wikipedia: Interpolation 3. Fundamental Numerical Methods and Data Analysis, George W. Collins, II, chapter Wikipedia: Numerical Methods/Numerical Integration 5. Wikipedia: Numerical Methods/Equation Solving

3 R e a d i n g ( s ) # 1

4 Numerical Methods/Errors Introduction From Wikibooks, the open-content textbooks collection To the student: the links in this document will not work unless you view the document on an internet enabled computer. When using numerical methods or algorithms and computing with finite precision, errors of approximation or rounding and truncation are introduced. It is important to have a notion of their nature and their order. A newly developed method is worthless without an error analysis. Neither does it make sense to use methods which introduce errors with magnitudes larger than the effects to be measured or simulated. On the other hand, using a method with very high accuracy might be computationally too expensive to justify the gain in accuracy. Accuracy and Precision Measurements and calculations can be characterized with regard to their accuracy and precision. Accuracy refers to how closely a value agrees with the true value. Precision refers to how closely values agree with each other. The following figures illustrate the different between accuracy and precision. In the first figure, the given values (black dots) are more accurate; whereas in the second figure, the given values are more precise. The term error represents the imprecision and inaccuracy of a numerical computation.

5 Absolute Error The error between two values is defined as where x denotes the exact value and its approximation. Relative Error The relative error of is the absolute error relative to the exact value. Look at it this way: if your measurement has an error of ± 1 inch, this seems to be a huge error when you try to measure something which is 3 in. long. However, when measuring distances on the order of miles, this error is mostly negligible. The definition of the relative error is

6 Sources of Error In a numerical computation, error may arise because of the following reasons: Truncation error Roundoff error Truncation Error Truncation error refers to the error in a method, which occurs because some series (finite or infinite) is truncated to a fewer number of terms. Such errors are essentially algorithmic errors and we can predict the extent of the error that will occur in the mehtod. Roundoff Error Roundoff error occurs because of the computing device's inability to deal with inexact numbers. Such numbers need to be rounded off to some near approximation which is dependent on the word size used to represent numbers of the device. Retrieved from: "

7 R e a d i n g ( s ) # 2

8 Interpolation From Wikipedia, the free encyclopedia To the student: the links in this document will not work unless you view the document on an internet enabled computer. In the mathematical subfield of numerical analysis, interpolation is a method of constructing new data points from a discrete set of known data points. In engineering and science one often has a number of data points, as obtained by sampling or experiment, and tries to construct a function which closely fits those data points. This is called curve fitting. Interpolation is a specific case of curve fitting, in which the function must go exactly through the data points. A different problem which is closely related to interpolation is the approximation of a complicated function by a simple function. Suppose we know the function but it is too complex to evaluate efficiently. Then we could pick a few known data points from the complicated function, creating a lookup table, and try to interpolate those data points to construct a simpler function. Of course when using the simple function to calculate new data points we usually do not receive the same result as when using the original function, but depending on the problem domain and the interpolation method used the gain in simplicity might offset the error. It should be mentioned that there is another very different kind of interpolation in mathematics, namely the "interpolation of operators". The classical results about interpolation of operators are the Riesz-Thorin theorem and the Marcinkiewicz theorem. There also are many other subsequent results. Definition Given a sequence of n distinct numbers x k called nodes and for each x k a second number y k, we are looking for a function f so that A pair x k,y k is called a data point and f is called an interpolant for the data points. When the numbers y k are given by a known function f, we sometimes write f k.

9 Example For example, suppose we have a table like this, which gives some values of an unknown function f. x f(x) Plot of the data points as given in the table. What value does the function have at, say, x = 2.5? Interpolation answers questions like this. There are many different interpolation methods, some of which are described below. Some of the concerns to take into account when choosing an appropriate algorithm are: How accurate is the method? How expensive is it? How smooth is the interpolant? How many data points are needed?

10 Linear interpolation Plot of the data with linear interpolation superimposed One of the simplest methods is linear interpolation (sometimes known as lerp). Consider the above example of determining f(2.5). Since 2.5 is midway between 2 and 3, it is reasonable to take f(2.5) midway between f(2) = and f(3) = , which yields Generally, linear interpolation takes two data points, say (x a,y a ) and (x b,y b ), and the interpolant is given by: at the point (x,y). Linear interpolation is quick and easy, but it is not very precise. Another disadvantage is that the interpolant is not differentiable at the point x k. The following error estimate shows that linear interpolation is not very precise. Denote the function which we want to interpolate by g, and suppose that x lies between x a and x b and that g is twice continuously differentiable. Then the linear interpolation error is In words, the error is proportional to the square of the distance between the data points. The error of some other methods, including polynomial interpolation and spline interpolation (described below), is proportional to higher powers of the distance between the data points. These methods also produce smoother interpolants.

11 Polynomial interpolation Plot of the data with polynomial interpolation applied Polynomial interpolation is a generalization of linear interpolation. Note that the linear interpolant is a linear function. We now replace this interpolant by a polynomial of higher degree. Consider again the problem given above. The following sixth degree polynomial goes through all the seven points: f(x) = x x x x x x. Substituting x = 2.5, we find that f(2.5) = Generally, if we have n data points, there is exactly one polynomial of degree n 1 going through all the data points. The interpolation error is proportional to the distance between the data points to the power n. Furthermore, the interpolant is a polynomial and thus infinitely differentiable. So, we see that polynomial interpolation solves all the problems of linear interpolation. However, polynomial interpolation also has some disadvantages. Calculating the interpolating polynomial is relatively very computationally expensive (see computational complexity). Furthermore, polynomial interpolation may not be so exact after all, especially at the end points (see Runge's phenomenon). These disadvantages can be avoided by using spline interpolation.

12 Spline interpolation Plot of the data with Spline interpolation applied Remember that linear interpolation uses a linear function for each of intervals [x k,x k+1 ]. Spline interpolation uses low-degree polynomials in each of the intervals, and chooses the polynomial pieces such that they fit smoothly together. The resulting function is called a spline. For instance, the natural cubic spline is piecewise cubic and twice continuously differentiable. Furthermore, its second derivative is zero at the end points. The natural cubic spline interpolating the points in the table above is given by In this case we get f(2.5)= Like polynomial interpolation, spline interpolation incurs a smaller error than linear interpolation and the interpolant is smoother. However, the interpolant is easier to evaluate than the high-degree polynomials used in polynomial interpolation. It also does not suffer from Runge's phenomenon.

13 Other forms of interpolation Other forms of interpolation can be constructed by picking a different class of interpolants. For instance, rational interpolation is interpolation by rational functions, and trigonometric interpolation is interpolation by trigonometric polynomials. The discrete Fourier transform is a special case of trigonometric interpolation. Another possibility is to use wavelets. The Whittaker Shannon interpolation formula can be used if the number of data points is infinite. Multivariate interpolation is the interpolation of functions of more than one variable. Methods include bilinear interpolation and bicubic interpolation in two dimensions, and trilinear interpolation in three dimensions. Sometimes, we know not only the value of the function that we want to interpolate, at some points, but also its derivative. This leads to Hermite interpolation problems. Related concepts The term extrapolation is used if we want to find the value of f at a point x which is outside of the points x k at which f is given. In curve fitting problems, the constraint that the interpolant has to go exactly through the data points is relaxed. It is only required to approach the data points as closely as possible. This requires parameterizing the potential interpolants and having some way of measuring the error. In the simplest case this leads to least squares approximation. Approximation theory studies how to find the best approximation to a given function by another function from some predetermined class, and how good this approximation is. This clearly yields a bound on how well the interpolant can approximate the unknown function. References David Kidner, Mark Dorey and Derek Smith (1999). What's the point? Interpolation and extrapolation with a regular grid DEM. IV International Conference on GeoComputation, Fredericksburg, VA, USA. Kincaid, David; Ward Cheney (2002). Numerical Analysis (3rd edition). Brooks/Cole. ISBN Chapter 6. Schatzman, Michelle (2002). Numerical Analysis: A Mathematical Introduction. Clarendon Press, Oxford. ISBN Chapters 4 and 6. Retrieved from "

14 Lagrange polynomial From Wikipedia, the free encyclopedia In numerical analysis, a Lagrange polynomial, named after Joseph Louis Lagrange, is the interpolation polynomial for a given set of data points in the Lagrange form. It was first discovered by Edward Waring in 1779 and later rediscovered by Leonhard Euler in As there is only one interpolation polynomial for a given set of data points it is a bit misleading to call the polynomial Lagrange interpolation polynomial. The more precise name is interpolation polynomial in the Lagrange form. This image shows, for four points (( 9, 5), ( 4, 2), ( 1, 2), (7, 9)), the (cubic) interpolation polynomial L(x), which is the sum of the scaled basis polynomials y 0 l 0 (x), y 1 l 1 (x), y 2 l 2 (x) and y 3 l 3 (x). The interpolation polynomial passes through all four control points, and each scaled basis polynomial passes through its respective control point and is 0 where x corresponds to the other three control points. Definition Given a set of k + 1 data points where no two x j are the same, the interpolation polynomial in the Lagrange form is a linear combination

15 of Lagrange basis polynomials Proof The function we are looking for has to be a polynomial function L(x) of degree k with The Lagrange polynomial is a solution to the interpolation problem: As can be easily seen Thus the function L(x) is a polynomial with degree k and 1. is a polynomial and has degree k. 2. There can be only one solution to the interpolation problem since the difference of two such solutions would be a polynomial with degree k and k+1 zeros. Therefore L(x) is our unique interpolation polynomial. Main idea Solving an interpolation problem leads to a problem in linear algebra where we have to solve a matrix. Using a standard monomial basis for our interpolation polynomial we get the very complicated Vandermonde matrix. By choosing another basis, the Lagrange basis, we get the much simpler identity matrix = δ i,j which we can solve instantly.

16 Usage Example The tangent function and its interpolant We wish to interpolate f(x) = tan(x) at the points x 0 = 1.5 f(x 0 ) = x 1 = 0.75 f(x 1 ) = x 2 = 0 f(x 2 ) = 0 x 3 = 0.75 f(x 3 ) = x 4 = 1.5 f(x 4 ) = Now, the basis polynomials are:

17 Thus the interpolating polynomial then is Notes The Lagrange form of the interpolation polynomial shows the linear character of polynomial interpolation and the uniqueness of the interpolation polynomial. Therefore, it is preferred in proofs and theoretical arguments. But, as can be seen from the construction, each time a node x k changes, all Lagrange basis polynomials have to be recalculated. A better form of the interpolation polynomial for practical (or computational) purposes is the Newton polynomial. Using nested multiplication amounts to the same idea. Lagrange and other interpolation at equally spaced points, as in the example above, yield a polynomial oscillating above and below the true function. This oscillation is lessened by choosing interpolation points at Chebyshev nodes. The Lagrange basis polynomials are used in numerical integration to derive the Newton- Cotes formulas. Lagrange interpolation is often used in digital signal processing of audio for the implementation of fractional delay FIR filters (e.g., to precisely tune digital waveguides in physical modelling synthesis). Retrieved from "

18 Divided differences From Wikipedia, the free encyclopedia In mathematics divided differences is a recursive division process. The method can be used to calculate the coefficients in the interpolation polynomial in the Newton form. Definition Given n data points the divided differences are defined as Notes If the data points are given as a function f(x) we sometimes write

19 Example For the first few [y ν ] this yields To make the recursive process more clear the divided differences can be put in a tabular form Expanded form Peano form The divided differences can be expressed as where B n 1 is a B-spline of degree n 1 for the data points and f (n) is the n- th derivative of the function f. This is called the Peano form of the divided differences and B n 1 is called the Peano kernel for the divided differences, both named after Giuseppe Peano.

20 Taylor form First order If nodes are cumulated, then the numerical computation of the divided differences is inaccurate, because you divide almost two zeros, each of which with a high relative error due to differences of similar values. However we know, that difference quotients approximate the derivative and vice versa: for This approximation can be turned into an identity whenever Taylor's theorem applies. You can eliminate the odd powers of y x by expanding the Taylor series at the center between x and y: x = m h,y = m + h, that is Higher order The Taylor series or any other representation with function series can in principle be used to approximate divided differences. Taylor series are infinite sums of power functions. The mapping from a function f to a divided difference is a linear functional. We can as well apply this functional to the function summands. Express power notation with an ordinary function: p n (x) = x n

21 Regular Taylor series is a weighted sum of power functions: Taylor series for divided differences: We know that the first n terms vanish, because we have a higher difference order than polynomial order, and in the following term the divided difference is one: It follows that the Taylor series for the divided difference essentially starts with which is also a simple approximation of the divided difference, according to the mean value theorem for divided differences. If we would have to compute the divided differences for the power functions in the usual way, we would encounter the same numerical problems that we had when computing the divided difference of f. The nice thing is, that there is a simpler way. It holds Consequently we can compute the divided differences of p n by a division of formal power series. See how this reduces to the successive computation of powers when we compute p n [h] for several n. Cf. an implementation in Haskell. Limits From the mean value theorem for divided differences it follows that

22 Forward differences When the data points are equidistantly distributed we get the special case called forward differences. They are easier to calculate than the more general divided differences. Definition Given n data points with the divided differences can be calculated via forward differences defined as Example Retrieved from "

23 Newton polynomial From Wikipedia, the free encyclopedia In the mathematical field of numerical analysis, a Newton polynomial, named after its inventor Isaac Newton, is the interpolation polynomial for a given set of data points in the Newton form. The Newton polynomial is sometimes called Newton's divided differences interpolation polynomial because the coefficients of the polynomial are calculated using divided differences. As there is only one interpolation polynomial for a given set of data points it is a bit misleading to call the polynomial Newton interpolation polynomial. The more precise name is interpolation polynomial in the Newton form. Definition Given a set of k + 1 data points where no two x j are the same, the interpolation polynomial in the Newton form is a linear combination of Newton basis polynomials with the Newton basis polynomials defined as and the coefficients defined as where is the notation for divided differences. Thus the Newton polynomial can be written as

24 General case For the special case of x i = i, there is a closely related set of polynomials, also called the Newton polynomials, that are simply the binomial coefficients for general argument. That is, one also has the Newton polynomials p n (z) given by In this form, the Newton polynomials generate the Newton series. These are in turn a special case of the general difference polynomials which allow the representation of analytic functions through generalized difference equations. Main idea Solving an interpolation problem leads to a problem in linear algebra where we have to solve a matrix. Using a standard monomial basis for our interpolation polynomial we get the very complicated Vandermonde matrix. By choosing another basis, the Newton basis, we get a much simpler lower triangular matrix which can be solved faster. For k + 1 data points we construct the Newton basis as Using these polynomials as a basis for Π k we have to solve to solve the polynomial interpolation problem. This matrix can be solved recursively by solving

25 Application As can be seen from the definition of the divided differences new data points can be added to the data set to create a new interpolation polynomial without recalculating the old coefficients. And when a data point changes we usually do not have to recalculate all coefficients. Furthermore if the x i are distributed equidistantly the calculation of the divided differences becomes significantly easier. Therefore the Newton form of the interpolation polynomial is usually preferred over the Lagrange form for practical purposes. Example The divided differences can be written in the form of a table. For example, for a function f is to be interpolated on points. Write the interpolating polynomial is formed as above using the topmost entries in each column as coefficients. For example, if we are to construct the interpolating polynomial to f(x) = tanx using divided differences, at the points x 0 = 1.5 x 1 = 0.75 x 2 = 0 x 3 = 0.75 x 4 = 1.5 f(x 0 ) = f(x 1 ) = f(x 2 ) = 0 f(x 3 ) = f(x 4 ) = we construct the table

26 Thus, the interpolating polynomial is Neglecting minor numerical stability problems in evaluating the entries of the table, this polynomial is essentially the same as that obtained with the same function and data points in the Lagrange polynomial article. Retrieved from "

27 R e a d i n g ( s ) # 3

28 Numerical Methods/Numerical Integration From Wikibooks, the open-content textbooks collection To the student: the links in this document will not work unless you view the document on an internet enabled computer. Often, we need to find the integral of a function that may be difficult to integrate analytically (ie, as a definite integral) or impossible (the function only existing as a table of values). Some methods of approximating said integral are listed below. Trapezoidal Rule Consider some function, possibly unknown, f(x), with known values over the interval [a,b] at n+1 evenly spaced points x i of spacing, x 0 = a and x n = b. Further, denote the function value at the ith mesh point as f(x i ). Using the notion of integration as "finding the area under the function curve", we can denote the integral over the ith segment of the interval, from x i - 1 to x i as: = (1) Since we may not know the antiderivative of f(x), we must approximate it. Such approximation in the Trapezoidal Rule, unsurprisingly, involves approximating (1) with a trapezoid of width h, left height f(x i - 1 ), right height f(x i ). Thus, (1) = (2) (2) gives us an approximation to the area under one interval of the curve, and must be repeated to cover the entire interval. For the case where n = 2, = (3)

29 Collecting like terms on the right hand side of (3) gives us: or Now, substituting in for h and cleaning up, To motivate the general version of the trapezoidal rule, now consider n = 4, Following a similar process as for the case when n=2, we obtain Proceeding to the general case where n = N, This is an example of what the trapezoidal rule would represent graphicly, here y = x

30 Example Approximate to within 5%. First, since the function can be exactly integrated, let us do so, to provide a check on our answer. = (4) We will start with an interval size of 1, only considering the end points. f(0) = 0 f(1) = 1 (4) Relative error = Hmm, a little high for our purposes. So, we halve the interval size to 0.5 and add to the list f(0.5) = (4) Relative error = Still above 0.01, but vastly improved from the initial step. We continue in the same fashion, calculating f(0.25) and f(0.75), rounding off to four decimal places.

31 f(0.25) = f(0.75) = (4) Relative error = We are well on our way. Continuing, with interval size and rounding as before, f(0.125) = f(0.375) = f(0.625) = f(0.875) = (4) Relative error = Since our relative error is less than 5%, we stop. Error Analysis Let y=f(x) be continuous,well-behaved and have continuous derivatives in [x 0,x n ].We expand y in a Taylor series about x=x 0,thus-

32 Simpson's Rule Consider some function y = f(x) possibily unknown with known values over the interval [a,b] at n+1 evently spaced points then it defined as where and x 0 = a and x n = b. Example Evaluate by taking n = 6 (n must be even) Solution: Here Since a = 0 & b = 1.2 so Now when a = x 0 = 0 then f(x 0 ) = 0 And since x n = x n 1 + h, therefore for x 1 = 0.2, x 2 = 0.4, x 3 = 0.6, x 4 = 0.8, x 5 = 1, x 6 = b = 1.2 the corresponding values are f(x 1 ) = , f(x 2 ) = , f(x 3 ) = , f(x 4 ) = , f(x 5 ) = , f(x 6 ) = References Eric W. Weisstein. "Trapezoidal Rule." From MathWorld--A Wolfram Web Resource. Retrieved from "

33 R e a d i n g ( s ) # 4

34 Numerical Methods/Equation Solving From Wikibooks, the open-content textbooks collection To the student: the links in this document will not work unless you view the document on an internet enabled computer. Solution of Algebraic and Transcendental Equations An equation of the type f(x) = 0 is either algebraic or transcendental. E.g, these equations are algebraic. and these are transcendental While roots can be found directly for algebraic equations of fourth order or lower, and for a few special transcendental equations, in practice we need to solve equations of higher order and also arbitary transcendental equations. As analytic solutions are often either too cumbersome or simply do not exist, we need to find an approximate method of solution. This is where numerical analysis comes into the picture. Some Useful Observations The total number of roots an algebraic equation can have is the same as its degree. An algebraic equation can have at most as many positive roots as the number of changes of sign in f(x). An algebraic equation can have at most as many negative roots as the number of changes of sign in f( - x). In an algebraic equation with real coefficients, complex roots occur in conjugate pairs If f(x) = a 0 x n + a 1 x n a 2 x n a n - 1 x + a n with roots α 1,α 2,...,α n then the following hold good: o o o If f(x) is continuous in the interval [a,b] and f(a)f(b) < 0 then a root must exist in the interval (a,b)

35 Initial Approximation The last point about the interval is one of the most useful properties numerical methods use to find the roots. All of them have in common the requirement that we need to make an initial guess for the root. Practically, this is easy to do graphically. Simply plot the equation and make a rough estimate of the solution. Analytically, we can usually choose any point in an interval where a change of sign takes place. However, this is subject to certain conditions that vary from method to method. Convergence A numerical method to solve equations will be a long process. We would like to know, if the method will lead to a solution (close to the exact solution) or will lead us away from the solution. If the method, leads to the solution, then we say that the method is convergent. Otherwise, the method is said to be divergent. Rate of Convergence Various methods converge to the root at different rates. That is, some methods are slow to converge and it takes a long time to arrive at the root, while other methods can lead us to the root faster. This is in general a compromise between ease of calculation and time. For a computer program however, it is generally better to look at methods which converge quickly. The rate of convergence could be linear or of some higher order. The higher the order, the faster the method converges. If e i is the magnitude of the error in the ith iteration, ignoring sign, then the order is n if is approximately constant. It is also important to note that the chosen method will converge only if e i + 1 < e i. Bisection Method This is one of the simplest methods and is strongly based on the property of intervals. To find a root using this method, the first thing to do is to find an interval [a,b] such that. Bisect this interval to get a point (c,f(c)). Choose one of a or b so that the sign of f(c) is opposite to the ordinate at that point. Use this as the new interval and proceed until you get the root within desired accuracy.

36 Error Analysis The maximum error after the ith iteration using this process will be given as As the interval at each iteration is halved, we have converges linearly. Thus this method Example Solve x 3-2x - 5 = 0 correct upto 2 decimal places. f(x) = x 3-2x - 5 f(x 1 ) = > 0 Regula Falsi Method This method is essentially same as the bisection method except that instead of bisecting the interval, we find where the chord joining the two points meets the X axis. The roots are calculated using the equation of the chord, i.e. putting y = 0 in The rate of convergence is still linear but faster than that of the bisection method. Both these methods will fail if f has a double root..

37 Example Consider f(x)=x 2-1. We already know the roots of this equation, so we can easily check how fast the regula falsi method converges. For our initial guess, we'll use the interval [0,2]. Since f is concave upwards and increasing, a quick sketch of the geometry shows that the chord will always intersect the x-axis to the left of the solution. This can be confirmed by a little algebra. We'll call our n th iteration of the interval [a n, 2] The chord intersects the x-axis when Rearranging and simplifying gives Since this is always less than the root, it is also a n+1 The difference between a n and the root is e n =a n -1, but This is always smaller than e n when a n is positive. When a n approaches 1, each extra iteration reduces the error by two-thirds, rather than one-half as the bisection method would. In this case, the lower end of the interval tends to the root, and the minimum error tends to zero, but the upper limit and maximum error remain fixed. This is not uncommon.

38 Fixed Point Iteration (or Staircase method) If we can write f(x)=0 in the form x=g(x), then the point x would be a fixed point of the function g (that is, the input of g is also the output). Then an obvious sequence to consider is If we look at this on a graph we can see how this could converge to the intersection. Any function can be written in this form if we define g(x)=f(x)+x, though in some cases other rearrangements may prove more useful. Error analysis We define the error at the n th step to be Then we have So, when g (x) <l, this sequence converges to a root, and the error will be approximately proportional to n. Because the relationship between e n+1 and e n is linear, we say that this method converges linearly, if it converges at all. When g(x)=f(x)+x this means that if converges to a root, x, of f then. Note that this convergence will only happen for a certain range of x. If the first estimate is outside that range then no solution will be found.

39 Also note that although this is a necessary condition for convergence, it does not guarantee convergence. In the error analysis we neglected higher powers of e n, but we can only do this if e n is small. If our initial error is large, the higher powers may prevent convergence, even when the condition is satisfied. If g (x) <l is true at the root, the iteration sequence will converge in some interval around the root, which may be smaller than the interval where g (x) <l. If g (x) isn't smaller than one at the root, the iteration will not converge to that root. Example Lets consider f(x) = x 3 + x - 2, which we can see has a single root at x=1. There are several ways f(x)=0 can be written in the desired form, x=g(x). The simplest is In this case, g'(x) = 3x 2 + 2, and the convergence condition is Since this is never true, this doesn't converge to the root. An alternate rearrangement is This converges when Since this range does not include the root, this method won't converge either. Another obvious rearrangement is In this case the convergence condition becomes

40 Again, this region excludes the root. Another possibility is obtained by dividing by x 2 +1 In this case the convergence condition becomes Consideration of this ineqaulity shows it is satisified if x>1, so if we start with such an x, this will converge to the root. Clearly, finding a method of this type which converges is not always straightforwards.

41 Newton-Raphson We can get better convergence if we know about the function's derivatives. Consider the tangent to the function: Near any point, the tangent at that point is approximately the same as f('x) itself, so we can use the tangent to approximate the function. The tangent through the point (x n, 'f(x n )) is The next approximation, x n+1, is where the tangent line intersects the axis, so where y=0. Rearranging, we find

42 Error analysis Again, we define the root to be x, and the error at the n th step to be Then the error at the next step is where we've written f as a Taylor series round its root, x. Rearranging this, and using f(x)=0, we get (1) where we've neglected cubic and higher powers of the error, since they will be much smaller than the squared term, when the error itself is small. Notice that the error is squared at each step. This means that the number of correct decimal places doubles with each step, much faster than linear convergence. This sequence will converge if If f isn't zero at the root, then there will always be a range round the root where this method converges. If f is zero at the root, then on looking again at (1) we see that we get and the convergence becomes merely linear. Overall, this method works well, provided f does not have a minimum near its root, but it can only be used if the derivative is known.

43 Example Let's consider f(x)=x 2 -a. Here, we know the roots exactly, so we can see better just how well the method converges. We have This method is easily implemented, even with just pen and paper, and has been used to rapidly estimate square roots since long before Newton. The n th error is e n =x n - a, so we have If a=0, this simplifies to e n /2, as expected. If a>0, e n+1 will be positive, provided e n is greater than - a, i.e provided x n is positive. Thus, starting from any positive number, all the errors, except perhaps the first will be positive. The method converges when, so, assuming e n is positive, it converges when which is always true. This method converges to the square root, starting from any positive number, and it does so quadratically.

44 Higher order methods There are methods that converge even faster than Newton-Raphson. e.g which converges cubicly, tripling the number of correct digits at each iteration, which is 50% faster than Newton-Raphson. However, if iterating each step takes 50% longer, due to the more complex formula, there is no net gain in speed. For this reason, methods such as this are seldom used.