CHAPTER 1. INTRODUCTION. ERRORS.

CHAPTER 1. INTRODUCTION. ERRORS. SEC. 1. INTRODUCTION. Frequently, in fact most commonly, practical problems do not have neat analytical solutions. As examples of analytical solutions to mathematical problems, consider the following 5 cases:- (i) 1 0 x2 dx = [ x3 3 ]1 0 = 1 3 (ii) d dx (x4 ) at x = 1; solution = 4x 3 = 4 (iii) Evaluate e 1 (solution from calculator). (iv) Find the roots of x 2 +5x+5 = 0 [solutions x = 5± 5 ]. 2 (v) Solve dy +2xy = 0 for y = 1 when x = 0 [solution y = ] dx e x2 Now consider the following rather similar problems:- (vi) 1 0 cos x dx. e x2 (vii) Findthederivative at x=1ofthefunctiongiven by thefollowing table:- x.7.8.9 1.0 1.1 1.2 1.3 1.4 f(x) 2.785 2.843 2.912 3.034 3.122 3.208 3.296 3.374 (viii) Evaluate J 0 (1.2345), using tables at intervals of.01; (ix) Find the roots of x 5 +x 4 +3x 3 +2x 2 +x+1 = 0; (x) Solve ( dy dx )2 = x 2 +y 2 ; The last 5 problems cannot be solved by the standard methods of Calculus, Algebra or Differential Equations, at least not by any obvious method. However, they can be solved by methods which deal entirely with actual numbers, avoiding direct use of the rules of Calculus etc, although they are based on quite advanced Calculus, etc.. These methods are therefore called NUMERICAL METHODS. Sometimes a problem can be solved by analytical methods, but can also be solved more easily by numerical methods. For example a cubic polynomial can be solved algebraically, i.e. by a formula involving the coefficients, but the resulting expressions are so cumbersome that a numerical method such as Newton s would be much quicker. Moreover, even a so-called analytic method often reduces to some kind of numerical method in the end, e.g. 10 0 cos x dx = sin 10 =.1736 by tables or calculator, but the table for sinx did not write itself; it had to be 1

calculated numerically by a formula such as sin x = x x3 6 + x5 120... for each value of x (and likewise for a calculator). Again, many functions in real life (e.g. Engineering and Science) are given only as a table, making analytical methods impossible. The above problems illustrate most of the methods we shall describe, such as integration, differentiation, interpolation, solution of non-linear equations, and differential equations. To show how numerical methods may be derived, consider ex (vi) above. This is 1 0 ydx, where y = cosx. We may form a table of y for x = 0.0, e x2.1,.2,...,1.0, and hence draw a graph of y. A B U X C V Y D W Z E F G H.18 0.1 1.0 2

Nowthe ydx=areaunder thegraph<(areaofrectanglesabfe+xcgf+...etc = S U ) but > (Sum of rectangles UXFE+VYGF+...etc = S L ) [provided curveissmooth]. Aswecancalculatealltheseareas, wecangetanupper and lower bound on our integral. A good approximation is the average of these two bounds, 1 2 (S U +S L ). Also the error ABXUetc = (S U S L ). By taking sufficiently small steps in the x-direction, we may estimate our integral as closely as we like. Similarly we may find the derivative in (vii) by drawing a graph and measuring the slope of the line between f at.9 and 1.1, for remember dy y lim x 0 BC =.210 x AC dy FE = tanθ = 1.0 dx DE dx =.2 = 1.05; or we may draw a tangent at P; then f B A P C F D θ E.9 1.0 1.1 x CLASS EXERCISE.Find e 1.2345 by graphical methods, given the folowing table:- x 1.22 1.23 1.24 1.25 e x 3.3872 3.4212 3.4556 3.4903 [HINT. Draw a graph near the point concerned]. 3

An example from algebra Solve a 1x+b 1 y = c 1 a 2 x+b 2 y = c 2 (i)we will give a general (analytic) solution by determinants (Cramer s rule) c x = 1 b 1 c 2 b 2 a 1 b 1, etc. This gives a formula for the solution in a 2 b 2 terms of the coefficients. (ii) for particular numerical values of a i,b i,c i we may eliminate x to give an equation in y, solve for y and then x; e.g.:- Hence:- 2x+3y = 5 3x+2y = 5 6x+9y = 15 6x+4y = 10 hence 5y = 5, hence y = 1, hence 2x = 5-3y = 2, hence x = 1. In this method we have no general formula for the solution, although we do have a general prescription, or method, for finding the numerical solution, dealing always with numbers as opposed to symbols. Hence it is commonly called a numerical method. The distinction between numerical and analytical methods is not always clear, but is often a matter of convention. An example from calculus illustrates the difference: a pure mathematician might tell you that e x = 1+x+ x2 +...+ xn +..( to ) = x n 2! n! n=0 n! and consider he has defined e x. A numerical analyst however might take the same formula, but would seek to know how many terms must be included, for a given x, to calculate e x to a specified degree of accuracy (i.e. correct to a certain number of decimal places). Furthermore he would seek if possible the most efficient formula to evaluate e x, i.e. the one with the fewest possible terms. 4

SEC. 2. TYPES OF ERRORS. In all numerical work, it is essential to know how accurate (or inaccurate) our result is; or putting it another way we must know what steps to take in order to get an answer within a specified tolerance(e.g. to 3 significant figures). There are at least 4 kinds of errors arising in numerical work:- 1. Plain mistakes, when a problem is done by hand; e.g. copying incorrectly, reversal of digits,wrong signs or decimal point. Such mistakes can be detected as they occur by means of careful checking procdures (NOT based on repetition which usually just repeats the mistake); e.g. in adding a column of figures I always add it from the top down and again from the bottom up. In spite of the great availability of cheap and high-speed computers, manual work is still necessary before testing programs and for small one-shot jobs. Also manual calculation greatly helps in understanding or finding pitfalls in a method which is going to be tried on a computer. (By manual we mean with the aid of a hand-held calculator only). 2. Rounding error. Numbers in a computer or calculator can only be stored to a finite number of significant figures(usually about 8 although in double precision we can have 15). Whenever two numbers are added, subtracted, multiplied or divided an additional error may be introduced, called rounding error. e.g. suppose we add together.9237 and.2347, in a computer which has only space for 4 decimal (base 10) digits per number. The correct answer is 1.1584 but since we have only room for 4 digits we must drop the last digit, introducing a rounding error of.0004. 3. Intrinsic error. If data is obtained by experimental observation, it is usually not very accurate (say it is known to only 2 or 3 significant figures). Even if our initial data is known exactly, it can only be stored in the computer to a finite (say 8) number of figures. Similarly, binary to decimal conversion gives an error up to half a unit in the last (binary) place. 5

Note that a rounding error in one operation could be regarded as an intrinsic error in the following one. Rounding and intrinsic errors can be made much worse by subtraction of nearly equal numbers. This is disastrous, e.g. given two 8-decimal accurate numbers 1.2345678 and 1.2345679, their difference(.00000001) has only one significant figure correct. That is to say, the relative error actual error actual number (= ) is greatly increased, although the actual error is still about the same. Usually it is the relative error which is important, e.g. if we are wrong by 1/2 a meter in measuring a room size it is quite serious, but not if we are measuring the distance between Toronto and Ottawa. Another example:- the root of a quadratic is usually given by x = b+ b 2 4ac; this leads to heavy cancellation if 4ac b 2 (in that 2a 2c case it is much better to use x = b+, which is mathematically b 2 4ac equivalent to the usual formula...but not numerically). CLASS EXERCISE (i) If the three numbers 12.91,.3281, and 1001., all accurate to 4 significant figures (i.e. error up to.5 in 4th figure counting from first non-zero figure in going from left to right), are added together, what is the maximum error in the result? (ii) Solve x 2 +100.0001x+.01 = 0 using 5 sig figs. (a) by x = b+ b 2 4ac (b) by x = 2a 2c b+ b 2 4ac Generally, suppose two numbers x and y are represented with errors ǫ and η, i.e. their representations x,y are given by x = x + ǫ and y = y + η; then x + y = x + y + ǫ + η + r a, i.e. error in sum = sum of separate errors + new rounding errors. For multiplication, x y = xy+ǫy+ηx+r m (neglecting term ǫη), hence error in product = ǫy +ηx+r m, hence relative error = error = ǫ + η + rm = sum xy x y xy of relative errors + relative rounding error. 4. Truncation error. This is the type of error which arises when we 6

use an approximate formula to represent some mathematical function or operation such as integration, e.g.(i) if we represent sin x by x x3 + x5 3! 5! there will be an error about x7, due to the chopping off (truncation) 7! of the infinite series. (ii) When we represent 1 0 f(x) by.1 9 r=0 f(.1r) there will beanerror. This kind of error isalso oftencalled discretization error. SEC. 3. PROPOGATION OF ERROR. If a numerical process is repeated many times (say n), the error produced at an early stage will likely be magnified. If it grows in proportion to the actual solution the process is usually said to be stable, but if it grows faster than the true solution it is unstable. Such a situation is usually disastrous, as the error will eventually become greater than the actual result itself. IGNORE REST OF THIS SECTION An Example. Suppose we wish to compute, for n = 0,1,2,...,5 the integral y n = 1 0 x n dx (1) x+5 One way to do this is to derive a recurrence relation connecting y n for different values of n, as follows:- y n +5y n 1 = 1 0 x n +5x n 1 dx = x+5 1 0 x n 1 dx = 1 n and y 0 = [ln(x+5)] 1 0 = ln6 ln5 = 1.791760 1.609438 =.182322 (3) Using the recurrence relation 2, and the value obtained for y 0, and working to 3 decimal places, we get:- y 1 = 1 5y 0 = 1.910 =.090; y 2 = 1 2 5y 1 =.050 (2) y 3 = 1 3 5y 2 =.083; y 4 = 1 4 5y 3 =.165 This is obviously quite wrong since y n > 0 for all n. 7

The reason for this totally wrong result is that the error in y n 1 (due to rounding) is multiplied by -5 at each stage. Thus after 4 stages error ( 5) 4 [error in y 0 ] = +5 4 (.3 10 3 ) =.187 (apart from additional rounding errors at each stage), whereas true result =.034. The problem can be avoided by working backwards, starting with (say) n = 10 and using y n 1 = 1 5n yn 5 ; and taking y 10 = 0; N.B. y 10 1 0 x10 5.5 dx = 1 11 5.5 =.017 We get y 9 = 1 50 0 =.020; y 8 = 1 45 y 9 5 =.022.004 =.018 y 7 = 1 40.018 5 =.025.004 =.021; y 6 = 1 35.021 5 y 5 = 1 30.025 =.033.005 =.028 5 Calculating y 5 by a different method we may proceed thus:- =.029.004 =.025 y 5 = 1 0 x 5 6 x+5 = (y 5) 5 5 y [with x+5 = y] = 6 5 (y 5... 5 5 ) dy y Integrating each term separately gives y 5 =.0284, in good agreement with the previous method by recurrence relations. The reason the backward recurrence relation method is so accurate is that the errors are divided by 5 at each stage; thus they are rapidly diminished. E.g. the error.02 at n = 10 is reduced by a factor 1 5 5 at n = 5 i.e. it =.02 3125 =.000006 SEC. 4. ILL-CONDITIONING. In real life we usually start with inaccurate data. In some problems the errors in the data may be greatly magnified by the computational process, even if we work to infinite precision, e.g. if we are solving a set of simultaneous equations, and the determinant of the coefficients is small compared to the coefficients themselves, our solution is probably rubbish. (In 2 dimensions, 8

this corresponds to finding the intersection of two nearly parallel lines; a small change in slope of the lines means a large change in the position of their intersection). This situation is known as ill-conditioning. A similar situation arises in finding roots of polynomials; sometimes a small change in the coefficients produces a large change in the computed roots. 9

EXERCISE 1. QU. 1. Find graphically (i.e. using a graph to help you know what to do) the derivative of e x cosx at x =.1 N.B. Measure x in radians QU. 2. Find graphically 1 0 e x2 cosx, by splitting the range of x into 10 equal steps of length.1 each, and adding the areas of the resulting rectangles. Give a bound to the error in your solution. QU. 3. Find the smaller root of x 2 +0.4002x+.00008 = 0 using (i) x = b+ b 2 4ac and (ii) x 2c = 2a b+, b 2 4ac rounding all numbers obtained at each step to 3 significant figures (NOT the same as 3 decimal places). Compare the results with the true solution x = -.0002 QU. 4 Write a program to add together the 1000 numbers.1,.101,.102,.103,...,1.098,1.099 (a) using single precision (i.e. REAL in Fortran) (b) using double precision (i.e. DOUBLE or REAL*8 in Fortran)) What is the effect of rounding error in (a)? 10

CHAPTER 2. REVIEW OF CALCULUS. Although numerical methods do not use algebra or calculus directly, the formulas used rely heavily on a number of definitions of the calculus for their derivation. Some of these will now be reviewed. Derivative Suppose that as x gets closer and closer to a value a ( tends to a ), then the function G(x) ges closer and closer to a value g ( tends to g ). In this case we say that the Limit of the function G(x) as x a (x tends to a) is g; or more concisely: lim G(x) = g x a This leads to the definition of the Derivative: f (x) = df dx = lim f x 0 x = Limit as x approaches 0, or as change in x gets smaller and smaller of change in f change in x 11

B (0) T B A C (1) B (2) (2) C (1) C (0) P Q (2) Q (1) Q (0) 12

df dx = Limit as x 0, i.e. as Q(n) P, of CB(n) (n = 0,1,2,...) AC (n) As PQ (n) gets smaller and smaller, the chords AB (n) (n=0,1,2,...) get closer to the tangent to the curve at A. So, df TC(0) = slope of tangent = dx AC (0) Example Suppose f = x n, then (x+ x) n x n lim x 0 x df dx = lim f x 0 x = = lim xn +nx n 1 x+n(n 1)x n 2 ( x) 2 +... x n x = lim x 0 (nxn 1 +n(n 1)x n 2 x+...) = nx n 1 The Integral of f = f(x)dx is defined as F(x) where F (x) or df = f. dx e.g. [ ] x n = xn+1 d (for then x n+1 n+1 dx n+1 = (n+1) x (n+1) 1 = x n ). n+1 Also b a f(x)dx = area under curve y = f(x) betweeen x=a and x=b. y=f(x) a b 13

DERIVATIVES OF SIMPLE FUNCTIONS. Function y x n dy dx y nx n 1 e x e x sin(x) cos(x) f(g(x)) cos(x) sin(x) df dg dg dx (1) e.g. (1+x 2 ) 2 2(1+x 2 ).2x uv u dv dx +vdu dx CLASS EXERCISE. Find the derivative of the following functions;- (i)x 3.5 (ii) 1 x (iii) sin(3x) (iv) xsin(x). (v) sin(1+x 2 ) 14

EXERCISE 2 Find the derivatives of the following functions: 1)x 5 x 2) 1 x x 3)cos(5x) 4)xe x 5) cos(2+x 3 ) 15