Chapter 1: Introduction and mathematical preliminaries

Transcription

1 Chapter 1: Introduction and mathematical preliminaries Evy Kersalé September 26, 2011

2 Motivation Most of the mathematical problems you have encountered so far can be solved analytically. However, in real-life, analytic solutions are rather rare, and therefore we must devise a way of approximating the solutions.

3 Motivation Most of the mathematical problems you have encountered so far can be solved analytically. However, in real-life, analytic solutions are rather rare, and therefore we must devise a way of approximating the solutions. For example, while 2 1 e x dx has a well known analytic solution, only be solved in terms of special functions and solution e x2 dx can e x3 dx has no analytic

4 Motivation Most of the mathematical problems you have encountered so far can be solved analytically. However, in real-life, analytic solutions are rather rare, and therefore we must devise a way of approximating the solutions. For example, while 2 1 e x dx has a well known analytic solution, only be solved in terms of special functions and solution e x2 dx can e x3 dx has no analytic These integrals exist as the area under the curves y = exp(x 2 ) and y = exp(x 3 ). We can obtain a numerical approximation by estimating this area, e.g. by dividing it into strips and using the trapezium rule. 1 2

5 Definition Numerical analysis is a part of mathematics concerned with

6 Definition Numerical analysis is a part of mathematics concerned with devising methods, called numerical algorithms, for obtaining numerical approximate solutions to mathematical problems;

7 Definition Numerical analysis is a part of mathematics concerned with devising methods, called numerical algorithms, for obtaining numerical approximate solutions to mathematical problems; being able to estimate the error involved.

8 Definition Numerical analysis is a part of mathematics concerned with devising methods, called numerical algorithms, for obtaining numerical approximate solutions to mathematical problems; being able to estimate the error involved. Traditionally, numerical algorithms are built upon the most simple arithmetic operations (+,, and ).

9 Definition Numerical analysis is a part of mathematics concerned with devising methods, called numerical algorithms, for obtaining numerical approximate solutions to mathematical problems; being able to estimate the error involved. Traditionally, numerical algorithms are built upon the most simple arithmetic operations (+,, and ). Interestingly, digital computers can only do these very basic operations. However, they are very fast and, hence, have led to major advances in applied mathematics in the last 60 years.

10 Computer arithmetic: Floating point numbers. Computers can store integers exactly but not real numbers in general. Instead, they approximate them as floating point numbers.

11 Computer arithmetic: Floating point numbers. Computers can store integers exactly but not real numbers in general. Instead, they approximate them as floating point numbers. A decimal floating point (or machine number) is a number of the form ± 0. d 1 d 2... d }{{ k 10 ±n, 0 d } i 9, d 1 0, m where the significand or mantissa m (i.e. the fractional part) and the exponent n are fixed-length integers. (m cannot start with a zero.)

12 Computer arithmetic: Floating point numbers. Computers can store integers exactly but not real numbers in general. Instead, they approximate them as floating point numbers. A decimal floating point (or machine number) is a number of the form ± 0. d 1 d 2... d }{{ k 10 ±n, 0 d } i 9, d 1 0, m where the significand or mantissa m (i.e. the fractional part) and the exponent n are fixed-length integers. (m cannot start with a zero.) In fact, computers use binary numbers (base 2) rather than decimal numbers (base 10) but the same principle applies (see handout).

13 Computer arithmetic: Machine ε. Consider a simple computer where m is 3 digits long and n is one digit long. The smallest positive number this computer can store is and the largest is

14 Computer arithmetic: Machine ε. Consider a simple computer where m is 3 digits long and n is one digit long. The smallest positive number this computer can store is and the largest is Thus, the length of the exponent determines the range of numbers that can be stored.

15 Computer arithmetic: Machine ε. Consider a simple computer where m is 3 digits long and n is one digit long. The smallest positive number this computer can store is and the largest is Thus, the length of the exponent determines the range of numbers that can be stored. However, not all values in the range can be distinguished: numbers can only be recorded to a certain relative accuracy ε.

16 Computer arithmetic: Machine ε. Consider a simple computer where m is 3 digits long and n is one digit long. The smallest positive number this computer can store is and the largest is Thus, the length of the exponent determines the range of numbers that can be stored. However, not all values in the range can be distinguished: numbers can only be recorded to a certain relative accuracy ε. For example, on our simple computer, the next floating point number after 1 = is = The quantity ε machine = 0.01 (machine ε) is the worst relative uncertainty in the floating point representation of a number.

17 Chopping and rounding There are two ways of terminating the mantissa of the k-digit decimal machine number approximating 0.d 1 d 2... d k d k+1 d k n, 0 d i 9, d 1 0,

18 Chopping and rounding There are two ways of terminating the mantissa of the k-digit decimal machine number approximating 0.d 1 d 2... d k d k+1 d k n, 0 d i 9, d 1 0, Chopping: chop off the digits d k+1, d k+2,... to get 0.d 1 d 2... d k 10 n.

19 Chopping and rounding There are two ways of terminating the mantissa of the k-digit decimal machine number approximating 0.d 1 d 2... d k d k+1 d k n, 0 d i 9, d 1 0, Chopping: chop off the digits d k+1, d k+2,... to get 0.d 1 d 2... d k 10 n. Rounding: add 5 10 n (k+1) and chop off the k + 1, k + 2,... digits. (If d k+1 5 we add 1 to d k before chopping.) Rounding is more accurate than chopping.

20 Chopping and rounding There are two ways of terminating the mantissa of the k-digit decimal machine number approximating 0.d 1 d 2... d k d k+1 d k n, 0 d i 9, d 1 0, Chopping: chop off the digits d k+1, d k+2,... to get 0.d 1 d 2... d k 10 n. Rounding: add 5 10 n (k+1) and chop off the k + 1, k + 2,... digits. (If d k+1 5 we add 1 to d k before chopping.) Rounding is more accurate than chopping. Example The five-digit floating-point form of π = is using chopping and using rounding.

21 Chopping and rounding There are two ways of terminating the mantissa of the k-digit decimal machine number approximating 0.d 1 d 2... d k d k+1 d k n, 0 d i 9, d 1 0, Chopping: chop off the digits d k+1, d k+2,... to get 0.d 1 d 2... d k 10 n. Rounding: add 5 10 n (k+1) and chop off the k + 1, k + 2,... digits. (If d k+1 5 we add 1 to d k before chopping.) Rounding is more accurate than chopping. Example The five-digit floating-point form of π = is using chopping and using rounding. Similarly, the five-digit floating-point form of 2/3 = is using chopping and using rounding but that of 1/3 = is using either chopping or rounding.

22 Measure of the error Much of numerical analysis is concerned with controlling the size of errors in calculations. These errors, quantified in two different ways, arise from two distinct sources.

23 Measure of the error Much of numerical analysis is concerned with controlling the size of errors in calculations. These errors, quantified in two different ways, arise from two distinct sources. Let p be the result of a numerical calculation and p the exact answer (i.e. p is an approximation to p). We define two measures of the error,

24 Measure of the error Much of numerical analysis is concerned with controlling the size of errors in calculations. These errors, quantified in two different ways, arise from two distinct sources. Let p be the result of a numerical calculation and p the exact answer (i.e. p is an approximation to p). We define two measures of the error, Absolute error: E = p p

25 Measure of the error Much of numerical analysis is concerned with controlling the size of errors in calculations. These errors, quantified in two different ways, arise from two distinct sources. Let p be the result of a numerical calculation and p the exact answer (i.e. p is an approximation to p). We define two measures of the error, Absolute error: E = p p Relative error: E r = p p / p (provided p 0) which takes into consideration the size of the value.

26 Measure of the error Much of numerical analysis is concerned with controlling the size of errors in calculations. These errors, quantified in two different ways, arise from two distinct sources. Let p be the result of a numerical calculation and p the exact answer (i.e. p is an approximation to p). We define two measures of the error, Absolute error: E = p p Relative error: E r = p p / p (provided p 0) which takes into consideration the size of the value. Example If p = 2 and p = 2.1, the absolute error E = 10 1 ; if p = and p = , E = 10 4 is smaller; and if p = and p = , E = 10 2 is larger but in all three cases the relative error remains the same, E r =

27 Round-off errors Caused by the imprecision of using finite-digit arithmetic in practical calculations (e.g. floating point numbers).

28 Round-off errors Caused by the imprecision of using finite-digit arithmetic in practical calculations (e.g. floating point numbers). Example The 4-digit representation of x = 2 = is x = = Using 4-digit arithmetic, we can evaluate x 2 = , due to round-off errors.

29 Round-off errors Caused by the imprecision of using finite-digit arithmetic in practical calculations (e.g. floating point numbers). Example The 4-digit representation of x = 2 = is x = = Using 4-digit arithmetic, we can evaluate x 2 = , due to round-off errors. Round-off errors can be minimised by reducing the number of arithmetic operations, particularly those that magnify errors.

30 Magnification of the error.

31 Magnification of the error. Computers store numbers to a relative accuracy ε. Thus, the true value of a floating point number x could be anywhere between x (1 ε) and x (1 + ε).

32 Magnification of the error. Computers store numbers to a relative accuracy ε. Thus, the true value of a floating point number x could be anywhere between x (1 ε) and x (1 + ε). Now, if we add two numbers together, x + y, the true value lies in the interval (x + y ε( x + y ), x + y + ε( x + y )).

33 Magnification of the error. Computers store numbers to a relative accuracy ε. Thus, the true value of a floating point number x could be anywhere between x (1 ε) and x (1 + ε). Now, if we add two numbers together, x + y, the true value lies in the interval (x + y ε( x + y ), x + y + ε( x + y )). Thus, the absolute error is the sum of the errors in x and y, E = ε( x + y ) but the relative error of the answer is E r = ε x + y x + y.

34 Magnification of the error. Computers store numbers to a relative accuracy ε. Thus, the true value of a floating point number x could be anywhere between x (1 ε) and x (1 + ε). Now, if we add two numbers together, x + y, the true value lies in the interval (x + y ε( x + y ), x + y + ε( x + y )). Thus, the absolute error is the sum of the errors in x and y, E = ε( x + y ) but the relative error of the answer is x + y E r = ε x + y. If x and y both have the same sign the relative accuracy remains equal to ε, but if the have opposite signs the relative error will be larger.

35 Magnification of the error. Computers store numbers to a relative accuracy ε. Thus, the true value of a floating point number x could be anywhere between x (1 ε) and x (1 + ε). Now, if we add two numbers together, x + y, the true value lies in the interval (x + y ε( x + y ), x + y + ε( x + y )). Thus, the absolute error is the sum of the errors in x and y, E = ε( x + y ) but the relative error of the answer is x + y E r = ε x + y. If x and y both have the same sign the relative accuracy remains equal to ε, but if the have opposite signs the relative error will be larger. This magnification becomes particularly significant when two very close numbers are subtracted.

36 Magnification of the error: Example Recall: the exact solutions of the quadratic equation ax 2 + bx + c = 0 are x 1 = b b 2 4ac 2a, x 2 = b + b 2 4ac. 2a

37 Magnification of the error: Example Recall: the exact solutions of the quadratic equation ax 2 + bx + c = 0 are x 1 = b b 2 4ac 2a, x 2 = b + b 2 4ac. 2a Using 4-digit rounding arithmetic, solve the quadratic equation x x + 1 = 0, with roots x and x

38 Magnification of the error: Example Recall: the exact solutions of the quadratic equation ax 2 + bx + c = 0 are x 1 = b b 2 4ac 2a, x 2 = b + b 2 4ac. 2a Using 4-digit rounding arithmetic, solve the quadratic equation x x + 1 = 0, with roots x and x The discriminant b 2 4ac = 3840 = is close to b = 62. Thus, x 1 = ( )/2 = 124.0/2 = 62, with a relative error E r = , but x 2 = ( )/2 = , with a much larger relative error E r =

39 Magnification of the error: Example Recall: the exact solutions of the quadratic equation ax 2 + bx + c = 0 are x 1 = b b 2 4ac 2a, x 2 = b + b 2 4ac. 2a Using 4-digit rounding arithmetic, solve the quadratic equation x x + 1 = 0, with roots x and x The discriminant b 2 4ac = 3840 = is close to b = 62. Thus, x 1 = ( )/2 = 124.0/2 = 62, with a relative error E r = , but x 2 = ( )/2 = , with a much larger relative error E r = Similarly, division by small numbers (or equivalently, multiplication by large numbers) magnifies the absolute error, leaving the relative error unchanged.

40 Truncation errors Caused by the approximations in the computational algorithm itself. (An algorithm only gives an approximate solution to a mathematical problem, even if the arithmetic is exact.)

41 Truncation errors Caused by the approximations in the computational algorithm itself. (An algorithm only gives an approximate solution to a mathematical problem, even if the arithmetic is exact.) Example Calculate the derivative of a function f (x) at the point x 0. Recall the definition of the derivative df (x0) = lim dx h 0 f (x 0 + h) f (x 0). h

42 Truncation errors Caused by the approximations in the computational algorithm itself. (An algorithm only gives an approximate solution to a mathematical problem, even if the arithmetic is exact.) Example Calculate the derivative of a function f (x) at the point x 0. Recall the definition of the derivative df (x0) = lim dx h 0 f (x 0 + h) f (x 0). h However, on a computer we cannot take h 0 (there exists a smallest positive floating point number), so h must take a finite value.

43 Truncation errors Caused by the approximations in the computational algorithm itself. (An algorithm only gives an approximate solution to a mathematical problem, even if the arithmetic is exact.) Example Calculate the derivative of a function f (x) at the point x 0. Recall the definition of the derivative df (x0) = lim dx h 0 f (x 0 + h) f (x 0). h However, on a computer we cannot take h 0 (there exists a smallest positive floating point number), so h must take a finite value.

44 Truncation errors Caused by the approximations in the computational algorithm itself. (An algorithm only gives an approximate solution to a mathematical problem, even if the arithmetic is exact.) Example Calculate the derivative of a function f (x) at the point x 0. Recall the definition of the derivative df (x0) = lim dx h 0 f (x 0 + h) f (x 0). h However, on a computer we cannot take h 0 (there exists a smallest positive floating point number), so h must take a finite value. Using Taylor s theorem, f (x 0 + h) = f (x 0) + hf (x 0) + h 2 /2 f (ξ), where x 0 < ξ < x 0 + h.

45 Truncation errors Caused by the approximations in the computational algorithm itself. (An algorithm only gives an approximate solution to a mathematical problem, even if the arithmetic is exact.) Example Calculate the derivative of a function f (x) at the point x 0. Recall the definition of the derivative df (x0) = lim dx h 0 f (x 0 + h) f (x 0). h However, on a computer we cannot take h 0 (there exists a smallest positive floating point number), so h must take a finite value. Using Taylor s theorem, f (x 0 + h) = f (x 0) + hf (x 0) + h 2 /2 f (ξ), where x 0 < ξ < x 0 + h. Therefore, f (x 0 + h) f (x 0) h = hf (x 0) + h 2 /2 f (ξ) h = f (x 0)+ h 2 f (ξ) f (x 0)+ h 2 f (x 0).

46 Truncation errors Caused by the approximations in the computational algorithm itself. (An algorithm only gives an approximate solution to a mathematical problem, even if the arithmetic is exact.) Example Calculate the derivative of a function f (x) at the point x 0. Recall the definition of the derivative df (x0) = lim dx h 0 f (x 0 + h) f (x 0). h However, on a computer we cannot take h 0 (there exists a smallest positive floating point number), so h must take a finite value. Using Taylor s theorem, f (x 0 + h) = f (x 0) + hf (x 0) + h 2 /2 f (ξ), where x 0 < ξ < x 0 + h. Therefore, f (x 0 + h) f (x 0) h = hf (x 0) + h 2 /2 f (ξ) h = f (x 0)+ h 2 f (ξ) f (x 0)+ h 2 f (x 0). Using a finite value of h leads to a truncation error of size h/2 f (x 0).

47 Truncation errors: Example continued Clearly, the truncation error h/2 f (x 0) decreases with decreasing h.

48 Truncation errors: Example continued Clearly, the truncation error h/2 f (x 0) decreases with decreasing h. The absolute round-off error in f (x 0 + h) f (x 0) is 2ε f (x 0) and that in the derivative (f (x 0 + h) f (x 0))/h is 2ε/h f (x 0). So, the round-off error increases with decreasing h.

49 Truncation errors: Example continued Clearly, the truncation error h/2 f (x 0) decreases with decreasing h. The absolute round-off error in f (x 0 + h) f (x 0) is 2ε f (x 0) and that in the derivative (f (x 0 + h) f (x 0))/h is 2ε/h f (x 0). So, the round-off error increases with decreasing h. The relative accuracy of the calculation of f (x 0) (i.e. the sum of the relative truncation and round-off errors) is E r 1 h h E r = h f 2 f + 2ε f h f, h m h which has a minimum, min(e r ) = 2 ε ff / f, for h m = 2 ε f /f.