Compute the behavior of reality even if it is impossible to observe the processes (for example a black hole in astrophysics).

Transcription

1 1 Introduction Read sections 1.1, , 1.2.6, 1.3.8, 1.3.9, 1.4. Review questions , , The subject of Scientific Computing is to simulate the reality. Simulation is the representation and emulation of a physical system or a physical process with the aid of a computer (the computational model). Hence, Scientific Computing is an integration of mathematics (numerical analysis), computer science, and engineering sciences. What can be done if we have a computational model of a process? Compute the behavior of reality even if it is impossible to observe the processes (for example a black hole in astrophysics). Make experiments considerably faster and cheaper (for example the development of integrated circuits, simulated crash tests for cars). The whole process of the solution of an applied problem can be subdivided into a number of subproblems: 1. Develop a mathematical model for the system under consideration. This consists most often of a system of equations and inequalities. 2. Develop algorithms for the numerical solution of the equations describing the mathematical model. This is conveniently considered being the numerical model. 3. Implement the resulting algorithms in a computer program. This should be an equivalent description of the numerical model. 4. Run the program on a computer. Because of compiler and hardware dependencies (for instance rounding errors), the results may differ from what is expected from the numerical model. 5. Represent the results in a form which is easily comprehensible. This is often done by visualizing the results graphically. 6. Interpret and validate the results. If necessary, repeat some or all of the previous steps. The first step is usually called mathematical modeling. Even if it is called mathematical, it is only the language which is mathematical. Indeed, mathematical modeling is a part of the engineering science which deals with the process under consideration. But it really important that the scientist or engineer who will develop the numerical methods takes part in this process. Usually there are many different ways of describing a process, but their ability to be efficiently approximated numerically may differ considerably. Numerical analysis consists of the steps two and three of the solution process. This is what we consider during the lectures. 1

2 The general strategy for the solution of a mathematical model consists of a replacement of harder problems by easier ones which are solvable on a computer and whose solution comes close to that of the original hard problem: Replace an infinite process by a finite one. Replace differential expressions by algebraic ones. Replace nonlinear models by linear ones. After the replacement we obtain only an approximate problem such that the solution is also only an approximate one. This is not necessarily a big problem because the complete solution process (starting with the mathematical formulation and ending with the solution representation) includes already a number of approximations: Before the real computation starts: model errors; measurement errors; results of previous computations which will be used in the present model. During computations: rounding errors; discretization errors; truncation errors during iterative processes. The accuracy of the final result depends on all these sources of errors: (i) The uncertainty in the input data may be amplified because of the properties of the mathematical model. (ii) Perturbations during the computations may be amplified because of the properties of the algorithm. Example 1.1. We want to compute the surface of the earth by using the formula Among the approximations are, for example: The earth is considered to be a sphere. A 4πr 2 The value for the radius is obtained by measurements and previous computations. The value for π requires the truncation of an infinite process. 2

3 The input data as well as the numerical computations lead to rounding errors. The problem can even become much harder if one tries to apply numerical algorithms. Example 1.2. The following identity holds true: 1 a b a c a b a c b We take for example a 1000, b 0 001, and c If we evaluate both expressions on a small hand-held computer with eight digits accuracy, then we obtain the following results: Using the left expression: 0 0. Using the right expression: Which result is more correct? The problem is that different algorithms for identical mathematical models may lead to large errors. For those of you who are curios: In MATLAB both expressions yield because of the double precision arithmetic used. Example 1.3. Consider the following expression: W b 6 a 2 11a 2 b 2 b 6 121b b 8 a This formula was implemented on an IBM 4381 computer using the Fortran programming language and the VS Fortran system in different precisions. Taking the input data a and b , the programs provided the following results: precision rounding unit result single double extended b The exact result is W Absolute And Relative Errors In order to be able to provide some estimations of the accuracy of the input data and the results of certain computations, it is usual to use the notion of the absolute and relative errors of a number, a vector, a matrix, and so on. The general rules are: absolute error approximate value true value relative error absolute error true value 3

4 The last equation can also be rewritten in the more convenient form approximate value true value 1 relative error Because the true value is not available, it is only possible to estimate, or bound, the errors. It is common to use often statements like three digits of accuracy (which amounts to a bound for the relative error of 10 3 ) or two decimals (which provides a bound for the absolute error of 10 2 ). Let x denote the true value and ˆx an approximation to x. Then the formulas are given by e x ˆx x (absolute error) r x e x x ˆx x x (relative error) Uncertainties of measurements are often described by expressions of the type x ˆx E x. This is meant to denote the estimate e x E x. Similarly, a statement that ˆx has d correct digits means that r x 10 d. Example 1.4. The following table illustrates the different notions. exact x π approximation ˆx absolute error e x relative error r x Example 1.5. Assume that we want to compute the sine of a value. Unfortunately, there is only a hand-held computer available which knows the four basic arithmetic operations, only. Therefore, we try to approximate the value of x sina for a given value of a. We know from analysis that Hence, we apply the approximation x sina a e x ˆx x max ã 0 5 a 3 6 a ˆx a 6 How large are the errors if we know that a o? It can be estimated that Similarly, the relative error can be estimated by So the error is much less than 0.1%! a 3 ã r x

5 1.2 Input Data and Computational Errors We consider a typical example. Assume that we want to compute the value of a scalar function for a given argument. We use the following notation: x true input data, f x true function value (provided by an analytic formula), ˆx approximate input data, ˆf approximate function evaluation (the algorithm used on the computer). Then the true result would be y f x. But what we really obtain is ŷ ˆf ˆx. So the absolute error amounts to e y ŷ y ˆf ˆx f x ˆf ˆx f ˆx computational error f ˆx f x error propagation The error propagation is a property of the problem at hand. In order to bound the error propagation the mathematical model must be constructed appropriately. The computational error is a property of the algorithm and the hardware (compiler, computer, computing environment, etc). While we do not have much influence on the latter, the correct design of the algorithm is the key to a successful solution of our problem. 1.3 Computational Errors We will observe two types of computational errors: truncation (or, discretization) errors and rounding errors. In order to analyze these errors we assume that even ˆx is given without errors, that is, we neglect the presence of errors in the input data. truncation error is the difference between the true result (for the actual input) and the result that would be produced by a given algorithm using exact arithmetic. It is due to approximations such as truncating an infinite series, replacing derivatives by finite differences, or terminating an iterative sequence before convergence. Rounding error is the difference between the result produced by a given algorithm using exact arithmetic and the result produced by the same algorithm using finite-precision, rounded arithmetic. Example 1.6. The derivative of a function f x at a point x is defined by f f x h f x x lim h 0 h 5

6 This infinite process must be approximated by a finite one. If h is sufficiently small, we expect the expression f a h f a D h f a h to be a good approximation of the derivative f a. We would even expect that, if h becomes smaller and smaller, this approximation becomes better and better. This intuitive behavior can be justified more rigorously. Assume that we write down the Taylor expansion of f at a: f a h f a h f a Inserting this expression in the definition of D h, we arrive at D h f a h 2 2 f a f a h O h 2 2 So we would expect that the error reduces proportional to h. Lets see what is happening on a real computer. Let us take the example f x e x and a 1. Then we have f 1 e The table below shows the results if we are using a calculator with eight digits of accuracy. h D h error The plot in Figure 1 shows the error curve. It is clearly seen that it consists of two parts. The above derive formula characterizes the truncation error. It is dominating as long as the step size h is not too small. Otherwise, the rounding error dominates. For completeness, both curves are added to the plot. The curve indicates that the rounding error behaves like O h 1. This is indeed expected. The difference f a h f a is computed up to a rounding error e in the order of magnitude of the machine accuracy. Therefore, the contribution of the rounding error can be estimated by e h O h Sensitivity and Conditioning Here we are interested in an estimation of the propagated error due to errors in the input data. This property is called sensitivity (if we think of the qualitative behavior). A quantification of 6

7 Computational error total truncation rounding error step size Figure 1: Error plot for the difference approximations in Example 1.6 the property is called conditioning (or, condition number) of a problem. The condition number is defined as the relation between the errors in the input data and those of the results: cond relative error of the result relative error in the input data The condition number is the factor by which errors in the input data are amplified because of the properties of the algorithm. A problem is called well-conditioned if the condition number is of moderate size. Otherwise, the problem is called ill-conditioned. Ill-conditioning is very often a severe problem in real life applications. If we consider our example of computing a function value of a scalar function, the condition number can be represented as follows: cond f ˆx f x f x ˆx x x y y x x x f x f x Here, we have used the estimation y x f x. In practice, the exact condition number is 7

8 usually not available. Instead, an estimation cond will be used such that we have the estimate relative error of the result cond relative error in the input data Example 1.7. Let us consider two simple examples: (i) f x x. A simple computation gives cond x 2 x x Consequently, the computation of the square root is very well conditioned. The relative error in the result is approximately halved. (ii) Consider the computation of f x tanx at x π 2. The derivative is given by f x 1 tan 2 x. This leads to a condition number cond x 1 tan 2 x tanx This condition number indicates that we will lose at least five digits. Since our input data has only five digits of accuracy, we expect that the result is completely unreliable. The following table shows that even the sign is not reliable: 1 2 ˆx tan ˆx Once the mathematical model is given, we do not any longer have any influence on its error propagation (or, with other words, its sensitivity). Let us now apply a numerical algorithm for solving it. Ideally, the final numerical result should not be much worse than the uncertainty which is already present in the input data. Therefore, an algorithm is called numerically stable if the computational errors are not much larger than the errors which are present because of the errors in the input data. Stability of an algorithm is, in some sense, analogous to the conditioning of a problem. We are interested in estimating the accuracy of a computed result. This means that we want to know how close the computed result comes to the true solution of the problem. From the discussion above, we find the following conclusions: Stability alone does not guarantee accuracy. Accuracy depends on conditioning of the problem as well as stability of the algorithm. Inaccuracy can result from applying a stable algorithm to an ill-conditioned problem or an unstable algorithm to a well-conditioned problem. Applying a stable algorithm to a well-conditioned problem yields an accurate solution. 8

9 1.5 A Few Words About Floating-Point Operations Computation on a computer means computation with finite precision. Almost all arithmetic operations lead to a rounding error. The representation of real numbers by floating point numbers leads to a common rounding error for all numbers as long as the numbers are not too large in magnitude (which is called overflow) or too small in absolute value (which is called underflow). The largest bound on the relative error is the accuracy of the computer. It is sometimes also called the machine epsilon ε mach. Today, the computer arithmetic is very much standardized by an ANSI-IEEE standard. The machine precision is (IEEE single precision), and (IEEE double precision), respectively. MATLAB uses internally IEEE double precision. But you should be aware of the fact that the numerical results can differ if you run your program on different hardware, even if all claim to use ANSI-IEEE arithmetic! According to its definition, the machine accuracy is the smallest rounding error in all arithmetic operations. But the more important question is how this errors propagate if very many (billions!!) of operations are carried out. For individual arithmetic operations, we have the following properties: Addition and subtraction ˆx ŷ x e x y e y x y e x e y The absolute errors in these operations will be added. This can lead to very large relative error if the final result x y is close to zero. This effect is called cancellation of leading digits. Consider as an example x , y Both values have a relative error of If we subtract these numbers, x y has a relative error of %! Multiplication ˆxŷ x 1 r x y 1 r y xy 1 r x 1 r y xy 1 r x r y Division Here, the relative errors are added. Hence, the multiplication is a very innocent operation with respect to error propagation. ˆx ŷ x y 1 r x 1 r y x y 1 r x r y The division has the same properties as the multiplication. Example 1.8. As an illustrative example of how to avoid catastrophic error propagation by cancellation, consider the solution of a quadratic equation ax 2 bx c 0. The well-known solution formula reads b b x 2 4ac 1 2 2a 9

10 Assume that there are two real roots. If b is positive, then the computation of the first root is subject to cancellation. If b is negative, the second root is subject to cancellation errors. The idea for constructing a better algorithm is to use Vieta s theorem: x 1 x 2 If b happens to be negative, compute x 1 by the former expression and then x 2 by x 2 c a c ax 1 If b is positive, the role of x 1 and x 2 will be exchanged. 1.6 Numerical Software High-quality mathematical software is available for solving most commonly occurring problems in scientific computing. The use of sophisticated, professionally written software has many advantages. We can concentrate ourselves on our problem. There will be no need to bother with highly difficult questions of numerical software architecture. Writing good software is an art. The following table contains an overview of sources for mathematical software. source description free? FMM From book by Forsyth/Malcolm/Moler yes HSL Harwell Subroutine Library partly IMSL Internat. Math. & Stat. Library no KMN From book Kahaner/Moler/Nash yes NAG Numerical Algorithms Group no Netlib Numerical library on the Internet yes NR From book Numerical Recipes no NUMAL Math. Centrum Amsterdam partly SLATEC From US Government Labs partly? SOL Systems Optimization Lab, Stanford no? TOMS ACM Trans. Math. Software yes It became more and more popular to use interactive environments for scientific computing. Such environments provide powerful mathematical capabilities, sophisticated graphics, and a high-level programming language for rapid prototyping. MATLAB is probably the most popular example. It is available for most personal computers, workstations, or even mainframes. There are also some free alternatives available. They include octave, RLaB, and Scilab. Sometimes symbolic computing environments are useful. They do not use numerical methods but abstract computer algebra tools. Prominent examples are Maple and Mathematica. There are also some free versions available. 10