Numerical Mathematical Analysis

Transcription

1 Numerical Mathematical Analysis Numerical Mathematical Analysis Catalin Trenchea Department of Mathematics University of Pittsburgh September 20, 2010 Numerical Mathematical Analysis Math 1070

2 Numerical Mathematical Analysis Outline 1 Introduction, matlab notes 2 Chapter 1: Taylor polynomials 3 Chapter 2: Error and Computer Arithmetic 4 Chapter 4: Interpolation 5 Chapter 5: Numerical integration and differentiation 6 Chapter 3: Rootfinding 7 Chapter 4: Approximation of functions Numerical Mathematical Analysis Math 1070

3 > Introduction Numerical Analysis Numerical Analysis: This refers to the analysis of mathematical problems by numerical means, especially mathematical problems arising from models based on calculus. Effective numerical analysis requires several things: An understanding of the computational tool being used, be it a calculator or a computer. An understanding of the problem to be solved. Construction of an algorithm which will solve the given mathematical problem to a given desired accuracy and within the limits of the resources (time, memory, etc) that are available. 1. Introduction, Matlab notes Math 1070

4 > Introduction This is a complex undertaking. Numerous people make this their lifes work, usually working on only a limited variety of mathematical problems. Within this course, we attempt to show the spirit of the subject. Most of our time will be taken up with looking at algorithms for solving basic problems such as rootfinding and numerical integration; but we will also look at the structure of computers and the implications of using them in numerical calculations. We begin by looking at the relationship of numerical analysis to the larger world of science and engineering. 1. Introduction, Matlab notes Math 1070

5 > Introduction > Science Traditionally, engineering and science had a two-sided approach to understanding a subject: the theoretical and the experimental. More recently, a third approach has become equally important: the computational. Traditionally we would build an understanding by building theoretical mathematical models, and we would solve these for special cases. For example, we would study the flow of an incompressible irrotational fluid past a sphere, obtaining some idea of the nature of fluid flow. But more practical situations could seldom be handled by direct means, because the needed equations were too difficult to solve. Thus we also used the experimental approach to obtain better information about the flow of practical fluids. The theory would suggest ideas to be tried in the laboratory, and the experimental results would often suggest directions for a further development of theory. 1. Introduction, Matlab notes Math 1070

6 > Introduction > Science With the rapid advance in powerful computers, we now can augment the study of fluid flow by directly solving the theoretical models of fluid flow as applied to more practical situations; and this area is often referred to as computational fluid dynamics. At the heart of computational science is numerical analysis; and to effectively carry out a computational science approach to studying a physical problem, we must understand the numerical analysis being used, especially if improvements are to be made to the computational techniques being used. 1. Introduction, Matlab notes Math 1070

7 > Introduction > Science Mathematical Models A mathematical model is a mathematical description of a physical situation. By means of studying the model, we hope to understand more about the physical situation. Such a model might be very simple. For example, A = 4πRe, 2 R e 6, 371 km is a formula for the surface area of the earth. How accurate is it? 1 First, it assumes the earth is sphere, which is only an approximation. At the equator, the radius is approximately 6,378 km; and at the poles, the radius is approximately 6,357 km. 2 Next, there is experimental error in determining the radius; and in addition, the earth is not perfectly smooth. Therefore, there are limits on the accuracy of this model for the surface area of the earth. 1. Introduction, Matlab notes Math 1070

8 > Introduction > Science An infectious disease model For rubella measles, we have the following model for the spread of the infection in a population (subject to certain assumptions). ds dt = asi di = asi bi dt dr dt = bi In this, s, i, and r refer, respectively, to the proportions of a total population that are susceptible, infectious, and removed (from the susceptible and infectious pool of people). All variables are functions of time t. The constants can be taken as a = , b = Introduction, Matlab notes Math 1070

9 > Introduction > Science Mathematical Models The same model works for some other diseases (e.g. flu), with a suitable change of the constants a and b. Again, this is an approximation of reality (and a useful one). But it has its limits. Solving a bad model will not give good results, no matter how accurately it is solved; and the person solving this model and using the results must know enough about the formation of the model to be able to correctly interpret the numerical results. 1. Introduction, Matlab notes Math 1070

10 > Introduction > Science The logistic equation This is the simplest model for population growth. Let N(t) denote the number of individuals in a population (rabbits, people, bacteria, etc). Then we model its growth by N (t) = cn(t), t 0, N(t 0 ) = N 0 The constant c is the growth constant, and it usually must be determined empirically. Over short periods of time, this is often an accurate model for population growth. For example, it accurately models the growth of US population over the period of 1790 to 1860, with c = Introduction, Matlab notes Math 1070

11 > Introduction > Science The predator-prey model Let F (t) denote the number of foxes at time t; and let R(t) denote the number of rabbits at time t. A simple model for these populations is called the Lotka-Volterra predator-prey model: dr = a[1 bf (t)]r(t) dt df = c[ 1 + dr(t)]f (t) dt with a, b, c, d positive constants. If one looks carefully at this, then one can see how it is built from the logistic equation. In some cases, this is a very useful model and agrees with physical experiments. Of course, we can substitute other interpretations, replacing foxes and rabbits with other predator and prey. The model will fail, however, when there are other populations that affect the first two populations in a significant way. 1. Introduction, Matlab notes Math 1070

12 > Introduction > Science 1. Introduction, Matlab notes Math 1070

13 > Introduction > Science Newton s second law Newtons second law states that the force acting on an object is directly proportional to the product of its mass and acceleration, F ma With a suitable choice of physical units, we usually write this in its scalar form as F = ma Newtons law of gravitation for a two-body situation, say the earth and an object moving about the earth is then m d2 r(t) dt 2 = Gm m e r(t) 2 r(t) r(t) with r(t) the vector from the center of the earth to the center of the object moving about the earth. The constant G is the gravitational constant, not dependent on the earth; and m and m e are the masses, respectively of the object and the earth. 1. Introduction, Matlab notes Math 1070

14 > Introduction > Science This is an accurate model for many purposes. But what are some physical situations under which it will fail? When the object is very close to the surface of the earth and does not move far from one spot, we take r(t) to be the radius of the earth. We obtain the new model m d2 r(t) dt 2 = mgk with k the unit vector directly upward from the earths surface at the location of the object. The gravitational constant g. = 9.8 meters/second 2 Again this is a model; it is not physical reality. 1. Introduction, Matlab notes Math 1070

15 > Matlab notes Matlab notes Matlab designed for numerical computing. Strongly oriented towards use of arrays, one and two dimensional. Excellent graphics that are easy to use. Powerful interactive facilities; and programs can also be written in it. It is a procedural language, not an object-oriented language. It has facilities for working with both Fortran and C language programs. 1. Introduction, Matlab notes Math 1070

16 > Matlab notes At the prompt in Unix or Linux, type Matlab. Or click the Red Hat, then DIVMS, then Mathematics, then Matlab. Run the demo program (simply type demo). Then select one of the many available demos. To seek help on any command, simply type help command or use the online Help command. To seek information on Matlab commands that involve a given word in their description, type lookfor word Look at the various online manuals available thru the help page. 1. Introduction, Matlab notes Math 1070

17 > Matlab notes MATLAB is an interactive computer language. For example, to evaluate use y = 6 4x + 7x 2 3x x + 2 y = 6 4 x + 7 x x 3 x 5 + 3/(x + 2); There are many built-in functions, e.g. exp(x), cos(x), x, log(x) The default arithmetic used in MATLAB is double precision (16 decimal digits and magnitude range : ) and real. However, complex arithmetic appears automatically when needed. sqrt(-4) results in an answer of 2i. 1. Introduction, Matlab notes Math 1070

18 > Matlab notes The default output to the screen is to have 4 digits to the right of the decimal point. To control the formatting of output to the screen, use the command format. The default formatting is obtained using format short To obtain the full accuracy available in a number, you can use format long The commands format short e format long e will use scientific notation for the output. Other format options are also available. 1. Introduction, Matlab notes Math 1070

19 > Matlab notes SEE plot trig.m MATLAB works very efficiently with arrays, and many tasks are best done with arrays. For example, plot sin x and cos x on the interval 0 x 10. t = 0 :.1 : 10; x = cos(t); y = sin(t); plot(t, x, t, y, LineWidth, 4) Introduction, Matlab notes Math 1070

20 > Matlab notes The statement t = a : h : b; with h > 0 creates a row vector of the form t = [a, a + h, a + 2h,...] giving all values a + jh that are less than b. When h is omitted, it is assumed to be 1. Thus n = 1 : 5 creates the row vector n = [1, 2, 3, 4, 5] 1. Introduction, Matlab notes Math 1070

21 > Matlab notes Arrays creates a row vector of length 3. creates the square matrix b = [1, 2, 3] A = [1 2 3; 4 5 6; 7 8 9] A = Spaces or commas can be used as delimiters in giving the components of an array; and a semicolon will separate the various rows of a matrix. For a column vector, b = [1 3 6] results in the column vector Introduction, Matlab notes Math 1070

22 > Matlab notes Array Operations Addition: Do componentwise addition. results in the answer A = [1, 2; 3, 2; 6, 1]; B = [2, 3; 3, 2; 2, 2]; C = A + B; C = Multiplication by a constant: Multiply the constant times each component of the array. D = 2 A; results in the answer D = Introduction, Matlab notes Math 1070

23 > Matlab notes Array Operations Matrix multiplication: This has the standard meaning. E = [1, 2; 2, 1; 3, 2] F = [2, 1, 3; 1, 2, 3]; G = E F ; results in the answer 1 2 [ G = A nonstandard notation: results in the computation [ H = ] [ + H = 3 + F ; ] = ] = [ Introduction, Matlab notes Math 1070 ]

24 > Matlab notes Componentwise operations Matlab also has component-wise operations for multiplication, division and exponentiation. These three operations are denoted by using a period to precede the usual symbol for the operation. With we have The expression a = [1 2 3]; b = [2, 1 4]; a. b = [2 2 12] a./b = [ ] a.ˆ3 = [1 8 27] 2.ˆa = [2 4 8] b.ˆa = [2 1 64] y = 6 4x + 7x 2 3x x + 2 can be evaluated at all of the elements of an array x using the command y = 6 4 x + 7 x. x 3 x.ˆ5 + 3./(x + 2); The output y is then an array of the same size as x. 1. Introduction, Matlab notes Math 1070

25 > Matlab notes Special arrays A = zeros(2, 3) produces an array with 2 rows and 3 columns, with all components set to zero, [ ] B = ones(2, 3) produces an array with 2 rows and 3 columns, with all components set to 1, [ ] eye(3) results in the 3 3 identity matrix, Introduction, Matlab notes Math 1070

26 > Matlab notes Array functions There are many MATLAB commands that operate on arrays, we include only a very few here. For a vector x, row or column, of length n, we have the following functions. max(x) = maximum component of x min(x) = minimum component of x abs(x) = vector of absolute values of components of x sum(x) = sum of the components of x norm(x)= x x n 2 1. Introduction, Matlab notes Math 1070

27 > Matlab notes Script files A list of interactive commands can be stored as a script file. For example, store t = 0 :.1 : 10; x = cos(t); y = sin(t); plot(t, x, t, y) with the file name plot trig.m. Then to run the program, give the command plot trig The variables used in the script file will be stored locally, and parameters given locally are available for use by the script file. 1. Introduction, Matlab notes Math 1070

28 > Matlab notes Functions To create a function, we proceed similarly, but now there are input and output parameters. Consider a function for evaluating the polynomial p(x) = a 1 + a 2 x + a 3 x a n x n 1 MATLAB does not allow zero subscripts for arrays. The following function would be stored under the name polyeval.m. The coefficients {a j } are given to the function in the array named coeff, and the polynomial is to be evaluated at all of the components of the array x. 1. Introduction, Matlab notes Math 1070

29 > Matlab notes Functions function value = polyeval(x,coeff); % % function value = polyeval(x,coeff) % % Evaluate a polynomial at the points given in x. % The coefficients are to be given in coeff. % The constant term in the polynomial is coeff(1). n = length(coeff) value = coeff(n)*ones(size(x)); for i = n-1:-1:1 value = coeff(i) + x.*value; end >> polyeval(3,[1,2]) yields n=2 ans = 7 1. Introduction, Matlab notes Math 1070

30 > 1. Taylor polynomials Taylor polynomials 1 Taylor polynomials 1 The Taylor polynomial 2 Error in Taylor s polynomial 3 Polynomial evaluation 1. Taylor polynomials Math 1070

31 > 1. Taylor polynomials > 1.1 The Taylor polynomial Let f(x) be a given function, for example e x, sin x, log(x). The Taylor polynomial mimics the behavior of f(x) near x = a: Example T (x) f(x), for all x close to a. Find a linear polynomial p 1 (x) for which { p1 (a) = f(a), p 1 (a) = f (a). p 1 is uniquely given by p 1 (x) = f(a) + (x a)f (a). The graph of y = p 1 (x) is tangent to that of y = f(x) at x = a. 1. Taylor polynomials Math 1070

32 > 1. Taylor polynomials > 1.1 The Taylor polynomial Example Let f(x) = e x, a = 0. Then p 1 (x) = 1 + x y=p_1(x) Figure: Linear Taylor Approximation e x 1. Taylor polynomials Math 1070

33 > 1. Taylor polynomials > 1.1 The Taylor polynomial Example Find a quadratic polynomial p 2 (x) to approximate f(x) near x = a. Since p 2 (x) = b 0 + b 1 x + b 2 x 2 we impose three conditions on p 2 (x) to determine the coefficients. To better mimic f(x) at x = a we require p 2 is uniquely given by p 2 (a) = f(a) p 2 (a) = f (a) p 2 (a) = f (a) p 2 (x) = f(a) + (x a)f (a) (x a)2 f (a). 1. Taylor polynomials Math 1070

34 > 1. Taylor polynomials > 1.1 The Taylor polynomial Example Let f(x) = e x, a = 0. Then p 2 (x) = 1 + x x y=p_2(x) 0.5 y=e^x 0.25 y=p_1(x) Figure: Linear and quadratic Taylor Approximation e x (see eval exp simple.m) 1. Taylor polynomials Math 1070

35 > 1. Taylor polynomials > 1.1 The Taylor polynomial Let p n (x) be a polynomial of degree n that mimics the behavior of f(x) at x = a. We require p (j) n (a) = f (j) (a), j = 0, 1,..., n where f (j) the j th derivative of f(x). The Taylor polynomial of degree n for the function f(x) at point a: p n (x) = f(a) + (x a)f (a) + = (x a)2 f (a) ! (x a)n f (n) (a) n! n (x a) j f (j) (a). (3.1) j! j=0 Recall the notations: f (0) (a) = f(a) and the factorial : { 1, j = 0 j! = j (j 1) 2 1, j = 1, 2, 3, 4, Taylor polynomials Math 1070

36 > 1. Taylor polynomials > 1.1 The Taylor polynomial Example Let f(x) = e x, a = 0. Since f (j) (x) = e x, f (j) (0) = 1, for all j 0, then p n (x) = 1 + x + 1 2! x n! xn = n j=0 x j j! (3.2) For a fixed x, the accuracy improves as the degree n increases. For a fixed degree n, the accuracy improves as x gets close to a = Taylor polynomials Math 1070

37 > 1. Taylor polynomials > 1.1 The Taylor polynomial Table. Taylor approximations to e x x p 1 (x) p 2 (x) p 3 (x) e x Taylor polynomials Math 1070

38 > 1. Taylor polynomials > 1.1 The Taylor polynomial Example Let f(x) = e x, a arbitrary, not necessarily 0. Since f (j) (x) = e x, f (j) (a) = e a, for all j 0, then p n (x; a) = e a ( 1 + (x a) + 1 2! (x a) n! (x a)n ) n = e a (x a) j j! j=0 1. Taylor polynomials Math 1070

39 > 1. Taylor polynomials > 1.1 The Taylor polynomial Example Let f(x) = ln(x), a = 1. Since f(1) = ln(1) = 0, and then the Taylor polynomial is f (j) (x) = ( 1) j 1 (j 1)! 1 x j f (j) (1) = ( 1) j 1 (j 1)! p n (x) = (x 1) 1 2 (x 1) (x 1) ( 1) (n 1) 1 (x 1)n n n ( 1) j 1 = (x 1) j j j=1 1. Taylor polynomials Math 1070

40 > 1. Taylor polynomials > 1.1 The Taylor polynomial 1 y=p_1(x) y=p_3 y=ln(x) 0.5 y=p_2(x) Figure: Taylor approximations of ln(x) about x = 1 (see plot log.m) 1. Taylor polynomials Math 1070

41 > 1. Taylor polynomials > 1.2 The error in Taylor s polynomial Theorem (Lagrange s form) Assume f C n [α, β], x [α, β]. The remainder R n (x) f(x) p n (x), or error in approximating n (x a) j f(x) by p n (x) = f (j) (a) satisfies j! j=0 R n (x) = (x a)n+1 f (n+1) (c x ), α x β (3.3) (n + 1)! where c x is an unknown point between a and x. [Exercise.] Derive the formal Taylor series for f(x) = ln(1 + x) at a = 0, and determine the range of positive x for which the series represents the function. Hint: f (k) (x) = ( 1) k 1 (k 1)! 1 (1+x) k, f (k) (0) = ( 1) k 1 (k 1)!, 0 ln(1 + x) = P n k=1 ( 1) k 1 xk k + ( 1)n n+1 x n+1 (1+cx) n+1 1+cx x 1, if x [0, 1]. 1. Taylor polynomials Math 1070

42 > 1. Taylor polynomials > 1.2 The error in Taylor s polynomial Example Let f(x) = e x and a = 0. n x j Recall that p n (x) = j! = 1 + x + 1 2! x n! xn. j=0 The approximation error is e x p n (x) = xn+1 (n + 1)! ec, n 0 (3.4) with c (0, x). [Exercise.] It can be proved that for each fixed x, lim R n(x) = 0, i.e., e x = lim n n n k=0 x k k! = k=0 x k k!. (See the case x 1). For each fixed n, R n becomes larger as x moves away from Taylor polynomials Math 1070

43 > 1. Taylor polynomials > 1.2 The error in Taylor s polynomial 0.75 y=r_1(x) 0.5 y=r_1(x) 0.25 y=r_2(x) y=r_3(x) y=r_3(x) y=r_4(x) 1 y=r_4(x) y=r_2(x) Figure: Error in Taylor polynomial approx. to e x (see plot exp simple.m) 1. Taylor polynomials Math 1070

44 > 1. Taylor polynomials > 1.2 The error in Taylor s polynomial Example Let x = 1 in (3.4), so from (3.2): and from (3.4) e p n (1) = ! + 1 3! n! R n (1) = e p n (1) = Since e < 3 and e 0 e c e 1 we have 1 (n + 1)! R n(1) e c (n + 1)!, 0 < c < 1 e (n + 1)! < 3 (n + 1)! 1. Taylor polynomials Math 1070

45 > 1. Taylor polynomials > 1.2 The error in Taylor s polynomial Suppose we want to approximate e by p n (1) with R n (1) We have to take n 12 to guarantee 3 (n + 1)! 10 9, i.e., to have p 12 as a sufficiently good approximation to e. 1. Taylor polynomials Math 1070

46 > 1. Taylor polynomials > 1.2 The error in Taylor s polynomial Exercise Expand 1 + h in powers of h, then compute If f(x) = x 1 2, then f (x) = 1 2 x 1 2, f (x) = 1 4 x 3 2, f (x) = 3 8 x 5 2, h = h 1 8 h h3 ɛ 5 2, 1 < ɛ < 1 + h, if h > 0. Let h = Then = Since 1 < ɛ < 1 + h, the absolute error does not exceed 1 16 h3 ɛ 5 2 < = and the numerical value is correct to all 15 decimal places shown. 1. Taylor polynomials Math 1070

47 > 1. Taylor polynomials > 1.2 The error in Taylor s polynomial Approximations and remainder formulae e x = 1 + x + x2 2! + + xn n! + xn+1 (n + 1)! ec (3.5) sin(x) = x x3 3! + x5 x2n 1 +( 1)n 1 5! (2n 1)! +( 1)n x2n+1 cos(c) (3.6) (2n+1)! cos(x) = 1 x2 2! + x4 x2n +( 1)n 4! (2n)! +( 1)n+1 x2n+2 cos(c) (3.7) (2n+2)! 1 1 x = 1 + x + x2 + + x n + xn+1 1 x, x 1 (3.8) ( ) ( ) ( ) (1 + x) α α α = 1 + x + x 2 α + + x n (3.9) 1 2 n ( ) α + x n+1 (1 + c) α n 1, α R n + 1 Recall that c is between 0 and x, and the binomial coefficients are ( ) α α(α 1) (α κ + 1) = κ κ! 1. Taylor polynomials Math 1070

48 > 1. Taylor polynomials > 1.2 The error in Taylor s polynomial Example Approximate cos(x) for x π 4 with an error no greater than Since cos(c) 1 and we must have R 2n+1 (x) x2n+2 (2n + 2)! 10 5 ( π ) 2n+2 4 (2n + 2)! 10 5 which is satisfied when n 3. Hence cos(x) 1 x2 2! + x4 4! x6 6! 1. Taylor polynomials Math 1070

49 > 1. Taylor polynomials > 1.2 The error in Taylor s polynomial Indirect construction of a Taylor polynomial approximation Remark From (3.5), by replacing x with t 2, we obtain e t2 =1 t 2 + t4 2! t6 3! + + ( 1)n x 2n n! t 2 c 0. + ( 1)(n+1) x 2n+2 e c, (n + 1)! (3.10) If we attempt to construct the Taylor approximation directly, the derivatives of e t2 become quickly too complicated. 1. Taylor polynomials Math 1070

50 > 1. Taylor polynomials > 1.2 The error in Taylor s polynomial Indirect construction of a Taylor polynomial approximation From (3.8) we can easily get: t 0 dx 1 x = ln(1 t) (3.12) = t 0 ( 1 + x + x x n) dx t ( t t n + 1 tn+1 ( ln(1 t) = t t n + 1 tn+1 where c t is a number between 0 and t, 1 t 1. Theorem (Integral Mean Value Theorem) 0 ) 1 1 c x n+1 1 x dx, t 0 x n+1 dx ) 1 1 c t t n+2 n + 2, (3.11) Let w(x) b a nonnegative integrable function on (a, b), and f C[a, b]. Then at least one point c [a, b] for which b a f(x)w(x)dx = f(c) b a w(x)dx (3.12) 1. Taylor polynomials Math 1070

51 > 1. Taylor polynomials > 1.2 The error in Taylor s polynomial Infinite Series By rearranging the terms in (3.8) we obtain the sum of a infinite geometric series 1 + x + x x n = 1 xn+1, x 1. (3.13) 1 x For x < 1, letting n we obtain the infinite geometric series 1 1 x = 1 + x + x2 + x 3 + = x j, x 1. (3.14) j=0 1. Taylor polynomials Math 1070

52 > 1. Taylor polynomials > 1.2 The error in Taylor s polynomial Infinite Series Definition The infinite series is convergent if the partial sums j=0 c j S n = form a convergent sequence, i.e., n c j, n 0 j=0 S = lim n S n and we then write S = c j j=0 1. Taylor polynomials Math 1070

53 > 1. Taylor polynomials > 1.2 The error in Taylor s polynomial Infinite Series For the infinite series (3.14) with x 1, the partial sums are given by (3.13): S n = 1 xn+1 1 x S n x <1 n 1 1 x S n is divergent when x > 1 What happens when x = 1? 1. Taylor polynomials Math 1070

54 > 1. Taylor polynomials > 1.2 The error in Taylor s polynomial Infinite Series Definition Assume that f(x) has derivatives of any order at x = a. The infinite series (x a) j f (j) (a) j! j=0 is called the Taylor series expansion of the function f(x) about the point x = a. The partial sum n (x a) j f (j) (a) j! j=0 is simply the Taylor polynomial p n (x). 1. Taylor polynomials Math 1070

55 > 1. Taylor polynomials > 1.2 The error in Taylor s polynomial Infinite Series If the sequence {p n (x)} has the limit f(x), i.e. the error tends to zero as n ( ) f(x) p n (x) = 0 then we can write lim n f(x) = (x a) j f (j) (a) j! j=0 1. Taylor polynomials Math 1070

56 > 1. Taylor polynomials > 1.2 The error in Taylor s polynomial Infinite series Actually, it can be shown that the errors terms in (3.5)-(3.9) and (3.11) tend to 0 as n for suitable values of x. Hence the Taylor expansions e x = j=0 sin x = x j j!, < x < j=0 ( 1) j x 2j+1, < x < (2j + 1)! ( 1) j x 2j cos x =, < x < (3.15) (2j)! j=0 ( ) (1 + x) α α = x j, 1 < x < 1 j j=0 ln(1 t) = t j j, 1 x < 1 j=0 1. Taylor polynomials Math 1070

57 > 1. Taylor polynomials > 1.2 The error in Taylor s polynomial Infinite series Definition Infinite series of the form are called power series. j=0 a j (x a) j (3.16) They can arise from Taylor s formulae or some other ways. Their convergence can be examined directly. Theorem (Comparison criterion) Assume the series (3.16) converges for some value x 0. Then the series (3.16) converges for all x satisfying x a x 0 a. Theorem (Quotient criterion) For the series (3.16), assume that the limit R = lim a n+1 n a n exists. Then for x satisfying x a < 1 R, the series (3.16) converges to a limit S(x). When R = 0, the series (3.16) converges for any x R. 1. Taylor polynomials Math 1070

58 > 1. Taylor polynomials > 1.2 The error in Taylor s polynomial Infinite series Example Consider the power series in (3.15). Letting t = x 2, we obtain the series j=0 ( 1) j t j (2j)! (3.17) Applying the quotient criterion with a j = ( 1)j (2j)! we find R = 0, so the series (3.17) converges for any value of t, hence the series in the formula (3.15) converges for any value of x. 1. Taylor polynomials Math 1070

59 > 1. Taylor polynomials > 1.3 Polynomial evaluation Consider the evaluation of the polynomial p(x) = 3 4x 5x 2 6x 3 + 7x 4 8x 5 1 simplest method: compute each term independently, i.e., c x k or c x k yielding = 15 multiplications 2 a more efficient method: compute each power of x using the preceding one: x 3 = x(x 2 ), x 4 = x(x 3 ), x 5 = x(x 4 ) (3.18) Since each term takes two multiplications for k > 1, the result will be 3 nested multiplication: = 9 multiplications p(x) = 3 + x( 4 + x( 5 + x( 6 + x(7 8x)))) with only 5 multiplications 1. Taylor polynomials Math 1070

60 > 1. Taylor polynomials > 1.3 Polynomial evaluation Consider now the general polynomial of degree n: p(x) = a 0 + a 1 x + + a n x n, a n 0 If we use the 2 nd method, with the powers of x computed as in (3.18), the number of multiplications in evaluating p(x) is 2n 1. For the nested multiplication, we write and evaluate p(x) in the form p(x) = a 0 + x(a 1 + x(a x(a n 1 + a n x) )) (3.19) using only n multiplications, saving about 50% over the 2 n d method. All methods use n additions. 1. Taylor polynomials Math 1070

61 > 1. Taylor polynomials > 1.3 Polynomial evaluation Example Evaluate the Taylor polynomial p 5 (x) for ln(x) about a = 1. A general formula is (3.11), with t replaced by (x 1), yielding p 5 (x) = (x }{{ 1 } ) 1 2 (x 1) (x 1)3 1 4 (x 1)4 + 1 (x 1)5 5 w ( = w 1 + w ( 12 ( 13 ( + w + w )))) w. 1. Taylor polynomials Math 1070

62 > 1. Taylor polynomials > 1.3 Polynomial evaluation A more formal algorithm than (3.19) Suppose we want to evaluate p(x) at number z. Let define the sequence of coefficients b i as: b n = a n (3.20) b n 1 = a n 1 + zb n (3.21) b n 2 = a n 2 + zb n 1 (3.22). (3.23) b 0 = a 0 + zb 1 (3.24) Now the nested multiplication is the Horner Method: p(z) = a 0 + z(a 1 + z(a z(a n 1 + a }{{} n z) )) b n }{{} b n 1 }{{} }{{} } {{ } b 0 1. Taylor polynomials Math 1070 b 1 b 2

63 > 1. Taylor polynomials > 1.3 Polynomial evaluation Hence p(z) = b 0 With the coefficients from (3.20), define the polynomial It can be shown that q(x) = b 1 + b 2 x + b 3 x b n x n 1 p(x) = b 0 + (x z)q(x), i.e., q(x) is the quotient from dividing p(x) by x z, and b 0 is the remainder. Remark This property we will use it later for polynomial rootfinding method to reduce the degree of a polynomial when a root z has been found, since then b 0 = 0 and p(x) = (x z)q(x). 1. Taylor polynomials Math 1070

64 > 2. Error and Computer Arithmetic Numerical analysis is concerned with how to solve a problem numerically, i.e., how to develop a sequence of numerical calculations to get a satisfactory answer. Part of this process is the consideration of the errors that arise in these calculations, from the errors in the arithmetic operations or from other sources. 2. Error and Computer Arithmetic Math 1070

65 > 2. Error and Computer Arithmetic Computers use binary arithmetic, representing each number as a binary number: a finite sum of integer powers of 2. Some numbers can be represented exactly, but others, such as 1 10, 1 100, ,..., cannot. For example, = has an exact representation in binary (base 2), but does not. And, of course, there are the transcendental numbers like π that have no finite representation in either decimal or binary number system. 2. Error and Computer Arithmetic Math 1070

66 > 2. Error and Computer Arithmetic Computers use 2 formats for numbers. Fixed-point numbers are used to store integers. Typically, each number is stored in a computer word of 32 binary digits (bits) with values of 0 and 1. at most 2 32 different numbers can be stored. If we allow for negative numbers, we can represent integers in the range 2 31 x , since there are 2 32 such numbers. Since , the range for fixed-point numbers is too limited for scientific computing. = always get an integer answer. the numbers that we can store are equally spaced. very limited range of numbers. Therefore they are used mostly for indices and counters. An alternative to fixed-point, floating-point numbers approximate real numbers. the numbers that we can store are NOT equally spaced. wide range of variably-spaced numbers that can be representeed exactly. 2. Error and Computer Arithmetic Math 1070

67 > 2. Error and Computer Arithmetic > 2.1 Floating-point numbers Numbers must be stores and used for arithmetic operations. Storing: 1 integer format 2 floating-point format Definition (decimal Floating-point representation) Let consider x 0 written in decimal system. Then it can be written uniquely as where σ = +1 or 1 is the sign e is an integer, the exponent 1 x < 10, the significand or mantissa Example ( = ( }{{} ) 10 2 ) x x = σ x 10 e (4.1) σ = +1, the exponent e = 2, the significand x = Error and Computer Arithmetic Math 1070

68 > 2. Error and Computer Arithmetic > 2.1 Floating-point numbers The decimal floating-point representation of x R is given in (4.1), with limitations on the 1 number of digits in mantissa x 2 size of e Example Suppose we limit 1 number of digits in x to e 99 We say that a computer with such a representation has a four-digit decimal floating point arithmetic. This implies that we cannot store accurately more than the first four digits of a number; and even the fourth digit may be changed by rounding. What is the next smallest number bigger than 1? What is the next smallest number bigger than 100? What are the errors and relative errors? What is the smallest positive number? 2. Error and Computer Arithmetic Math 1070

69 > 2. Error and Computer Arithmetic > 2.1 Floating-point numbers Definition (Floating-point representation of a binary number x) Let consider x written in binary format. Analogous to (4.1) x = σ x 2 e (4.2) where σ = +1 or 1 is the sign e is an integer, the exponent x is a binary fraction satisfying (1) 2 x < (10) 2 (in decimal:1 x < 2) For example, if x = ( ) 2 then σ = +1, e = 4 = (100) 2 and x = ( ) 2 2. Error and Computer Arithmetic Math 1070

70 > 2. Error and Computer Arithmetic > 2.1 Floating-point numbers The floating-point representation of a binary number x is given by (4.2) with a restriction on 1 number of digits in x: the precision of the binary floating-point representation of x 2 size of e The IEEE floating-point arithmetic standard is the format for floating point numbers used in almost all computers. the IEEE single precision floating-point representation of x has a precision of 24 binary digits, and the exponent e is limited by 126 e 127: where, in binary x = σ (1.a 1 a 2... a 23 ) 2 e ( ) 2 e ( ) 2 the IEEE double precision floating-point representation of x has a precision of 53 binary digits, and the exponent e is limited by 1022 e 1023: x = σ (1.a 1 a 2... a 52 ) 2 e 2. Error and Computer Arithmetic Math 1070

71 > 2. Error and Computer Arithmetic > 2.1 Floating-point numbers the IEEE single precision floating-point representation of x has a precision of 24 binary digits, and the exponent e is limited by 126 e 127: stored on 4 bytes (32 bits) b 1 }{{} σ x = σ (1.a 1 a 2... a 23 ) 2 e b 2 b 3 b 9 }{{} E=e+127 b 10 b 11 b 32 }{{} x the IEEE double precision floating-point representation of x has a precision of 53 binary digits, and the exponent e is limited by 1022 e 1023: stored on 8 bytes (64 bits) x = σ (1.a 1 a 2... a 52 ) 2 e b 1 }{{} σ b 2 b 3 b 12 }{{} E=e+1023 b 13 b 14 b 64 }{{} x 2. Error and Computer Arithmetic Math 1070

72 > 2. Error and Computer Arithmetic > Accuracy of floating-point representation Epsilon machine How accurate can a number be stored in the floating point representation? How can this be measured? (1) Machine epsilon Machine epsilon For any format, the machine epsilon is the difference between 1 and the next larger number that can be stored in that format. In single precision IEEE, the next larger binary number is }{{} a 23 ( cannot be stored exactly) Then the machine epsilon in single precision IEEE format is 2 23 = i.e., we can store approximately 7 decimal digits of a number x in decimal format. 2. Error and Computer Arithmetic Math 1070

73 > 2. Error and Computer Arithmetic > Accuracy of floating-point representation Then the machine epsilon in double precision IEEE format is 2 52 = IEEE double-precision format can be used to store approximately 16 decimal digits of a number x in decimal format. MATLAB: machine epsilon is available as the constant eps. 2. Error and Computer Arithmetic Math 1070

74 > 2. Error and Computer Arithmetic > Accuracy of floating-point representation Another way to measure the accuracy of floating-point format: (2) look for the largest integer M such that any integer x such that 0 x M can be stored and represented exactly in floating point form. If n is the number of binary digits in the significand x, all integers less or equal to ( ( ) 2 2 n 1 = (n 1)) 2 n 1 ( 1 1 ) 2 = n n 1 = 2 n 1 2 can be represented exactly. In IEEE single precision format M = 2 24 = and all 7-digit decimal integers will store exactly. In IEEE double precision format M = 2 53 = and all 15-digit decimal integers and most 16 digit ones will store exactly. 2. Error and Computer Arithmetic Math 1070

75 > 2. Error and Computer Arithmetic > Rounding and chopping Let say that a number x has a significand x = 1.a 1 a 2... a n 1 a n a n+1 but the floating-point representation may contain only n binary digits. Then x must be shortened when stored. Definition We denote the machine floating-point version of x by fl(x). 1 truncate or chop x to n binary digits, ignoring the remaining digits 2 round x to n binary digits, based on the size of the part of x following digit n: (a) if digit n + 1 is 0, chop x to n digits (b) if digit n + 1 is 1, chop x to n digits and add 1 to the last digit of the result 2. Error and Computer Arithmetic Math 1070

76 > 2. Error and Computer Arithmetic > Rounding and chopping It can be shown that where ɛ is a small number depending of x. (a) If chopping is used (b) If rounding is used Characteristics of chopping: fl(x) = x (1 + ɛ) (4.3) 2 n+1 ɛ 0 (4.4) 2 n ɛ 2 n (4.5) 1 the worst possible error is twice as large as when rounding is used 2 the sign of the error x fl(x) is the same as the sign of x The worst of the two: no possibility of cancellation of errors. Characteristics of rounding: 1 the worst possible error is only half as large as when chopping is used 2 More important: the error x fl(x) is negative for only half the cases, which leads to better error propagation behavior 2. Error and Computer Arithmetic Math 1070

77 > 2. Error and Computer Arithmetic > Rounding and chopping For single precision IEEE floating-point rounding arithmetic (there are n = 24 digits in the significand): 1 chopping ( rounding towards zero ): 2 23 ɛ 0 (4.6) 2 standard rounding: 2 24 ɛ 2 24 (4.7) For double precision IEEE floating-point rounding arithmetic: 1 chopping: 2 52 ɛ 0 (4.8) 2 rounding: 2 53 ɛ 2 53 (4.9) 2. Error and Computer Arithmetic Math 1070

78 > 2. Error and Computer Arithmetic >Conseq. for programming of floating-point arithm. Numbers that have finite decimal expressions may have infinite binary expansions. For example (0.1) 10 = ( ) 2 Hence (0.1) 10 cannot be represented exactly in binary floating-point arithmetic. Possible problems: Run into infinite loops pay attention to the language used: Fortran and C have both single and double precision, specify double precision constants correctly MATLAB does all computations in double precision 2. Error and Computer Arithmetic Math 1070

79 > 2. Error and Computer Arithmetic > 2.2 Errors: Definitions, Sources and Examples Definition The error in a computed quantity is defined as Error(x A ) = x T - x A where x T =true value, x A =approximate value. This is called also absolute error. The relative error Rel(x A ) is a measure off error related to the size of the true value Rel(x A ) = true error value = x T x A x T For example, for the approximation π = 22 7 we have x T = π = and x A = 22 7 = ( ) 22 Error = π ( ) 22 Rel = π π = = Error and Computer Arithmetic Math 1070

80 > 2. Error and Computer Arithmetic > 2.2 Errors: Definitions, Sources and Examples The notion of relative error is a more intrinsic error measure. 1 The exact distance between 2 cities: x 1 T = 100km and the measured distance is x 1 A = 99km Error ( x 1 ) T = x 1 T x 1 A = 1km Rel ( x 1 ) Error ( x 1 ) T = T = 0.01 = 1% x 1 T 2 The exact distance between 2 cities: x 2 T distance is x 2 A = 1km = 2km and the measured Error ( x 2 ) T = x 2 T x 2 A = 1km Rel ( x 2 ) Error ( x 2 ) T = T = 0.5 = 50% x 2 T 2. Error and Computer Arithmetic Math 1070

81 > 2. Error and Computer Arithmetic > 2.2 Errors: Definitions, Sources and Examples Definition (significant digits) The number of significant digits in x A is number of its leading digits that are correct relative to the corresponding digits in the true value x T More precisely, if x A, x T are written in decimal form; compute the error x T = a 1 a 2.a 3 a m a m+1 a m+2 x T x A = b m+1 b m+2 If the error is 5 units in the (m + 1) th digit of x T, counting rightward from the first nonzero digit, then we say that x A has, at least, m significant digits of accuracy relative to x T. Example 1 x A = 0.222, x T = 2 9 : x T = x T x A = Error and Computer Arithmetic Math 1070

82 > 2. Error and Computer Arithmetic > 2.2 Errors: Definitions, Sources and Examples Example 1 x A = , x T = : x T = x T x A = x A = , x T = : x T = x T x A = x A = 22 7 = , x T = π = : x T = x T x A = Error and Computer Arithmetic Math 1070

83 > 2. Error and Computer Arithmetic > Sources of error Errors in a scientific-mathematical-computational problem: 1 Original errors (E1) Modeling errors (E2) Blunders and mistakes (E3) Physical measurement errors (E4) Machine representation and arithmetic errors (E5) Mathematical approximation errors 2 Consequences of errors (F1) Loss-of-significance errors (F2) Noise in function evaluation (F3) Underflow and overflow erros 2. Error and Computer Arithmetic Math 1070

84 > 2. Error and Computer Arithmetic > Sources of error (E1) Modeling errors: the mathematical equations are used to represent physical reality - mathematical model. Malthusian growth model (it can be accurate for some stages of growth of a population, with unlimited resources) N(t) = N 0 e kt, N 0, k 0 where N(t) = population at time t. For large t the model overestimates the actual population: accurately models the growth of US population for 1790 t 1860, with k = , N 0 = 3, 929, 000 e 1790k but considerably overestimates the actual population for Error and Computer Arithmetic Math 1070

85 > 2. Error and Computer Arithmetic > Sources of error (E2) Blunders and mistakes: mostly programming errors test by using cases where you know the solution break into small subprograms that can be tested separately (E2) Physical measurement errors. For example, the speed of light in vacuum c = ( ɛ) cm/sec, ɛ Due to the error in data, the calculations will contain the effect of this observational error. Numerical analysis cannot remove the error in the data, but it can look at its propagated effect in a calculation and suggest the best form for a calculation that will minimize the propagated effect of errors in data 2. Error and Computer Arithmetic Math 1070

86 > 2. Error and Computer Arithmetic > Sources of error (E4) Machine representation and arithmetics errors. For example errors from rounding and chopping. they are inevitable when using floating-point arithmetic they form the main source of errors with some problems (solving systems of linear equations). We will look at the effect of rounding errors for some summation procedures (E4) Mathematical approximation errors: major forms of error that we will look at. For example, when evaluating the integral I = 1 0 e x2 dx, since there is no antiderivative for e x2, I cannot be evaluated explicitly. Instead, we approximate it with a quantity that can be computed. 2. Error and Computer Arithmetic Math 1070

87 > 2. Error and Computer Arithmetic > Sources of error Using the Taylor approximation we can easily evaluate I e x2 1 x 2 + x4 2! x6 3! + x8 4! 1 0 ) (1 x 2 + x4 2! x6 3! + x8 dx 4! with the truncation error being evaluated by (3.10) 2. Error and Computer Arithmetic Math 1070

88 > 2. Error and Computer Arithmetic > Loss-of-significance errors Loss of significant digits Example Consider the evaluation of f(x) = x[ x + 1 x] for an increasing sequence of values of x. x 0 Computed f(x) True f(x) , , Table: results of using a 6-digit decimal calculator As x increases, there are fewer digits of accuracy in the computed value f(x) 2. Error and Computer Arithmetic Math 1070

89 > 2. Error and Computer Arithmetic > Loss-of-significance errors What happens? For x = 100: 100 = }{{}, exact 101 = }{{} rounded where 101 is correctly rounded to 6 significant digits of accuracy. x + 1 x = = while the true value should be The calculation has a loss-of-significance error. Three digits of accuracy in x + 1 = 101 were canceled by subtraction of the corresponding digits in x = 100. The loss of accuracy was a by-product of the form of f(x) and the finite precision 6-digit decimal arithmetic being used 2. Error and Computer Arithmetic Math 1070

90 > 2. Error and Computer Arithmetic > Loss-of-significance errors For this particular f, there is a simple way to reformulate it and avoid the loss-of-significance error: f(x) = x x + 1 x which on a 6 digit decimal calculator will imply the correct answer to six digits. f(100) = Error and Computer Arithmetic Math 1070

91 > 2. Error and Computer Arithmetic > Loss-of-significance errors Example Consider the evaluation of f(x) = 1 cos(x) x 2 for a sequence approaching 0. x 0 Computed f(x) True f(x) Table: results of using a 10-digit decimal calculator 2. Error and Computer Arithmetic Math 1070

92 > 2. Error and Computer Arithmetic > Loss-of-significance errors Look at the calculation when x = 0.01: cos(0.01) = (= ) has nine significant digits of accuracy, being off in the tenth digit by two units. Next 1 cos(0.01) = (= e 05) which has only five significant digits, with four digits being lost in the subtraction. To avoid the loss of significant digits, due to the subtraction of nearly equal quantities, we use the Taylor approximation (3.7) for cos(x) about x = 0: cos(x) = 1 x2 2! + x4 4! x6 6! + R 6(x) R 6 (x) = x8 cos(ξ), ξ unknown number between 0 and x. 8! 2. Error and Computer Arithmetic Math 1070

93 > 2. Error and Computer Arithmetic > Loss-of-significance errors Hence f(x) = 1 ]} {1 x 2 [1 x2 2! + x4 4! x6 6! + R 6(x) = 1 2! x2 4! + x4 6! x6 8! cos(ξ) giving f(0) = 1 2. For x 0.1 x 6 8! cos(ξ) ! Therefore, with this accuracy = f(x) 1 2! x2 4! + x4, x < 0.1 6! 2. Error and Computer Arithmetic Math 1070

94 > 2. Error and Computer Arithmetic > Loss-of-significance errors Remark When two nearly equal quantities are subtracted, leading significant digits will be lost. In the previous two examples, this was easy to recognize and we found ways to avoid the loss of significance. More often, the loss of significance is subtle and difficult to detect, as in calculating sums (for example, in approximating a function f(x) by a Taylor polynomial). If the value of the sum is relatively small compared to the terms being summed, then there are probably some significant digits of accuracy being lost in the summation process. 2. Error and Computer Arithmetic Math 1070

95 > 2. Error and Computer Arithmetic > Loss-of-significance errors Example Consider using the Taylor series approximation for e x to evaluate e 5 : e 5 = 1 + ( 5) + ( 5)2 + ( 5)3 + ( 5) ! 2! 3! 4! Degree Term Sum Degree Term Sum E E E E E E E E E E E E Table. Calculation of e 5 = using four-digit decimal arithmetic 2. Error and Computer Arithmetic Math 1070

96 > 2. Error and Computer Arithmetic > Loss-of-significance errors There are loss-of-significance errors in the calculation of the sum. To avoid the loss of significance is simple in this case: e 5 = 1 e 5 = 1 series for e 5 and form e 5 with a series not involving cancellation of positive and negative terms. 2. Error and Computer Arithmetic Math 1070

97 > 2. Error and Computer Arithmetic > Noise in function evaluation Consider evaluating a continuous function f for all x [a, b]. The graph is a continuous curve. When evaluating f on a computer using floating-point arithmetic (with rounding or chopping), the errors from arithmetic operations cause the graph to cease being a continuous curve. Let look at on [0, 2]. f(x) = (x 1) 3 = 1 + x(3 + x( 3 + x)) 2. Error and Computer Arithmetic Math 1070

98 > 2. Error and Computer Arithmetic > Noise in function evaluation Figure: f(x) = x 3 3x 2 + 3x 1 2. Error and Computer Arithmetic Math 1070

99 > 2. Error and Computer Arithmetic > Noise in function evaluation Figure: Detailed graph of f(x) = x 3 3x 2 + 3x 1 near x = 1 Here is a plot of the computed values of f(x) for 81 evenly spaced values of x [ , ]. A rootfinding program might consider f(x) t have a very large number of solutions near 1 based on the many sign changes!!! 2. Error and Computer Arithmetic Math 1070

100 > 2. Error and Computer Arithmetic > Underflow and overflow errors From the definition of floating-point number, there are upper and lower limits for the magnitudes of the numbers that can be expressed in a floating-point form. Attempts to create numbers that are too small underflow errors: the default option is to set the number to zero and proceed that are too large overflow errors: generally fatal errors on most computers. With the IEEE floating-point format, overflow errors can be carried along as having a value of ± or NaN, depending on the context. Usually, an overflow error is an indication of a more significant problem or error in the program. 2. Error and Computer Arithmetic Math 1070

101 > 2. Error and Computer Arithmetic > Underflow and overflow errors Example (underflow errors) Consider evaluating f(x) = x 10 for x near 0. With the IEEE single precision arithmetic, the smallest nonzero positive number expressible in normalized floating point arithmetic is So f(x) is set to zero if m = = x 10 < m x < 10 m = < x < Error and Computer Arithmetic Math 1070

102 > 2. Error and Computer Arithmetic > Underflow and overflow errors Sometimes is possible to eliminate the overflow error by just reformulating the expression being evaluated. Example (overflow errors) Consider evaluating z = x 2 + y 2. If x or y is very large, then x 2 + y 2 might create an overflow error, even though z might be within the floating-point range of the machine. x 1 + ( y 2, x) 0 y x z = ( ) y 1 + x 2, y 0 x y In both cases, the argument of 1 + w 2 has w 1, which will not cause any overflow error (except when z is too large to be expressed in the floating-point format used). 2. Error and Computer Arithmetic Math 1070

103 > 2. Error and Computer Arithmetic > 2.3 Propagation of Error When doing calculations with numbers that contain an error, the result will be effected by these errors. If x A, y A denote numbers used in a calculation, corresponding to x T, y T, we wish to bound the propagated error E = (x T ωy T ) (x A ωy A ) where ω denotes: +,,,. The first technique used to bound E is interval arithmetic. Suppose we know bounds on x T x A and y T y A. Using these bounds and x A ωy A, we look for an interval guaranteed to contain x T ωy T. 2. Error and Computer Arithmetic Math 1070