Numerical Methods - Lecture 2. Numerical Methods. Lecture 2. Analysis of errors in numerical methods

Transcription

1 Numerical Methods - Lecture 1 Numerical Methods Lecture. Analysis o errors in numerical methods

2 Numerical Methods - Lecture Why represent numbers in loating point ormat? Eample 1. How a number can be represented using 5-places? What is the smallest number that can be represented in this ormat? What is the largest number that can be represented in this ormat?

3 Numerical Methods - Lecture 3 Why represent numbers in loating point ormat? Eample. How a number can be represented using 5-places? rounded o chopped Conclusion: The error is smaller than 0.01

4 Numerical Methods - Lecture 4 Calculation o errors True Error o accurate or true value o o Relative True Error o o Calculations: t o o % 100% %

5 Numerical Methods - Lecture 5 Calculation o errors Relative errors o small numbers are large. Compare: t o o % 100% % t o o % 100% % Absolute true errors are the same: o

6 Numerical Methods - Lecture 6 How to keep relative true errors at a constant level? The number can be represented as: sign mantissa 10 eponent or sign mantissa eponent is written as is written as is written as

7 Numerical Methods - Lecture 7 Floating point arithmetic decimal representation Form o a number e m10 an integer eponent sign (-1 or +1) mantissa (1) 10 m<(10) 10 Eample m.5678 e

8 Numerical Methods - Lecture 8 Floating point arithmetic binary representation Form o a number e m an integer eponent sign (0 positive or 1 negative) mantisses (1) m<() Eample: (101) 1 is not represented 0 m e 101

9 Numerical Methods - Lecture 9 What do we gain by using the loating point representation? The rage o the numbers that can be epressed has increased I we use only 5 places to represent a positive number with a positive eponent, the smallest number that can be epressed is 1 and the largest is mantissa eponent The range o numbers that can be represented has increased rom to

10 Numerical Methods - Lecture 10 What do we lose by using a loating point representation? Precision Why? The number will be represented as on ive places: mantissa eponent There will be a round-o error.

11 Numerical Methods - Lecture 11 Test problem 1 1. Write down the number on ive places in a similar manner as that discussed in the previous eample: mantissa eponent. Calculate the true error and the relative true error o round o 3. Compare the results with the previous eample (56.78).

12 Numerical Methods - Lecture 1 Test problem We have 9-bits irst bit represents the sign o the number, second bit represents the sign o the eponent, the net our bits represent the mantissa, the last three bits represent the eponent 0 0 sign eponent sign mantissa eponent Find the number (in decimal), which is presented above

13 Numerical Methods - Lecture 13 Solution o the problem 1 1. The number written on ive places is: mantissa eponent. The true error is 9.78 and the relative true error is % 3. For the number these errors are: 0.0 (smaller) and % (comparable)

14 Numerical Methods - Lecture 14 Solution o the problem is not represented 54 10

15 Numerical Methods - Lecture 15 What is the accuracy ε? For each digital machine epsilon ε is deined as a parameter determining the accuracy o the calculations: t ε N where: N= (in binary), N=10 (in decimal), t is the number o bits representing the mantissa 5 ε is lowered when more bits are allocated to represent the mantissa M Epsilon ε can be regarded as a parameter characterizing the accuracy o the computing machine (small ε = more accurate calculations). Double precision (Fortran) DP

16 Numerical Methods - Lecture 16 What is the accuracy ε? Epsilon ε is the smallest number that when added to produces a number that can be represented as dierent rom Eample: 10-bit word M N w sign eponent mantissa eponent sign the net number 10 mach

17 Numerical Methods - Lecture 17 Single precision in IEEE-754 ormat (Institute o Electrical and Electronics Engineers) 3 bits or single precision sign (s) eponent (e ) mantissa (m) Number e' 17 1 m. s ( 1)

18 Numerical Methods - Lecture 18 Eample Sign (s) Biased Eponent (e ) Mantissa (m) Value s e' m 1 ( )

19 Numerical Methods - Lecture 19 The eponent or the 3-bit IEEE -754 standard 8-bit eponent means 17 e 18 0 e 55 The ied eponent shit is 17 and thereore In act, 1 e 54 e 0 e 55 Numbers and are reserved or special cases Eponent range 16 e 17

20 Numerical Methods - Lecture 0 Representation o special numbers e e 0 55 all zeros all ones e s m Represents 0 zeros zeros 0 1 zeros zeros -0 0 ones zeros 1 ones zeros 0 or 1 ones nonzero NaN

21 1 IEEE-754 Format The largest number The smallest number Epsilon mach

22 Numerical Methods - Lecture Analysis o errors I we do not know the eact value o o we calculate the approimate error as the dierence in the values obtained in the consecutive approimations: n n1 Relative error ε a : a n n1 n

23 Numerical Methods - Lecture 3 Eample For 0.5 ( ) 7e at a) () or h 0. 3 () b) or h ind c) approimate error Solution ' ( ) ( h) h ( ) a) h 0.3 '() ( 0,3) 0,3 () 7e 0,5(,3) 7e 0,3 0,5() 10,65

24 Numerical Methods - Lecture 4 Eample (cont.) b) h 0.15 '() ( 0,15) 0,15 () 7e 0,5(,15) 0,15 7e 0,5() 9,880 c) a n n1 n a 9,880 10,65 9,8800 0,0389 Error 3,89%

25 Numerical Methods - Lecture 5 The relative error as a criterion or ending the iterative procedure I the relative error is less than or equal to a predetermined number then urther iterations are no longer required a s I we require at least m signiicant digits in the result then ε a m

26 Numerical Methods - Lecture 6 Relative error and signiicant digits ε a m h () N/A a m The eact value is 9.514

27 Numerical Methods - Lecture 7 Sources o errors in numerical calculations 1. Input errors (input data errors). Truncation error 3. Round o error Input errors occur when the input data entered into the computer memory are dierent rom the eact values. Truncation errors are errors due to numerical procedures caused by reducing the number o operations. Round o errors are errors that usually can not be avoided. They arise in the course o the calculations, and can be reduced by setting skillully the method and sequence o tasks.

28 Numerical Methods - Lecture 8 Input errors Sources o input errors: the input data are the result o measurement o physical quantities a inite length o binary words and thereore prerounding is needed preliminary rounding o irrational numbers Rounding o numbers that cannot be epressed eactly is accomplished by: chopping rounding o

29 Numerical Methods - Lecture 9 Eample: 3,1415 3, ,1416 chopping rounding o Rounding o introduces smaller error than chopping.

30 Numerical Methods - Lecture 30 Truncation error Caused by the use o the approimate ormula instead o a ull mathematical operation: when calculating the sum o the ininite series when calculating integral, derivative W lim 0 1 F 1 Fd work

31 Numerical Methods - Lecture 31 Taylor series I the unction is continuous and all derivatives ', '',... n eist within the interval [, + h] the value o the unction at the point + h can be calculated as: h h! h 3! h 3 The Maclaurin series is simply the Taylor series about the point =0 0 h 0 0h 0 0 h! h 3 3!

32 Numerical Methods - Lecture 3 Eamples Some eamples o Taylor series which you must have seen cos( ) 1! 4 4! 6 6! sin( ) 3 3! 5 5! 7 7! e 1! 3 3!

33 33 Truncation error in the Taylor series R n h h h h n n n!! ' ' remainder c n h R n n n 1 1 1)! ( ) ( h c Numerical Methods - Lecture

34 Eample The Taylor series or e at point =0 is given by e 1! 3 3! 4 4! 5 5! It can be seen that as the number o terms used increases, the error decreases and hence a better estimate can be ound. Question: How many terms would it require to get an approimation o e 1 with a true error o less than 10-6? 1 e 1 11! Numerical Methods - Lecture 1 3 3! 1 4! 1 5!

35 Numerical Methods - Lecture 35 Solution 0, h 1, ( ) e R n h n1 c ( n n1 1)! R n 0 1 n n1 1! n1 c since c 0 c c 1 h ( n 1 1)! n 1 1 e n 1! R n c 0 ( n e 1)!

36 Numerical Methods - Lecture 36 Solution e ( n 1)! 10 6 true error ( n 1)! 6 10 e ( n 1)! n 9 It means that 9 terms or more are needed to get a true error less than 10-6

37 Numerical Methods - Lecture 37 Arithmetic operations 1. Addition and subtraction To add or subtract two standard loating-point numbers, the eponents should be made equal with an adequate shit o the mantissa. Eample: Add 0, to 0, shit 0, , =0, Answer: We loose some signiicant digits

38 Numerical Methods - Lecture 38 Arithmetic operations. Multiplication Eample: Multiply 0, by 0, Multiply mantissas and add the eponents. 0, , =0, =0, Division Eample: Divide 0, by 0, , /0, =0, All the time we loose some signiicant digits which is the source o the error

39 Numerical Methods - Lecture 39 Sequence o operations (a+b)-c (a-c)+b lack o associativity a(b-c) (ab-ac) lack o distributive property Eample: a= 0, , b=0, , c=0, (a+b)=0, , =0, , =0, (a+b)-c=0, , =0, (a-c)=0, , =0, =0, (a-c)+b=0, , =0,

40 Numerical Methods - Lecture 40 Wilkinson Lemma Rounding o errors occurring during loating point operations are equivalent to substitution o accurate numbers by slightly perturbed numbers on which we act eactly. For individual arithmetical operations: l( 1 ) 1(1 1) (1 ) l( 1 ) 1 (1 3) 1 (1 3) l( 1 / ) 1 (1 4) / 1 /( (1 5)) symbol o operation perormed on loating-point data and ε i

41 Numerical Methods - Lecture 41 Tragic eample o the rounding o error On February 5th, 1991 in Dhahran, Saudi Arabia, 8 American soldiers were killed in the attack o the Iraqi Scud missiles. The deense system Patriot did not detect any assault. Why? The system calculates the area, which should be scanned based on the speed o the object and the last detection. The internal clock was set to measure every 1/10 second and 4-bit word length was assumed. Due to rounding o, the absolute error was s which ater 100 hours amounted to: sec Based on this, the calculated displacement is 687 m. Object is considered outside the range when the displacement is 137 m

42 Numerical Methods - Lecture 4 Useul hints: In the numerical calculations it is advantageous: To solve the same problem by another method, or by the same method, but with a dierent order o operations To solve the problem on slightly perturbed input data

43 y Numerical Methods - Lecture 44 Propagation o errors u(y) unction y = () tangent dy/d u (y) u() u() dy d 0 0 4

44 Numerical Methods - Lecture 45 Total dierential in application to errors For y=( 1,,... n ) where maimum uncertainties 1,,... n are small compared with the values o the variables 1,,... n the maimum uncertainty o y can be epressed as: y y y y n n

45 Numerical Methods - Lecture 46 Eample Estimate the uncertainty o density ρ o a sphere o mass m and radius R ( m, R) (4 m 3)πR 3 absolute error m m R R but m πR R 3m πr relative error 3 m R

46 Numerical Methods - Lecture 47 Errors o arithmetic operations Addition A a a B b b absolute errors A B a b a b a b ( a b) the addition absolute error Thereore, the addition (subtraction) absolute error is equal to the sum o absolute errors o components. ( a b) a b

47 Numerical Methods - Lecture 48 Errors o arithmetic operations the addition relative error ab the subtraction relative error ab a a a a b b b b Subtraction relative error can be large even i the relative errors o minuend and subtrahend are small. Subtracting o nearly equal numbers should be avoided! This phenomenon is called reduction o signiicant digits It is important or the calculation o Newton s dierence quotients approimating derivatives o unctions, roots o a quadratic equation with the dominant actor o the irst power, etc.

48 The concept o zero We lose the precise meaning o the number 0 i we perorm numerical calculations Eact roots are Approimate solutions 1 3 0, o Veriy that upon substitution o the approimate solutions to +- you will not get 0!!! You should thereore avoid subtraction and the loop condition should not be set to "zero", 0 Numerical Methods - Lecture i a-b<ε 49

49 Numerical Methods - Lecture 50 Tasks and numerical algorithms Numerical task requires a clear and unambiguous description o the unctional relationship between the input or "independent variables" o the task and output data, i.e. searched results. Numerical task is a problem determining the results vector w on the basis o data vector a a Projection W task well-deined w W (a) D w unambiguous assignment

50 Numerical Methods - Lecture 51 Tasks and numerical algorithms Numerical algorithm is a ull description o the operations correctly transorming the input data vector to the vector o output data. The algorithm is ormulated correctly when the number o necessary actions is inite DN D w WN ( a, ) a DN Projection WN w Resulting vector depends on the computing accuracy ε o the machine

51 Numerical Methods - Lecture 5 Eamples o algorithms Given a comple number a=+iy. Calculate 1/a Algorithm I: 1. t y / (tangent phase o number a). 3. a 1 Re a y 1 / a / (module o number a) 1 t 1 t 1 Im a 1 / a / t 1 t The task is well-deined i: y 0 that is: D R (0,0) This algorithm is ormulated correctly (11 necessary steps)

52 Numerical Methods - Lecture 53 Eamples o algorithms Not or each data pair (, y) 0 the solution to the problem can be ound using the algorithm I. 1. There will be an overlow o loating point numbers (or = 0, also because o rounding to zero). The overlow may happen even in the irst step when = 10-5 and y = 10 5 because o the division y / 3. For = 0, and y 0 the solution cannot be determined using this algorithm. The increase in the accuracy o the calculation does not change this act. Algorithm I is not numerically stable

53 Numerical Methods - Lecture 54 Eamples o algorithms Given a comple number a=+iy. Calculate 1/a Algorithm II: 1. r Re 1 a y y. u Im 1 a y y Algorithm II is ormulated correctly (9 necessary steps) Algorithm II is numerically stable due to the continuity o equations or y 0

54 Numerical Methods - Lecture 55 Conditioning and stability The calculation algorithm is numerically stable i or any selected data vector a 0 D there eists an accuracy o the calculations ε 0, that or ε<ε 0 we have a 0 DN( ) and WN ( a, ) W ( a ) lim The algorithm is numerically stable when increasing the accuracy o the calculations any eisting solution to the problem can be ound.

55 Numerical Methods - Lecture 56 Conditioning and stability Due to the rounding o error WN ( a, ) W ( a) By lemma o Wilkinson disturbed data vector can be selected a a D or which WN ( a, ) W ( a a) Size o the disturbance -the data vector a a depends on: -the number o perormed operations -the accuracy o calculations

56 Numerical Methods - Lecture 57 Conditioning and stability Based on Wilkinson s Lemma: when 0 0 Conditioning tells us how much the result o the disturbed vector data diers rom the eact result or vector data that is: a a W(a a) W(a) Condition number B(a) is the number or which the ollowing ormula is satisied: w w B(a) a a w WN ( a, ) W ( a)

57 58 Let us assume the relative error o Condition number ~ ~ The relative error o () ) ~ ( ) ~ )( ~ ( ' ) ~ ( ) ~ ( ) ( Condition number: ) ~ ( ) ~ ( ' ~ Numerical Methods - Lecture

58 Numerical Methods - Lecture 59 Condition number Eample ( ) Condition number: ~ '( ~ ) ( ~ ) 1 1 task is well-conditioned when condition number is small

59 Numerical Methods - Lecture 60 Condition number Eample ( ) 10 1 Condition number: ~ '( ~ ) ( ~ ) 1 ill-conditioned task near =1 i =-1