Numerical Methods - Lecture 1 Numerical Methods Lecture. Analysis o errors in numerical methods
Numerical Methods - Lecture Why represent numbers in loating point ormat? Eample 1. How a number 56.78 can be represented using 5-places? 5 6. 7 8 What is the smallest number that can be represented in this ormat? 0 0 0. 0 0 What is the largest number that can be represented in this ormat? 9 9 9. 9 9
Numerical Methods - Lecture 3 Why represent numbers in loating point ormat? Eample. How a number 56.786 can be represented using 5-places? rounded o 5 6. 7 9 chopped 5 6. 7 8 Conclusion: The error is smaller than 0.01
Numerical Methods - Lecture 4 Calculation o errors True Error o accurate or true value o o Relative True Error o o Calculations: t o o 56.79 56.786 100% 100% 56.786 0.001558%
Numerical Methods - Lecture 5 Calculation o errors Relative errors o small numbers are large. Compare: t o o 56.79 56.786 100% 100% 56.786 0.001558% t o o 3.55 3.546 100% 100% 3.546 0.1180% Absolute true errors are the same: o 56.786 56.79 3.546 3.55 0.004
Numerical Methods - Lecture 6 How to keep relative true errors at a constant level? The number can be represented as: sign mantissa 10 eponent or sign mantissa eponent 56.78 is written as 0.003678 is written as 56.78 is written as.567810 3.67810 3.567810
Numerical Methods - Lecture 7 Floating point arithmetic decimal representation Form o a number e m10 an integer eponent sign (-1 or +1) mantissa (1) 10 m<(10) 10 Eample.567810 1 m.5678 e
Numerical Methods - Lecture 8 Floating point arithmetic binary representation Form o a number e m an integer eponent sign (0 positive or 1 negative) mantisses (1) m<() Eample: 1.1011011 (101) 1 is not represented 0 m 1011011 e 101
Numerical Methods - Lecture 9 What do we gain by using the loating point representation? The rage o the numbers that can be epressed has increased I we use only 5 places to represent a positive number with a positive eponent, the smallest number that can be epressed is 1 and the largest is 9.999 10 9 9 9 9 9 9 mantissa eponent The range o numbers that can be represented has increased rom 999.99 to 9.999 10 9.
Numerical Methods - Lecture 10 What do we lose by using a loating point representation? Precision Why? The number 56.78 will be represented as.5678 10 on ive places: 5 6 8 mantissa eponent There will be a round-o error.
Numerical Methods - Lecture 11 Test problem 1 1. Write down the number 57639.78 on ive places in a similar manner as that discussed in the previous eample: mantissa eponent. Calculate the true error and the relative true error o round o 3. Compare the results with the previous eample (56.78).
Numerical Methods - Lecture 1 Test problem We have 9-bits irst bit represents the sign o the number, second bit represents the sign o the eponent, the net our bits represent the mantissa, the last three bits represent the eponent 0 0 sign eponent sign mantissa eponent Find the number (in decimal), which is presented above
Numerical Methods - Lecture 13 Solution o the problem 1 1. The number 57639.78 written on ive places is: 5 7 6 3 5 mantissa eponent. The true error is 9.78 and the relative true error is 0.005167% 3. For the number 56.78 these errors are: 0.0 (smaller) and 0.0077888% (comparable)
Numerical Methods - Lecture 14 Solution o the problem 54.75 110110.11 1.1011011 10 1.1011 101 5 0 0 1 0 1 1 1 0 1 is not represented 54 10
Numerical Methods - Lecture 15 What is the accuracy ε? For each digital machine epsilon ε is deined as a parameter determining the accuracy o the calculations: 54.75 t 10 110110.11 1.1011011 ε N 1.1011 101 where: N= (in binary), N=10 (in decimal), t is the number o bits representing the mantissa 5 ε is lowered when more bits are allocated to represent the mantissa M Epsilon ε can be regarded as a parameter characterizing the accuracy o the computing machine (small ε = more accurate calculations). Double precision (Fortran) DP
Numerical Methods - Lecture 16 What is the accuracy ε? Epsilon ε is the smallest number that when added to 1.000 produces a number that can be represented as dierent rom 1.000. Eample: 10-bit word 0 0 0 0 0 0 0 0 0 0 1 10 M N w sign eponent mantissa eponent sign 0 0 0 0 0 0 0 0 0 1 1.0001 1. 065 the net number 10 mach 1.065 1 4
Numerical Methods - Lecture 17 Single precision in IEEE-754 ormat (Institute o Electrical and Electronics Engineers) 3 bits or single precision 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 sign (s) eponent (e ) mantissa (m) Number e' 17 1 m. s ( 1)
Numerical Methods - Lecture 18 Eample 1 1 0 1 0 0 0 1 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Sign (s) Biased Eponent (e ) Mantissa (m) Value s e' 17 1 1. m 1 (10100010 ) 17 1 1.10100000 1617 1 1.65 35 10 1 1.65 5.583410
Numerical Methods - Lecture 19 The eponent or the 3-bit IEEE -754 standard 8-bit eponent means 17 e 18 0 e 55 The ied eponent shit is 17 and thereore In act, 1 e 54 e 0 e 55 Numbers and are reserved or special cases Eponent range 16 e 17
Numerical Methods - Lecture 0 Representation o special numbers e e 0 55 all zeros all ones e s m Represents 0 zeros zeros 0 1 zeros zeros -0 0 ones zeros 1 ones zeros 0 or 1 ones nonzero NaN
1 IEEE-754 Format The largest number 17 38 1.1...1 3.40 The smallest number 10 16 38 1.00...0.18 10 Epsilon mach 3 1.19 10 7
Numerical Methods - Lecture Analysis o errors I we do not know the eact value o o we calculate the approimate error as the dierence in the values obtained in the consecutive approimations: n n1 Relative error ε a : a n n1 n
Numerical Methods - Lecture 3 Eample For 0.5 ( ) 7e at a) () or h 0. 3 () b) or h 0. 15 ind c) approimate error Solution ' ( ) ( h) h ( ) a) h 0.3 '() ( 0,3) 0,3 () 7e 0,5(,3) 7e 0,3 0,5() 10,65
Numerical Methods - Lecture 4 Eample (cont.) b) h 0.15 '() ( 0,15) 0,15 () 7e 0,5(,15) 0,15 7e 0,5() 9,880 c) a n n1 n a 9,880 10,65 9,8800 0,0389 Error 3,89%
Numerical Methods - Lecture 5 The relative error as a criterion or ending the iterative procedure I the relative error is less than or equal to a predetermined number then urther iterations are no longer required a s I we require at least m signiicant digits in the result then ε a 0.510 m
Numerical Methods - Lecture 6 Relative error and signiicant digits ε a 0.510 m h () 0.3 10.65 N/A 0 0.15 9.8800 0.03894 1 0.10 9.7559 0.0171 1 0.01 9.5378 0.086 1 0.001 9.5164 0.005 a m The eact value is 9.514
Numerical Methods - Lecture 7 Sources o errors in numerical calculations 1. Input errors (input data errors). Truncation error 3. Round o error Input errors occur when the input data entered into the computer memory are dierent rom the eact values. Truncation errors are errors due to numerical procedures caused by reducing the number o operations. Round o errors are errors that usually can not be avoided. They arise in the course o the calculations, and can be reduced by setting skillully the method and sequence o tasks.
Numerical Methods - Lecture 8 Input errors Sources o input errors: the input data are the result o measurement o physical quantities a inite length o binary words and thereore prerounding is needed preliminary rounding o irrational numbers Rounding o numbers that cannot be epressed eactly is accomplished by: chopping rounding o
Numerical Methods - Lecture 9 Eample: 3,1415 3,1415965359 3,1416 chopping rounding o Rounding o introduces smaller error than chopping.
Numerical Methods - Lecture 30 Truncation error Caused by the use o the approimate ormula instead o a ull mathematical operation: when calculating the sum o the ininite series when calculating integral, derivative W lim 0 1 F 1 Fd work
Numerical Methods - Lecture 31 Taylor series I the unction is continuous and all derivatives ', '',... n eist within the interval [, + h] the value o the unction at the point + h can be calculated as: h h! h 3! h 3 The Maclaurin series is simply the Taylor series about the point =0 0 h 0 0h 0 0 h! h 3 3!
Numerical Methods - Lecture 3 Eamples Some eamples o Taylor series which you must have seen cos( ) 1! 4 4! 6 6! sin( ) 3 3! 5 5! 7 7! e 1! 3 3!
33 Truncation error in the Taylor series R n h h h h n n n!! ' ' remainder c n h R n n n 1 1 1)! ( ) ( h c Numerical Methods - Lecture
Eample The Taylor series or e at point =0 is given by e 1! 3 3! 4 4! 5 5! It can be seen that as the number o terms used increases, the error decreases and hence a better estimate can be ound. Question: How many terms would it require to get an approimation o e 1 with a true error o less than 10-6? 1 e 1 11! Numerical Methods - Lecture 1 3 3! 1 4! 1 5! 1 1 1 1 6 4 10 4 5 34
Numerical Methods - Lecture 35 Solution 0, h 1, ( ) e R n h n1 c ( n n1 1)! R n 0 1 n n1 1! n1 c since c 0 c 0 1 0 c 1 h ( n 1 1)! n 1 1 e n 1! R n c 0 ( n e 1)!
Numerical Methods - Lecture 36 Solution e ( n 1)! 10 6 true error ( n 1)! 6 10 e ( n 1)! 10 6 3 n 9 It means that 9 terms or more are needed to get a true error less than 10-6
Numerical Methods - Lecture 37 Arithmetic operations 1. Addition and subtraction To add or subtract two standard loating-point numbers, the eponents should be made equal with an adequate shit o the mantissa. Eample: Add 0,4546 10 5 to 0,5433 10 7 shit 0,0045 10 7 +0,5433 10 7 =0,5478 10 7 Answer: We loose some signiicant digits
Numerical Methods - Lecture 38 Arithmetic operations. Multiplication Eample: Multiply 0,5543 10 1 by 0,4111 10-15 Multiply mantissas and add the eponents. 0,5543 10 1 0,4111 10-15 =0,7873 10-3 =0,78 10-3 3. Division Eample: Divide 0,1000 10 5 by 0,9999 10 3 0,1000 10 5 /0,9999 10 3 =0,1000 10 All the time we loose some signiicant digits which is the source o the error
Numerical Methods - Lecture 39 Sequence o operations (a+b)-c (a-c)+b lack o associativity a(b-c) (ab-ac) lack o distributive property Eample: a= 0,5665 10 1, b=0,5556 10-1, c=0,5644 10 1 (a+b)=0,5665 10 1 +0,5556 10-1 =0,5665 10 1 +0,0055 10 1 =0,570 10 1 (a+b)-c=0,570 10 1-0,5644 10 1 =0,7600 10-1 (a-c)=0,5665 10 1-0,5644 10 1 =0,001 10 1 =0,100 10-1 (a-c)+b=0,100 10-1 +0,5556 10-1 =0,7656 10-1
Numerical Methods - Lecture 40 Wilkinson Lemma Rounding o errors occurring during loating point operations are equivalent to substitution o accurate numbers by slightly perturbed numbers on which we act eactly. For individual arithmetical operations: l( 1 ) 1(1 1) (1 ) l( 1 ) 1 (1 3) 1 (1 3) l( 1 / ) 1 (1 4) / 1 /( (1 5)) symbol o operation perormed on loating-point data and ε i
Numerical Methods - Lecture 41 Tragic eample o the rounding o error On February 5th, 1991 in Dhahran, Saudi Arabia, 8 American soldiers were killed in the attack o the Iraqi Scud missiles. The deense system Patriot did not detect any assault. Why? The system calculates the area, which should be scanned based on the speed o the object and the last detection. The internal clock was set to measure every 1/10 second and 4-bit word length was assumed. Due to rounding o, the absolute error was 9.5 10-8 s which ater 100 hours amounted to: 9.510 8 10 60 60100 0.34 sec Based on this, the calculated displacement is 687 m. Object is considered outside the range when the displacement is 137 m
Numerical Methods - Lecture 4 Useul hints: In the numerical calculations it is advantageous: To solve the same problem by another method, or by the same method, but with a dierent order o operations To solve the problem on slightly perturbed input data
y Numerical Methods - Lecture 44 Propagation o errors 140 10 100 80 60 40 0 u(y) unction y = () tangent dy/d u (y) u() u() dy d 0 0 4
Numerical Methods - Lecture 45 Total dierential in application to errors For y=( 1,,... n ) where maimum uncertainties 1,,... n are small compared with the values o the variables 1,,... n the maimum uncertainty o y can be epressed as: y y y 1... 1 y n n
Numerical Methods - Lecture 46 Eample Estimate the uncertainty o density ρ o a sphere o mass m and radius R ( m, R) (4 m 3)πR 3 absolute error m m R R but m 1 3 4 3πR R 3m 4 4 3 πr relative error 3 m R
Numerical Methods - Lecture 47 Errors o arithmetic operations Addition A a a B b b absolute errors A B a b a b a b ( a b) the addition absolute error Thereore, the addition (subtraction) absolute error is equal to the sum o absolute errors o components. ( a b) a b
Numerical Methods - Lecture 48 Errors o arithmetic operations the addition relative error ab the subtraction relative error ab a a a a b b b b Subtraction relative error can be large even i the relative errors o minuend and subtrahend are small. Subtracting o nearly equal numbers should be avoided! This phenomenon is called reduction o signiicant digits It is important or the calculation o Newton s dierence quotients approimating derivatives o unctions, roots o a quadratic equation with the dominant actor o the irst power, etc.
The concept o zero We lose the precise meaning o the number 0 i we perorm numerical calculations Eact roots are Approimate solutions 1 3 0,730 10 o -0.73 10 1 Veriy that upon substitution o the approimate solutions to +- you will not get 0!!! You should thereore avoid subtraction and the loop condition should not be set to "zero", 0 Numerical Methods - Lecture i a-b<ε 49
Numerical Methods - Lecture 50 Tasks and numerical algorithms Numerical task requires a clear and unambiguous description o the unctional relationship between the input or "independent variables" o the task and output data, i.e. searched results. Numerical task is a problem determining the results vector w on the basis o data vector a a Projection W task well-deined w W (a) D w unambiguous assignment
Numerical Methods - Lecture 51 Tasks and numerical algorithms Numerical algorithm is a ull description o the operations correctly transorming the input data vector to the vector o output data. The algorithm is ormulated correctly when the number o necessary actions is inite DN D w WN ( a, ) a DN Projection WN w Resulting vector depends on the computing accuracy ε o the machine
Numerical Methods - Lecture 5 Eamples o algorithms Given a comple number a=+iy. Calculate 1/a Algorithm I: 1. t y / (tangent phase o number a). 3. a 1 Re a y 1 / a / (module o number a) 1 t 1 t 1 Im a 1 / a / t 1 t The task is well-deined i: y 0 that is: D R (0,0) This algorithm is ormulated correctly (11 necessary steps)
Numerical Methods - Lecture 53 Eamples o algorithms Not or each data pair (, y) 0 the solution to the problem can be ound using the algorithm I. 1. There will be an overlow o loating point numbers (or = 0, also because o rounding to zero). The overlow may happen even in the irst step when = 10-5 and y = 10 5 because o the division y / 3. For = 0, and y 0 the solution cannot be determined using this algorithm. The increase in the accuracy o the calculation does not change this act. Algorithm I is not numerically stable
Numerical Methods - Lecture 54 Eamples o algorithms Given a comple number a=+iy. Calculate 1/a Algorithm II: 1. r Re 1 a y y. u Im 1 a y y Algorithm II is ormulated correctly (9 necessary steps) Algorithm II is numerically stable due to the continuity o equations or y 0
Numerical Methods - Lecture 55 Conditioning and stability The calculation algorithm is numerically stable i or any selected data vector a 0 D there eists an accuracy o the calculations ε 0, that or ε<ε 0 we have a 0 DN( ) and WN ( a, ) W ( a ) lim 0 0 0 The algorithm is numerically stable when increasing the accuracy o the calculations any eisting solution to the problem can be ound.
Numerical Methods - Lecture 56 Conditioning and stability Due to the rounding o error WN ( a, ) W ( a) By lemma o Wilkinson disturbed data vector can be selected a a D or which WN ( a, ) W ( a a) Size o the disturbance -the data vector a a depends on: -the number o perormed operations -the accuracy o calculations
Numerical Methods - Lecture 57 Conditioning and stability Based on Wilkinson s Lemma: when 0 0 Conditioning tells us how much the result o the disturbed vector data diers rom the eact result or vector data that is: a a W(a a) W(a) Condition number B(a) is the number or which the ollowing ormula is satisied: w w B(a) a a w WN ( a, ) W ( a)
58 Let us assume the relative error o Condition number ~ ~ The relative error o () ) ~ ( ) ~ )( ~ ( ' ) ~ ( ) ~ ( ) ( Condition number: ) ~ ( ) ~ ( ' ~ Numerical Methods - Lecture
Numerical Methods - Lecture 59 Condition number Eample ( ) Condition number: ~ '( ~ ) ( ~ ) 1 1 task is well-conditioned when condition number is small
Numerical Methods - Lecture 60 Condition number Eample ( ) 10 1 Condition number: ~ '( ~ ) ( ~ ) 1 ill-conditioned task near =1 i =-1