Introduction to Scientific Computing Languages

Size: px

Start display at page:

Download "Introduction to Scientific Computing Languages"

Hugh Ellis
6 years ago
Views:

1 1 / 19 Introduction to Scientific Computing Languages Prof. Paolo Bientinesi pauldj@aices.rwth-aachen.de

2 Numerical Representation 2 / 19 Numbers 123 = (first 40 digits) π = In general: Infinite number of digits

3 Numerical Representation 2 / 19 Numbers 123 = (first 40 digits) π = In general: Infinite number of digits Computers Finite memory

4 Computers: Inexact Numbers 3 / 19 Infinite numbers vs. finite memory Approximated numbers

5 Computers: Inexact Numbers 3 / 19 Infinite numbers vs. finite memory Approximated numbers How many digits? A pre-determined amount π=

6 Computers: Inexact Numbers 3 / 19 Infinite numbers vs. finite memory Approximated numbers How many digits? A pre-determined amount π= Using 4 digits: π = Modern computers: normally 8 or 16 digits, single / double precision.

7 Computers: Inexact Numbers 3 / 19 Infinite numbers vs. finite memory Approximated numbers How many digits? A pre-determined amount π= Using 4 digits: π = Modern computers: normally 8 or 16 digits, single / double precision. Alternatively? Extended precision Variable precision Symbolic representation

8 4 / 19 Computers: Approximated Computations 4-digit representation Inexact Arithmetic = =

9 4 / 19 Computers: Approximated Computations 4-digit representation Inexact Arithmetic = =

10 4 / 19 Computers: Approximated Computations 4-digit representation Inexact Arithmetic = = Truncated Rounded 124.0

11 4 / 19 Computers: Approximated Computations 4-digit representation Inexact Arithmetic = = Truncated Rounded Associativity? Exact arithmetic: ( ) = ( )

12 4 / 19 Computers: Approximated Computations 4-digit representation Inexact Arithmetic = = Truncated Rounded Associativity? Exact arithmetic: ( ) = ( ) Inexact arithmetic: ( ) = 124.4

13 4 / 19 Computers: Approximated Computations 4-digit representation Inexact Arithmetic = = Truncated Rounded Associativity? No! Exact arithmetic: ( ) = ( ) Inexact arithmetic: ( ) = ( ) = 124.5

14 Error Analysis 5 / 19 f : A B, y = f(x) Es.: f(x) = x 2 + sin(2 x) x = π, f(x) =? 123

15 Error Analysis 5 / 19 f : A B, y = f(x) Es.: f(x) = x 2 + sin(2 x) x = π, f(x) =? 123 Exact arithmetic: ( π ) 2 ( + sin 2 π ) =...

16 Error Analysis 5 / 19 f : A B, y = f(x) Es.: f(x) = x 2 + sin(2 x) x = π, f(x) =? 123 Exact arithmetic: ( π ) 2 ( + sin 2 π ) =... Inexact arithmetic: x ˆx, f ˆf ˆf(ˆx) instead of f(x)

17 Errors 6 / 19 Representation Errors Roundoff Errors Algorithmic Errors

18 Known Disasters 7 / 19

19 Known Disasters 7 / 19 Patriot Missile, 1991 Scud launched from Iraq againt US military base in South Arabia. US Patriot s missile missed the incoming Scud. 28 casualties. Cancellation

20 Known Disasters 7 / 19 Patriot Missile, 1991 Scud launched from Iraq againt US military base in South Arabia. US Patriot s missile missed the incoming Scud. 28 casualties. Cancellation Spaceship Ariane 5 launched in 1996, destroyed 37 seconds after liftoff. Overflow

21 Floating Point Numbers 8 / 19 y = ±d 0.d 1 d 2... d t 1 β e

22 Floating Point Numbers 8 / 19 β: base (radix) y = ±d 0.d 1 d 2... d t 1 β e

23 Floating Point Numbers 8 / 19 β: base (radix) y = ±d 0.d 1 d 2... d t 1 β e t: precision = # slots for the mantissa (d 0.d 1... d t 1 )

24 Floating Point Numbers 8 / 19 y = ±d 0.d 1 d 2... d t 1 β e β: base (radix) t: precision = # slots for the mantissa (d 0.d 1... d t 1 ) 0 d i β 1: digits

25 Floating Point Numbers 8 / 19 y = ±d 0.d 1 d 2... d t 1 β e β: base (radix) t: precision = # slots for the mantissa (d 0.d 1... d t 1 ) 0 d i β 1: digits e min e e max : exponent range

26 Floating Point Numbers 8 / 19 y = ±d 0.d 1 d 2... d t 1 β e β: base (radix) t: precision = # slots for the mantissa (d 0.d 1... d t 1 ) 0 d i β 1: digits e min e e max : exponent range Normalization: d 0 = 1; d 0 used for the sign ±

27 Floating Point Numbers 8 / 19 y = ±d 0.d 1 d 2... d t 1 β e β: base (radix) t: precision = # slots for the mantissa (d 0.d 1... d t 1 ) 0 d i β 1: digits e min e e max : exponent range Normalization: d 0 = 1; d 0 used for the sign ± Arithmetic β t e min e max Single precision Double precision

28 Floating Point Numbers 8 / 19 y = ±d 0.d 1 d 2... d t 1 β e β: base (radix) t: precision = # slots for the mantissa (d 0.d 1... d t 1 ) 0 d i β 1: digits e min e e max : exponent range Normalization: d 0 = 1; d 0 used for the sign ± Arithmetic β t e min e max Single precision Double precision HP calculator IBM Setun 3 Quadruple prec

29 Floating Point Numbers (2) 9 / 19 What are the nonnegative points in β = 2, t = 3, e min = 1, e max = 3?

30 Floating Point Numbers (2) 9 / 19 What are the nonnegative points in β = 2, t = 3, e min = 1, e max = 3? Floating point numbers are non-equidistant!

31 Floating Point Numbers (2) 9 / 19 What are the nonnegative points in β = 2, t = 3, e min = 1, e max = 3? Floating point numbers are non-equidistant! How to represent 0?

32 Floating Point Numbers (2) 9 / 19 What are the nonnegative points in β = 2, t = 3, e min = 1, e max = 3? Floating point numbers are non-equidistant! How to represent 0? underflow? overflow? NaN?

33 IEEE single precision FYI 10 / 19

34 IEEE double precision FYI 11 / 19

35 Machine Precision 12 / 19 Machine Precision u Smallest positive number such that [1 + u] 1

36 Machine Precision 12 / 19 Machine Precision u Smallest positive number such that [1 + u] 1 Largest positive number such that [1 + u] = 1

37 Machine Precision 12 / 19 Machine Precision u Smallest positive number such that [1 + u] 1 Largest positive number such that [1 + u] = β1 t

38 Machine Precision 12 / 19 Machine Precision u Smallest positive number such that [1 + u] 1 Largest positive number such that [1 + u] = β1 t Distance between 1 and the next floating point number Machine epsilon, ɛ M, u

39 Representation Error 13 / 19 f min = smallest positive floating point number f max = largest positive floating point number x = [x] = floating point representation of x Theorem: Let x R and x [f min, f max ] Then x = x(1 + δ 1 ) where δ 1 u

40 Representation Error 13 / 19 f min = smallest positive floating point number f max = largest positive floating point number x = [x] = floating point representation of x Theorem: Let x R and x [f min, f max ] Then x = x(1 + δ 1 ) where δ 1 u Also, x = x/(1 + δ 2 ) where δ 2 u Note: δ 1 and δ 2 are functions of x

41 Smallest distance 0 14 / 19

42 Smallest distance 0 14 / 19 x and y are floating point numbers, x y; how small can z be? z := x y

43 Smallest distance 0 14 / 19 x and y are floating point numbers, x y; how small can z be? z := x y vs. z := x y max( x, y )

44 Smallest distance 0 14 / 19 x and y are floating point numbers, x y; how small can z be? z := x y vs. z := x y max( x, y ) z := x y 1) = 2 1 t 2) emin = 2 emin t+1 3) emin+t 1 = 2 emin

45 Smallest distance 0 14 / 19 x and y are floating point numbers, x y; how small can z be? z := x y vs. z := x y max( x, y ) z := x y 1) = 2 1 t 2) emin = 2 emin t+1 3) emin+t 1 = 2 emin

46 Smallest distance 0 14 / 19 x and y are floating point numbers, x y; how small can z be? z := x y vs. z := x y max( x, y ) z := x y 1) = 2 1 t 2) emin = 2 emin t+1 3) emin+t 1 = 2 emin z := x y max( x, y ) 1) = 2 1 t 2) emin = 2 emin t+1 3) emin+t 1 = 2 emin

47 Smallest distance 0 14 / 19 x and y are floating point numbers, x y; how small can z be? z := x y vs. z := x y max( x, y ) z := x y 1) = 2 1 t 2) emin = 2 emin t+1 3) emin+t 1 = 2 emin z := x y max( x, y ) 1) = 2 1 t 2) emin = 2 emin t+1 3) emin+t 1 = 2 emin

48 Roundoff Error 15 / 19 Notation: [exp] denotes the evaluation of exp in floating point arithmetic. Assuming a left-to-right evaluation, it holds [ [[x] ] [ ] ] [x + y + z/w] = + [y] + [z]/[w]

49 Roundoff Error 15 / 19 Notation: [exp] denotes the evaluation of exp in floating point arithmetic. Assuming a left-to-right evaluation, it holds [ [[x] ] [ ] ] [x + y + z/w] = + [y] + [z]/[w] Floating Point Arithmetic Theorem: (Standard and Alternative Computational Models) Let x and y be floating point numbers Then [x op y] = (x op y)(1 + ɛ 1 ), where ɛ 1 u, and op {+,,, /}

50 Roundoff Error 15 / 19 Notation: [exp] denotes the evaluation of exp in floating point arithmetic. Assuming a left-to-right evaluation, it holds [ [[x] ] [ ] ] [x + y + z/w] = + [y] + [z]/[w] Floating Point Arithmetic Theorem: (Standard and Alternative Computational Models) Let x and y be floating point numbers Then [x op y] = (x op y)(1 + ɛ 1 ), where ɛ 1 u, and op {+,,, /} Also, [x op y] = x op y (1 + ɛ 2 ), where ɛ 2 u, and op {+,,, /} Note: ɛ 1 and ɛ 2 are functions of x, y and op

51 Example: Dot Product 16 / 19 x, y R n ; κ := x T y ( ((χ0 κ := ψ 0 + χ 1 ψ 1 ) + ) ) + χ n 2 ψ n 2 + χ n 1 ψ n 1 ˇκ = = ( ((χ0 ψ 0 (1 + ɛ (0) ) + χ 1 ψ 1 (1 + ɛ (1) ) ) (1 + ɛ (1) + ) + ) +χ n 1 ψ n 1 (1 + ɛ (n 1) ) (1 + ɛ (n 1) + ) n 1 n 1 χ i ψ i (1 + ɛ (i) ) (1 + ɛ (j) i=0 j=i + ) ) (1 + ɛ (n 2) + ) where ɛ (0) + = 0 and ɛ (0), ɛ (j), ɛ (j) + u for j = 1,..., n 1

52 Case Studies Bisection Recursive implementation Base case? 17 / 19

53 Case Studies Bisection Recursive implementation Base case? Π 2 / i=1 1 i 2 vs. 1 i= i 2 17 / 19

54 Case Studies Bisection Recursive implementation Base case? Π 2 / i=1 1 i 2 vs. 1 i= i 2 sqsqrt vs. sqrtsq 17 / 19

55 Case Studies Bisection Recursive implementation Base case? Π 2 / i=1 1 i 2 vs. 1 i= i 2 sqsqrt vs. sqrtsq Ax = b A = [ ɛ ] 17 / 19

56 Backward Stability 18 / 19 Let f : D R be a map from the domain D to the range R. Let ˆf : D R represent the execution in floating point arithmetic of a given algorithm A that computes f. A is said to be backward stable if for all x D there exists a perturbed input x D, close to x, such that ˆf(x) = f( x).

57 Backward Stability 18 / 19 Let f : D R be a map from the domain D to the range R. Let ˆf : D R represent the execution in floating point arithmetic of a given algorithm A that computes f. A is said to be backward stable if for all x D there exists a perturbed input x D, close to x, such that ˆf(x) = f( x). I.e., the result computed in floating point arithmetic ( ˆf(x)) equals the result obtained when the mathematically exact function (f) is applied to slightly perturbed data ( x). The difference between x and x, is the perturbation to the original input x.

58 References 19 / 19 IEEE and IEEE : Standard for Floating-Point Arithmetic Book: Accuracy and Stability of Numerical Algorithms, by Nick Higham Article: What every computer scientist should know about floating-point arithmetic, by David Goldberg Book: Numerical Computing with IEEE Floating Point Arithmetic, by Michael Overton

Introduction to Scientific Computing Languages

1 / 21 Introduction to Scientific Computing Languages Prof. Paolo Bientinesi pauldj@aices.rwth-aachen.de Numerical Representation 2 / 21 Numbers 123 = (first 40 digits) 29 4.241379310344827586206896551724137931034...