CME 302: NUMERICAL LINEAR ALGEBRA FALL 2005/06 LECTURE 5. Ax = b.
|
|
- Angelica Gilbert
- 5 years ago
- Views:
Transcription
1 CME 302: NUMERICAL LINEAR ALGEBRA FALL 2005/06 LECTURE 5 GENE H GOLUB Suppose we want to solve We actually have an approximation ξ such that 1 Perturbation Theory Ax = b x = ξ + e The question is, how can we use norms to bound the relative error in ξ? We define the residual r by r = b Aξ = A(x ξ) = Ae Note that r = 0 if Ax = b From the relations r A e and e A 1 r, we obtain r A e A 1 r It follows that the relative error is bounded as follows: since A x b The quantity r A x e x A 1 r A 1 A r x b κ(a) = A 1 A is called the condition number of A The condition number serves as a measure of how perturbation in the data of the problem Ax = b affects the solution How does a perturbation in A affect the solution x? To answer this question, we define A(ɛ) to be a function of the size of the perturbation ɛ, with A(0) = A Starting with and differentiating with respect to epsilon yields or Now, suppose that x(ɛ) satisfies A(ɛ) da 1 (ɛ) da 1 (ɛ) A(ɛ)A 1 (ɛ) = I + da(ɛ) A 1 (ɛ) = 0 = A 1 (ɛ) da(ɛ) A 1 (ɛ) (A + ɛe)x(ɛ) = b Date: October 26, 2005, version 10 Notes originally due to James Lambers Minor editing by Lek-Heng Lim 1
2 Using Taylor series, we obtain Taking norms, we obtain x(ɛ) = (A + ɛe) 1 b = x(0) + ɛ dx(ɛ) + O(ɛ 2 ) ( ɛ=0 = x(0) + ɛ A 1 (ɛ) da(ɛ) ) A 1 (ɛ) b + O(ɛ 2 ) = x(0) + ɛ( A 1 EA 1 )b + O(ɛ 2 ) = x(0) + ɛ( A 1 E)x + O(ɛ 2 ) x(ɛ) x(0) ɛ A 1 2 E b + O(ɛ 2 ) from which it follows that the relative perturbation in x is x(ɛ) x x ɛ A 1 A E A + O(ɛ2 ) κ(a)ρ + O(ɛ 2 ) where ρ = ɛe / A is the relative perturbation in A Since the exact solution to Ax = b is given by x = A 1 b, we are also interested in examining (A + E) 1 where E is some perturbation Can we say something about (A + E) 1 A 1? We assume that A 1 E = r < 1 We have which yields From we obtain or A + E = A(I + A 1 E) = A(I F ), (I F ) r (A + E) 1 A 1 = (I + A 1 E) 1 A 1 A 1 F = A 1 E = (I + A 1 E) 1 (A 1 (I + A 1 E)A 1 ) = (I + A 1 E) 1 ( A 1 EA 1 ) (A + E) 1 A r A 1 2 E (A + E) 1 A 1 A r κ(a) E A 2 Inexact Computation 21 Number representation Real numbers can be represented using floating-point notation: a floating-point representation such as y = ±d 1 d s d s+1 10 e A real number may not necessarily have a unique floating-point representation, as the following examples indicate: or even worse, y = ± = ± y = =
3 How can we represent y? We can use a chopped representation or possibly where ỹ = ±d 1 d s 10 3 ỹ = ±d 1 d s 1 ds 10ē d s d s+1 < 5, d s = (d s + 1) mod 9 d s+1 > 5,? d s+1 = 5 where the value of d s when d s = 5 depends on what convention is used for rounding Note that e can change if d s+1 > 5 We often write ỹ = fl(y) where fl( ) means floating-point representation of y Floating point numbers can be represented in base β as where m e M and the digits d j satisfy y = ±d 1 d s β e 1 d 1 β 1, 0 d j β 1, j = 2,, s This is a normalized floating-point number The sequence of significant digits d 1 d s is called the mantissa, and the number e is called the exponent Suppose s = 1, m = 1, and M = 1 Then the representable numbers are and 27 negative numbers Note that the distribution is not uniform In the previous example, the representable numbers are 9, 8,, 1, 09, 08,, 01, 009,, 001, 0, 01, Note that there are large gaps between integers Now, suppose that s = 3, and that we multiply two numbers y 1 = 0999 and y 2 = 0999 The exact product is y 1 y 2 = , but the computed product is fl(y 1 y 2 ) = 0998 We will write fl(y 1 op y 2 ) to indicate some numerical calculation On the IBM 360, we have base 16 (hexadecimal) In general, y = (±d 1 d s ) β β e, ( d1 = ± β + d 2 β d ) s β s β 3, β = 16, with 1 d 1 β 1 and 0 d j β 1 for 2 j s For the IBM 360, floating-point numbers are represented using s = 6, m = 64 and M = 63, while double-precision floating-point numbers are represented using s = 14, m = 64, and M = 63 If, for the result of any operation, e < 64, then underflow has occurred On the other hand, the scenario e > 63 is called overflow 22 Roundoff Error in Arithmetic Operations We need to consider the error arising from computing the results of arithmetic expressions In general, z = fl(x op y) = (x op y)(1 + δ) where δ β (s 1) u when results are truncated, or δ 1 2 β (s 1) when results are rounded Therefore the relative error in fl(x op y) is fl(x op y) (x op y) x op y = δ u where u is known as the unit roundoff 3
4 We can use this idea sequentially to bound the error in more complex computations Suppose we want to compute s n = x 1 + x x n If we use the numerical algorithm the corresponding computer algorithm is s 0 = 0 s 1 = s 0 + x 1 s 2 = s 1 + x 2 s n = s n 1 + x n σ 0 = 0 σ 1 = fl(σ 0 + x 1 ) σ 2 = fl(σ 1 + x 2 ) Expanding the expressions for the σ i, we obtain σ 1 = fl(σ 0 + x 1 ) = (σ 0 + x 1 )(1 + δ 1 ) σ n = fl(σ n 1 + x n ) σ 2 = fl(σ 1 + x 2 ) = (σ 1 + x 2 )(1 + δ 2 ) = σ 1 (1 + δ 2 ) + x 2 (1 + δ 2 ) = (σ 1 + x 1 )(1 + δ 1 )(1 + δ 2 ) + x 2 (1 + δ 2 ) σ n = x 1 (1 + δ 1 ) (1 + δ n ) + x 2 (1 + δ 2 ) (1 + δ n ) + + x n (1 + δ n ) = where In summary, σ n (1 + η j ) = x i n (1 + δ k ) = x x n + k=j x j η j x j n k=j η j x j, η j (n j + 1)u + O(u) 2 (1 + δ k ) = x j (1 + η j ) if 0 < x i1 x i2 x in It would seem to imply that we should add the numbers in that order since the numbers are weighted A better procedure is to add the numbers in pairs, resulting in the bound 1 + η i 1 + pu where n = 2 p In this case, the error is uniformly distributed For the product of n numbers p = x 1 x n, we obtain the computed results Π 1 = fl(x 1 x 2 ) = (x 1 x 2 )(1 + ɛ 1 ) Π 2 = fl(π 2 x 3 ) = (x 1 x 2 x 3 )(1 + ɛ 1 )(1 + ɛ 2 ) where Π n = fl(π n 1 x n ) = (x 1 x n )(1 + ɛ 1 ) (1 + ɛ n 1 ) = p(1 + η n 1 ) η n 1 (n 1)u + O(u 2 ) 4
5 23 Common Computations It is frequently necessary to compute the inner product s = u i v i Now Therefore, if we wish to compute fl(u i v i ) = (u i v i )(1 + γ i ), γ i u fl(s) = x j y j (1 + η j ) and (1 + η j ) will depend on how the computation is performed If we do serial addition, then n (1 + η j ) = (1 + δ k )(1 + γ j ) If we do pairwise addition and n = 2 p, then ( p ) (1 + η j ) = (1 + δ ikj ) (1 + γ j ) k=j k=1 In computing inner products one must be quite careful; otherwise the underflow or overflow problem can be quite serious Suppose one wishes to compute s = x 2 i It is quite possible that overflow will occur One solution is to scale Suppose and x ± = αβ p Then one should compute [ s = x ± x j, j = 1, 2,, n ( xi This requires two passes; there are better ways of performing the calculation We can build up analysis of many problems This has been done extensively for linear algebra by J H Wilkinson The analysis can be automated; R Moore has used interval arithmetic for determining bounds on the solution It is a general principle that one should add terms of a similar sign Suppose we wish to compute s 2 = (x i x) 2, x = 1 x i (21) n It is well known that s 2 = β p ) 2 ] β 2p x 2 i n x 2 (22) and this formula is frequently used But it is very bad, especially if the x i s are large but s 2 is small Then, you can be subtracting two large numbers Formula (21) is more stable but requires two passes First, you must compute the mean and then the quantity s 2 We could try writing s 2 = (x i ν) 2 + η(ν 2 x 2 ) where ν is some approximation to x 5
6 3 IEEE floating point numbers These notes are due to Lieven Vandenberghe of UCLA 31 Floating point numbers with base 10 Notation x = ±(d 1 d 2 d n ) e d 1 d 2 d n is the mantissa (d i integer, 0 d i 9, d 1 0 if x 0) n is the mantissa length (or precision) e is the exponent (e min e e max ) Interpretation Example (with n = 7) = +( ) x = ±(d d d n 10 n ) 10 e = +( ) 10 2 used in pocket calculators Properties a finite set of numbers unequally spaced distance between floating point numbers varies the smallest number greater than 1 is n+1 the smallest number greater than 10 is n+2, largest positive number: smallest positive number: 32 Floating point numbers with base 2 Notation +(999 9) emax = (1 10 n )10 emax x min = +(100 0) e min = 10 e min 1 x = ±(d 1 d 2 d n ) 2 2 e d 1 d 2 d n is the mantissa (d i {0, 1}, d 1 = 1 if x 0) n is the mantissa length (or precision) e is the exponent (e min e e max ) Interpretation Example (with n = 8): = +( ) x = ±(d d d n 2 n ) 2 e = +( ) 2 4 used in almost all computers Properties a finite set of unequally spaced numbers largest positive number: smallest positive number: x max = +(1 1) 2 2 emax = (1 2 n )2 emax x min = +(100 0) 2 2 e min = 2 e min 1 6
7 in practice, the number system includes subnormal numbers: unnormalized small numbers (d 1 = 0, e = e min ), and the number 0 33 IEEE floating point standard Specifies two binary floating point number formats: IEEE standard single precision: n = 24, e min = 125, e max = 128 requires 32 bits: 1 sign bit, 23 bits for mantissa, 8 bits for exponent IEEE standard double precision: n = 53, e min = 1021, e max = 1024 requires 64 bits: 1 sign bit, 52 bits for mantissa, 11 bits for exponent; used in almost all modern computers 34 Machine precision Definition The machine precision of a binary floating point number system with mantissa length n is defined as ɛ M = 2 n Example IEEE standard double precision (n = 53): ɛ M = Interpretation 1 + 2ɛ M is the smallest floating point number greater than 1: (10 01) = n = 1 + 2ɛ M 35 Rounding error A floating-point number system is a finite set of numbers; all other numbers must be rounded Notation f l(x) is the floating-point representation of x Rounding rules used in practice: numbers are rounded to the nearest floating-point number in case of a tie: round to the number with least significant bit 0 ( round to nearest even ) Example numbers x (1, 1 + 2ɛ M ) are rounded to 1 or 1 + 2ɛ M : fl(x) = 1 if 1 < x 1 + ɛ M fl(x) = 1 + 2ɛ M if 1 + ɛ M < x < 1 + 2ɛ M gives another interpretation of ɛ M : numbers between 1 and 1 + ɛ M are indistinguishable from 1 36 Rounding error and machine precision General result (no proof): fl(x) x x ɛ M machine precision gives a bound on the relative error due to rounding number of correct (decimal) digits in fl(x) is roughly log 10 ɛ M ie, about 15 or 16 in IEEE precision fundamental limit on accuracy of numerical computation Department of Computer Science, Gates Building 2B, Room 280, Stanford, CA address: golub@stanfordedu 7
FLOATING POINT ARITHMETHIC - ERROR ANALYSIS
FLOATING POINT ARITHMETHIC - ERROR ANALYSIS Brief review of floating point arithmetic Model of floating point arithmetic Notation, backward and forward errors Roundoff errors and floating-point arithmetic
More informationACM 106a: Lecture 1 Agenda
1 ACM 106a: Lecture 1 Agenda Introduction to numerical linear algebra Common problems First examples Inexact computation What is this course about? 2 Typical numerical linear algebra problems Systems of
More informationFLOATING POINT ARITHMETHIC - ERROR ANALYSIS
FLOATING POINT ARITHMETHIC - ERROR ANALYSIS Brief review of floating point arithmetic Model of floating point arithmetic Notation, backward and forward errors 3-1 Roundoff errors and floating-point arithmetic
More informationLecture 7. Floating point arithmetic and stability
Lecture 7 Floating point arithmetic and stability 2.5 Machine representation of numbers Scientific notation: 23 }{{} }{{} } 3.14159265 {{} }{{} 10 sign mantissa base exponent (significand) s m β e A floating
More information1 Floating point arithmetic
Introduction to Floating Point Arithmetic Floating point arithmetic Floating point representation (scientific notation) of numbers, for example, takes the following form.346 0 sign fraction base exponent
More informationCME 302: NUMERICAL LINEAR ALGEBRA FALL 2005/06 LECTURE 9
CME 302: NUMERICAL LINEAR ALGEBRA FALL 2005/06 LECTURE 9 GENE H GOLUB 1 Error Analysis of Gaussian Elimination In this section, we will consider the case of Gaussian elimination and perform a detailed
More informationECS 231 Computer Arithmetic 1 / 27
ECS 231 Computer Arithmetic 1 / 27 Outline 1 Floating-point numbers and representations 2 Floating-point arithmetic 3 Floating-point error analysis 4 Further reading 2 / 27 Outline 1 Floating-point numbers
More informationFloating Point Number Systems. Simon Fraser University Surrey Campus MACM 316 Spring 2005 Instructor: Ha Le
Floating Point Number Systems Simon Fraser University Surrey Campus MACM 316 Spring 2005 Instructor: Ha Le 1 Overview Real number system Examples Absolute and relative errors Floating point numbers Roundoff
More informationNotes on floating point number, numerical computations and pitfalls
Notes on floating point number, numerical computations and pitfalls November 6, 212 1 Floating point numbers An n-digit floating point number in base β has the form x = ±(.d 1 d 2 d n ) β β e where.d 1
More informationJim Lambers MAT 610 Summer Session Lecture 2 Notes
Jim Lambers MAT 610 Summer Session 2009-10 Lecture 2 Notes These notes correspond to Sections 2.2-2.4 in the text. Vector Norms Given vectors x and y of length one, which are simply scalars x and y, the
More informationMathematical preliminaries and error analysis
Mathematical preliminaries and error analysis Tsung-Ming Huang Department of Mathematics National Taiwan Normal University, Taiwan September 12, 2015 Outline 1 Round-off errors and computer arithmetic
More informationCME 302: NUMERICAL LINEAR ALGEBRA FALL 2005/06 LECTURE 6
CME 302: NUMERICAL LINEAR ALGEBRA FALL 2005/06 LECTURE 6 GENE H GOLUB Issues with Floating-point Arithmetic We conclude our discussion of floating-point arithmetic by highlighting two issues that frequently
More informationFloating-point Computation
Chapter 2 Floating-point Computation 21 Positional Number System An integer N in a number system of base (or radix) β may be written as N = a n β n + a n 1 β n 1 + + a 1 β + a 0 = P n (β) where a i are
More informationNotes for Chapter 1 of. Scientific Computing with Case Studies
Notes for Chapter 1 of Scientific Computing with Case Studies Dianne P. O Leary SIAM Press, 2008 Mathematical modeling Computer arithmetic Errors 1999-2008 Dianne P. O'Leary 1 Arithmetic and Error What
More informationArithmetic and Error. How does error arise? How does error arise? Notes for Part 1 of CMSC 460
Notes for Part 1 of CMSC 460 Dianne P. O Leary Preliminaries: Mathematical modeling Computer arithmetic Errors 1999-2006 Dianne P. O'Leary 1 Arithmetic and Error What we need to know about error: -- how
More informationComputer Arithmetic. MATH 375 Numerical Analysis. J. Robert Buchanan. Fall Department of Mathematics. J. Robert Buchanan Computer Arithmetic
Computer Arithmetic MATH 375 Numerical Analysis J. Robert Buchanan Department of Mathematics Fall 2013 Machine Numbers When performing arithmetic on a computer (laptop, desktop, mainframe, cell phone,
More informationIntroduction CSE 541
Introduction CSE 541 1 Numerical methods Solving scientific/engineering problems using computers. Root finding, Chapter 3 Polynomial Interpolation, Chapter 4 Differentiation, Chapter 4 Integration, Chapters
More informationLecture Notes 7, Math/Comp 128, Math 250
Lecture Notes 7, Math/Comp 128, Math 250 Misha Kilmer Tufts University October 23, 2005 Floating Point Arithmetic We talked last time about how the computer represents floating point numbers. In a floating
More informationIntroduction to Scientific Computing Languages
1 / 21 Introduction to Scientific Computing Languages Prof. Paolo Bientinesi pauldj@aices.rwth-aachen.de Numerical Representation 2 / 21 Numbers 123 = (first 40 digits) 29 4.241379310344827586206896551724137931034...
More informationChapter 1 Error Analysis
Chapter 1 Error Analysis Several sources of errors are important for numerical data processing: Experimental uncertainty: Input data from an experiment have a limited precision. Instead of the vector of
More informationElements of Floating-point Arithmetic
Elements of Floating-point Arithmetic Sanzheng Qiao Department of Computing and Software McMaster University July, 2012 Outline 1 Floating-point Numbers Representations IEEE Floating-point Standards Underflow
More informationChapter 1: Introduction and mathematical preliminaries
Chapter 1: Introduction and mathematical preliminaries Evy Kersalé September 26, 2011 Motivation Most of the mathematical problems you have encountered so far can be solved analytically. However, in real-life,
More informationChapter 1 Computer Arithmetic
Numerical Analysis (Math 9372) 2017-2016 Chapter 1 Computer Arithmetic 1.1 Introduction Numerical analysis is a way to solve mathematical problems by special procedures which use arithmetic operations
More informationNumerical Algorithms. IE 496 Lecture 20
Numerical Algorithms IE 496 Lecture 20 Reading for This Lecture Primary Miller and Boxer, Pages 124-128 Forsythe and Mohler, Sections 1 and 2 Numerical Algorithms Numerical Analysis So far, we have looked
More informationMAT 460: Numerical Analysis I. James V. Lambers
MAT 460: Numerical Analysis I James V. Lambers January 31, 2013 2 Contents 1 Mathematical Preliminaries and Error Analysis 7 1.1 Introduction............................ 7 1.1.1 Error Analysis......................
More informationElements of Floating-point Arithmetic
Elements of Floating-point Arithmetic Sanzheng Qiao Department of Computing and Software McMaster University July, 2012 Outline 1 Floating-point Numbers Representations IEEE Floating-point Standards Underflow
More informationIntroduction to Scientific Computing Languages
1 / 19 Introduction to Scientific Computing Languages Prof. Paolo Bientinesi pauldj@aices.rwth-aachen.de Numerical Representation 2 / 19 Numbers 123 = (first 40 digits) 29 4.241379310344827586206896551724137931034...
More information1.1 COMPUTER REPRESENTATION OF NUM- BERS, REPRESENTATION ERRORS
Chapter 1 NUMBER REPRESENTATION, ERROR ANALYSIS 1.1 COMPUTER REPRESENTATION OF NUM- BERS, REPRESENTATION ERRORS Floating-point representation x t,r of a number x: x t,r = m t P cr, where: P - base (the
More informationHow do computers represent numbers?
How do computers represent numbers? Tips & Tricks Week 1 Topics in Scientific Computing QMUL Semester A 2017/18 1/10 What does digital mean? The term DIGITAL refers to any device that operates on discrete
More informationERROR AND SENSITIVTY ANALYSIS FOR SYSTEMS OF LINEAR EQUATIONS. Perturbation analysis for linear systems (Ax = b)
ERROR AND SENSITIVTY ANALYSIS FOR SYSTEMS OF LINEAR EQUATIONS Conditioning of linear systems. Estimating errors for solutions of linear systems Backward error analysis Perturbation analysis for linear
More informationRound-off Errors and Computer Arithmetic - (1.2)
Round-off Errors and Comuter Arithmetic - (.). Round-off Errors: Round-off errors is roduced when a calculator or comuter is used to erform real number calculations. That is because the arithmetic erformed
More informationHomework 2 Foundations of Computational Math 1 Fall 2018
Homework 2 Foundations of Computational Math 1 Fall 2018 Note that Problems 2 and 8 have coding in them. Problem 2 is a simple task while Problem 8 is very involved (and has in fact been given as a programming
More informationNumerical Methods - Preliminaries
Numerical Methods - Preliminaries Y. K. Goh Universiti Tunku Abdul Rahman 2013 Y. K. Goh (UTAR) Numerical Methods - Preliminaries 2013 1 / 58 Table of Contents 1 Introduction to Numerical Methods Numerical
More informationEAD 115. Numerical Solution of Engineering and Scientific Problems. David M. Rocke Department of Applied Science
EAD 115 Numerical Solution of Engineering and Scientific Problems David M. Rocke Department of Applied Science Computer Representation of Numbers Counting numbers (unsigned integers) are the numbers 0,
More informationErrors. Intensive Computation. Annalisa Massini 2017/2018
Errors Intensive Computation Annalisa Massini 2017/2018 Intensive Computation - 2017/2018 2 References Scientific Computing: An Introductory Survey - Chapter 1 M.T. Heath http://heath.cs.illinois.edu/scicomp/notes/index.html
More information1 ERROR ANALYSIS IN COMPUTATION
1 ERROR ANALYSIS IN COMPUTATION 1.2 Round-Off Errors & Computer Arithmetic (a) Computer Representation of Numbers Two types: integer mode (not used in MATLAB) floating-point mode x R ˆx F(β, t, l, u),
More informationOutline. Math Numerical Analysis. Errors. Lecture Notes Linear Algebra: Part B. Joseph M. Mahaffy,
Math 54 - Numerical Analysis Lecture Notes Linear Algebra: Part B Outline Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Department of Mathematics and Statistics Dynamical Systems Group Computational Sciences
More informationChapter 1 Mathematical Preliminaries and Error Analysis
Numerical Analysis (Math 3313) 2019-2018 Chapter 1 Mathematical Preliminaries and Error Analysis Intended learning outcomes: Upon successful completion of this chapter, a student will be able to (1) list
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra)
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 09: Accuracy and Stability Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 12 Outline 1 Condition Number of Matrices
More informationBinary floating point
Binary floating point Notes for 2017-02-03 Why do we study conditioning of problems? One reason is that we may have input data contaminated by noise, resulting in a bad solution even if the intermediate
More informationTu: 9/3/13 Math 471, Fall 2013, Section 001 Lecture 1
Tu: 9/3/13 Math 71, Fall 2013, Section 001 Lecture 1 1 Course intro Notes : Take attendance. Instructor introduction. Handout : Course description. Note the exam days (and don t be absent). Bookmark the
More informationALU (3) - Division Algorithms
HUMBOLDT-UNIVERSITÄT ZU BERLIN INSTITUT FÜR INFORMATIK Lecture 12 ALU (3) - Division Algorithms Sommersemester 2002 Leitung: Prof. Dr. Miroslaw Malek www.informatik.hu-berlin.de/rok/ca CA - XII - ALU(3)
More informationAn Introduction to Numerical Analysis
An Introduction to Numerical Analysis Department of Mathematical Sciences, NTNU 21st august 2012 Practical issues webpage: http://wiki.math.ntnu.no/tma4215/2012h/start Lecturer:, elenac@math.ntnu.no Lecures:
More informationApplied Mathematics 205. Unit 0: Overview of Scientific Computing. Lecturer: Dr. David Knezevic
Applied Mathematics 205 Unit 0: Overview of Scientific Computing Lecturer: Dr. David Knezevic Scientific Computing Computation is now recognized as the third pillar of science (along with theory and experiment)
More informationNumbering Systems. Contents: Binary & Decimal. Converting From: B D, D B. Arithmetic operation on Binary.
Numbering Systems Contents: Binary & Decimal. Converting From: B D, D B. Arithmetic operation on Binary. Addition & Subtraction using Octal & Hexadecimal 2 s Complement, Subtraction Using 2 s Complement.
More informationEE731 Lecture Notes: Matrix Computations for Signal Processing
EE731 Lecture Notes: Matrix Computations for Signal Processing James P. Reilly c Department of Electrical and Computer Engineering McMaster University November 2, 2006 Lecture 4 In this lecture, we first
More informationMath 411 Preliminaries
Math 411 Preliminaries Provide a list of preliminary vocabulary and concepts Preliminary Basic Netwon s method, Taylor series expansion (for single and multiple variables), Eigenvalue, Eigenvector, Vector
More informationChapter 1 Mathematical Preliminaries and Error Analysis
Chapter 1 Mathematical Preliminaries and Error Analysis Per-Olof Persson persson@berkeley.edu Department of Mathematics University of California, Berkeley Math 128A Numerical Analysis Limits and Continuity
More informationGaussian Elimination for Linear Systems
Gaussian Elimination for Linear Systems Tsung-Ming Huang Department of Mathematics National Taiwan Normal University October 3, 2011 1/56 Outline 1 Elementary matrices 2 LR-factorization 3 Gaussian elimination
More informationChapter 1: Preliminaries and Error Analysis
Chapter 1: Error Analysis Peter W. White white@tarleton.edu Department of Tarleton State University Summer 2015 / Numerical Analysis Overview We All Remember Calculus Derivatives: limit definition, sum
More informationAccurate polynomial evaluation in floating point arithmetic
in floating point arithmetic Université de Perpignan Via Domitia Laboratoire LP2A Équipe de recherche en Informatique DALI MIMS Seminar, February, 10 th 2006 General motivation Provide numerical algorithms
More informationChapter 4 Number Representations
Chapter 4 Number Representations SKEE2263 Digital Systems Mun im/ismahani/izam {munim@utm.my,e-izam@utm.my,ismahani@fke.utm.my} February 9, 2016 Table of Contents 1 Fundamentals 2 Signed Numbers 3 Fixed-Point
More informationAn Introduction to Numerical Analysis. James Brannick. The Pennsylvania State University
An Introduction to Numerical Analysis James Brannick The Pennsylvania State University Contents Chapter 1. Introduction 5 Chapter 2. Computer arithmetic and Error Analysis 7 Chapter 3. Approximation and
More informationMATH ASSIGNMENT 03 SOLUTIONS
MATH444.0 ASSIGNMENT 03 SOLUTIONS 4.3 Newton s method can be used to compute reciprocals, without division. To compute /R, let fx) = x R so that fx) = 0 when x = /R. Write down the Newton iteration for
More informationQUADRATIC PROGRAMMING?
QUADRATIC PROGRAMMING? WILLIAM Y. SIT Department of Mathematics, The City College of The City University of New York, New York, NY 10031, USA E-mail: wyscc@cunyvm.cuny.edu This is a talk on how to program
More informationNumerical Analysis. Yutian LI. 2018/19 Term 1 CUHKSZ. Yutian LI (CUHKSZ) Numerical Analysis 2018/19 1 / 41
Numerical Analysis Yutian LI CUHKSZ 2018/19 Term 1 Yutian LI (CUHKSZ) Numerical Analysis 2018/19 1 / 41 Reference Books BF R. L. Burden and J. D. Faires, Numerical Analysis, 9th edition, Thomsom Brooks/Cole,
More informationIntroduction and mathematical preliminaries
Chapter Introduction and mathematical preliminaries Contents. Motivation..................................2 Finite-digit arithmetic.......................... 2.3 Errors in numerical calculations.....................
More informationNumerical Linear Algebra Primer. Ryan Tibshirani Convex Optimization /36-725
Numerical Linear Algebra Primer Ryan Tibshirani Convex Optimization 10-725/36-725 Last time: proximal gradient descent Consider the problem min g(x) + h(x) with g, h convex, g differentiable, and h simple
More informationCompute the behavior of reality even if it is impossible to observe the processes (for example a black hole in astrophysics).
1 Introduction Read sections 1.1, 1.2.1 1.2.4, 1.2.6, 1.3.8, 1.3.9, 1.4. Review questions 1.1 1.6, 1.12 1.21, 1.37. The subject of Scientific Computing is to simulate the reality. Simulation is the representation
More informationINTRODUCTION TO COMPUTATIONAL MATHEMATICS
INTRODUCTION TO COMPUTATIONAL MATHEMATICS Course Notes for CM 271 / AMATH 341 / CS 371 Fall 2007 Instructor: Prof. Justin Wan School of Computer Science University of Waterloo Course notes by Prof. Hans
More informationNumerical Methods - Lecture 2. Numerical Methods. Lecture 2. Analysis of errors in numerical methods
Numerical Methods - Lecture 1 Numerical Methods Lecture. Analysis o errors in numerical methods Numerical Methods - Lecture Why represent numbers in loating point ormat? Eample 1. How a number 56.78 can
More information1 Error analysis for linear systems
Notes for 2016-09-16 1 Error analysis for linear systems We now discuss the sensitivity of linear systems to perturbations. This is relevant for two reasons: 1. Our standard recipe for getting an error
More informationNumerical Analysis and Computing
Numerical Analysis and Computing Lecture Notes #02 Calculus Review; Computer Artihmetic and Finite Precision; and Convergence; Joe Mahaffy, mahaffy@math.sdsu.edu Department of Mathematics Dynamical Systems
More informationComputation of dot products in finite fields with floating-point arithmetic
Computation of dot products in finite fields with floating-point arithmetic Stef Graillat Joint work with Jérémy Jean LIP6/PEQUAN - Université Pierre et Marie Curie (Paris 6) - CNRS Computer-assisted proofs
More informationNumerical Linear Algebra Primer. Ryan Tibshirani Convex Optimization
Numerical Linear Algebra Primer Ryan Tibshirani Convex Optimization 10-725 Consider Last time: proximal Newton method min x g(x) + h(x) where g, h convex, g twice differentiable, and h simple. Proximal
More informationMAT128A: Numerical Analysis Lecture Three: Condition Numbers
MAT128A: Numerical Analysis Lecture Three: Condition Numbers October 1, 2018 Lecture 1 October 1, 2018 1 / 26 An auspicious example Last time, we saw that the naive evaluation of the function f (x) = 1
More informationLecture 1. Introduction to Numerical Methods. T. Gambill. Department of Computer Science University of Illinois at Urbana-Champaign.
Lecture 1 Introduction to Numerical Methods T. Gambill Department of Computer Science University of Illinois at Urbana-Champaign June 10, 2013 T. Gambill (UIUC) CS 357 June 10, 2013 1 / 52 Course Info
More informationLet x be an approximate solution for Ax = b, e.g., obtained by Gaussian elimination. Let x denote the exact solution. Call. r := b A x.
ESTIMATION OF ERROR Let x be an approximate solution for Ax = b, e.g., obtained by Gaussian elimination. Let x denote the exact solution. Call the residual for x. Then r := b A x r = b A x = Ax A x = A
More informationLecture notes to the course. Numerical Methods I. Clemens Kirisits
Lecture notes to the course Numerical Methods I Clemens Kirisits November 8, 08 ii Preface These lecture notes are intended as a written companion to the course Numerical Methods I taught from 06 to 08
More informationBASIC COMPUTER ARITHMETIC
BASIC COMPUTER ARITHMETIC TSOGTGEREL GATUMUR Abstract. First, we consider how integers and fractional numbers are represented and manipulated internally on a computer. Then we develop a basic theoretical
More informationLecture 2 MODES OF NUMERICAL COMPUTATION
1. Diversity of Numbers Lecture 2 Page 1 It is better to solve the right problem the wrong way than to solve the wrong problem the right way. The purpose of computing is insight, not numbers. Richard Wesley
More informationNumber Representation and Waveform Quantization
1 Number Representation and Waveform Quantization 1 Introduction This lab presents two important concepts for working with digital signals. The first section discusses how numbers are stored in memory.
More informationMATH Dr. Halimah Alshehri Dr. Halimah Alshehri
MATH 1101 haalshehri@ksu.edu.sa 1 Introduction To Number Systems First Section: Binary System Second Section: Octal Number System Third Section: Hexadecimal System 2 Binary System 3 Binary System The binary
More informationPowerPoints organized by Dr. Michael R. Gustafson II, Duke University
Part 1 Chapter 4 Roundoff and Truncation Errors PowerPoints organized by Dr. Michael R. Gustafson II, Duke University All images copyright The McGraw-Hill Companies, Inc. Permission required for reproduction
More informationECE260: Fundamentals of Computer Engineering
Data Representation & 2 s Complement James Moscola Dept. of Engineering & Computer Science York College of Pennsylvania Based on Computer Organization and Design, 5th Edition by Patterson & Hennessy Data
More informationROUNDOFF ERRORS; BACKWARD STABILITY
SECTION.5 ROUNDOFF ERRORS; BACKWARD STABILITY ROUNDOFF ERROR -- error due to the finite representation (usually in floatingpoint form) of real (and complex) numers in digital computers. FLOATING-POINT
More informationBinary Floating-Point Numbers
Binary Floating-Point Numbers S exponent E significand M F=(-1) s M β E Significand M pure fraction [0, 1-ulp] or [1, 2) for β=2 Normalized form significand has no leading zeros maximum # of significant
More informationThe numerical stability of barycentric Lagrange interpolation
IMA Journal of Numerical Analysis (2004) 24, 547 556 The numerical stability of barycentric Lagrange interpolation NICHOLAS J. HIGHAM Department of Mathematics, University of Manchester, Manchester M13
More informationIntroduction to Scientific Computing
(Lecture 2: Machine precision and condition number) B. Rosić, T.Moshagen Institute of Scientific Computing General information :) 13 homeworks (HW) Work in groups of 2 or 3 people Each HW brings maximally
More informationNumerical Methods. Dr Dana Mackey. School of Mathematical Sciences Room A305 A Dana Mackey (DIT) Numerical Methods 1 / 12
Numerical Methods Dr Dana Mackey School of Mathematical Sciences Room A305 A Email: Dana.Mackey@dit.ie Dana Mackey (DIT) Numerical Methods 1 / 12 Practical Information The typed notes will be available
More informationMath 128A: Homework 2 Solutions
Math 128A: Homework 2 Solutions Due: June 28 1. In problems where high precision is not needed, the IEEE standard provides a specification for single precision numbers, which occupy 32 bits of storage.
More information1 Finding and fixing floating point problems
Notes for 2016-09-09 1 Finding and fixing floating point problems Floating point arithmetic is not the same as real arithmetic. Even simple properties like associativity or distributivity of addition and
More information8/13/16. Data analysis and modeling: the tools of the trade. Ø Set of numbers. Ø Binary representation of numbers. Ø Floating points.
Data analysis and modeling: the tools of the trade Patrice Koehl Department of Biological Sciences National University of Singapore http://www.cs.ucdavis.edu/~koehl/teaching/bl5229 koehl@cs.ucdavis.edu
More informationApplied Numerical Analysis (AE2220-I) R. Klees and R.P. Dwight
Applied Numerical Analysis (AE0-I) R. Klees and R.P. Dwight February 018 Contents 1 Preliminaries: Motivation, Computer arithmetic, Taylor series 1 1.1 Numerical Analysis Motivation..........................
More informationDense LU factorization and its error analysis
Dense LU factorization and its error analysis Laura Grigori INRIA and LJLL, UPMC February 2016 Plan Basis of floating point arithmetic and stability analysis Notation, results, proofs taken from [N.J.Higham,
More information1 Backward and Forward Error
Math 515 Fall, 2008 Brief Notes on Conditioning, Stability and Finite Precision Arithmetic Most books on numerical analysis, numerical linear algebra, and matrix computations have a lot of material covering
More informationThe Euclidean Division Implemented with a Floating-Point Multiplication and a Floor
The Euclidean Division Implemented with a Floating-Point Multiplication and a Floor Vincent Lefèvre 11th July 2005 Abstract This paper is a complement of the research report The Euclidean division implemented
More informationLecture 9: Numerical Linear Algebra Primer (February 11st)
10-725/36-725: Convex Optimization Spring 2015 Lecture 9: Numerical Linear Algebra Primer (February 11st) Lecturer: Ryan Tibshirani Scribes: Avinash Siravuru, Guofan Wu, Maosheng Liu Note: LaTeX template
More informationLecture 28 The Main Sources of Error
Lecture 28 The Main Sources of Error Truncation Error Truncation error is defined as the error caused directly by an approximation method For instance, all numerical integration methods are approximations
More informationThe answer in each case is the error in evaluating the taylor series for ln(1 x) for x = which is 6.9.
Brad Nelson Math 26 Homework #2 /23/2. a MATLAB outputs: >> a=(+3.4e-6)-.e-6;a- ans = 4.449e-6 >> a=+(3.4e-6-.e-6);a- ans = 2.224e-6 And the exact answer for both operations is 2.3e-6. The reason why way
More information4.2 Floating-Point Numbers
101 Approximation 4.2 Floating-Point Numbers 4.2 Floating-Point Numbers The number 3.1416 in scientific notation is 0.31416 10 1 or (as computer output) -0.31416E01..31416 10 1 exponent sign mantissa base
More informationLecture Notes to be used in conjunction with. 233 Computational Techniques
Lecture Notes to be used in conjunction with 233 Computational Techniques István Maros Department of Computing Imperial College London V29e January 2008 CONTENTS i Contents 1 Introduction 1 2 Computation
More informationMA580: Numerical Analysis: I. Zhilin Li
MA580: Numerical Analysis: I Zhilin Li Chapter 1 Introduction Why is this course important (motivations)? What is the role of this class in the problem solving process using mathematics and computers?
More informationThe Classical Relative Error Bounds for Computing a2 + b 2 and c/ a 2 + b 2 in Binary Floating-Point Arithmetic are Asymptotically Optimal
-1- The Classical Relative Error Bounds for Computing a2 + b 2 and c/ a 2 + b 2 in Binary Floating-Point Arithmetic are Asymptotically Optimal Claude-Pierre Jeannerod Jean-Michel Muller Antoine Plet Inria,
More informationMathematics for Engineers. Numerical mathematics
Mathematics for Engineers Numerical mathematics Integers Determine the largest representable integer with the intmax command. intmax ans = int32 2147483647 2147483647+1 ans = 2.1475e+09 Remark The set
More informationDSP Design Lecture 2. Fredrik Edman.
DSP Design Lecture Number representation, scaling, quantization and round-off Noise Fredrik Edman fredrik.edman@eit.lth.se Representation of Numbers Numbers is a way to use symbols to describe and model
More information14:332:231 DIGITAL LOGIC DESIGN. Why Binary Number System?
:33:3 DIGITAL LOGIC DESIGN Ivan Marsic, Rutgers University Electrical & Computer Engineering Fall 3 Lecture #: Binary Number System Complement Number Representation X Y Why Binary Number System? Because
More informationRoundoff Analysis of Gaussian Elimination
Jim Lambers MAT 60 Summer Session 2009-0 Lecture 5 Notes These notes correspond to Sections 33 and 34 in the text Roundoff Analysis of Gaussian Elimination In this section, we will perform a detailed error
More informationWhat is it we are looking for in these algorithms? We want algorithms that are
Fundamentals. Preliminaries The first question we want to answer is: What is computational mathematics? One possible definition is: The study of algorithms for the solution of computational problems in
More informationCISE-301: Numerical Methods Topic 1:
CISE-3: Numerical Methods Topic : Introduction to Numerical Methods and Taylor Series Lectures -4: KFUPM Term 9 Section 8 CISE3_Topic KFUPM - T9 - Section 8 Lecture Introduction to Numerical Methods What
More information