Dense LU factorization and its error analysis
|
|
- Gary Walton
- 5 years ago
- Views:
Transcription
1 Dense LU factorization and its error analysis Laura Grigori INRIA and LJLL, UPMC February 2016
2 Plan Basis of floating point arithmetic and stability analysis Notation, results, proofs taken from [N.J.Higham, 2002] Direct methods of factorization LU factorization Error analysis of LU factorization - main results Block LU factorization 2 of 33
3 Plan Basis of floating point arithmetic and stability analysis Notation, results, proofs taken from [N.J.Higham, 2002] Direct methods of factorization 3 of 33
4 Norms and other notations A F = n i=1 j=1 n a ij 2 A 2 = σ max (A) n A = max 1 i n a ij A 1 = max 1 j n j=1 n a ij i=1 Inequalities x y and A B hold componentwise. 4 of 33
5 Floating point arithmetic The machine precision or unit roundoff is u The maximum relative error for a given rounding procedure u is of order 10 8 in single precision, in double precision Another definition: the smallest number that added to one gives a result different from one The evaluation involving basic arithmetic operations +,,, / in floating point satisfies fl(x op y) = (x op y)(1 + δ), δ u 5 of 33
6 Relative error Given a real number x and its approximation ˆx, the absolute error and the relative errors are E abs (ˆx) = x ˆx, E rel (ˆx) = x ˆx x (1) The relative error is scale independent Some examples, outline the difference with correct significant digits x = , ˆx = , E rel (ˆx) = x = , ˆx = , E rel (ˆx) = When x is a vector, the componentwise relative error is max i x i ˆx i x i 6 of 33
7 Backward and Forward errors Consider y = f (x) a scalar function of a real scalar variable and ŷ its approximation. Ideally we would like the forward error E rel (ŷ) u Instead we focus on the backward error, For what set of data we have solved the problem? that is we look for min x such that ŷ = f (x + x) 7 of 33
8 Condition number Assume f is twice continuously differentiable, then ŷ y = f (x + x) f (x) = f (x) x + ( ŷ y xf ) (x) x = y f (x) x + O(( x)2 ) f (x + τ x) ( x) 2, τ (0, 1) 2! The condition number is c(x) = xf (x) f (x) Rule of thumb When consistently defined, we have forward error condition number backward error 8 of 33
9 Preliminaries Lemma (Lemma 3.1 in [N.J.Higham, 2002]) If δ i u and ρ i = ±1 for i = 1 : n, and nu < 1, then n (1 + δ i ) ρ i = 1 + Θ n, Θ n nu 1 nu = γ n i=1 Other notations γ n = cnu 1 cnu 9 of 33
10 Inner product in floating point arithmetic Consider computing s n = x T y, with an evaluation from left to right. We denote different errors as 1 + δ i 1 ± δ ŝ 1 = fl(x 1 y 1 ) = x 1 y 1 (1 ± δ) ŝ 2 = fl(ŝ 1 + x 2 y 2 ) = (ŝ 1 + x 2 y 2 (1 ± δ))(1 ± δ) = x 1 y 1 (1 ± δ) 2 + x 2 y 2 (1 ± δ) 2. ŝ n = x 1 y 1 (1 ± δ) n + x 2 y 2 (1 ± δ) n + x 3 y 3 (1 ± δ) n x n y n (1 ± δ) 2 After applying the previous lemma, we obtain ŝ n = x 1 y 1 (1 + Θ n ) + x 2 y 2 (1 + Θ n) x n y n (1 + Θ 2 ) 10 of 33
11 Inner product in FP arithmetic - error bounds We obtain the following backward and forward errors ŝ n = x 1 y 1 (1 + Θ n ) + x 2 y 2 (1 + Θ n) x n y n (1 + Θ 2 ) fl(x T y) = (x + x) T y = x T (y + y), x γ n x, y γ n y, x T y fl(x T y) n γ n x i y i = γ n x T y i=1 High relative accuracy is obtained when computing x T x No guarantee of high accuracy when x T y << x T y 11 of 33
12 Plan Basis of floating point arithmetic and stability analysis Direct methods of factorization LU factorization Block LU factorization 12 of 33
13 Algebra of the LU factorization LU factorization Compute the factorization PA = LU Example Given the matrix Let M 1 = A = , M 1 A = of 33
14 Algebra of the LU factorization In general I k 1 1 A (k+1) = M k A (k) := m k+1,k 1 A (k), where m n,k 1 M k = I m k e T k, M 1 k = I + m k e T k where e k is the k-th unit vector, ei T m k = 0, i k The factorization can be written as M n 1... M 1 A = A (n) = U 14 of 33
15 Algebra of the LU factorization We obtain A = M M 1 n 1 U = (I + m 1 e1 T )... (I + m n 1 en 1)U T ( ) n 1 = I + m i ei T U = i=1 1 m U = LU m n1 m n of 33
16 The need for pivoting For stability, avoid division by small diagonal elements For example A = (2) has an LU factorization if we permute the rows of matrix A PA = = (3) Partial pivoting allows to bound the multipliers m ik 1 and hence L 1 16 of 33
17 Existence of the LU factorization Theorem Given a full rank matrix A of size m n, m n, the matrix A can be decomposed as A = PLU where P is a permutation matrix of size m m, L is a unit lower triangular matrix of size m n and U is a nonsingular upper triangular matrix of size n n. Proof: simpler proof for the square case. Since A is full rank, there is a permutation P 1 such that P 1 a 11 is nonzero. Write the factorization as ( ) ( ) ( ) a11 A P 1 A = a11 A = 12 A 21 A 22 A 21 /a 11 I 0 A 22 a 1 11 A, 21A 12 where S = A 22 a 1 11 A 21A 12. Since det(a) 0, then det(s) 0. Continue the proof by induction on S. 17 of 33
18 Solving Ax = b by using Gaussian elimination Composed of 4 steps 1. Factor A = PLU, (2/3)n 3 ) flops 2. Compute P T b to solve LUx = P T b 3. Forward substitution: solve Ly = P T b, n 2 flops 4. Backward substitution: solve Ux = y, n 2 flops 18 of 33
19 Algorithm to compute the LU factorization Algorithm for computing the in place LU factorization of a matrix of size n n. #flops = 2n 3 /3 1: for k = 1:n-1 do 2: Let a ik be the element of maximum magnitude in A(k : n, k) 3: Permute row i and row k 4: A(k + 1 : n, k) = A(k + 1 : n, k)/a kk 5: for i = k + 1 : n do 6: for j = k + 1 : n do 7: a ij = a ij a ik a kj 8: end for 9: end for 10: end for 19 of 33
20 Algorithm to compute the LU factorization Left looking approach, pivoting ignored, A of size m n #flops = n 2 m n 3 /3 1: for k = 1:n do 2: for j = k:n do 3: u kj = a kj k 1 i=1 l kiu ij 4: end for 5: for i = k+1:m do 6: l ik = (a ik k 1 j=1 l iju jk )/u kk 7: end for 8: end for 20 of 33
21 Error analysis of the LU factorization Given the first k 1 columns of L and k 1 rows of U were computed, we have a kj = l k1 u 1j l k,k 1 u k 1,j + u kj, j = k : n a ik = l i1 u 1k l ik u kk, i = k + 1 : m The computed elements of ˆL and Û satisfy: a k 1 k kj ˆl ki û ij û kj γ k ˆl ki û ij, j k, i=1 i=1 k k a ik ˆl ij û jk γ k ˆl ij û jk, i > k. j=1 j=1 21 of 33
22 Error analysis of the LU factorization (continued) Theorem (Theorem 9.3 in [N.J.Higham, 2002]) Let A R m n, m n and let ˆL R m n and Û Rn n be its computed LU factors obtained by Gaussian elimination (suppose there was no failure during GE). Then, ˆLÛ = A + A, A γ n ˆL Û. Theorem (Theorem 9.4 in [N.J.Higham, 2002]) Let A R m n, m n and let ˆx be the computed solution to Ax = b obtained by using the computed LU factors of A obtained by Gaussian elimination. Then (A + A)ˆx = b, A γ 3n ˆL Û. 22 of 33
23 Error analysis of Ax = b Theorem (Theorem 9.4 in [N.J.Higham, 2002] continued) Proof. We have the following: Thus (A + A)ˆx = b, A γ 3n ˆL Û. ˆLÛ = A + A, A γn ˆL Û, (ˆL + L)ŷ = b, L γ n ˆL, (Û + U)ˆx = ŷ, U γn Û. b = (ˆL + L)(Û + U)ˆx = (A + A 1 + ˆL U + LÛ + L U)ˆx = (A + A)ˆx, where 2 A (3γ n + γn) ˆL Û γ3n ˆL Û. 23 of 33
24 Wilkinson s backward error stability result Growth factor g W defined as Note that g W = max i,j,k a k ij max i,j a ij u ij = aij i g W max a ij i,j Theorem (Wilkinson s backward error stability result, see also [N.J.Higham, 2002] for more details) Let A R n n and let ˆx be the computed solution of Ax = b obtained by using GEPP. Then (A + A)ˆx = b, A n 2 γ 3n g W (n) A. 24 of 33
25 The growth factor The LU factorization is backward stable if the growth factor is small (grows linearly with n). For partial pivoting, the growth factor g(n) 2 n 1, and this bound is attainable. In practice it is on the order of n 2/3 n 1/2 Exponential growth factor for Wilkinson matrix A = diag(±1) of 33
26 Experimental results for special matrices Several errror bounds for GEPP, the normwise backward error η and the componentwise backward error w (r = b Ax). η = w = max i r 1 A 1 x 1 + b 1, r i ( A x + b ) i. matrix cond(a,2) g W L 1 cond(u,1) PA LU F A F η w b hadamard 1.0E+0 4.1E+3 4.1E+3 5.3E+5 0.0E+0 3.3E E-15 randsvd 6.7E+7 4.7E+0 9.9E+2 1.4E E E E-15 chebvand 3.8E E+2 2.2E+3 4.8E E E E-16 frank 1.7E E+0 2.0E+0 1.9E E E E-23 hilb 8.0E E+0 3.1E+3 2.2E E E E of 33
27 Block formulation of the LU factorization Partitioning of matrix A of size n n [ ] A11 A A = 12 A 21 A 22 where A 11 is of size b b, A 21 is of size (m b) b, A 12 is of size b (n b) and A 22 is of size (m b) (n b). Block LU algebra The first iteration computes the factorization: [ ] P1 T L11 A = L 21 I n b [ Ib A 1 ] [ ] U11 U 12 I n b The algorithm continues recursively on the trailing matrix A of 33
28 Block LU factorization - the algorithm 1. Compute the LU factorization with partial pivoting of the first block column ( ) ( ) A11 L11 P 1 = U A L Pivot by applying the permutation matrix P T 1 3. Solve the triangular system 4. Update the trailing matrix, Ā = P T 1 A. L 11 U 12 = Ā12 A 1 = Ā 22 L 21 U Compute recursively the block LU factorization of A 1. on the entire matrix, 28 of 33
29 LU Factorization as in ScaLAPACK LU factorization on a P = Pr x Pc grid of processors For ib = 1 to n-1 step b A(ib) = A(ib : n, ib : n) 1. Compute panel factorization find pivot in each column, swap rows 2. Apply all row permutations broadcast pivot information along the rows swap rows at left and right 3. Compute block row of U broadcast right diagonal block of L of current panel 4. Update trailing matrix broadcast right block column of L broadcast down block row of U 29 of 33
30 Cost of LU Factorization in ScaLAPACK LU factorization on a P = Pr x Pc grid of processors For ib = 1 to n-1 step b A(ib) = A(ib : n, ib : n) 1. Compute panel factorization #messages = O(n log 2 P r ) 2. Apply all row permutations #messages = O(n/b(log 2 P r + log 2 P c)) 3. Compute block row of U #messages = O(n/b log 2 P c) 4. Update trailing matrix #messages = O(n/b(log 2 P r + log 2 P c) 30 of 33
31 Cost of parallel block LU Consider that we have a P P grid, block size b ( ) 2/3n 3 γ + n2 b + β n2 log P + P P P α ( 1.5n log P + 3.5n b log P ). 31 of 33
32 Acknowledgement Stability analysis results presented from [N.J.Higham, 2002] Some of the examples taken from [Golub and Van Loan, 1996] 32 of 33
33 References (1) Golub, G. H. and Van Loan, C. F. (1996). Matrix Computations (3rd Ed.). Johns Hopkins University Press, Baltimore, MD, USA. N.J.Higham (2002). Accuracy and Stability of Numerical Algorithms. SIAM, second edition. Schreiber, R. and Loan, C. V. (1989). A storage efficient WY representation for products of Householder transformations. SIAM J. Sci. Stat. Comput., 10(1): of 33
14.2 QR Factorization with Column Pivoting
page 531 Chapter 14 Special Topics Background Material Needed Vector and Matrix Norms (Section 25) Rounding Errors in Basic Floating Point Operations (Section 33 37) Forward Elimination and Back Substitution
More informationScientific Computing
Scientific Computing Direct solution methods Martin van Gijzen Delft University of Technology October 3, 2018 1 Program October 3 Matrix norms LU decomposition Basic algorithm Cost Stability Pivoting Pivoting
More informationLU factorization with Panel Rank Revealing Pivoting and its Communication Avoiding version
1 LU factorization with Panel Rank Revealing Pivoting and its Communication Avoiding version Amal Khabou Advisor: Laura Grigori Université Paris Sud 11, INRIA Saclay France SIAMPP12 February 17, 2012 2
More informationDirect Methods for Solving Linear Systems. Simon Fraser University Surrey Campus MACM 316 Spring 2005 Instructor: Ha Le
Direct Methods for Solving Linear Systems Simon Fraser University Surrey Campus MACM 316 Spring 2005 Instructor: Ha Le 1 Overview General Linear Systems Gaussian Elimination Triangular Systems The LU Factorization
More informationc 2013 Society for Industrial and Applied Mathematics
SIAM J. MATRIX ANAL. APPL. Vol. 34, No. 3, pp. 1401 1429 c 2013 Society for Industrial and Applied Mathematics LU FACTORIZATION WITH PANEL RANK REVEALING PIVOTING AND ITS COMMUNICATION AVOIDING VERSION
More information2.1 Gaussian Elimination
2. Gaussian Elimination A common problem encountered in numerical models is the one in which there are n equations and n unknowns. The following is a description of the Gaussian elimination method for
More informationCSE 160 Lecture 13. Numerical Linear Algebra
CSE 16 Lecture 13 Numerical Linear Algebra Announcements Section will be held on Friday as announced on Moodle Midterm Return 213 Scott B Baden / CSE 16 / Fall 213 2 Today s lecture Gaussian Elimination
More informationGaussian Elimination for Linear Systems
Gaussian Elimination for Linear Systems Tsung-Ming Huang Department of Mathematics National Taiwan Normal University October 3, 2011 1/56 Outline 1 Elementary matrices 2 LR-factorization 3 Gaussian elimination
More informationDirect Methods for Solving Linear Systems. Matrix Factorization
Direct Methods for Solving Linear Systems Matrix Factorization Numerical Analysis (9th Edition) R L Burden & J D Faires Beamer Presentation Slides prepared by John Carroll Dublin City University c 2011
More informationLU Factorization. Marco Chiarandini. DM559 Linear and Integer Programming. Department of Mathematics & Computer Science University of Southern Denmark
DM559 Linear and Integer Programming LU Factorization Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark [Based on slides by Lieven Vandenberghe, UCLA] Outline
More informationNumerical Linear Algebra
Numerical Linear Algebra Decompositions, numerical aspects Gerard Sleijpen and Martin van Gijzen September 27, 2017 1 Delft University of Technology Program Lecture 2 LU-decomposition Basic algorithm Cost
More informationProgram Lecture 2. Numerical Linear Algebra. Gaussian elimination (2) Gaussian elimination. Decompositions, numerical aspects
Numerical Linear Algebra Decompositions, numerical aspects Program Lecture 2 LU-decomposition Basic algorithm Cost Stability Pivoting Cholesky decomposition Sparse matrices and reorderings Gerard Sleijpen
More informationLecture 12 (Tue, Mar 5) Gaussian elimination and LU factorization (II)
Math 59 Lecture 2 (Tue Mar 5) Gaussian elimination and LU factorization (II) 2 Gaussian elimination - LU factorization For a general n n matrix A the Gaussian elimination produces an LU factorization if
More information1 Multiply Eq. E i by λ 0: (λe i ) (E i ) 2 Multiply Eq. E j by λ and add to Eq. E i : (E i + λe j ) (E i )
Direct Methods for Linear Systems Chapter Direct Methods for Solving Linear Systems Per-Olof Persson persson@berkeleyedu Department of Mathematics University of California, Berkeley Math 18A Numerical
More informationLU Factorization with Panel Rank Revealing Pivoting and Its Communication Avoiding Version
LU Factorization with Panel Rank Revealing Pivoting and Its Communication Avoiding Version Amal Khabou James Demmel Laura Grigori Ming Gu Electrical Engineering and Computer Sciences University of California
More informationThis can be accomplished by left matrix multiplication as follows: I
1 Numerical Linear Algebra 11 The LU Factorization Recall from linear algebra that Gaussian elimination is a method for solving linear systems of the form Ax = b, where A R m n and bran(a) In this method
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra)
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 12: Gaussian Elimination and LU Factorization Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 10 Gaussian Elimination
More information7. LU factorization. factor-solve method. LU factorization. solving Ax = b with A nonsingular. the inverse of a nonsingular matrix
EE507 - Computational Techniques for EE 7. LU factorization Jitkomut Songsiri factor-solve method LU factorization solving Ax = b with A nonsingular the inverse of a nonsingular matrix LU factorization
More informationGaussian Elimination and Back Substitution
Jim Lambers MAT 610 Summer Session 2009-10 Lecture 4 Notes These notes correspond to Sections 31 and 32 in the text Gaussian Elimination and Back Substitution The basic idea behind methods for solving
More informationThe Solution of Linear Systems AX = B
Chapter 2 The Solution of Linear Systems AX = B 21 Upper-triangular Linear Systems We will now develop the back-substitution algorithm, which is useful for solving a linear system of equations that has
More informationReview Questions REVIEW QUESTIONS 71
REVIEW QUESTIONS 71 MATLAB, is [42]. For a comprehensive treatment of error analysis and perturbation theory for linear systems and many other problems in linear algebra, see [126, 241]. An overview of
More informationCS412: Lecture #17. Mridul Aanjaneya. March 19, 2015
CS: Lecture #7 Mridul Aanjaneya March 9, 5 Solving linear systems of equations Consider a lower triangular matrix L: l l l L = l 3 l 3 l 33 l n l nn A procedure similar to that for upper triangular systems
More informationCHAPTER 6. Direct Methods for Solving Linear Systems
CHAPTER 6 Direct Methods for Solving Linear Systems. Introduction A direct method for approximating the solution of a system of n linear equations in n unknowns is one that gives the exact solution to
More informationAnalysis of Block LDL T Factorizations for Symmetric Indefinite Matrices
Analysis of Block LDL T Factorizations for Symmetric Indefinite Matrices Haw-ren Fang August 24, 2007 Abstract We consider the block LDL T factorizations for symmetric indefinite matrices in the form LBL
More informationScientific Computing: An Introductory Survey
Scientific Computing: An Introductory Survey Chapter 2 Systems of Linear Equations Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction
More informationLecture Notes to Accompany. Scientific Computing An Introductory Survey. by Michael T. Heath. Chapter 2. Systems of Linear Equations
Lecture Notes to Accompany Scientific Computing An Introductory Survey Second Edition by Michael T. Heath Chapter 2 Systems of Linear Equations Copyright c 2001. Reproduction permitted only for noncommercial,
More informationMatrix decompositions
Matrix decompositions How can we solve Ax = b? 1 Linear algebra Typical linear system of equations : x 1 x +x = x 1 +x +9x = 0 x 1 +x x = The variables x 1, x, and x only appear as linear terms (no powers
More informationNumerical Methods I Solving Square Linear Systems: GEM and LU factorization
Numerical Methods I Solving Square Linear Systems: GEM and LU factorization Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 MATH-GA 2011.003 / CSCI-GA 2945.003, Fall 2014 September 18th,
More informationLecture Note 2: The Gaussian Elimination and LU Decomposition
MATH 5330: Computational Methods of Linear Algebra Lecture Note 2: The Gaussian Elimination and LU Decomposition The Gaussian elimination Xianyi Zeng Department of Mathematical Sciences, UTEP The method
More informationAx = b. Systems of Linear Equations. Lecture Notes to Accompany. Given m n matrix A and m-vector b, find unknown n-vector x satisfying
Lecture Notes to Accompany Scientific Computing An Introductory Survey Second Edition by Michael T Heath Chapter Systems of Linear Equations Systems of Linear Equations Given m n matrix A and m-vector
More informationCALU: A Communication Optimal LU Factorization Algorithm
CALU: A Communication Optimal LU Factorization Algorithm James Demmel Laura Grigori Hua Xiang Electrical Engineering and Computer Sciences University of California at Berkeley Technical Report No. UCB/EECS-010-9
More informationLinear Algebraic Equations
Linear Algebraic Equations 1 Fundamentals Consider the set of linear algebraic equations n a ij x i b i represented by Ax b j with [A b ] [A b] and (1a) r(a) rank of A (1b) Then Axb has a solution iff
More informationI-v k e k. (I-e k h kt ) = Stability of Gauss-Huard Elimination for Solving Linear Systems. 1 x 1 x x x x
Technical Report CS-93-08 Department of Computer Systems Faculty of Mathematics and Computer Science University of Amsterdam Stability of Gauss-Huard Elimination for Solving Linear Systems T. J. Dekker
More informationComputational Methods. Systems of Linear Equations
Computational Methods Systems of Linear Equations Manfred Huber 2010 1 Systems of Equations Often a system model contains multiple variables (parameters) and contains multiple equations Multiple equations
More informationCME 302: NUMERICAL LINEAR ALGEBRA FALL 2005/06 LECTURE 6
CME 302: NUMERICAL LINEAR ALGEBRA FALL 2005/06 LECTURE 6 GENE H GOLUB Issues with Floating-point Arithmetic We conclude our discussion of floating-point arithmetic by highlighting two issues that frequently
More informationNumerical methods for solving linear systems
Chapter 2 Numerical methods for solving linear systems Let A C n n be a nonsingular matrix We want to solve the linear system Ax = b by (a) Direct methods (finite steps); Iterative methods (convergence)
More informationScientific Computing: Dense Linear Systems
Scientific Computing: Dense Linear Systems Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 Course MATH-GA.2043 or CSCI-GA.2112, Spring 2012 February 9th, 2012 A. Donev (Courant Institute)
More informationMTH 464: Computational Linear Algebra
MTH 464: Computational Linear Algebra Lecture Outlines Exam 2 Material Prof. M. Beauregard Department of Mathematics & Statistics Stephen F. Austin State University February 6, 2018 Linear Algebra (MTH
More informationSolving Linear Systems Using Gaussian Elimination. How can we solve
Solving Linear Systems Using Gaussian Elimination How can we solve? 1 Gaussian elimination Consider the general augmented system: Gaussian elimination Step 1: Eliminate first column below the main diagonal.
More informationLU Factorization. LU factorization is the most common way of solving linear systems! Ax = b LUx = b
AM 205: lecture 7 Last time: LU factorization Today s lecture: Cholesky factorization, timing, QR factorization Reminder: assignment 1 due at 5 PM on Friday September 22 LU Factorization LU factorization
More informationCOURSE Numerical methods for solving linear systems. Practical solving of many problems eventually leads to solving linear systems.
COURSE 9 4 Numerical methods for solving linear systems Practical solving of many problems eventually leads to solving linear systems Classification of the methods: - direct methods - with low number of
More informationSince the determinant of a diagonal matrix is the product of its diagonal elements it is trivial to see that det(a) = α 2. = max. A 1 x.
APPM 4720/5720 Problem Set 2 Solutions This assignment is due at the start of class on Wednesday, February 9th. Minimal credit will be given for incomplete solutions or solutions that do not provide details
More informationReview of matrices. Let m, n IN. A rectangle of numbers written like A =
Review of matrices Let m, n IN. A rectangle of numbers written like a 11 a 12... a 1n a 21 a 22... a 2n A =...... a m1 a m2... a mn where each a ij IR is called a matrix with m rows and n columns or an
More informationLAPACK-Style Codes for Pivoted Cholesky and QR Updating. Hammarling, Sven and Higham, Nicholas J. and Lucas, Craig. MIMS EPrint: 2006.
LAPACK-Style Codes for Pivoted Cholesky and QR Updating Hammarling, Sven and Higham, Nicholas J. and Lucas, Craig 2007 MIMS EPrint: 2006.385 Manchester Institute for Mathematical Sciences School of Mathematics
More informationComputational Linear Algebra
Computational Linear Algebra PD Dr. rer. nat. habil. Ralf Peter Mundani Computation in Engineering / BGU Scientific Computing in Computer Science / INF Winter Term 2017/18 Part 2: Direct Methods PD Dr.
More informationNext topics: Solving systems of linear equations
Next topics: Solving systems of linear equations 1 Gaussian elimination (today) 2 Gaussian elimination with partial pivoting (Week 9) 3 The method of LU-decomposition (Week 10) 4 Iterative techniques:
More informationLinear Algebra Section 2.6 : LU Decomposition Section 2.7 : Permutations and transposes Wednesday, February 13th Math 301 Week #4
Linear Algebra Section. : LU Decomposition Section. : Permutations and transposes Wednesday, February 1th Math 01 Week # 1 The LU Decomposition We learned last time that we can factor a invertible matrix
More informationBidiagonal decompositions, minors and applications
Electronic Journal of Linear Algebra Volume 25 Volume 25 (2012) Article 6 2012 Bidiagonal decompositions, minors and applications A. Barreras J. M. Pena jmpena@unizar.es Follow this and additional works
More informationRoundoff Analysis of Gaussian Elimination
Jim Lambers MAT 60 Summer Session 2009-0 Lecture 5 Notes These notes correspond to Sections 33 and 34 in the text Roundoff Analysis of Gaussian Elimination In this section, we will perform a detailed error
More information9. Numerical linear algebra background
Convex Optimization Boyd & Vandenberghe 9. Numerical linear algebra background matrix structure and algorithm complexity solving linear equations with factored matrices LU, Cholesky, LDL T factorization
More informationBLAS: Basic Linear Algebra Subroutines Analysis of the Matrix-Vector-Product Analysis of Matrix-Matrix Product
Level-1 BLAS: SAXPY BLAS-Notation: S single precision (D for double, C for complex) A α scalar X vector P plus operation Y vector SAXPY: y = αx + y Vectorization of SAXPY (αx + y) by pipelining: page 8
More informationCME 302: NUMERICAL LINEAR ALGEBRA FALL 2005/06 LECTURE 9
CME 302: NUMERICAL LINEAR ALGEBRA FALL 2005/06 LECTURE 9 GENE H GOLUB 1 Error Analysis of Gaussian Elimination In this section, we will consider the case of Gaussian elimination and perform a detailed
More informationA Backward Stable Hyperbolic QR Factorization Method for Solving Indefinite Least Squares Problem
A Backward Stable Hyperbolic QR Factorization Method for Solving Indefinite Least Suares Problem Hongguo Xu Dedicated to Professor Erxiong Jiang on the occasion of his 7th birthday. Abstract We present
More informationBindel, Fall 2016 Matrix Computations (CS 6210) Notes for
1 Logistics Notes for 2016-09-14 1. There was a goof in HW 2, problem 1 (now fixed) please re-download if you have already started looking at it. 2. CS colloquium (4:15 in Gates G01) this Thurs is Margaret
More informationSolving Linear Systems of Equations
Solving Linear Systems of Equations Gerald Recktenwald Portland State University Mechanical Engineering Department gerry@me.pdx.edu These slides are a supplement to the book Numerical Methods with Matlab:
More informationLAPACK-Style Codes for Pivoted Cholesky and QR Updating
LAPACK-Style Codes for Pivoted Cholesky and QR Updating Sven Hammarling 1, Nicholas J. Higham 2, and Craig Lucas 3 1 NAG Ltd.,Wilkinson House, Jordan Hill Road, Oxford, OX2 8DR, England, sven@nag.co.uk,
More informationMatrix decompositions
Matrix decompositions How can we solve Ax = b? 1 Linear algebra Typical linear system of equations : x 1 x +x = x 1 +x +9x = 0 x 1 +x x = The variables x 1, x, and x only appear as linear terms (no powers
More informationRank revealing factorizations, and low rank approximations
Rank revealing factorizations, and low rank approximations L. Grigori Inria Paris, UPMC January 2018 Plan Low rank matrix approximation Rank revealing QR factorization LU CRTP: Truncated LU factorization
More informationCommunication avoiding parallel algorithms for dense matrix factorizations
Communication avoiding parallel dense matrix factorizations 1/ 44 Communication avoiding parallel algorithms for dense matrix factorizations Edgar Solomonik Department of EECS, UC Berkeley October 2013
More informationLinear Systems of n equations for n unknowns
Linear Systems of n equations for n unknowns In many application problems we want to find n unknowns, and we have n linear equations Example: Find x,x,x such that the following three equations hold: x
More informationMATH 3511 Lecture 1. Solving Linear Systems 1
MATH 3511 Lecture 1 Solving Linear Systems 1 Dmitriy Leykekhman Spring 2012 Goals Review of basic linear algebra Solution of simple linear systems Gaussian elimination D Leykekhman - MATH 3511 Introduction
More informationPivoting. Reading: GV96 Section 3.4, Stew98 Chapter 3: 1.3
Pivoting Reading: GV96 Section 3.4, Stew98 Chapter 3: 1.3 In the previous discussions we have assumed that the LU factorization of A existed and the various versions could compute it in a stable manner.
More informationApplied Mathematics 205. Unit II: Numerical Linear Algebra. Lecturer: Dr. David Knezevic
Applied Mathematics 205 Unit II: Numerical Linear Algebra Lecturer: Dr. David Knezevic Unit II: Numerical Linear Algebra Chapter II.2: LU and Cholesky Factorizations 2 / 82 Preliminaries 3 / 82 Preliminaries
More informationComputation of the mtx-vec product based on storage scheme on vector CPUs
BLAS: Basic Linear Algebra Subroutines BLAS: Basic Linear Algebra Subroutines BLAS: Basic Linear Algebra Subroutines Analysis of the Matrix Computation of the mtx-vec product based on storage scheme on
More informationLecture 7. Gaussian Elimination with Pivoting. David Semeraro. University of Illinois at Urbana-Champaign. February 11, 2014
Lecture 7 Gaussian Elimination with Pivoting David Semeraro University of Illinois at Urbana-Champaign February 11, 2014 David Semeraro (NCSA) CS 357 February 11, 2014 1 / 41 Naive Gaussian Elimination
More informationPractical Linear Algebra: A Geometry Toolbox
Practical Linear Algebra: A Geometry Toolbox Third edition Chapter 12: Gauss for Linear Systems Gerald Farin & Dianne Hansford CRC Press, Taylor & Francis Group, An A K Peters Book www.farinhansford.com/books/pla
More informationChapter 4 No. 4.0 Answer True or False to the following. Give reasons for your answers.
MATH 434/534 Theoretical Assignment 3 Solution Chapter 4 No 40 Answer True or False to the following Give reasons for your answers If a backward stable algorithm is applied to a computational problem,
More informationANALYTICAL MATHEMATICS FOR APPLICATIONS 2018 LECTURE NOTES 3
ANALYTICAL MATHEMATICS FOR APPLICATIONS 2018 LECTURE NOTES 3 ISSUED 24 FEBRUARY 2018 1 Gaussian elimination Let A be an (m n)-matrix Consider the following row operations on A (1) Swap the positions any
More informationNumerical Linear Algebra
Numerical Linear Algebra Carlos Hurtado Department of Economics University of Illinois at Urbana-Champaign hrtdmrt2@illinois.edu Sep 19th, 2017 C. Hurtado (UIUC - Economics) Numerical Methods On the Agenda
More informationNumerical Linear Algebra
Numerical Linear Algebra Direct Methods Philippe B. Laval KSU Fall 2017 Philippe B. Laval (KSU) Linear Systems: Direct Solution Methods Fall 2017 1 / 14 Introduction The solution of linear systems is one
More informationLU Factorization. LU Decomposition. LU Decomposition. LU Decomposition: Motivation A = LU
LU Factorization To further improve the efficiency of solving linear systems Factorizations of matrix A : LU and QR LU Factorization Methods: Using basic Gaussian Elimination (GE) Factorization of Tridiagonal
More informationSolving Linear Systems of Equations
1 Solving Linear Systems of Equations Many practical problems could be reduced to solving a linear system of equations formulated as Ax = b This chapter studies the computational issues about directly
More informationNumerical Methods - Numerical Linear Algebra
Numerical Methods - Numerical Linear Algebra Y. K. Goh Universiti Tunku Abdul Rahman 2013 Y. K. Goh (UTAR) Numerical Methods - Numerical Linear Algebra I 2013 1 / 62 Outline 1 Motivation 2 Solving Linear
More informationA Review of Matrix Analysis
Matrix Notation Part Matrix Operations Matrices are simply rectangular arrays of quantities Each quantity in the array is called an element of the matrix and an element can be either a numerical value
More informationInteger Least Squares: Sphere Decoding and the LLL Algorithm
Integer Least Squares: Sphere Decoding and the LLL Algorithm Sanzheng Qiao Department of Computing and Software McMaster University 28 Main St. West Hamilton Ontario L8S 4L7 Canada. ABSTRACT This paper
More informationSherman-Morrison-Woodbury
Week 5: Wednesday, Sep 23 Sherman-Mrison-Woodbury The Sherman-Mrison fmula describes the solution of A+uv T when there is already a factization f A. An easy way to derive the fmula is through block Gaussian
More informationSOLVING LINEAR SYSTEMS
SOLVING LINEAR SYSTEMS We want to solve the linear system a, x + + a,n x n = b a n, x + + a n,n x n = b n This will be done by the method used in beginning algebra, by successively eliminating unknowns
More informationSolution of Linear Equations
Solution of Linear Equations (Com S 477/577 Notes) Yan-Bin Jia Sep 7, 07 We have discussed general methods for solving arbitrary equations, and looked at the special class of polynomial equations A subclass
More informationOn the Skeel condition number, growth factor and pivoting strategies for Gaussian elimination
On the Skeel condition number, growth factor and pivoting strategies for Gaussian elimination J.M. Peña 1 Introduction Gaussian elimination (GE) with a given pivoting strategy, for nonsingular matrices
More informationCommunication-avoiding LU and QR factorizations for multicore architectures
Communication-avoiding LU and QR factorizations for multicore architectures DONFACK Simplice INRIA Saclay Joint work with Laura Grigori INRIA Saclay Alok Kumar Gupta BCCS,Norway-5075 16th April 2010 Communication-avoiding
More informationSolving Linear Systems of Equations
November 6, 2013 Introduction The type of problems that we have to solve are: Solve the system: A x = B, where a 11 a 1N a 12 a 2N A =.. a 1N a NN x = x 1 x 2. x N B = b 1 b 2. b N To find A 1 (inverse
More informationLet x be an approximate solution for Ax = b, e.g., obtained by Gaussian elimination. Let x denote the exact solution. Call. r := b A x.
ESTIMATION OF ERROR Let x be an approximate solution for Ax = b, e.g., obtained by Gaussian elimination. Let x denote the exact solution. Call the residual for x. Then r := b A x r = b A x = Ax A x = A
More informationLecture 2 Decompositions, perturbations
March 26, 2018 Lecture 2 Decompositions, perturbations A Triangular systems Exercise 2.1. Let L = (L ij ) be an n n lower triangular matrix (L ij = 0 if i > j). (a) Prove that L is non-singular if and
More information9. Numerical linear algebra background
Convex Optimization Boyd & Vandenberghe 9. Numerical linear algebra background matrix structure and algorithm complexity solving linear equations with factored matrices LU, Cholesky, LDL T factorization
More informationMathematical Optimisation, Chpt 2: Linear Equations and inequalities
Mathematical Optimisation, Chpt 2: Linear Equations and inequalities Peter J.C. Dickinson p.j.c.dickinson@utwente.nl http://dickinson.website version: 12/02/18 Monday 5th February 2018 Peter J.C. Dickinson
More informationChapter 2 - Linear Equations
Chapter 2 - Linear Equations 2. Solving Linear Equations One of the most common problems in scientific computing is the solution of linear equations. It is a problem in its own right, but it also occurs
More informationHybrid static/dynamic scheduling for already optimized dense matrix factorization. Joint Laboratory for Petascale Computing, INRIA-UIUC
Hybrid static/dynamic scheduling for already optimized dense matrix factorization Simplice Donfack, Laura Grigori, INRIA, France Bill Gropp, Vivek Kale UIUC, USA Joint Laboratory for Petascale Computing,
More informationFAST VERIFICATION OF SOLUTIONS OF MATRIX EQUATIONS
published in Numer. Math., 90(4):755 773, 2002 FAST VERIFICATION OF SOLUTIONS OF MATRIX EQUATIONS SHIN ICHI OISHI AND SIEGFRIED M. RUMP Abstract. In this paper, we are concerned with a matrix equation
More informationAMS 209, Fall 2015 Final Project Type A Numerical Linear Algebra: Gaussian Elimination with Pivoting for Solving Linear Systems
AMS 209, Fall 205 Final Project Type A Numerical Linear Algebra: Gaussian Elimination with Pivoting for Solving Linear Systems. Overview We are interested in solving a well-defined linear system given
More informationMATH2210 Notebook 2 Spring 2018
MATH2210 Notebook 2 Spring 2018 prepared by Professor Jenny Baglivo c Copyright 2009 2018 by Jenny A. Baglivo. All Rights Reserved. 2 MATH2210 Notebook 2 3 2.1 Matrices and Their Operations................................
More informationCS475: Linear Equations Gaussian Elimination LU Decomposition Wim Bohm Colorado State University
CS475: Linear Equations Gaussian Elimination LU Decomposition Wim Bohm Colorado State University Except as otherwise noted, the content of this presentation is licensed under the Creative Commons Attribution
More informationLinear Analysis Lecture 16
Linear Analysis Lecture 16 The QR Factorization Recall the Gram-Schmidt orthogonalization process. Let V be an inner product space, and suppose a 1,..., a n V are linearly independent. Define q 1,...,
More informationLecture 2 INF-MAT : , LU, symmetric LU, Positve (semi)definite, Cholesky, Semi-Cholesky
Lecture 2 INF-MAT 4350 2009: 7.1-7.6, LU, symmetric LU, Positve (semi)definite, Cholesky, Semi-Cholesky Tom Lyche and Michael Floater Centre of Mathematics for Applications, Department of Informatics,
More informationECE133A Applied Numerical Computing Additional Lecture Notes
Winter Quarter 2018 ECE133A Applied Numerical Computing Additional Lecture Notes L. Vandenberghe ii Contents 1 LU factorization 1 1.1 Definition................................. 1 1.2 Nonsingular sets
More informationNumerical Linear Algebra
Numerical Linear Algebra R. J. Renka Department of Computer Science & Engineering University of North Texas 02/03/2015 Notation and Terminology R n is the Euclidean n-dimensional linear space over the
More informationDraft. Lecture 12 Gaussian Elimination and LU Factorization. MATH 562 Numerical Analysis II. Songting Luo
Lecture 12 Gaussian Elimination and LU Factorization Songting Luo Department of Mathematics Iowa State University MATH 562 Numerical Analysis II ongting Luo ( Department of Mathematics Iowa State University[0.5in]
More informationExample: Current in an Electrical Circuit. Solving Linear Systems:Direct Methods. Linear Systems of Equations. Solving Linear Systems: Direct Methods
Example: Current in an Electrical Circuit Solving Linear Systems:Direct Methods A number of engineering problems or models can be formulated in terms of systems of equations Examples: Electrical Circuit
More informationIntroduction to communication avoiding algorithms for direct methods of factorization in Linear Algebra
Introduction to communication avoiding algorithms for direct methods of factorization in Linear Algebra Laura Grigori Abstract Modern, massively parallel computers play a fundamental role in a large and
More informationProcess Model Formulation and Solution, 3E4
Process Model Formulation and Solution, 3E4 Section B: Linear Algebraic Equations Instructor: Kevin Dunn dunnkg@mcmasterca Department of Chemical Engineering Course notes: Dr Benoît Chachuat 06 October
More informationDEN: Linear algebra numerical view (GEM: Gauss elimination method for reducing a full rank matrix to upper-triangular
form) Given: matrix C = (c i,j ) n,m i,j=1 ODE and num math: Linear algebra (N) [lectures] c phabala 2016 DEN: Linear algebra numerical view (GEM: Gauss elimination method for reducing a full rank matrix
More information