Computational Linear Algebra
|
|
- Nickolas Lucas
- 6 years ago
- Views:
Transcription
1 Computational Linear Algebra PD Dr. rer. nat. habil. Ralf Peter Mundani Computation in Engineering / BGU Scientific Computing in Computer Science / INF Winter Term 2017/18
2 Part 2: Direct Methods PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 2
3 overview definitions GAUSSIAN elimination CHOLESKY decomposition QR decomposition PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 3
4 Definitions direct methods algorithms for exactly solving linear systems (avoiding round off errors) within finite amount of steps nowadays seldom used for solving huge linear systems (due to their large complexities) nevertheless, frequently used (in incomplete form) as preconditioners within iterative methods typical methods GAUSSIAN elimination CHOLESKY decomposition QR decomposition GRAM SCHMIDT method GIVENS method PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 4
5 overview definitions GAUSSIAN elimination CHOLESKY decomposition QR decomposition PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 5
6 GAUSSIAN Elimination basic concept GAUSSIAN elimination (GE) successively transforms a linear system Ax b into an equivalent system LRx b with upper triangular matrix R and lower triangular matrix L that can be solved via simple forward / backward substitution Definition 2.1 The decomposition of a matrix A into a product A LR consisting of a lower triangular matrix L and an upper triangular matrix R is called LR decomposition. In literature this is also referred to as LU (lower, upper) decomposition. PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 6
7 GAUSSIAN Elimination basic concept (cont d) we denote k th row P kj (2.1.1) j th row as permutation matrix that emerges from the identity matrix I via transposition of j th and k th row (j k) for k j follows P kj I PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 7
8 GAUSSIAN Elimination basic concept (cont d) furthermore, the lower triangular matrix L to be represented via multiplicative combination of matrices L k (2.1.2) those matrices differ at most in one column from identity matrix with (2.1.1) and (2.1.2) we are able to formulate the essential part of GE PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 8
9 GAUSSIAN Elimination algorithm: GAUSSIAN elimination (LR decomposition) A (1) : A for k 1,..., n 1 choose from k th column of A (k) some arbitrary element 0 with j k define P kj with above j and k according to (2.1.1) à (k) : P kj A (k) define L k according to (2.1.2) with, i k 1,..., n A (k 1) : L k à (k) with A (n) we get an upper triangular matrix R that can be used for a simple solution of the system question: where to get matrix L from? PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 9
10 GAUSSIAN Elimination existence and uniqueness of LR decomposition simple examples such as A show not each regular matrix necessarily exhibits an LR decomposition let A be regular, then A exhibits an LR decomposition if and only if det A[k] 0 k 1,..., n with A[k] : for k 1,..., n being the principal k k submatrix of A and det A[k] the principal determinant of A proof is lengthy PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 10
11 GAUSSIAN Elimination algorithm: GAUSSIAN elimination w/o pivoting for k 1,..., n 1 for i k 1,..., n a ik : a ik / a kk for j k 1,..., n a ij : a ij a ik a kj for k 2,..., n for i 1,..., k 1 b k : b k a ki b i for k n,..., 1 for i k 1,..., n b k : b k a ki x i LR decomposition A : LR forward substitution b : L 1 b backward substitution x : R 1 b x k : b k / a kk PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 11
12 GAUSSIAN Elimination complexity only expensive multiplications and divisions to be considered for k 1,..., n 1 for i k 1,..., n a ik : a ik / a kk for j k 1,..., n a ij : a ij a ik a kj for k 2,..., n for i 1,..., k 1 b k : b k a ki b i for k n,..., 1 for i k 1,..., n b k : b k a ki x i x k : b k / a kk (n k) times (inner loop) for k 1,..., n 1 (n k) 2 times (inner loops) for k 1,..., n 1 (k 1) times (inner loop) for k 2,..., n (n k) times (inner loop) for k 1,..., n n times PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 12
13 GAUSSIAN Elimination complexity (cont d) only expensive multiplications and divisions to be considered #divisions #multiplications hence, #divisions #multiplications such that the total complexity of GAUSSIAN elimination can be estimated as (n 3 ) on a standard computer (3 GHz) we need for n 10 4 approx. 5 minutes PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 13
14 GAUSSIAN Elimination error analysis let 1, such that (according to machine precision) resp. applies consider the following linear system Ax b with and its exact solution PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 14
15 GAUSSIAN Elimination error analysis (cont d) with GAUSSIAN elimination we get (1) and due to present computational accuracy substituting this into first equation of (1) yields hence x 1 0 follows PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 15
16 GAUSSIAN Elimination error analysis (cont d) a previous row interchange yields the following linear system hence, with GAUSSIAN elimination follows and we get x and x 1 1 x as (correct) solution permutation of rows and columns not only makes sense in case such strategies are called pivoting PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 16
17 GAUSSIAN Elimination pivoting we consider three different types of pivoting: column pivoting define P kj according to (2.1.1) with j index and consider for A (k) x b the equivalent linear system P kj A (k) x P kj b PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 17
18 GAUSSIAN Elimination pivoting (cont d) we consider three different types of pivoting: row pivoting define P kj according to (2.1.1) with j index and consider the linear system A (k) P kj y b x P kj y PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 18
19 GAUSSIAN Elimination pivoting (cont d) we consider three different types of pivoting: total pivoting define P kj1, P kj2 according to (2.1.1) with j 1 index j 2 index and consider the linear system P kj1 A (k) P kj2 y P kj1 b x P kj2 y PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 19
20 overview definitions GAUSSIAN elimination CHOLESKY decomposition QR decomposition PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 20
21 CHOLESKY Decomposition basic concept complexity for LR decomposition of symmetric positive definite (SPD) matrices during GAUSSIAN elimination can be further reduced Definition 2.2 The decomposition of a matrix A into a product A LL T with a lower triangular matrix L is called CHOLESKY decomposition. for each SPD matrix A exists exactly one lower triangular matrix L with l ii 0, i 1,..., n, such that A LL T applies PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 21
22 CHOLESKY Decomposition basic concept (cont d) let s consider a column wise computation of matrix coefficients we assume all l ij for i 1,..., n and j k 1 are known hence, from follows the relation thus l kk can be computed according to (2.2.1) PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 22
23 CHOLESKY Decomposition basic concept (cont d) from for i k 1,..., n follows a rule for computing elements of k th column (below diagonal) via for i k 1,..., n (2.2.2) with (2.2.1) and (2.2.2) we are able to formulate the algorithm for the CHOLESKY decomposition PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 23
24 CHOLESKY Decomposition algorithm CHOLESKY decomposition A : LL T forward substitution b : L 1 b backward substitution x : L T b for k 1,..., n for j 1,..., k 1 a kk : a kk a kj a kj a kk : for i k 1,..., n for j 1,..., k 1 a ik : a ik a ij a kj a ik : a ik / a kk for k 1,..., n for i 1,..., k 1 b k : b k a ki b i b k : b k / a kk for k n,..., 1 for i k 1,..., n b k : b k a ik x i x k : b k / a kk (2.2.1) (2.2.2) PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 24
25 CHOLESKY Decomposition complexity solely decomposition (as most complex part) to be considered only expensive multiplications, divisions, and roots for k 1,..., n for j 1,..., k 1 a kk : a kk a kj a kj a kk : for i k 1,..., n for j 1,..., k 1 a ik : a ik a ij a kj a ik : a ik / a kk (k 1) times (inner loop) for k 1,..., n n times (n k) (k 1) times (inner loops) for k 1,..., n (n k) times (inner loop) for k 1,..., n PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 25
26 CHOLESKY Decomposition complexity (cont d) #multiplications #divisions #roots 0 0 hence, for large n CHOLESKY decomposition needs approximately only half of the expensive operations of GAUSSIAN elimination PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 26
27 overview definitions GAUSSIAN elimination CHOLESKY decomposition QR decomposition PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 27
28 QR Decomposition basic concept essential foundation for GMRES ( iterative method) and many methods for solving eigenvalue problems or linear regressions due to Q Q 1, the linear system Ax b is easily to be solved via Ax b QRx b Rx Q b the three most well known methods for this decomposition are GIVENS GRAM SCHMIDT or HOUSEHOLDER (both not to be considered here) Definition 2.3 The decomposition of a matrix A into a product A QR with a unitary matrix Q and an upper triangular matrix R is called QR decomposition. PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 28
29 QR Decomposition GIVENS method let s assume the following matrix (confined to the case A ) A j th row i th column PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 29
30 QR Decomposition GIVENS method (cont d) idea: successively eliminate elements below main diagonal starting from first column, sub diagonal elements of each column become nullified in ascending order via orthogonal rotation matrices for previous matrix A applies a kl 0 l 1,..., i 1 with l k 1,..., n, a i 1,i... a j 1,i 0 (2.3.1) (2.3.2) and a ji 0 in order to nullify a ji, we look for an orthogonal matrix G ji PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 30
31 QR Decomposition GIVENS method (cont d) let G ji be an orthogonal matrix PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 31
32 QR Decomposition GIVENS method (cont d) such that for à G ji A in addition to ã kl 0 l 1,..., i 1 with l k 1,..., n, (2.3.3) and ã i 1,i... ã j 1,i 0 (2.3.4) also ã ji 0 (2.3.5) applies PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 32
33 QR Decomposition GIVENS method (cont d) at first, Ã and A solely differ in i th and j th row, and for l 1,..., n ã il g ii a il g ij a jl ã jl g ji a il g jj a jl applies with(2.3.1) follows a il a jl 0 for l i j, thus ã il ã jl 0 for l 1,..., i 1 and hence the requirements (2.3.3) and (2.3.4) are fulfilled well defined via a ji 0 we set g ij g ji and g ii g jj PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 33
34 QR Decomposition GIVENS method (cont d) thus, G ji represents an orthogonal rotation matrix with angle arccos g ii and the following applies defining G ji I in case of a matrix A that satisfies (2.3.1) and (2.3.2) and furthermore implies a ji 0, then with ~ Q : G ji : G n,n 1... G 3,2 G n,1... G 3,1 G 2,1 we get an orthogonal matrix for which R QA yields an upper triangular matrix with Q Q~ T follows A QR ~ PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 34
35 QR Decomposition algorithm for i 1,..., n 1 for j i 1,..., n Y a ji 0 t : 1 / s : ta ji c : ta ii for k i,..., n 1 t : ca ik sa jk Y k i N N QR decomposition A : QR By using GIVENS method, there is no need to explicitly store the orthogonal matrix. Here, A will be extended by the right hand side b according to a n 1 b. for i n,..., 1 for j i 1,..., n a i,n 1 : a i,n 1 a ij x j a ik : t a ji : 0 a jk : sa ik ca jk x i : a i,n 1 / a ii back substitution x : R 1 b PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 35
36 QR Decomposition complexity for i 1,..., n 1 for j i 1,..., n Y a ji 0 t : 1 / s : ta ji c : ta ii for k i,..., n 1 t : ca ik sa jk Y a ik : t a ji : 0 k i N N a jk : sa ik ca jk solely decomposition (as most complex part) to be considered w/o right hand side b only expensive multiplications, divisions, and roots (n i) times (inner loop) for i 1,..., n 1 (n i) times (inner loop) for i 1,..., n 1 (n i) times (inner loop) for i 1,..., n 1 (n i) times (inner loop) for i 1,..., n 1 2(n i 1)(n i) times (inner loops) for i 1,..., n 1 2(n i 1)(n i) times (inner loops) for i 1,..., n 1 PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 36
37 QR Decomposition complexity (cont d) #multiplications #divisions #roots hence, the complexity is approximately four times larger than GAUSSIAN elimination PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 37
38 overview definitions GAUSSIAN elimination CHOLESKY decomposition QR decomposition PD Dr. Ralf Peter Mundani Computational Linear Algebra Winter Term 2017/18 38
Computational Linear Algebra
Computational Linear Algebra PD Dr. rer. nat. habil. Ralf Peter Mundani Computation in Engineering / BGU Scientific Computing in Computer Science / INF Winter Term 2017/18 Part 3: Iterative Methods PD
More information1 Multiply Eq. E i by λ 0: (λe i ) (E i ) 2 Multiply Eq. E j by λ and add to Eq. E i : (E i + λe j ) (E i )
Direct Methods for Linear Systems Chapter Direct Methods for Solving Linear Systems Per-Olof Persson persson@berkeleyedu Department of Mathematics University of California, Berkeley Math 18A Numerical
More informationDirect Methods for Solving Linear Systems. Simon Fraser University Surrey Campus MACM 316 Spring 2005 Instructor: Ha Le
Direct Methods for Solving Linear Systems Simon Fraser University Surrey Campus MACM 316 Spring 2005 Instructor: Ha Le 1 Overview General Linear Systems Gaussian Elimination Triangular Systems The LU Factorization
More informationLU Factorization. LU Decomposition. LU Decomposition. LU Decomposition: Motivation A = LU
LU Factorization To further improve the efficiency of solving linear systems Factorizations of matrix A : LU and QR LU Factorization Methods: Using basic Gaussian Elimination (GE) Factorization of Tridiagonal
More informationMatrix Factorization and Analysis
Chapter 7 Matrix Factorization and Analysis Matrix factorizations are an important part of the practice and analysis of signal processing. They are at the heart of many signal-processing algorithms. Their
More informationLecture 11. Linear systems: Cholesky method. Eigensystems: Terminology. Jacobi transformations QR transformation
Lecture Cholesky method QR decomposition Terminology Linear systems: Eigensystems: Jacobi transformations QR transformation Cholesky method: For a symmetric positive definite matrix, one can do an LU decomposition
More informationDirect Methods for Solving Linear Systems. Matrix Factorization
Direct Methods for Solving Linear Systems Matrix Factorization Numerical Analysis (9th Edition) R L Burden & J D Faires Beamer Presentation Slides prepared by John Carroll Dublin City University c 2011
More informationComputational Linear Algebra
Computational Linear Algebra PD Dr. rer. nat. habil. Ralf-Peter Mundani Computation in Engineering / BGU Scientific Computing in Computer Science / INF Winter Term 2018/19 Part 4: Iterative Methods PD
More informationAlgebra C Numerical Linear Algebra Sample Exam Problems
Algebra C Numerical Linear Algebra Sample Exam Problems Notation. Denote by V a finite-dimensional Hilbert space with inner product (, ) and corresponding norm. The abbreviation SPD is used for symmetric
More information5.6. PSEUDOINVERSES 101. A H w.
5.6. PSEUDOINVERSES 0 Corollary 5.6.4. If A is a matrix such that A H A is invertible, then the least-squares solution to Av = w is v = A H A ) A H w. The matrix A H A ) A H is the left inverse of A and
More informationMatrix decompositions
Matrix decompositions Zdeněk Dvořák May 19, 2015 Lemma 1 (Schur decomposition). If A is a symmetric real matrix, then there exists an orthogonal matrix Q and a diagonal matrix D such that A = QDQ T. The
More informationNumerical Methods - Numerical Linear Algebra
Numerical Methods - Numerical Linear Algebra Y. K. Goh Universiti Tunku Abdul Rahman 2013 Y. K. Goh (UTAR) Numerical Methods - Numerical Linear Algebra I 2013 1 / 62 Outline 1 Motivation 2 Solving Linear
More informationLecture 3: QR-Factorization
Lecture 3: QR-Factorization This lecture introduces the Gram Schmidt orthonormalization process and the associated QR-factorization of matrices It also outlines some applications of this factorization
More informationCS412: Lecture #17. Mridul Aanjaneya. March 19, 2015
CS: Lecture #7 Mridul Aanjaneya March 9, 5 Solving linear systems of equations Consider a lower triangular matrix L: l l l L = l 3 l 3 l 33 l n l nn A procedure similar to that for upper triangular systems
More informationDEN: Linear algebra numerical view (GEM: Gauss elimination method for reducing a full rank matrix to upper-triangular
form) Given: matrix C = (c i,j ) n,m i,j=1 ODE and num math: Linear algebra (N) [lectures] c phabala 2016 DEN: Linear algebra numerical view (GEM: Gauss elimination method for reducing a full rank matrix
More informationLinear Analysis Lecture 16
Linear Analysis Lecture 16 The QR Factorization Recall the Gram-Schmidt orthogonalization process. Let V be an inner product space, and suppose a 1,..., a n V are linearly independent. Define q 1,...,
More informationNumerical Linear Algebra
Numerical Linear Algebra Direct Methods Philippe B. Laval KSU Fall 2017 Philippe B. Laval (KSU) Linear Systems: Direct Solution Methods Fall 2017 1 / 14 Introduction The solution of linear systems is one
More informationThis can be accomplished by left matrix multiplication as follows: I
1 Numerical Linear Algebra 11 The LU Factorization Recall from linear algebra that Gaussian elimination is a method for solving linear systems of the form Ax = b, where A R m n and bran(a) In this method
More informationSolution of Linear Equations
Solution of Linear Equations (Com S 477/577 Notes) Yan-Bin Jia Sep 7, 07 We have discussed general methods for solving arbitrary equations, and looked at the special class of polynomial equations A subclass
More informationSTAT 309: MATHEMATICAL COMPUTATIONS I FALL 2018 LECTURE 13
STAT 309: MATHEMATICAL COMPUTATIONS I FALL 208 LECTURE 3 need for pivoting we saw that under proper circumstances, we can write A LU where 0 0 0 u u 2 u n l 2 0 0 0 u 22 u 2n L l 3 l 32, U 0 0 0 l n l
More informationCHAPTER 6. Direct Methods for Solving Linear Systems
CHAPTER 6 Direct Methods for Solving Linear Systems. Introduction A direct method for approximating the solution of a system of n linear equations in n unknowns is one that gives the exact solution to
More informationMATH 3511 Lecture 1. Solving Linear Systems 1
MATH 3511 Lecture 1 Solving Linear Systems 1 Dmitriy Leykekhman Spring 2012 Goals Review of basic linear algebra Solution of simple linear systems Gaussian elimination D Leykekhman - MATH 3511 Introduction
More informationLU Factorization. LU factorization is the most common way of solving linear systems! Ax = b LUx = b
AM 205: lecture 7 Last time: LU factorization Today s lecture: Cholesky factorization, timing, QR factorization Reminder: assignment 1 due at 5 PM on Friday September 22 LU Factorization LU factorization
More informationSolving Dense Linear Systems I
Solving Dense Linear Systems I Solving Ax = b is an important numerical method Triangular system: [ l11 l 21 if l 11, l 22 0, ] [ ] [ ] x1 b1 = l 22 x 2 b 2 x 1 = b 1 /l 11 x 2 = (b 2 l 21 x 1 )/l 22 Chih-Jen
More informationComputational Linear Algebra
Computational Linear Algebra PD Dr. rer. nat. habil. Ralf Peter Mundani Computation in Engineering / BGU Scientific Computing in Computer Science / INF Winter Term 2017/18 Part 3: Iterative Methods PD
More informationComputation of the mtx-vec product based on storage scheme on vector CPUs
BLAS: Basic Linear Algebra Subroutines BLAS: Basic Linear Algebra Subroutines BLAS: Basic Linear Algebra Subroutines Analysis of the Matrix Computation of the mtx-vec product based on storage scheme on
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra)
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 12: Gaussian Elimination and LU Factorization Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 10 Gaussian Elimination
More informationFundamentals of Engineering Analysis (650163)
Philadelphia University Faculty of Engineering Communications and Electronics Engineering Fundamentals of Engineering Analysis (6563) Part Dr. Omar R Daoud Matrices: Introduction DEFINITION A matrix is
More informationComputational Linear Algebra
Computational Linear Algebra PD Dr. rer. nat. habil. Ralf-Peter Mundani Computation in Engineering / BGU Scientific Computing in Computer Science / INF Winter Term 2018/19 Part 6: Some Other Stuff PD Dr.
More informationLinear Algebra Section 2.6 : LU Decomposition Section 2.7 : Permutations and transposes Wednesday, February 13th Math 301 Week #4
Linear Algebra Section. : LU Decomposition Section. : Permutations and transposes Wednesday, February 1th Math 01 Week # 1 The LU Decomposition We learned last time that we can factor a invertible matrix
More informationGaussian Elimination and Back Substitution
Jim Lambers MAT 610 Summer Session 2009-10 Lecture 4 Notes These notes correspond to Sections 31 and 32 in the text Gaussian Elimination and Back Substitution The basic idea behind methods for solving
More informationToday s class. Linear Algebraic Equations LU Decomposition. Numerical Methods, Fall 2011 Lecture 8. Prof. Jinbo Bi CSE, UConn
Today s class Linear Algebraic Equations LU Decomposition 1 Linear Algebraic Equations Gaussian Elimination works well for solving linear systems of the form: AX = B What if you have to solve the linear
More informationLinear Algebraic Equations
Linear Algebraic Equations 1 Fundamentals Consider the set of linear algebraic equations n a ij x i b i represented by Ax b j with [A b ] [A b] and (1a) r(a) rank of A (1b) Then Axb has a solution iff
More informationGaussian Elimination without/with Pivoting and Cholesky Decomposition
Gaussian Elimination without/with Pivoting and Cholesky Decomposition Gaussian Elimination WITHOUT pivoting Notation: For a matrix A R n n we define for k {,,n} the leading principal submatrix a a k A
More informationGaussian Elimination for Linear Systems
Gaussian Elimination for Linear Systems Tsung-Ming Huang Department of Mathematics National Taiwan Normal University October 3, 2011 1/56 Outline 1 Elementary matrices 2 LR-factorization 3 Gaussian elimination
More informationLinear Algebra, part 3 QR and SVD
Linear Algebra, part 3 QR and SVD Anna-Karin Tornberg Mathematical Models, Analysis and Simulation Fall semester, 2012 Going back to least squares (Section 1.4 from Strang, now also see section 5.2). We
More informationSOLVING LINEAR SYSTEMS
SOLVING LINEAR SYSTEMS We want to solve the linear system a, x + + a,n x n = b a n, x + + a n,n x n = b n This will be done by the method used in beginning algebra, by successively eliminating unknowns
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra)
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 3: Positive-Definite Systems; Cholesky Factorization Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical Analysis I 1 / 11 Symmetric
More informationNumerical Linear Algebra
Numerical Linear Algebra Decompositions, numerical aspects Gerard Sleijpen and Martin van Gijzen September 27, 2017 1 Delft University of Technology Program Lecture 2 LU-decomposition Basic algorithm Cost
More informationProgram Lecture 2. Numerical Linear Algebra. Gaussian elimination (2) Gaussian elimination. Decompositions, numerical aspects
Numerical Linear Algebra Decompositions, numerical aspects Program Lecture 2 LU-decomposition Basic algorithm Cost Stability Pivoting Cholesky decomposition Sparse matrices and reorderings Gerard Sleijpen
More informationApril 26, Applied mathematics PhD candidate, physics MA UC Berkeley. Lecture 4/26/2013. Jed Duersch. Spd matrices. Cholesky decomposition
Applied mathematics PhD candidate, physics MA UC Berkeley April 26, 2013 UCB 1/19 Symmetric positive-definite I Definition A symmetric matrix A R n n is positive definite iff x T Ax > 0 holds x 0 R n.
More informationMatrix decompositions
Matrix decompositions How can we solve Ax = b? 1 Linear algebra Typical linear system of equations : x 1 x +x = x 1 +x +9x = 0 x 1 +x x = The variables x 1, x, and x only appear as linear terms (no powers
More information5. Direct Methods for Solving Systems of Linear Equations. They are all over the place...
5 Direct Methods for Solving Systems of Linear Equations They are all over the place Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13, 2012
More informationOrthogonal Transformations
Orthogonal Transformations Tom Lyche University of Oslo Norway Orthogonal Transformations p. 1/3 Applications of Qx with Q T Q = I 1. solving least squares problems (today) 2. solving linear equations
More informationBLAS: Basic Linear Algebra Subroutines Analysis of the Matrix-Vector-Product Analysis of Matrix-Matrix Product
Level-1 BLAS: SAXPY BLAS-Notation: S single precision (D for double, C for complex) A α scalar X vector P plus operation Y vector SAXPY: y = αx + y Vectorization of SAXPY (αx + y) by pipelining: page 8
More informationV C V L T I 0 C V B 1 V T 0 I. l nk
Multifrontal Method Kailai Xu September 16, 2017 Main observation. Consider the LDL T decomposition of a SPD matrix [ ] [ ] [ ] [ ] B V T L 0 I 0 L T L A = = 1 V T V C V L T I 0 C V B 1 V T, 0 I where
More informationAM205: Assignment 2. i=1
AM05: Assignment Question 1 [10 points] (a) [4 points] For p 1, the p-norm for a vector x R n is defined as: ( n ) 1/p x p x i p ( ) i=1 This definition is in fact meaningful for p < 1 as well, although
More informationSolving linear equations with Gaussian Elimination (I)
Term Projects Solving linear equations with Gaussian Elimination The QR Algorithm for Symmetric Eigenvalue Problem The QR Algorithm for The SVD Quasi-Newton Methods Solving linear equations with Gaussian
More informationComputational Linear Algebra
Computational Linear Algebra PD Dr. rer. nat. habil. Ralf-Peter Mundani Computation in Engineering / BGU Scientific Computing in Computer Science / INF Winter Term 2018/19 Part 4: Iterative Methods PD
More informationSolving Linear Systems Using Gaussian Elimination. How can we solve
Solving Linear Systems Using Gaussian Elimination How can we solve? 1 Gaussian elimination Consider the general augmented system: Gaussian elimination Step 1: Eliminate first column below the main diagonal.
More information2.1 Gaussian Elimination
2. Gaussian Elimination A common problem encountered in numerical models is the one in which there are n equations and n unknowns. The following is a description of the Gaussian elimination method for
More informationNumerical Methods I: Numerical linear algebra
1/3 Numerical Methods I: Numerical linear algebra Georg Stadler Courant Institute, NYU stadler@cimsnyuedu September 1, 017 /3 We study the solution of linear systems of the form Ax = b with A R n n, x,
More informationAM 205: lecture 8. Last time: Cholesky factorization, QR factorization Today: how to compute the QR factorization, the Singular Value Decomposition
AM 205: lecture 8 Last time: Cholesky factorization, QR factorization Today: how to compute the QR factorization, the Singular Value Decomposition QR Factorization A matrix A R m n, m n, can be factorized
More information. =. a i1 x 1 + a i2 x 2 + a in x n = b i. a 11 a 12 a 1n a 21 a 22 a 1n. i1 a i2 a in
Vectors and Matrices Continued Remember that our goal is to write a system of algebraic equations as a matrix equation. Suppose we have the n linear algebraic equations a x + a 2 x 2 + a n x n = b a 2
More informationCOURSE Numerical methods for solving linear systems. Practical solving of many problems eventually leads to solving linear systems.
COURSE 9 4 Numerical methods for solving linear systems Practical solving of many problems eventually leads to solving linear systems Classification of the methods: - direct methods - with low number of
More information(17) (18)
Module 4 : Solving Linear Algebraic Equations Section 3 : Direct Solution Techniques 3 Direct Solution Techniques Methods for solving linear algebraic equations can be categorized as direct and iterative
More informationMatrix decompositions
Matrix decompositions How can we solve Ax = b? 1 Linear algebra Typical linear system of equations : x 1 x +x = x 1 +x +9x = 0 x 1 +x x = The variables x 1, x, and x only appear as linear terms (no powers
More informationEE731 Lecture Notes: Matrix Computations for Signal Processing
EE731 Lecture Notes: Matrix Computations for Signal Processing James P. Reilly c Department of Electrical and Computer Engineering McMaster University September 22, 2005 0 Preface This collection of ten
More informationAMS 209, Fall 2015 Final Project Type A Numerical Linear Algebra: Gaussian Elimination with Pivoting for Solving Linear Systems
AMS 209, Fall 205 Final Project Type A Numerical Linear Algebra: Gaussian Elimination with Pivoting for Solving Linear Systems. Overview We are interested in solving a well-defined linear system given
More informationScientific Computing
Scientific Computing Direct solution methods Martin van Gijzen Delft University of Technology October 3, 2018 1 Program October 3 Matrix norms LU decomposition Basic algorithm Cost Stability Pivoting Pivoting
More informationLinear Algebra in Actuarial Science: Slides to the lecture
Linear Algebra in Actuarial Science: Slides to the lecture Fall Semester 2010/2011 Linear Algebra is a Tool-Box Linear Equation Systems Discretization of differential equations: solving linear equations
More informationDense LU factorization and its error analysis
Dense LU factorization and its error analysis Laura Grigori INRIA and LJLL, UPMC February 2016 Plan Basis of floating point arithmetic and stability analysis Notation, results, proofs taken from [N.J.Higham,
More informationThe QR Factorization
The QR Factorization How to Make Matrices Nicer Radu Trîmbiţaş Babeş-Bolyai University March 11, 2009 Radu Trîmbiţaş ( Babeş-Bolyai University) The QR Factorization March 11, 2009 1 / 25 Projectors A projector
More informationLU Factorization. Marco Chiarandini. DM559 Linear and Integer Programming. Department of Mathematics & Computer Science University of Southern Denmark
DM559 Linear and Integer Programming LU Factorization Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark [Based on slides by Lieven Vandenberghe, UCLA] Outline
More informationThe Solution of Linear Systems AX = B
Chapter 2 The Solution of Linear Systems AX = B 21 Upper-triangular Linear Systems We will now develop the back-substitution algorithm, which is useful for solving a linear system of equations that has
More informationMODULE 7. where A is an m n real (or complex) matrix. 2) Let K(t, s) be a function of two variables which is continuous on the square [0, 1] [0, 1].
Topics: Linear operators MODULE 7 We are going to discuss functions = mappings = transformations = operators from one vector space V 1 into another vector space V 2. However, we shall restrict our sights
More informationLinear Algebra, part 3. Going back to least squares. Mathematical Models, Analysis and Simulation = 0. a T 1 e. a T n e. Anna-Karin Tornberg
Linear Algebra, part 3 Anna-Karin Tornberg Mathematical Models, Analysis and Simulation Fall semester, 2010 Going back to least squares (Sections 1.7 and 2.3 from Strang). We know from before: The vector
More informationLecture 2 INF-MAT : , LU, symmetric LU, Positve (semi)definite, Cholesky, Semi-Cholesky
Lecture 2 INF-MAT 4350 2009: 7.1-7.6, LU, symmetric LU, Positve (semi)definite, Cholesky, Semi-Cholesky Tom Lyche and Michael Floater Centre of Mathematics for Applications, Department of Informatics,
More informationforms Christopher Engström November 14, 2014 MAA704: Matrix factorization and canonical forms Matrix properties Matrix factorization Canonical forms
Christopher Engström November 14, 2014 Hermitian LU QR echelon Contents of todays lecture Some interesting / useful / important of matrices Hermitian LU QR echelon Rewriting a as a product of several matrices.
More informationNumerical Methods in Matrix Computations
Ake Bjorck Numerical Methods in Matrix Computations Springer Contents 1 Direct Methods for Linear Systems 1 1.1 Elements of Matrix Theory 1 1.1.1 Matrix Algebra 2 1.1.2 Vector Spaces 6 1.1.3 Submatrices
More informationLecture 12 (Tue, Mar 5) Gaussian elimination and LU factorization (II)
Math 59 Lecture 2 (Tue Mar 5) Gaussian elimination and LU factorization (II) 2 Gaussian elimination - LU factorization For a general n n matrix A the Gaussian elimination produces an LU factorization if
More informationLinear Least squares
Linear Least squares Method of least squares Measurement errors are inevitable in observational and experimental sciences Errors can be smoothed out by averaging over more measurements than necessary to
More information14.2 QR Factorization with Column Pivoting
page 531 Chapter 14 Special Topics Background Material Needed Vector and Matrix Norms (Section 25) Rounding Errors in Basic Floating Point Operations (Section 33 37) Forward Elimination and Back Substitution
More informationNumerical Linear Algebra
Chapter 3 Numerical Linear Algebra We review some techniques used to solve Ax = b where A is an n n matrix, and x and b are n 1 vectors (column vectors). We then review eigenvalues and eigenvectors and
More informationThe System of Linear Equations. Direct Methods. Xiaozhou Li.
1/16 The Direct Methods xiaozhouli@uestc.edu.cn http://xiaozhouli.com School of Mathematical Sciences University of Electronic Science and Technology of China Chengdu, China Does the LU factorization always
More informationSolving Linear Systems of Equations
November 6, 2013 Introduction The type of problems that we have to solve are: Solve the system: A x = B, where a 11 a 1N a 12 a 2N A =.. a 1N a NN x = x 1 x 2. x N B = b 1 b 2. b N To find A 1 (inverse
More informationPractical Linear Algebra: A Geometry Toolbox
Practical Linear Algebra: A Geometry Toolbox Third edition Chapter 12: Gauss for Linear Systems Gerald Farin & Dianne Hansford CRC Press, Taylor & Francis Group, An A K Peters Book www.farinhansford.com/books/pla
More informationChapter 2. Solving Systems of Equations. 2.1 Gaussian elimination
Chapter 2 Solving Systems of Equations A large number of real life applications which are resolved through mathematical modeling will end up taking the form of the following very simple looking matrix
More informationCSE 160 Lecture 13. Numerical Linear Algebra
CSE 16 Lecture 13 Numerical Linear Algebra Announcements Section will be held on Friday as announced on Moodle Midterm Return 213 Scott B Baden / CSE 16 / Fall 213 2 Today s lecture Gaussian Elimination
More informationCME 302: NUMERICAL LINEAR ALGEBRA FALL 2005/06 LECTURE 6
CME 302: NUMERICAL LINEAR ALGEBRA FALL 2005/06 LECTURE 6 GENE H GOLUB Issues with Floating-point Arithmetic We conclude our discussion of floating-point arithmetic by highlighting two issues that frequently
More informationL2-7 Some very stylish matrix decompositions for solving Ax = b 10 Oct 2015
L-7 Some very stylish matrix decompositions for solving Ax = b 10 Oct 015 Marty McFly: Wait a minute, Doc. Ah... Are you telling me you built a time machine... out of a DeLorean? Doc Brown: The way I see
More informationlecture 2 and 3: algorithms for linear algebra
lecture 2 and 3: algorithms for linear algebra STAT 545: Introduction to computational statistics Vinayak Rao Department of Statistics, Purdue University August 27, 2018 Solving a system of linear equations
More informationLecture Notes to Accompany. Scientific Computing An Introductory Survey. by Michael T. Heath. Chapter 2. Systems of Linear Equations
Lecture Notes to Accompany Scientific Computing An Introductory Survey Second Edition by Michael T. Heath Chapter 2 Systems of Linear Equations Copyright c 2001. Reproduction permitted only for noncommercial,
More informationCheat Sheet for MATH461
Cheat Sheet for MATH46 Here is the stuff you really need to remember for the exams Linear systems Ax = b Problem: We consider a linear system of m equations for n unknowns x,,x n : For a given matrix A
More informationAx = b. Systems of Linear Equations. Lecture Notes to Accompany. Given m n matrix A and m-vector b, find unknown n-vector x satisfying
Lecture Notes to Accompany Scientific Computing An Introductory Survey Second Edition by Michael T Heath Chapter Systems of Linear Equations Systems of Linear Equations Given m n matrix A and m-vector
More informationPreliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012
Instructions Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012 The exam consists of four problems, each having multiple parts. You should attempt to solve all four problems. 1.
More informationPowerPoints organized by Dr. Michael R. Gustafson II, Duke University
Part 3 Chapter 10 LU Factorization PowerPoints organized by Dr. Michael R. Gustafson II, Duke University All images copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
More informationNumerical Analysis: Solving Systems of Linear Equations
Numerical Analysis: Solving Systems of Linear Equations Mirko Navara http://cmpfelkcvutcz/ navara/ Center for Machine Perception, Department of Cybernetics, FEE, CTU Karlovo náměstí, building G, office
More informationReview Questions REVIEW QUESTIONS 71
REVIEW QUESTIONS 71 MATLAB, is [42]. For a comprehensive treatment of error analysis and perturbation theory for linear systems and many other problems in linear algebra, see [126, 241]. An overview of
More informationOrthogonalization and least squares methods
Chapter 3 Orthogonalization and least squares methods 31 QR-factorization (QR-decomposition) 311 Householder transformation Definition 311 A complex m n-matrix R = [r ij is called an upper (lower) triangular
More informationMAA507, Power method, QR-method and sparse matrix representation.
,, and representation. February 11, 2014 Lecture 7: Overview, Today we will look at:.. If time: A look at representation and fill in. Why do we need numerical s? I think everyone have seen how time consuming
More informationVector and Matrix Norms. Vector and Matrix Norms
Vector and Matrix Norms Vector Space Algebra Matrix Algebra: We let x x and A A, where, if x is an element of an abstract vector space n, and A = A: n m, then x is a complex column vector of length n whose
More informationLinear Algebra Linear Algebra : Matrix decompositions Monday, February 11th Math 365 Week #4
Linear Algebra Linear Algebra : Matrix decompositions Monday, February 11th Math Week # 1 Saturday, February 1, 1 Linear algebra Typical linear system of equations : x 1 x +x = x 1 +x +9x = 0 x 1 +x x
More informationApplied Linear Algebra
Applied Linear Algebra Gábor P. Nagy and Viktor Vígh University of Szeged Bolyai Institute Winter 2014 1 / 262 Table of contents I 1 Introduction, review Complex numbers Vectors and matrices Determinants
More informationNumerical Methods I Solving Square Linear Systems: GEM and LU factorization
Numerical Methods I Solving Square Linear Systems: GEM and LU factorization Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 MATH-GA 2011.003 / CSCI-GA 2945.003, Fall 2014 September 18th,
More informationLINEAR SYSTEMS (11) Intensive Computation
LINEAR SYSTEMS () Intensive Computation 27-8 prof. Annalisa Massini Viviana Arrigoni EXACT METHODS:. GAUSSIAN ELIMINATION. 2. CHOLESKY DECOMPOSITION. ITERATIVE METHODS:. JACOBI. 2. GAUSS-SEIDEL 2 CHOLESKY
More informationDeterminants. Chia-Ping Chen. Linear Algebra. Professor Department of Computer Science and Engineering National Sun Yat-sen University 1/40
1/40 Determinants Chia-Ping Chen Professor Department of Computer Science and Engineering National Sun Yat-sen University Linear Algebra About Determinant A scalar function on the set of square matrices
More information9. Numerical linear algebra background
Convex Optimization Boyd & Vandenberghe 9. Numerical linear algebra background matrix structure and algorithm complexity solving linear equations with factored matrices LU, Cholesky, LDL T factorization
More informationScientific Computing WS 2018/2019. Lecture 9. Jürgen Fuhrmann Lecture 9 Slide 1
Scientific Computing WS 2018/2019 Lecture 9 Jürgen Fuhrmann juergen.fuhrmann@wias-berlin.de Lecture 9 Slide 1 Lecture 9 Slide 2 Simple iteration with preconditioning Idea: Aû = b iterative scheme û = û
More informationMATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 1 x 2. x n 8 (4) 3 4 2
MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS SYSTEMS OF EQUATIONS AND MATRICES Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a
More information