Solving Dense Linear Systems I
|
|
|
- Melissa Patterson
- 5 years ago
- Views:
Transcription
1 Solving Dense Linear Systems I Solving Ax = b is an important numerical method Triangular system: [ l11 l 21 if l 11, l 22 0, ] [ ] [ ] x1 b1 = l 22 x 2 b 2 x 1 = b 1 /l 11 x 2 = (b 2 l 21 x 1 )/l 22 Chih-Jen Lin (National Taiwan Univ.) 1 / 43
2 Solving Dense Linear Systems II In general i 1 x i = (b i l ij x j )/l ii j=1 Require O(n 2 ) operations This is forward substitution Back substitution or upper system x i = (b i n j=i+1 u ij x j )/u jj Chih-Jen Lin (National Taiwan Univ.) 2 / 43
3 Solving Dense Linear Systems III Column oriented version x 1 x 2 = x 3 x 1 = 3 [ ] [ ] [ [ 5 0 x2 2 6 = 3 = 8 8 x 3 5] 7] [ ] Chih-Jen Lin (National Taiwan Univ.) 3 / 43
4 LU factorization I An example By Gaussian elimination: Actually [ ] 3 5 = 6 7 3x 1 + 5x 2 = 9 6x 1 + 7x 2 = 4 3x 1 + 5x 2 = 9 3x 2 = 14 [ ] [ ] Chih-Jen Lin (National Taiwan Univ.) 4 / 43
5 LU factorization II Then solve [ ] [ ] 1 0 y1 2 1 y 2 Gauss transformation [ 9 = 4] [ y1 ] = y 2 [ ] [ ] [ ] 3 5 x1 9 = 0 3 x 2 14 A = [ ] 9 14 Chih-Jen Lin (National Taiwan Univ.) 5 / 43
6 LU factorization III If M 1 = then M 1 A = Here 1 is called the pivot Likewise, M 2 = Chih-Jen Lin (National Taiwan Univ.) 6 / 43
7 LU factorization IV M 2 (M 1 A) = Now 3 is the pivot Actually the whole Gaussian elimination is U: upper triangular U = M n 1 M 2 M 1 A Chih-Jen Lin (National Taiwan Univ.) 7 / 43
8 LU factorization V A = (M1 1 (M 1 2 (Mn 1 1 U))) = LU M1 1 Mn 1 1 are lower triangular matrices M 1 = , M1 1 = M 2 = , M = Chih-Jen Lin (National Taiwan Univ.) 8 / 43
9 LU factorization VI M 1 1 M 1 2 = = You do not really store M 1,..., M n 1 Why LU factorization but not Gaussian elimination? Gaussian elimination overwrites A. If you do not overwrite A, need another array (the same size as A and L + U) Chih-Jen Lin (National Taiwan Univ.) 9 / 43
10 LU factorization VII Gaussian elimination update right-hand side together 3x 1 + 5x 2 = 9 6x 1 + 7x 2 = 4 3x 1 + 5x 2 = 9 3x 2 = 14 Chih-Jen Lin (National Taiwan Univ.) 10 / 43
11 LU factorization VIII To handle multiple right-hand sides, LU is the most effective way Number of operations n j=1 2j 2 n(n + 1)(2n + 1) 2 6 2n3 3 Why 2j 2 : each element: one multiplication and one subtraction Chih-Jen Lin (National Taiwan Univ.) 11 / 43
12 Problems of LU Factorization I Zero diagonal elements: A = [ ] Cannot generate M 1 Numerical error of LU factorization From Example of Golub and Van Loan Assume β = 10, p = 3, no guard digit Solve [ ] [ ] x1 = x 2 [ Chih-Jen Lin (National Taiwan Univ.) 12 / 43 ]
13 Problems of LU Factorization II LU factorization [ ] 1 0 L =, U = [ ] The real solution [ ] [ ] [ ] x1 1 = x x 2 = 997/998 x 1 = 1000(1 997/998) = 1000/998 Chih-Jen Lin (National Taiwan Univ.) 13 / 43
14 Problems of LU Factorization III Using our system: [ ] [ ] 1 0 y1 = y 2 [ ] y 1 = 1.00, y 2 = = 1000 [ ] [ ] [ ] x = x x 1 = 0, x 2 = 1 (0, 1) very different from (1000/998, 997/998) Conclusion: (1) Chih-Jen Lin (National Taiwan Univ.) 14 / 43
15 Problems of LU Factorization IV A small pivot may cause large numerical errors Think about (1). If x 2 contains errors, because , x 1 is seriously affected. Chih-Jen Lin (National Taiwan Univ.) 15 / 43
16 Problems of LU Factorization V If interchanging rows: [ ] [ ] L = U = [ ] [ ] [ 1 0 y1 3 = y 2 1] y 1 = 3, y 2 = 1 [ ] [ ] [ 1 2 x1 3 = 0 1 x 2 1] x 1 = 1, x 2 = 1 (1, 1) much closer to (1000/998, 997/998) Chih-Jen Lin (National Taiwan Univ.) 16 / 43
17 Pivoting: Avoid Small Pivots I How to interchange rows: [ ] 0 1 P = 1 0 PA = [ ] Chih-Jen Lin (National Taiwan Univ.) 17 / 43
18 Pivoting: Avoid Small Pivots II Permutation matrix: , 1 2, 3 3, 2 4 Chih-Jen Lin (National Taiwan Univ.) 18 / 43
19 Pivoting: Avoid Small Pivots III Pivoting in LU: A = Largest absolute value in the first column: 6 P 1 = Chih-Jen Lin (National Taiwan Univ.) 19 / 43
20 Pivoting: Avoid Small Pivots IV Then P 1 A = M 1 = / / M 1 P 1 A = Chih-Jen Lin (National Taiwan Univ.) 20 / 43
21 Pivoting: Avoid Small Pivots V Swap 2nd and 3rd rows: P 2 = M 2 = / M 2 P 2 M 1 P 1 A = In general M n 1 P n 1 M 1 P 1 A = U Chih-Jen Lin (National Taiwan Univ.) 21 / 43
22 Pivoting: Avoid Small Pivots VI If A is overwritten p(1) = swap two rows Chih-Jen Lin (National Taiwan Univ.) 22 / 43
23 Pivoting: Avoid Small Pivots VII Then / / / p(2) = 3 1/ / /3 1/4 6 Chih-Jen Lin (National Taiwan Univ.) 23 / 43
24 Pivoting: Avoid Small Pivots VIII Using an array p to store the permutation That is / A 0 1/ / = Chih-Jen Lin (National Taiwan Univ.) 24 / 43
25 Pivoting: Avoid Small Pivots IX Solve the linear system: y = M n 1 P n 1 M 1 P 1 b Ux = y Memory issue: how do we store M 1,..., M n 1? Notice that = / /3 1/ Chih-Jen Lin (National Taiwan Univ.) 25 / 43
26 Pivoting: Avoid Small Pivots X Algorithm: for k=1:n-1 find m swap A(k,1:n) and A(m, 1:n) p(k) = m; A(k+1:n,k) = A(k+1:n,k)/A(k,k); A(k+1:n, k+1:n) = A(k+1:n, k+1:n) - A(k+1:n,k)*A(k,k+1:n); end We swap the whole two rows Chih-Jen Lin (National Taiwan Univ.) 26 / 43
27 Pivoting: Avoid Small Pivots XI General form: M n 1 P n 1 M 1 P 1 A = U Then PA = LU and P = P n 1 P 1 We have P 2 P 1 = = = P Chih-Jen Lin (National Taiwan Univ.) 27 / 43
28 Pivoting: Avoid Small Pivots XII Explanation: P n 1 P 1 A =P n 1 P 1 P 1 1 M 1 1 P 1 2 M 1 2 P 1 n 1 M 1 n 1 U We claim that P n 1 P 1 P 1 1 M 1 1 P 1 2 P 1 n 1 becomes our first column, and P n 1 P 1 P 1 1 P 1 2 M 1 2 P 1 3 P 1 n 1 is the second column Chih-Jen Lin (National Taiwan Univ.) 28 / 43
29 Pivoting: Avoid Small Pivots XIII Note that P 1 i = P i. In our earlier example, P 2 P 1 P1 1 M 1 1 P 1 2 = P 2 M1 1 P 2 = / / = /3 0 1 = / / /3 0 1 Chih-Jen Lin (National Taiwan Univ.) 29 / 43
30 Special Matrices and Cholesky Factorization I There are many special matrices. For example, diagonal, tri-diagonal, banded matrices, positive definite matrices, etc. A banded matrix Chih-Jen Lin (National Taiwan Univ.) 30 / 43
31 Special Matrices and Cholesky Factorization II Many applications involve positive definite matrices An n n matrix A is positive definite if x T Ax > 0, x R n, x 0 A is positive definite if and only if all A s eigenvalues are positive A is PD all diagonal elements are positive, all principle sub-matrices are positive definite Chih-Jen Lin (National Taiwan Univ.) 31 / 43
32 Special Matrices and Cholesky Factorization III A is symmetric positive definite (SPD) there is a lower triangular matrix L s.t. A = LL T Cholesky factorization Reason of studying SPD matrices: 2 f x i x j is symmetric in optimization Chih-Jen Lin (National Taiwan Univ.) 32 / 43
33 Cholesky Factorization I We have [ ] α v T A = (α > 0) v B [ ] [ α 0 α v = v/ T / ] α α I 0 B vv T α [ ] [ ] [ α = v/ α v T / ] α α I 0 B vv T 0 I α Chih-Jen Lin (National Taiwan Univ.) 33 / 43
34 Cholesky Factorization II If B vv T α = L L T, [ ] [ α ] [ α v T / ] α v/ α I 0 L L T 0 I [ ] [ ] [ α 0 = ( v/ α I 0 L )( 0 L T [ ] [ α 0 = v/ α v T / ] α. α L 0 LT ] [ α v T / α 0 I If we can prove B vv T α is also PD, the same procedure could be applied to B vv T Chih-Jen Lin (National Taiwan Univ.) 34 / 43 α. ] )
35 Cholesky Factorization III [ ] α v T If A = v B For any x 0 is PD, B vv T α is PD. = = x T (B vv T α )x = x T Bx x T ( vv T α )x [ ] [ ] v T x α x T 0 v T x α v + Bx [ ] [ ] [ ] v T x α x T α v T v T x α v B x Chih-Jen Lin (National Taiwan Univ.) 35 / 43
36 Cholesky Factorization IV Implementation: A = LL T n n n A ij = L ik (L T ) kj = L ik L jk = L jk L ik A :,j = k=1 j L jk L :,k k=1 j 1 L jj L :,j = A :,j L jk L :,k k=1 k=1 k=1 (lower triangular) right-hand side: involve 1st to (j 1)st columns Chih-Jen Lin (National Taiwan Univ.) 36 / 43
37 Cholesky Factorization V left-hand side: jth column jth column unknown, 1st to (j 1)st columns known. Let Then j 1 L jj L jj = A j,j L jk L j,k k=1 j 1 L j+1:n,j = (A j+1:n,j L jk L j+1:n,k )/L jj k=1 Chih-Jen Lin (National Taiwan Univ.) 37 / 43
38 Cholesky Factorization VI for j=1:n v(j:n) = A(j:n,j) for k=1:j-1 v(j:n) = v(j:n) - L(j,k)L(j:n,k) end L(j:n,j) = v(j:n)/sqrt(v(j)) end Calculate A j:n,j j 1 k=1 L jkl j:n,k first Note that v(j:n) = v(j:n) - L(j,k)L(j:n,k) is a axpy operation Chih-Jen Lin (National Taiwan Univ.) 38 / 43
39 Cholesky Factorization VII Operations: a 3-level for loop n j=1 = 2 n = 2 n 2(n j=1 j 1 k=1 2(n j + 1) (j 1)(n j + 1) (nj n j 2 + j + j 1) j=1 n(n + 1) n(n + 1)(2n + 1) ) = O( n ) Cholesky factorization: half operations of LU factorization Chih-Jen Lin (National Taiwan Univ.) 39 / 43
40 Cholesky Factorization VIII A different form of Cholesky factorization: Outer product [ ] α v T A = (α > 0) v B [ ] [ ] [ α = v/ α v T / ] α α I 0 B vv T 0 I α Chih-Jen Lin (National Taiwan Univ.) 40 / 43
41 Cholesky Factorization IX for k=1:n A(k,k) = sqrt(a(k,k)) A(k+1:n,k) = A(k+1:n,k)/A(k,k) for j=k+1:n A(j:n,j) = A(j:n,j) - A(j:n,k)A(j,k) end end Chih-Jen Lin (National Taiwan Univ.) 41 / 43
42 HW3-2: Use Matlab for Cholesky Factorization I Randomly generate a symmetric positive definite matrix Example: randomly generate A and then AA T Implement Cholesky factorization in outer product form with three variants: 1 three-level for loop 2 two-level for loop 3 one-level for loop Chih-Jen Lin (National Taiwan Univ.) 42 / 43
43 HW3-2: Use Matlab for Cholesky Factorization II Validate your program by comparing with the chol command and compare the efficiency of your variants Timing in Matlab >> help cputime Chih-Jen Lin (National Taiwan Univ.) 43 / 43
Direct Methods for Solving Linear Systems. Simon Fraser University Surrey Campus MACM 316 Spring 2005 Instructor: Ha Le
Direct Methods for Solving Linear Systems Simon Fraser University Surrey Campus MACM 316 Spring 2005 Instructor: Ha Le 1 Overview General Linear Systems Gaussian Elimination Triangular Systems The LU Factorization
CS412: Lecture #17. Mridul Aanjaneya. March 19, 2015
CS: Lecture #7 Mridul Aanjaneya March 9, 5 Solving linear systems of equations Consider a lower triangular matrix L: l l l L = l 3 l 3 l 33 l n l nn A procedure similar to that for upper triangular systems
Numerical Linear Algebra
Numerical Linear Algebra Direct Methods Philippe B. Laval KSU Fall 2017 Philippe B. Laval (KSU) Linear Systems: Direct Solution Methods Fall 2017 1 / 14 Introduction The solution of linear systems is one
2.1 Gaussian Elimination
2. Gaussian Elimination A common problem encountered in numerical models is the one in which there are n equations and n unknowns. The following is a description of the Gaussian elimination method for
The Solution of Linear Systems AX = B
Chapter 2 The Solution of Linear Systems AX = B 21 Upper-triangular Linear Systems We will now develop the back-substitution algorithm, which is useful for solving a linear system of equations that has
Computational Linear Algebra
Computational Linear Algebra PD Dr. rer. nat. habil. Ralf Peter Mundani Computation in Engineering / BGU Scientific Computing in Computer Science / INF Winter Term 2017/18 Part 2: Direct Methods PD Dr.
Numerical Linear Algebra
Numerical Linear Algebra Decompositions, numerical aspects Gerard Sleijpen and Martin van Gijzen September 27, 2017 1 Delft University of Technology Program Lecture 2 LU-decomposition Basic algorithm Cost
Program Lecture 2. Numerical Linear Algebra. Gaussian elimination (2) Gaussian elimination. Decompositions, numerical aspects
Numerical Linear Algebra Decompositions, numerical aspects Program Lecture 2 LU-decomposition Basic algorithm Cost Stability Pivoting Cholesky decomposition Sparse matrices and reorderings Gerard Sleijpen
LU Factorization. LU Decomposition. LU Decomposition. LU Decomposition: Motivation A = LU
LU Factorization To further improve the efficiency of solving linear systems Factorizations of matrix A : LU and QR LU Factorization Methods: Using basic Gaussian Elimination (GE) Factorization of Tridiagonal
Scientific Computing
Scientific Computing Direct solution methods Martin van Gijzen Delft University of Technology October 3, 2018 1 Program October 3 Matrix norms LU decomposition Basic algorithm Cost Stability Pivoting Pivoting
MODULE 7. where A is an m n real (or complex) matrix. 2) Let K(t, s) be a function of two variables which is continuous on the square [0, 1] [0, 1].
![MODULE 7. where A is an m n real (or complex) matrix. 2) Let K(t, s) be a function of two variables which is continuous on the square [0, 1] [0, 1].](/thumbs/95/122529712.jpg "MODULE 7. where A is an m n real (or complex) matrix. 2) Let K(t, s) be a function of two variables which is continuous on the square [0, 1] [0, 1].")
Topics: Linear operators MODULE 7 We are going to discuss functions = mappings = transformations = operators from one vector space V 1 into another vector space V 2. However, we shall restrict our sights
1 Multiply Eq. E i by λ 0: (λe i ) (E i ) 2 Multiply Eq. E j by λ and add to Eq. E i : (E i + λe j ) (E i )
Direct Methods for Linear Systems Chapter Direct Methods for Solving Linear Systems Per-Olof Persson persson@berkeleyedu Department of Mathematics University of California, Berkeley Math 18A Numerical
Linear Algebra Section 2.6 : LU Decomposition Section 2.7 : Permutations and transposes Wednesday, February 13th Math 301 Week #4
Linear Algebra Section. : LU Decomposition Section. : Permutations and transposes Wednesday, February 1th Math 01 Week # 1 The LU Decomposition We learned last time that we can factor a invertible matrix
Systems of Linear Equations
Systems of Linear Equations Last time, we found that solving equations such as Poisson s equation or Laplace s equation on a grid is equivalent to solving a system of linear equations. There are many other
1.Chapter Objectives
LU Factorization INDEX 1.Chapter objectives 2.Overview of LU factorization 2.1GAUSS ELIMINATION AS LU FACTORIZATION 2.2LU Factorization with Pivoting 2.3 MATLAB Function: lu 3. CHOLESKY FACTORIZATION 3.1
CS475: Linear Equations Gaussian Elimination LU Decomposition Wim Bohm Colorado State University
CS475: Linear Equations Gaussian Elimination LU Decomposition Wim Bohm Colorado State University Except as otherwise noted, the content of this presentation is licensed under the Creative Commons Attribution
LU Factorization. LU factorization is the most common way of solving linear systems! Ax = b LUx = b
AM 205: lecture 7 Last time: LU factorization Today s lecture: Cholesky factorization, timing, QR factorization Reminder: assignment 1 due at 5 PM on Friday September 22 LU Factorization LU factorization
AMS526: Numerical Analysis I (Numerical Linear Algebra)
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 12: Gaussian Elimination and LU Factorization Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 10 Gaussian Elimination
CSE 160 Lecture 13. Numerical Linear Algebra
CSE 16 Lecture 13 Numerical Linear Algebra Announcements Section will be held on Friday as announced on Moodle Midterm Return 213 Scott B Baden / CSE 16 / Fall 213 2 Today s lecture Gaussian Elimination
Linear Systems of n equations for n unknowns
Linear Systems of n equations for n unknowns In many application problems we want to find n unknowns, and we have n linear equations Example: Find x,x,x such that the following three equations hold: x
Solution of Linear Equations
Solution of Linear Equations (Com S 477/577 Notes) Yan-Bin Jia Sep 7, 07 We have discussed general methods for solving arbitrary equations, and looked at the special class of polynomial equations A subclass
Lecture 12 (Tue, Mar 5) Gaussian elimination and LU factorization (II)
Math 59 Lecture 2 (Tue Mar 5) Gaussian elimination and LU factorization (II) 2 Gaussian elimination - LU factorization For a general n n matrix A the Gaussian elimination produces an LU factorization if
Direct Methods for Solving Linear Systems. Matrix Factorization
Direct Methods for Solving Linear Systems Matrix Factorization Numerical Analysis (9th Edition) R L Burden & J D Faires Beamer Presentation Slides prepared by John Carroll Dublin City University c 2011
Linear Equations and Matrix
1/60 Chia-Ping Chen Professor Department of Computer Science and Engineering National Sun Yat-sen University Linear Algebra Gaussian Elimination 2/60 Alpha Go Linear algebra begins with a system of linear
Gaussian Elimination and Back Substitution
Jim Lambers MAT 610 Summer Session 2009-10 Lecture 4 Notes These notes correspond to Sections 31 and 32 in the text Gaussian Elimination and Back Substitution The basic idea behind methods for solving
Lecture 12. Linear systems of equations II. a 13. a 12. a 14. a a 22. a 23. a 34 a 41. a 32. a 33. a 42. a 43. a 44)
1 Introduction Lecture 12 Linear systems of equations II We have looked at Gauss-Jordan elimination and Gaussian elimination as ways to solve a linear system Ax=b. We now turn to the LU decomposition,
1 GSW Sets of Systems
1 Often, we have to solve a whole series of sets of simultaneous equations of the form y Ax, all of which have the same matrix A, but each of which has a different known vector y, and a different unknown
Illustration of Gaussian elimination to find LU factorization. A = a 11 a 12 a 13 a 14 a 21 a 22 a 23 a 24 a 31 a 32 a 33 a 34 a 41 a 42 a 43 a 44
Illustration of Gaussian elimination to find LU factorization. A = a 21 a a a a 31 a 32 a a a 41 a 42 a 43 a 1 Compute multipliers : Eliminate entries in first column: m i1 = a i1 a 11, i = 2, 3, 4 ith
Linear Algebra and Matrix Inversion
Jim Lambers MAT 46/56 Spring Semester 29- Lecture 2 Notes These notes correspond to Section 63 in the text Linear Algebra and Matrix Inversion Vector Spaces and Linear Transformations Matrices are much
1.5 Gaussian Elimination With Partial Pivoting.
Gaussian Elimination With Partial Pivoting In the previous section we discussed Gaussian elimination In that discussion we used equation to eliminate x from equations through n Then we used equation to
Gaussian Elimination without/with Pivoting and Cholesky Decomposition
Gaussian Elimination without/with Pivoting and Cholesky Decomposition Gaussian Elimination WITHOUT pivoting Notation: For a matrix A R n n we define for k {,,n} the leading principal submatrix a a k A
PowerPoints organized by Dr. Michael R. Gustafson II, Duke University
Part 3 Chapter 10 LU Factorization PowerPoints organized by Dr. Michael R. Gustafson II, Duke University All images copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
MATH 3511 Lecture 1. Solving Linear Systems 1
MATH 3511 Lecture 1 Solving Linear Systems 1 Dmitriy Leykekhman Spring 2012 Goals Review of basic linear algebra Solution of simple linear systems Gaussian elimination D Leykekhman - MATH 3511 Introduction
Numerical Methods I Solving Square Linear Systems: GEM and LU factorization
Numerical Methods I Solving Square Linear Systems: GEM and LU factorization Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 MATH-GA 2011.003 / CSCI-GA 2945.003, Fall 2014 September 18th,
Chapter 3 - From Gaussian Elimination to LU Factorization
Chapter 3 - From Gaussian Elimination to LU Factorization Maggie Myers Robert A. van de Geijn The University of Texas at Austin Practical Linear Algebra Fall 29 http://z.cs.utexas.edu/wiki/pla.wiki/ 1
Pivoting. Reading: GV96 Section 3.4, Stew98 Chapter 3: 1.3
Pivoting Reading: GV96 Section 3.4, Stew98 Chapter 3: 1.3 In the previous discussions we have assumed that the LU factorization of A existed and the various versions could compute it in a stable manner.
Gaussian Elimination for Linear Systems
Gaussian Elimination for Linear Systems Tsung-Ming Huang Department of Mathematics National Taiwan Normal University October 3, 2011 1/56 Outline 1 Elementary matrices 2 LR-factorization 3 Gaussian elimination
5 Solving Systems of Linear Equations
106 Systems of LE 5.1 Systems of Linear Equations 5 Solving Systems of Linear Equations 5.1 Systems of Linear Equations System of linear equations: a 11 x 1 + a 12 x 2 +... + a 1n x n = b 1 a 21 x 1 +
7.2 Linear equation systems. 7.3 Linear least square fit
72 Linear equation systems In the following sections, we will spend some time to solve linear systems of equations This is a tool that will come in handy in many di erent places during this course For
SOLVING LINEAR SYSTEMS
SOLVING LINEAR SYSTEMS We want to solve the linear system a, x + + a,n x n = b a n, x + + a n,n x n = b n This will be done by the method used in beginning algebra, by successively eliminating unknowns
GAUSSIAN ELIMINATION AND LU DECOMPOSITION (SUPPLEMENT FOR MA511)
GAUSSIAN ELIMINATION AND LU DECOMPOSITION (SUPPLEMENT FOR MA511) D. ARAPURA Gaussian elimination is the go to method for all basic linear classes including this one. We go summarize the main ideas. 1.
CS513, Spring 2007 Prof. Amos Ron Assignment #5 Solutions Prepared by Houssain Kettani. a mj i,j [2,n] a 11
![CS513, Spring 2007 Prof. Amos Ron Assignment #5 Solutions Prepared by Houssain Kettani. a mj i,j [2,n] a 11](/thumbs/72/67273900.jpg "CS513, Spring 2007 Prof. Amos Ron Assignment #5 Solutions Prepared by Houssain Kettani. a mj i,j [2,n] a 11")
CS513, Spring 2007 Prof. Amos Ron Assignment #5 Solutions Prepared by Houssain Kettani 1 Question 1 1. Let a ij denote the entries of the matrix A. Let A (m) denote the matrix A after m Gaussian elimination
Today s class. Linear Algebraic Equations LU Decomposition. Numerical Methods, Fall 2011 Lecture 8. Prof. Jinbo Bi CSE, UConn
Today s class Linear Algebraic Equations LU Decomposition 1 Linear Algebraic Equations Gaussian Elimination works well for solving linear systems of the form: AX = B What if you have to solve the linear
COURSE Numerical methods for solving linear systems. Practical solving of many problems eventually leads to solving linear systems.
COURSE 9 4 Numerical methods for solving linear systems Practical solving of many problems eventually leads to solving linear systems Classification of the methods: - direct methods - with low number of
Solving linear equations with Gaussian Elimination (I)
Term Projects Solving linear equations with Gaussian Elimination The QR Algorithm for Symmetric Eigenvalue Problem The QR Algorithm for The SVD Quasi-Newton Methods Solving linear equations with Gaussian
Linear Algebraic Equations
Linear Algebraic Equations 1 Fundamentals Consider the set of linear algebraic equations n a ij x i b i represented by Ax b j with [A b ] [A b] and (1a) r(a) rank of A (1b) Then Axb has a solution iff
ACM106a - Homework 2 Solutions
ACM06a - Homework 2 Solutions prepared by Svitlana Vyetrenko October 7, 2006. Chapter 2, problem 2.2 (solution adapted from Golub, Van Loan, pp.52-54): For the proof we will use the fact that if A C m
Numerical Linear Algebra
Numerical Linear Algebra The two principal problems in linear algebra are: Linear system Given an n n matrix A and an n-vector b, determine x IR n such that A x = b Eigenvalue problem Given an n n matrix
Numerical Linear Algebra
Chapter 3 Numerical Linear Algebra We review some techniques used to solve Ax = b where A is an n n matrix, and x and b are n 1 vectors (column vectors). We then review eigenvalues and eigenvectors and
LINEAR SYSTEMS (11) Intensive Computation
LINEAR SYSTEMS () Intensive Computation 27-8 prof. Annalisa Massini Viviana Arrigoni EXACT METHODS:. GAUSSIAN ELIMINATION. 2. CHOLESKY DECOMPOSITION. ITERATIVE METHODS:. JACOBI. 2. GAUSS-SEIDEL 2 CHOLESKY
Fundamentals of Engineering Analysis (650163)
Philadelphia University Faculty of Engineering Communications and Electronics Engineering Fundamentals of Engineering Analysis (6563) Part Dr. Omar R Daoud Matrices: Introduction DEFINITION A matrix is
The System of Linear Equations. Direct Methods. Xiaozhou Li.
1/16 The Direct Methods xiaozhouli@uestc.edu.cn http://xiaozhouli.com School of Mathematical Sciences University of Electronic Science and Technology of China Chengdu, China Does the LU factorization always
Math 471 (Numerical methods) Chapter 3 (second half). System of equations
Math 47 (Numerical methods) Chapter 3 (second half). System of equations Overlap 3.5 3.8 of Bradie 3.5 LU factorization w/o pivoting. Motivation: ( ) A I Gaussian Elimination (U L ) where U is upper triangular
CHAPTER 6. Direct Methods for Solving Linear Systems
CHAPTER 6 Direct Methods for Solving Linear Systems. Introduction A direct method for approximating the solution of a system of n linear equations in n unknowns is one that gives the exact solution to
4.2 Floating-Point Numbers
101 Approximation 4.2 Floating-Point Numbers 4.2 Floating-Point Numbers The number 3.1416 in scientific notation is 0.31416 10 1 or (as computer output) -0.31416E01..31416 10 1 exponent sign mantissa base
STAT 309: MATHEMATICAL COMPUTATIONS I FALL 2018 LECTURE 13
STAT 309: MATHEMATICAL COMPUTATIONS I FALL 208 LECTURE 3 need for pivoting we saw that under proper circumstances, we can write A LU where 0 0 0 u u 2 u n l 2 0 0 0 u 22 u 2n L l 3 l 32, U 0 0 0 l n l
AMS526: Numerical Analysis I (Numerical Linear Algebra)
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 3: Positive-Definite Systems; Cholesky Factorization Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical Analysis I 1 / 11 Symmetric
Solving Linear Systems of Equations
1 Solving Linear Systems of Equations Many practical problems could be reduced to solving a linear system of equations formulated as Ax = b This chapter studies the computational issues about directly
Solving Linear Systems Using Gaussian Elimination. How can we solve
Solving Linear Systems Using Gaussian Elimination How can we solve? 1 Gaussian elimination Consider the general augmented system: Gaussian elimination Step 1: Eliminate first column below the main diagonal.
Lecture 2 INF-MAT : , LU, symmetric LU, Positve (semi)definite, Cholesky, Semi-Cholesky
Lecture 2 INF-MAT 4350 2009: 7.1-7.6, LU, symmetric LU, Positve (semi)definite, Cholesky, Semi-Cholesky Tom Lyche and Michael Floater Centre of Mathematics for Applications, Department of Informatics,
Math 552 Scientific Computing II Spring SOLUTIONS: Homework Set 1
Math 552 Scientific Computing II Spring 21 SOLUTIONS: Homework Set 1 ( ) a b 1 Let A be the 2 2 matrix A = By hand, use Gaussian elimination with back c d substitution to obtain A 1 by solving the two
Chapter 2. Solving Systems of Equations. 2.1 Gaussian elimination
Chapter 2 Solving Systems of Equations A large number of real life applications which are resolved through mathematical modeling will end up taking the form of the following very simple looking matrix
Bindel, Fall 2016 Matrix Computations (CS 6210) Notes for
1 Logistics Notes for 2016-09-14 1. There was a goof in HW 2, problem 1 (now fixed) please re-download if you have already started looking at it. 2. CS colloquium (4:15 in Gates G01) this Thurs is Margaret
DEN: Linear algebra numerical view (GEM: Gauss elimination method for reducing a full rank matrix to upper-triangular
form) Given: matrix C = (c i,j ) n,m i,j=1 ODE and num math: Linear algebra (N) [lectures] c phabala 2016 DEN: Linear algebra numerical view (GEM: Gauss elimination method for reducing a full rank matrix
Phys 201. Matrices and Determinants
Phys 201 Matrices and Determinants 1 1.1 Matrices 1.2 Operations of matrices 1.3 Types of matrices 1.4 Properties of matrices 1.5 Determinants 1.6 Inverse of a 3 3 matrix 2 1.1 Matrices A 2 3 7 =! " 1
. =. a i1 x 1 + a i2 x 2 + a in x n = b i. a 11 a 12 a 1n a 21 a 22 a 1n. i1 a i2 a in
Vectors and Matrices Continued Remember that our goal is to write a system of algebraic equations as a matrix equation. Suppose we have the n linear algebraic equations a x + a 2 x 2 + a n x n = b a 2
MA2501 Numerical Methods Spring 2015
Norwegian University of Science and Technology Department of Mathematics MA2501 Numerical Methods Spring 2015 Solutions to exercise set 3 1 Attempt to verify experimentally the calculation from class that
Matrix decompositions
Matrix decompositions How can we solve Ax = b? 1 Linear algebra Typical linear system of equations : x 1 x +x = x 1 +x +9x = 0 x 1 +x x = The variables x 1, x, and x only appear as linear terms (no powers
April 26, Applied mathematics PhD candidate, physics MA UC Berkeley. Lecture 4/26/2013. Jed Duersch. Spd matrices. Cholesky decomposition
Applied mathematics PhD candidate, physics MA UC Berkeley April 26, 2013 UCB 1/19 Symmetric positive-definite I Definition A symmetric matrix A R n n is positive definite iff x T Ax > 0 holds x 0 R n.
Parallel Scientific Computing
IV-1 Parallel Scientific Computing Matrix-vector multiplication. Matrix-matrix multiplication. Direct method for solving a linear equation. Gaussian Elimination. Iterative method for solving a linear equation.
V C V L T I 0 C V B 1 V T 0 I. l nk
Multifrontal Method Kailai Xu September 16, 2017 Main observation. Consider the LDL T decomposition of a SPD matrix [ ] [ ] [ ] [ ] B V T L 0 I 0 L T L A = = 1 V T V C V L T I 0 C V B 1 V T, 0 I where
CME 302: NUMERICAL LINEAR ALGEBRA FALL 2005/06 LECTURE 6
CME 302: NUMERICAL LINEAR ALGEBRA FALL 2005/06 LECTURE 6 GENE H GOLUB Issues with Floating-point Arithmetic We conclude our discussion of floating-point arithmetic by highlighting two issues that frequently
Linear Algebra. Matrices Operations. Consider, for example, a system of equations such as x + 2y z + 4w = 0, 3x 4y + 2z 6w = 0, x 3y 2z + w = 0.
Matrices Operations Linear Algebra Consider, for example, a system of equations such as x + 2y z + 4w = 0, 3x 4y + 2z 6w = 0, x 3y 2z + w = 0 The rectangular array 1 2 1 4 3 4 2 6 1 3 2 1 in which the
Matrix decompositions
Matrix decompositions How can we solve Ax = b? 1 Linear algebra Typical linear system of equations : x 1 x +x = x 1 +x +9x = 0 x 1 +x x = The variables x 1, x, and x only appear as linear terms (no powers
Matrices: 2.1 Operations with Matrices
Goals In this chapter and section we study matrix operations: Define matrix addition Define multiplication of matrix by a scalar, to be called scalar multiplication. Define multiplication of two matrices,
Numerical Methods I: Numerical linear algebra
1/3 Numerical Methods I: Numerical linear algebra Georg Stadler Courant Institute, NYU stadler@cimsnyuedu September 1, 017 /3 We study the solution of linear systems of the form Ax = b with A R n n, x,
Section 9.2: Matrices. Definition: A matrix A consists of a rectangular array of numbers, or elements, arranged in m rows and n columns.
Section 9.2: Matrices Definition: A matrix A consists of a rectangular array of numbers, or elements, arranged in m rows and n columns. That is, a 11 a 12 a 1n a 21 a 22 a 2n A =...... a m1 a m2 a mn A
Numerical Methods - Numerical Linear Algebra
Numerical Methods - Numerical Linear Algebra Y. K. Goh Universiti Tunku Abdul Rahman 2013 Y. K. Goh (UTAR) Numerical Methods - Numerical Linear Algebra I 2013 1 / 62 Outline 1 Motivation 2 Solving Linear
Linear Algebra. Solving Linear Systems. Copyright 2005, W.R. Winfrey
Copyright 2005, W.R. Winfrey Topics Preliminaries Echelon Form of a Matrix Elementary Matrices; Finding A -1 Equivalent Matrices LU-Factorization Topics Preliminaries Echelon Form of a Matrix Elementary
Applied Linear Algebra in Geoscience Using MATLAB
Applied Linear Algebra in Geoscience Using MATLAB Contents Getting Started Creating Arrays Mathematical Operations with Arrays Using Script Files and Managing Data Two-Dimensional Plots Programming in
Applied Mathematics 205. Unit II: Numerical Linear Algebra. Lecturer: Dr. David Knezevic
Applied Mathematics 205 Unit II: Numerical Linear Algebra Lecturer: Dr. David Knezevic Unit II: Numerical Linear Algebra Chapter II.2: LU and Cholesky Factorizations 2 / 82 Preliminaries 3 / 82 Preliminaries
Linear System of Equations
Linear System of Equations Linear systems are perhaps the most widely applied numerical procedures when real-world situation are to be simulated. Example: computing the forces in a TRUSS. F F 5. 77F F.
Hani Mehrpouyan, California State University, Bakersfield. Signals and Systems
Hani Mehrpouyan, Department of Electrical and Computer Engineering, Lecture 26 (LU Factorization) May 30 th, 2013 The material in these lectures is partly taken from the books: Elementary Numerical Analysis,
Numerical Methods Lecture 2 Simultaneous Equations
CGN 42 - Computer Methods Numerical Methods Lecture 2 Simultaneous Equations Topics: matrix operations solving systems of equations Matrix operations: Adding / subtracting Transpose Multiplication Adding
MATLAB Project: LU Factorization
Name Purpose: To practice Lay's LU Factorization Algorithm and see how it is related to MATLAB's lu function. Prerequisite: Section 2.5 MATLAB functions used: *, lu; and ludat and gauss from Laydata4 Toolbox
Solving linear systems (6 lectures)
Chapter 2 Solving linear systems (6 lectures) 2.1 Solving linear systems: LU factorization (1 lectures) Reference: [Trefethen, Bau III] Lecture 20, 21 How do you solve Ax = b? (2.1.1) In numerical linear
Solution of Linear Systems
Solution of Linear Systems Parallel and Distributed Computing Department of Computer Science and Engineering (DEI) Instituto Superior Técnico May 12, 2016 CPD (DEI / IST) Parallel and Distributed Computing
Lemma 8: Suppose the N by N matrix A has the following block upper triangular form:
17 4 Determinants and the Inverse of a Square Matrix In this section, we are going to use our knowledge of determinants and their properties to derive an explicit formula for the inverse of a square matrix
Computational Methods. Systems of Linear Equations
Computational Methods Systems of Linear Equations Manfred Huber 2010 1 Systems of Equations Often a system model contains multiple variables (parameters) and contains multiple equations Multiple equations
LU Factorization. Marco Chiarandini. DM559 Linear and Integer Programming. Department of Mathematics & Computer Science University of Southern Denmark
DM559 Linear and Integer Programming LU Factorization Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark [Based on slides by Lieven Vandenberghe, UCLA] Outline
BLAS: Basic Linear Algebra Subroutines Analysis of the Matrix-Vector-Product Analysis of Matrix-Matrix Product
Level-1 BLAS: SAXPY BLAS-Notation: S single precision (D for double, C for complex) A α scalar X vector P plus operation Y vector SAXPY: y = αx + y Vectorization of SAXPY (αx + y) by pipelining: page 8
A Review of Matrix Analysis
Matrix Notation Part Matrix Operations Matrices are simply rectangular arrays of quantities Each quantity in the array is called an element of the matrix and an element can be either a numerical value
MTH 464: Computational Linear Algebra
MTH 464: Computational Linear Algebra Lecture Outlines Exam 2 Material Prof. M. Beauregard Department of Mathematics & Statistics Stephen F. Austin State University February 6, 2018 Linear Algebra (MTH
14.2 QR Factorization with Column Pivoting
page 531 Chapter 14 Special Topics Background Material Needed Vector and Matrix Norms (Section 25) Rounding Errors in Basic Floating Point Operations (Section 33 37) Forward Elimination and Back Substitution
Introduction to Mathematical Programming
Introduction to Mathematical Programming Ming Zhong Lecture 6 September 12, 2018 Ming Zhong (JHU) AMS Fall 2018 1 / 20 Table of Contents 1 Ming Zhong (JHU) AMS Fall 2018 2 / 20 Solving Linear Systems A
NUMERICAL MATHEMATICS & COMPUTING 7th Edition
NUMERICAL MATHEMATICS & COMPUTING 7th Edition Ward Cheney/David Kincaid c UT Austin Engage Learning: Thomson-Brooks/Cole wwwengagecom wwwmautexasedu/cna/nmc6 October 16, 2011 Ward Cheney/David Kincaid
Determinants. Chia-Ping Chen. Linear Algebra. Professor Department of Computer Science and Engineering National Sun Yat-sen University 1/40
1/40 Determinants Chia-Ping Chen Professor Department of Computer Science and Engineering National Sun Yat-sen University Linear Algebra About Determinant A scalar function on the set of square matrices
Scientific Computing: Dense Linear Systems
Scientific Computing: Dense Linear Systems Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 Course MATH-GA.2043 or CSCI-GA.2112, Spring 2012 February 9th, 2012 A. Donev (Courant Institute)
7. LU factorization. factor-solve method. LU factorization. solving Ax = b with A nonsingular. the inverse of a nonsingular matrix
EE507 - Computational Techniques for EE 7. LU factorization Jitkomut Songsiri factor-solve method LU factorization solving Ax = b with A nonsingular the inverse of a nonsingular matrix LU factorization
The determinant. Motivation: area of parallelograms, volume of parallepipeds. Two vectors in R 2 : oriented area of a parallelogram
The determinant Motivation: area of parallelograms, volume of parallepipeds Two vectors in R 2 : oriented area of a parallelogram Consider two vectors a (1),a (2) R 2 which are linearly independent We