Illustration of Gaussian elimination to find LU factorization. A = a 11 a 12 a 13 a 14 a 21 a 22 a 23 a 24 a 31 a 32 a 33 a 34 a 41 a 42 a 43 a 44

Transcription

1 Illustration of Gaussian elimination to find LU factorization. A = a 21 a a a a 31 a 32 a a a 41 a 42 a 43 a 1

2 Compute multipliers : Eliminate entries in first column: m i1 = a i1 a 11, i = 2, 3, 4 ith row = ith row m i1 1st row A = a 21 a a a a 31 a 32 a a a 41 a 42 a 43 a where ij = a ij m i1 a 1j 2

3 Compute multipliers : m i2 = a(1) i2 Eliminate entries in second column:, i = 3, 4 ith row = ith row m i2 2nd row A = a 21 a a a a 31 a 32 a a a 41 a 42 a 43 a where ij = ij m i2 2j 3

4 Compute multipliers : m i3 = a(2) i3 Eliminate entries in third column:, i = 4 ith row = ith row m i3 3rd row A = a 21 a a a a 31 a 32 a a a 41 a 42 a 43 a a (3) where a (3) ij = ij m i3 3j 4

5 Observation: Gaussian elimination reduces A to an upper triangular matrix: A = a 21 a a a a 31 a 32 a a a 41 a 42 a 43 a a (3) Guess: This probably gives the U in the LU factorization. Question: If that s true, where is L? 5

6 Consider the matrix: M 1 = m m m , m i1 = multipliers for 1st column = a i1 a 11 Then observe that: A 1 = M 1 A = m m m a 21 a a a a 31 a 32 a a a 41 a 42 a 43 a =

7 Similarly, if M 2 = m m , m i2 = multipliers for 2nd column = a i2 a Then A 2 = M 2 (M 1 A) = M 2 A = 0 m m =

8 Finally, if M 3 = m 43 1, m i3 = multiplier for 3rd column = a i3 a Then A 3 = M 3 (M 2 M 1 A) = M 3 A = m = a (3) 8

9 Therefore, we have: M 3 M 2 M 1 A = U Observations: Each M i is nonsingular, so Each M i is unit lower triangular, so A = M 1 1 M 1 2 M 1 3 U M 1 i is unit lower triangular The product of unit lower triangular matrices is unit lower triangular, so is unit lower triangular. L = M1 1 M 2 1 M 3 1 Therefore, Gaussian elimination can be used to get A = LU where L is unit lower triangular, and U is upper triangular. 9

10 One final observation: We do not need to form M i or Mi 1 L is simply the matrix with: to get L. In fact, Ones on the main diagonal Zeros above the main diagonal The entries below the main diagonal are the multipliers, m ij. That is, m L = m 31 m m 41 m 42 m 43 1 This can be proved directly by verifying that M i = M m i+1,i 1 1 i =.... m n,i 1 Then verify that L = M 1 1 M 1 2 M 1 n has the above form. 1 m i+1,i m n,i 1 10

11 An algorithm for computing the LU factorization could look like: Initialize L to be an n-by-n identity matrix for k = 1, 2,..., n-1 for i = k+1,..., n L(i,k) = A(i,k) / A(k,k) for j = k+1,..., n for i = k+1,..., n A(i,j) = A(i,j) - L(i,k)*A(k,j) Set U = upper triangular part of A 11

12 One common improvement typically done is to conserve storage by observing: The entries below the main diagonal of U are 0, so they do not need to be explicitly stored. The same thing can be said for the entries of L above the main diagonal. Furthermore, the entries on the main diagonal of L are always 1, so they don t need to be explicitly stored. Therefore, we can conserve storage by overwriting A with U on and above the main diagonal, and L below the main diagonal. That is, A = u 11 u 12 u 13 u 14 l 21 u u u l 31 l 32 u u l 41 l 42 l 43 u 12

13 An algorithm for computing the LU factorization, overwriting A with the factors, could look like: for k = 1, 2,..., n-1 for i = k+1,..., n A(i,k) = A(i,k) / A(k,k) for j = k+1,..., n for i = k+1,..., n A(i,j) = A(i,j) - A(i,k)*A(k,j) 13

14 An Matlab function, exploiting the vector and array operations available in Matlab, could look like: function A = MyLU(A) Overwrite A with L and U, where A = LU. more comments... [m,n] = size(a); if ( m ~= n) error( MyLU only works for square matrices. ) for k = 1:n-1 A(k+1:n,k) = A(k+1:n,k) / A(k,k); A(k+1:n,k+1:n) = A(k+1:n,k+1:n) - A(k+1:n,k) * A(k,k+1:n); 14

15 An Matlab function, exploiting the vector and array operations available in Matlab, could look like: function [L, U] = MyLU(A) Explicitly compute L and U, where A = LU. more comments... [m,n] = size(a); if ( m ~= n) error( MyLU only works for square matrices. ) for k = 1:n-1 A(k+1:n,k) = A(k+1:n,k) / A(k,k); A(k+1:n,k+1:n) = A(k+1:n,k+1:n) - A(k+1:n,k) * A(k,k+1:n); L = eye(n) + tril(a, -1); U = triu(a); 15

16 An Matlab function, exploiting the vector and array operations available in Matlab, could look like: function [L, U] = MyLU(A) Either overwrite A or explicitly compute L and U, where A = LU. more comments... [m,n] = size(a); if ( m ~= n) error( MyLU only works for square matrices. ) for k = 1:n-1 A(k+1:n,k) = A(k+1:n,k) / A(k,k); A(k+1:n,k+1:n) = A(k+1:n,k+1:n) - A(k+1:n,k) * A(k,k+1:n); if nargout == 2 L = eye(n) + tril(a, -1); U = triu(a); else L = A; 16

17 Using nargout, If you want to overwrite A with L and U: >> A = MyLU(A); If you don t want to overwrite A, you can use it like: >> [L, U] = MyLU(A); 17

18 Some final remarks on A = LU The computational cost of computing the LU factorization is 2n 3 /3 flops. (Check!) Recall that forward and backward solves with the L and U factors costs n 2 flops. Thus, if we have several systems to solve, with the same matrix A, but different right hand side vectors, it is cheapr to compute A = LU once and use the forward and backward solves, rather than to use Gaussian (or Gauss-Jordan) elimination on each system. Our approach to computing A = LU only works if each of the pivots,, is nonzero. a (k 1) kk Theorem: If all of the principal submatrices, A k = a 11 a 1k.. a k1 a kk are nonsingular, then each pivot a (k 1) kk factorization. is nonzero, and thus A has an LU 18

19 In our code, we could check for zero pivots: function [L, U] = MyLU(A) some comments here... [m,n] = size(a); if ( m ~= n) error( MyLU only works for square matrices. ) for k = 1:n-1 if A(k,k) == 0 error( MyLU fails! ) A(k+1:n,k) = A(k+1:n,k) / A(k,k); A(k+1:n,k+1:n) = A(k+1:n,k+1:n) - A(k+1:n,k) * A(k,k+1:n); if nargout == 2 L = eye(n) + tril(a, -1); U = triu(a); else L = A; 19

20 Remarks: Zero pivots are easy to recognize. The real danger is if a pivot is very small, but not zero. What does very small mean? 20