.. Factorizations Reading: Trefethen and Bau (1997), Lecture 0 Solve the n n linear system by Gaussian elimination Ax = b (1) { Gaussian elimination is a direct method The solution is found after a nite number of operations { Iterative methods nd a solution as the number of operations t to innity Iteration is superior for large sparse systems Gaussian elimination is a factorization of A as A = LU (a) { L is unit lower triangular and U is upper triangular 1
LU Factorization a 11 a 1 a 1n A = a 1 a a n...... 75 (b) a n1 a n a nn L = 1 0 0 l 1 1 0 l 1 l 0...... ln1 ln 1 75 U = u 11 u 1 u 1n 0 u u n 0 u n...... 0 0 unn 75 (c) Once L and U have been determined { Let Ax = LUx = b Ux = y (a) { Then Ly = b (b) { Solve (b) for y by forward substitution and (a) for x by backward substitution
LU Factorization Classical Gaussian elimination { Use row saxpys to reduce A to upper triangular form U { Show that these row operations are the product of A and lower triangular matrices L k, k = 1 : n ; 1, i.e., L n;1 L L 1 A = U (4a) { Thus L = L ;1 1 L;1 L ;1 n;1 (4b) { cf. Trefethen and Bau (1997), Lecture 0 We use a direct factorization { Multiply the rst row of L by U to obtain the rst row of A a 1j = (1)u 1j j = 1 : n Thus u 1j = a 1j j = 1 : n { Multiply L by the rst column of U to obtain the rst column of A li1 u 11 = ai1 i = : n or l i1 = a i1 =u 11 i = : n Redundant information with i = 1 is not written The algorithm fails if u 11 = a 11 = 0
LU Factorization Continuing { Multiply the second row of L by U to obtain the second row of A l 1 u 1j + u j = a j j = : n Thus, u j = a j ; l 1 u 1j { The general procedure is uij = aij ; Xi;1 j = : n likukj j = i : n (5a) lji = 1 k=1 Xi;1 (aji ; ljk u ki) u ii k=1 j =i +1: n i =1 : n ; 1 (5b) The sums are zero when the lower limit exceeds the upper one The procedure fails if uii = 0 for some i The matrices L and U can be stored in the locations used for the same elements of A 4
LU Factorization function A = lufactor(a) % lufactor: Compute the LU decomposition of an n-by-n % matrix A by direct factorization. On return, L is % stored in the lower triangular part of A and U is % stored in the upper triangular part. [n n] = size(a) % Loop over the rows for i = 1: n - 1 % Calculate the i th column of L for j = i + 1: n for k = 1: i - 1 A(j,i) = A(j,i) - A(j,k)*A(k,i) A(j,i) = A(j,i)/A(i,i) % Calculate the (i + 1) th row of U for j = i+1: n for k = 1: i A(i+1,j) = A(i+1,j) - A(i+1,k)*A(k,j) 5
LU Factorization Observe: i. The rst row of U needs no explicit calculation ii. The 1s on the diagonal of L are not stored iii. MATLAB Loops do not execute when the lower index exceeds the upper one iv. Classical Gausian elimination corresponds to a dierent loop ordering { cf. Trefethen and Bau (1997), Lecture 0 Example 1. Factor A = 1 0 ;4 ;1 0 6 ; ;5 0 ; ; 4 4 75 { i = 1: Calculate the rst column of L and the second row of U 1 0 ;4 ;1 6 ;75 A ) ; ;5 0 ; ; 4 4 6
LU Factorization { i = : Calculate the second column of L and the third row of U 1 0 ;4 ;1 6 ;75 A ) ;4 ;1 4 ; 1= 4 4 { i = : Calculate the third column of L and the fourth row of U 1 0 ;4 ;1 6 ;75 A ) ;4 ;1 4 ; 1= ;1 1 The L and U factors L = 1 0 0 0 ;1 1 0 0 ;4 1 0 ; 1= ;1 1 75 U = 1 0 ;4 0 6 ; 0 0 ;1 4 0 0 0 1 75 7
Elementary transformations L 1 = 1 0 0 0 1 1 0 0 ; 0 1 0 0 0 1 L Exposed L = 75 L = 1 0 0 0 0 1 0 0 0 0 1 0 0 0 1 1 1 0 0 0 0 1 0 0 0 4 1 0 0 ;1= 0 1 { L k is the identity matrix with its k th column replaced by the k th column of L with the negative of the subdiagonal elements L ;1 = L L L 1 75 1 0 0 0 1 1 0 0 1 4 1 0 :5 :5 1 1 75 75 8
Forward and Backward Substitution Solution of (b) by forward substitution y i = b i ; Xi;1 j=1 l ij y j (6a) function y = forward(l, b) % forward: Solution of a n-by-n lower triangular system % Ly = b by forward substitution. [n n] = size(l) y(1) = b(1) for i = :n y(i) = b(i) - dot(l(i,1:i-1)', y(1:i-1)) Solve (a) by backward substitution xi = 1 nx [yi ; uij xj] u ii j=i+1 (6b) function x = backward(u, y) % backward: Solution of a n-by-n upper triangular system % Ux = y by backward substitution. [n n] = size(u) x(n) = y(n)/u(n,n) for i = n - 1:-1:1 x(i) = (y(i) - dot(u(i,i+1:n)', x(i+1:n)))/u(i,i) 9
Forward and Backward Substitution Example. Solve the linear system Ax = b for x when A is as given in Example 1 and b = [;1 7 ;4 ] T { Using the forward substitution algorithm y = [;1 6 1] T { Using backward substitution x = [1 1 1 1] T 10
Factorization: (cf. p. 5) { Inner column loop: Operation Count i ; 1 multiplications and subtractions 1 division { Inner row loop: i multiplications and subtractions { The summations on j give (i ; 1)(n ; i) multiplications and subtractions n ; i divisions { The summations on i give Multiplications and subtractions: Xn;1 i=1 Divisions: (i ; 1)(n ; i) = Xn;1 i=1 (n ; 1) = n(n ; 1)(n ; 1) 6 n(n ; 1) Formulas (1..1, ) were used to obtain the result { A oating point operation (FLOP) is any+ ; = p For large n the factorization has about n = FLOPs 11
Operation Count Forward Substitution: (cf. p. 9) { i;1 multiplications and additions/subtractions in the dot product Total multiplications and additions/subtractions: nx i= Backward Substitution: (i ; 1) = n(n ; 1) { n(n;1)=multiplications and additions/subtractions and n divisions Forward and Backward Substitution: { For large n, there are about n FLOPs Factorization dominates forward and backward substitution for large n 1