Section 3.5 LU Decomposition (Factorization) Key terms Matrix factorization Forward and back substitution LU-decomposition Storage economization
In matrix analysis as implemented in modern software the idea of factoring a matrix into a product of matrices of special form is particularly important. The strategy used is to factor the matrix and then use the factors to efficiently and quickly solve the problem. Case of a General Nonsingular Linear System Ax = b As we saw with GEM, triangular matrices are important. The notion of using row operations to transform the augmented matrix [A b] to upper triangular form [U c] and then use back substitution provides a reliable technique, especially when combined with a pivoting strategy. Related Idea: If L is a lower triangular matrix, then linear system Lx = b can solved by forward substitution. Assuming that no diagonal entry l ii is zero we can proceed as follows.
For many nonsingular linear systems Ax = b it can be shown that the coefficient matrix can be factored as a product of a lower triangular matrix and an upper triangular matrix. That is, A = LU, and we say we have an LU-factorization or LU-decomposition of A. If any row interchanges are required to perform the factorization or partial pivoting is incorporated, then the equivalent linear system must be expressed as LUx = Pb, where P is a permutation matrix that embodies the row interchanges that were used. If no row interchanges are used then the equivalent system is LUx = b. In either case, the equivalent system is easily solved as we now show. The solution of a linear system LUx = c is done as follows: 1. Name Ux to be z. 2. Solve Lz = c by forward substitution. We now have vector z. 3.Solve system Ux = z by back substitution. We say that LUx = c is solved by a forward substitution followed by a back substitution.
Start with Lz = b 3 z 10 1 Ux z augmented matrix Do back substitution. augmented matrix 1 0 0 3 2 1 0 4 0 1 2 8 2 0 1 3 0 3 1 10 0 0 1 1 Do forward substitution first. 1 3 2 3 z1 3 x 3 1 2z z 4 z2 10 1 2 z 3x x 10 2z 8 z3 1 2 3 2x x 3 x1 1 x2 3 x3 1 1 x 3 1
Comment: There can be more than one LU-factorization for a matrix A. Constructing an LU-factorization We develop an LU-factorization procedure that utilizes row operations in the same way as we applied them to GEM. Initially we assume that no row interchanges are required to get a nonzero pivot and later generalize the procedure to incorporate row interchanges so that partial pivoting can be used. Case of NO row interchanges: We use row operations to transform just the coefficient matrix A to upper triangular form U. As we proceed we construct the lower triangular matrix L using the negatives of the multipliers k of the row operations k*row(i) + Row(j) ==> Row(j). The negative of the multiplier is stored in row-column position in L that was zeroed out in A by the row operation. We illustrate this process in Example 2.
Case of INCORPORATING row interchanges: As in the case of NO row interchanges we build L and U using row operations. However, we must employ an indirect addressing scheme for the rows since we physically do want to make row interchanges. To implement this we use the pivot vector idea. In this case L and U are truly not triangular in general, rather their rows can be interchanged to get triangular form.
Recall that the pivot vector is 3 p 2 1
To determine the second pivot we inspect the entries in column 2 of U, except for the entry in the previous pivot row. So here we are looking at In the pivot vector the candidates for the second pivot are in rows 2 and 3; that is, in rows p 2 = 2 and p 3 = 1 of column 2 of U. Since the magnitude of these entries is the same we will choose the first one encountered. Thus the second pivot is the (p 2, 2) entry of U. So the pivot vector contents need not be rearranged. We insert a 1 into the (p 2, 2) entry of L and zeros into the other entries of row p 2 that are blank. We have 1/ 3 L 2 / 3 1 0 1 0 0 4 3 4 3
The permutation matrix is an identity matrix with rows interchanged; we need to inter change rows 1 and 3. So
Comment: It is not necessary to have a separate matrix L. The contents of L can be stored in the entries of A that are zeroed out as a result of the row operations k*row(i) + Row(j) ==> Row(j). In this regard we "remember" that L is to have 1's in diagonal entries if no interchanges are made and correspondingly when a strategy such as partial pivoting is used, the pivot vector contains information on where the 1's should appear. Hence there is storage economization with this device. MATLAB s backslash command If we have a square linear system Ax = b and coefficient matrix A doesn t have a special structure (like triangular & several others we haven t discussed) then MATLAB performs a general triangular factorization using LU factorization with partial pivoting. Any necessary permutations to avoid explicit row interchanges are included as part of the code. To solve Ax = b in MATLAB use the command >> x = A\b. Of course matrices A and b must be defined in MATLAB with appropriate sizes.
To practice lu-factorization we have routine lupr Perform LU-factorization on matrix A by explicitly choosing row operations to use. No row interchanges are permitted, hence it is possible that the factorization can not be found. It is recommended that the multilpiers be constructed in terms of the elements of matrix U, like -U(3,2)/U(2,2), since the displays of matrices L and U do not show all the decimal places available. A row operation can be "undone", but this feature cannot be used in succession. This routine uses the utilities mat2strh and blkmat. Use in the form ==> [L,U] = lupr(a) <== By: David R. Hill, Mathematics Dept., Temple Univ. Philadelphia, PA. 19122 Email: dhill001@temple.edu
LU-factorization in MATLAB The MATLAB command is lu. It can be used in a variety of ways to obtain different sets of output. We will consider just one variation. >> help lu lu lu factorization. The command [L,U,P] = lu(a) returns a unit lower triangular matrix L, upper triangular matrix U, and permutation matrix P so that P*A = L*U. Suppose we have a linear system Ax = b. Then command [L,U,P] = lu(a) matrices L, U, and P. So to solve the system we can think along the following lines: PAx = Pb LUx = Pb (theory) now use a forward and back substitution to solve for x; y = L\(P*b) and then x = U\y MATLAB commands Since the backslash command recognizes triangular structure of the coefficient matrix we need not have special m-files for forward or back substitution for efficiency issues.
Example: Use lu to solve the system Ax = b where >> [L,U,P]=lu(A) L = 1.0000 0 0 0.1429 1.0000 0 0.5714 0.5000 1.0000 U = 7.0000 8.0000 0 0 0.8571 3.0000 0 0 4.5000 P = 0 0 1 1 0 0 0 1 0 >> y=l\(p*b) y = 22.0000-14.1429-22.5000 >> x=u\y x = 2.0000 1.0000-5.0000 1 2 3 11 A 4 5 6, 17 b 7 8 0 22 >> format long e,x x = 2.000000000000004e+00 9.999999999999963e-01-4.999999999999999e+00 The same as obtained from x = A\b. The exact solution is 2 x 1 5
Routine lufact Performs an implicit LU-factorization on matrix A using Gaussian Elimination with partial pivoting. No row interchanges are performed so the output implicitly contains an upper triangular matrix U and a lower triangular matrix L with 1's on the diagonal. The pivot vector for "decoding" to explicitly form L and U is returned in column npiv. Use in the form --> [A,npiv] = lufact(a) <-- By: David R. Hill, MATH Dept, Temple University Philadelphia, Pa. 19122 Email: dhill001@temple.edu
MATLAB s linsolve linsolve Solve linear system A*X=B. X = linsolve(a,b) solves the linear system A*X=B using LU factorization with partial pivoting when A is square, and QR factorization with column pivoting otherwise. Warning is given if A is ill conditioned for square matrices and rank deficient for rectangular matrices. This routine has many features. Some of them are explored in Numerical Analysis II. More topics need to be developed than we have time for in Numerical Analysis I.