Draft. Lecture 12 Gaussian Elimination and LU Factorization. MATH 562 Numerical Analysis II. Songting Luo

Transcription

1 Lecture 12 Gaussian Elimination and LU Factorization Songting Luo Department of Mathematics Iowa State University MATH 562 Numerical Analysis II ongting Luo ( Department of Mathematics Iowa State University[0.5in] MATH562 MATH 562 Numerical Analys 1 / 12

2 Outline 1 Gaussian Elimination and LU Factorization Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH562 MATH 562 Numerical Analys 2 / 12

3 Outline 1 Gaussian Elimination and LU Factorization Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH562 MATH 562 Numerical Analys 3 / 12

4 Gaussian Elimination and LU Factorization Gaussian elimination can be viewed as triangular triangularization of nonsingular A P C mˆm Llooooooomooooooon m 1 L 2 L 1 A U L 1 analogous to Householder QR factorization of matrix A P C mˆm Q loooooomoooooon n Q 2 Q 1 A R Q Example of LU factorization of 4 ˆ 4 matrix A ˆ ˆ ˆ ˆ ˆ ˆ ˆ ˆ ˆ ˆ ˆ ˆ L Ñ 1 0 ˆ ˆ ˆ L 2 0 ˆ ˆ ˆ fl Ñ 0 ˆ ˆ ˆ L ˆ ˆ fl Ñ 0 ˆ ˆ ˆ 0 0 ˆ ˆ fl looooooooooomooooooooooon 0 ˆ ˆ ˆ looooooooooomooooooooooon 0 0 ˆ ˆ looooooooooomooooooooooon ˆ L 1 A L 2 L 1 A L 3 L 2 L 1 A Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH562 MATH 562 Numerical Analys 4 / 12

5 What is Matrices L k At step k, eliminate entries below a kk : let x k be kth column of L k 1 L 1 A, x k rx 1,k, x 2,k,..., x k,k, x k`1,k,..., x m,k s T L k x k rx 1,k, x 2,k,..., x k,k, 0,, 0s T The multiplier l jk x jk {x kk appear in L k 1... L k 1 l k`1,k fl l m,k 1 Let l k r0,, 0, l k`1,k,, l m,k s T and e k r0,, 0, 1,, 0s T, then L k I l k e k. Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH562 MATH 562 Numerical Analys 5 / 12

6 Forming L Luckily, the L matrix contains the multipliers l jk x jk {x kk 1 l 21 1 L L 1 1 L 1 2 L 1 m 1 l 31 l fl l m1 l m2 l m,m 1 1 and is said to be a unit lower triangular matrix First, L 1 k I ` l k e k, because e k l k 0 and pi l k e k qpi ` l ke k q I l ke k l ke k I Second, L 1 1 L 1 2 L 1 k`1 I ` řk`1 j 1 l je j, since (prove by induction) pi ` řk j 1 l je j qpi ` l k`1e k`1 q I ` řk`1 j 1 l je j ` řk j 1 l jpe j l k`1qe k`1 where e j l k`1 0 for j ă k ` 1 In other words, L is union of L 1 1, L 1 2,, L 1 m 1 Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH562 MATH 562 Numerical Analys 6 / 12

7 Gaussian Elimination without Pivoting Factorize A P C mˆm into A LU Gaussian elimination without pivoting U A, L I for k 1 to m 1 for j k ` 1 to m l jk u jk {u kk u j,k:m u j,k:m l jk u k,k:m Flop count ř m k 1 2pm kqpm kq 2 ř m k 1 k2 2m 3 {3 In practice, L often overwrites lower-triangular part of A and U overwrites upper-triangular part of A Question: What if u kk is 0? Answer: The algorithm would break. Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH562 MATH 562 Numerical Analys 7 / 12

8 Partial Pivoting At step k, we divide by u kk, which would break if u kk is 0 (or close to 0), which can happen even if A is nonsingular ˆ ˆ ˆ ˆ ˆ ˆ ˆ ˆ ˆ ˆ 0 x kk ˆ ˆ ˆ 0 ˆ ˆ ˆ ˆ 0 ˆ ˆ ˆ ˆ fl Ñ 0 x kk ˆ ˆ ˆ 0 0 ˆ ˆ ˆ 0 0 ˆ ˆ ˆ fl Ñ 0 ˆ ˆ ˆ ˆ 0 0 ˆ ˆ ˆ However, any nonzero entry in kth column below diagonal can also be used as pivot ( In general, we take nonzero entry with largest absolute value) ˆ ˆ ˆ ˆ ˆ ˆ ˆ ˆ ˆ ˆ 0 ˆ ˆ ˆ ˆ 0 ˆ ˆ ˆ ˆ 0 x ik ˆ ˆ ˆ fl Ñ 0 0 ˆ ˆ ˆ 0 0 ˆ ˆ ˆ 0 x ik ˆ ˆ ˆ fl 0 ˆ ˆ ˆ ˆ 0 0 ˆ ˆ ˆ Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH562 MATH 562 Numerical Analys 8 / 12

9 More on Partial Pivoting kth step of Gaussian elimination of partial pivoting ˆ ˆ ˆ ˆ 0 ˆ ˆ ˆ 0 x ik ˆ ˆ 0 ˆ ˆ ˆ Pivot selection fl P k Ñ ˆ ˆ ˆ ˆ 0 x kk 0 0 ˆ ˆ ˆ Row Interchange fl L k Ñ ˆ ˆ ˆ ˆ 0 x kk ˆ ˆ Elimimation and we interchange row i with row k In terms of matrices, it becomes Llooooooooooooooomooooooooooooooon m 1 P m 1 L 2 P 2 L 1 P 1 A U L 1 P P P m 1 P 2 P 1 and L pl 1 m 1 L 1 2L 1 1q 1, where L 1 k P m 1 P k`1 L P 1 k k`1 P 1 m 1 It is easy to verify that L m 1 P m 1 L 2 P 2 L 1 P 1 pl 1 m 1 L 1 2L 1 1qpP m 1 P 2 P 1 q L 1 k I P m 1 P k`1 l k e k and L is union of pl 1 k q 1 I ` P m 1 P k`1 l k e k Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH562 MATH 562 Numerical Analys 9 / 12 fl

10 Algorithm of Gaussian Elimination with Partial Pivoting Factorize A P C mˆm into PA LU Gaussian elimination with partial pivoting U A, L I, P I for k 1 to m 1 i Ð argmax iěk u ij u k,k:m Ø u i,k:m l k,1:k 1 Ø l i,1:k 1 p k,: Ø p i,: for j k ` 1 to m l jk u jk {u kk u j,k:m u j,k:m l jk u k,k:m Question: What if u kk is 0? Flot count ř m k 1 2pm kqpm kq 2 ř m k 1 k2 2m 3 {3, same as without pivoting. Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH562 MATH 562 Numerical Analys 10 / 12

11 An Alternative Implementation In practice, L and U overwrite A and P is represented by a vector Gaussian elimination with partial pivoting (alternative) p r1, 2,, ms; for k 1 to m 1 i Ð argmax iěk u ij a k,1:m Ø a i,1:m p k Ø p i a k`1:m,k Ð a k`1:m,k {a k,k A k`1:m,k`1:m Ð A k`1:m,k`1:m a k`1:m,k ˆ a k,k`1:m Using LU factorization to solve Ax b: PA LU (LU factorization with partial pivoting) Ly Pb (Forward substitution) Ux y (Back substitution) Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH562 MATH 562 Numerical Analys 11 / 12

12 Complete Pivoting More generally, we can use any nonzero entry In theory, any nonzero entry pi, jq, i ě k, j ě k ˆ ˆ ˆ ˆ ˆ ˆ ˆ ˆ 0 ˆ ˆ ˆ 0 ˆ x ij ˆ fl Ñ ˆ x ij ˆ fl 0 ˆ ˆ ˆ 0 0 and we then permute row i with row k, column j with column k In matrix operations, it can be expressed as Llooooooooooooooomooooooooooooooon m 1 P m 1 L 2 P 2 L 1 P 1 A Qloooooooomoooooooon 1 Q 2 Q m 1 U L 1 P Therefore, PAQ LU where P P m 1 P 2 P 1 and L pl 1 m 1 L 1 2L 1 1q 1 Q However, complete pivoting is typically not used in practice because it increases cost in search of pivot and complexity of implementation Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH562 MATH 562 Numerical Analys 12 / 12