1 Multiply Eq. E i by λ 0: (λe i ) (E i ) 2 Multiply Eq. E j by λ and add to Eq. E i : (E i + λe j ) (E i )

Transcription

1 Direct Methods for Linear Systems Chapter Direct Methods for Solving Linear Systems Per-Olof Persson Department of Mathematics University of California, Berkeley Math 18A Numerical Analysis Consider solving a linear system of the form: E 1 : a 11 x 1 + a 1 x + + a 1n x n = b 1, E : a 1 x 1 + a x + + a n x n = b, E n : a n1 x 1 + a n x + + a nn x n = b n, for x 1,, x n Direct methods give an answer in a fixed number of steps, subject only to round-off errors We use three row operations to simplify the linear system: 1 Multiply Eq E i by λ 0: (λe i ) (E i ) Multiply Eq E j by λ and add to Eq E i : (E i + λe j ) (E i ) 3 Exchange Eq E i and Eq E j : (E i ) (E j ) Gaussian Elimination Gaussian Elimination with Backward Substitution Reduce a linear system to triangular form by introducing zeros using the row operations (E i + λe j ) (E i ) Solve the triangular form using backward-substitution Row Exchanges If a pivot element on the diagonal is zero, the reduction to triangular form fails Find a nonzero element below the diagonal and exchange the two rows An n m matrix is a rectangular array of elements with n rows and m columns in which both value and position of an element is important Count the number of arithmetic operations performed Use the formulas m j = m(m + 1), Reduction to Triangular Form m j = m(m + 1)(m + 1) (n i)(n i + ) = = n3 + 3n 5n (n i)(n i + 1) = = n3 + 3n 5n Backward Substitution 1 + ((n i) + 1) = n + n ((n i 1) + 1) = n n Gaussian Elimination Total Operation Count n n n 3 n n 5n

2 Partial Pivoting Scaled Partial Pivoting In Gaussian elimination, if a pivot element a (k) kk is small compared to an element a (k) jk below, the multiplier m jk = a(k) jk a (k) kk will be large, resulting in round-off errors Partial pivoting finds the smallest p k such that a (k) pk = max k i n a(k) ik and interchanges the rows (E k ) (E p ) If there are large variations in magnitude of the elements within a row, scaled partial pivoting can be used Define a scale factor s i for each row At step i, find p such that s i = max 1 j n a ij a pi s p a ki = max i k n s k and interchange the rows (E i ) (E p ) Linear Algebra Properties Two matrices A and B are equal if they have the same number of rows and columns n m and if a ij = b ij If A and B are n m matrices, the sum A + B is the n m matrix with entries a ij + b ij If A is n m and λ a real number, the scalar multiplication λa is the n m matrix with entries λa ij Let A, B, C be n m matrices, λ, µ real numbers (a) A + B = B + A (b) (A + B) + C = A + (B + C) (c) A + 0 = 0 + A = A (d) A + ( A) = A + A = 0 (e) λ(a + B) = λa + λb (f) (λ + µ)a = λa + µa (g) λ(µa) = (λµ)a (h) 1A = A Matrix Multiplication Let A be n m and B be m p The matrix product C = AB is the n p matrix with entries c ij = m a ik b kj = a i1 b 1j + a i b j + + a im b mj k=1 Special Matrices A square matrix has m = n A diagonal matrix D = [d ij ] is square with d ij = 0 when i j The identity matrix of order n, I n = [δ ij ], is diagonal with { 1, if i = j, δ ij = 0, if i j An upper-triangular n n matrix U = [u ij ] has u ij = 0, if i = j + 1,, n A lower-triangular n n matrix L = [l ij ] has l ij = 0, if i = 1,, j 1

3 Properties Matrix Inversion Let A be n m, B be m k, C be k p, D be m k, and λ a real number (a) A(BC) = (AB)C (b) A(B + D) = AB + AD (c) I m B = B and BI k = B (d) λ(ab) = (λa)b = A(λB) An n n matrix A is nonsingular or invertible if n n A 1 exists with AA 1 = A 1 A = I The matrix A 1 is called the inverse of A A matrix without an inverse is called singular or noninvertible For any nonsingular n n matrix A, (a) A 1 is unique (b) A 1 is nonsingular and (A 1 ) 1 = A (c) If B is nonsingular n n, then (AB) 1 = B 1 A 1 Matrix Transpose Determinants The transpose of n m A = [a ij ] is m n A t = [a ji ] A square matrix A is called symmetric if A = A t (a) (A t ) t = A (b) (A + B) t = A t + B t (c) (AB) t = B t A t (d) if A 1 exists, then (A 1 ) t = (A t ) 1 (a) If A = [a] is a 1 1 matrix, then det A = a (b) If A is n n, the minor M ij is the determinant of the (n 1) (n 1) submatrix deleting row i and column j of A (c) The cofactor A ij associated with M ij is A ij = ( 1) i+j M ij (d) The determinant of n n matrix A for n > 1 is or det A = det A = a ij A ij = a ij A ij = ( 1) i+j a ij M ij ( 1) i+j a ij M ij Properties Linear Systems and Determinants (a) If any row or column of A has all zeros, then det A = 0 (b) If A has two rows or two columns equal, then det A = 0 (c) If à comes from (E i) (E j ) on A, then det à = det A (d) If à comes from (λe i) (E i ) on A, then det à = λ det A (e) If à comes from (E i + λe j ) (E i ) on A, with i j, then det à = det A (f) If B is also n n, then det AB = det A det B (g) det A t = det A (h) When A 1 exists, det A 1 = (det A) 1 (i) If A is upper/lower triangular or diagonal, then det A = n a ii The following statements are equivalent for any n n matrix A: (a) The equation Ax = 0 has the unique solution x = 0 (b) The system Ax = b has a unique solution for any b (c) The matrix A is nonsingular; that is, A 1 exists (d) det A 0 (e) Gaussian elimination with row interchanges can be performed on the system Ax = b for any b

4 The kth Gaussian transformation matrix is defined by M (k) 0 = m k+1,k m n,k Gaussian elimination can be written as a (1) 11 a (1) 1 a (1) 1n A (n) = M () M (1) A = 0 a () () a,n 0 0 a (n) nn Reversing the elimination steps gives the inverses: L (k) = [M (k) ] 1 0 = m k+1,k m n,k and we have If Gaussian elimination can be performed on the linear system Ax = b without row interchanges, A can be factored into the product of lower-triangular L and upper-triangular U as A = LU, where m ji = a (i) ji /a(i) ii : a (1) 11 a (1) 1 a (1) 1n U = 0 a () () a, L = m 1 1,n a (n) nn m n1 m n, 1 LU = L (1) L () M () M (1) A = [M (1) ] 1 [M () ] 1 M () M (1) A = A Permutation Matrices Suppose k 1,, k n is a permutation of 1,, n The permutation matrix P = (p ij ) is defined by { 1, if j = k i, p ij = 0, otherwise (i) P A permutes the rows of A: a k11 a k1n P A = a kn1 a knn (ii) P 1 exists and P 1 = P t Gaussian elimination with row interchanges then becomes: Diagonally Dominant Matrices The n n matrix A is said to be strictly diagonally dominant when a ii > j i a ij A strictly diagonally dominant matrix A is nonsingular, Gaussian elimination can be performed on Ax = b without row interchanges, and the computations will be stable A = P 1 LU = (P t L)U

5 Positive Definite Matrices Principal Submatrices A matrix A is positive definite if it is symmetric and if x t Ax > 0 for every x 0 If A is an n n positive definite matrix, then (a) A has an inverse (b) a ii > 0 (c) max 1 k,j n a kj max 1 i n a ii (d) (a ij ) < a ii a jj for i j A leading principal submatrix of a matrix A is a matrix of the form a 11 a 1 a 1k a 1 a a k A k = a k1 a k a kk for some 1 k n A symmetric matrix A is positive definite if and only if each of its leading principal submatrices has a positive determinant SPD and Gaussian Elimination Band Matrices The symmetric matrix A is positive definite if and only if Gaussian elimination without row interchanges can be done on Ax = b with all pivot elements positive, and the computations are then stable Corollary The matrix A is positive definite if and only if it can be factored A = LDL t where L is lower triangular with 1 s on its diagonal and D is diagonal with positive diagonal entries Corollary The matrix A is positive definite if and only if it can be factored A = LL t, where L is lower triangular with nonzero diagonal entries An n n matrix is called a band matrix if p, q exist with 1 < p, q < n and a ij = 0 when p j i or q i j The bandwidth is w = p + q 1 A tridiagonal matrix has p = q = and bandwidth 3 Suppose A = [a ij ] is tridiagonal with a i,i 1 a i,i+1 0 If a 11 > a 1, a ii a i,i 1 + a i,i+1, and a nn > a n,, then A is nonsingular