Matrix Multiplication Chapter IV Special Linear Systems

Transcription

1 Matrix Multiplication Chapter IV Special Linear Systems By Gokturk Poyrazoglu The State University of New York at Buffalo BEST Group Winter Lecture Series

2 Outline 1. Diagonal Dominance and Symmetry a. LDL T Factorization 2. Positive Definite Systems a. Cholesky Factorization 3. Banded Systems 4. Symmetric Indefinite Systems

3 Diagonal Dominance Row Diagonally Dominant Matrix: Column Diagonally Dominant Matrix : The magnitude of diagonal component is larger than the sum of all off-diagonal element magnitudes in the same column. Properties: 1. The matrix can be singular. 2. If it is non-singular, LU factorization is SAFE.

4 Bounded Entries of L Example : where v/α is always less than 1 (one).

5 LDL T Factorization Consider a symmetric, nonsingular, square matrix A; Solving Ax=b by LDL T factorization Requires (n 3 /3)flops; half as many flops to compute LU

6 Positive Definite Matrix Definition: Properties: Consider a symmetric matrix A; Then we have the followings:

7 Properties of Positive Definite The last two equations imply Results: 1. The largest element in A is on the diagonal and it is positive. 2. A symmetric positive definite matrix does NOT need pivoting, and a special factorization (Cholesky) is available.

8 Positive Definiteness A positive definite matrix is always Nonsingular

9 Positive Definiteness Computation Safety : 1. Matrix-A is positive definite, but pivoting is required for safe computation.

10 Unsymmetric Positive Definite Systems Consider a general matrix A; Symmetric part of A: Skew-symmetric part of A: where Matrix-A is positive definite iff matrix-t is positive definite.

11 Symmetric Positive Definite Systems If matrix A is symmetric positive definite. A=LU exists and is stable to compute. A=LDL T is also stable and exploits symmetry. A variation of LDL T is often handier. (Cholesky factorization) Cholesky Factorization:

12 Cholesky Factorization G is the Cholesky factor. Solve triangular systems and is known as the Cholesky factorization Algorithm:

13 Stability of Cholesky Process In exact arithmetic, a symmetric positive definite matrix has a Cholesky factorization. Challenges: 1. Small Diagonal elements and 2. small minimum eigenvalue of A may jeopardize the factorization process. Hence, LDL T can be used to handle ill-conditioned matrix.

14 LDL T with Symmetric Pivoting Consider a symmetric matrix A and a permutation P. is not symmetric, but is. where Choose symmetric pivoting matrix P such that α is the largest component of A s diagonal entries.

15 Why LDL T versus Cholesky 1. LDL T is more efficient in narrow band situations because it avoids square roots. 2. LDL T can also handle symmetric semidefinite and symmetric indefinite matrix factorizations.

16 Positive Semidefinite Case Positive Semidefinite Matrix: for every vector x. Symmetric Positive Semidefinite Properties:

17 Symmetric Semidefinite Case Rank estimation by LDL T : Consider a symmetric positive semidefinite matrix A Rank(A) =r and k<=r After k th step of factorization, we have If d k =0; then A k =0 because of symmetric positive definiteness Meaning: If d k =0; then k=r, and rank(a)=k=r Note: In practice, a threshold tolerance for small diagonal entries is needed to identify zero at the diagonal.

18 Sum of Rank-1 Matrices Consider a symmetric positive definite matrix A LDLT factorization of A is; Rewrite as a sum of rank-1 matrices Note : Relatively cheap alternative to SVD rank-1 expansion for symmetric positive semidefinite matrices.

19 Tridiagonal System Solving Consider a tridiagonal symmetric positive definite matrix A; LDLT factorization is in the form: From the equation of ; we can get the followings:

20 Tridiagonal System Solving Algorithm: The solution of the system:

21 Symmetric Indefinite Systems LDL T Challenges: 1. Without pivoting, no stability 2. Even with pivoting, no stability 3. Pivoting destroys symmetry. Question: Is there any other way to compute LDLT safely while maintaining symmetry?

22 Stability for Symmetric Indefinite Systems 1. Aasen Method: 2. Diagonal Pivoting Method:

23 Aasen s Method Method is also known as The Parlett-Reid Algorithm. Consider a symmetric indefinite matrix A; At the 2 nd step, we have Scan the vector [v 3 v 4 v 5 ] T for its largest entry. Determine P 2 such that;

24 Aasen s Method Hence; So that; After n-2 steps;

25 Extra Proof Slides Chapter IV Special Linear Systems

26 Proof of Slide 4

27 Proof of Slide 5

28 Proof of Slide 8

29 Proofs of Slide 9

30 Proof of Slide 16