Matrix Multiplication Chapter IV Special Linear Systems

Matrix Multiplication Chapter IV Special Linear Systems By Gokturk Poyrazoglu The State University of New York at Buffalo BEST Group Winter Lecture Series

Outline 1. Diagonal Dominance and Symmetry a. LDL T Factorization 2. Positive Definite Systems a. Cholesky Factorization 3. Banded Systems 4. Symmetric Indefinite Systems

Diagonal Dominance Row Diagonally Dominant Matrix: Column Diagonally Dominant Matrix : The magnitude of diagonal component is larger than the sum of all off-diagonal element magnitudes in the same column. Properties: 1. The matrix can be singular. 2. If it is non-singular, LU factorization is SAFE.

Bounded Entries of L Example : where v/α is always less than 1 (one).

LDL T Factorization Consider a symmetric, nonsingular, square matrix A; Solving Ax=b by LDL T factorization Requires (n 3 /3)flops; half as many flops to compute LU

Positive Definite Matrix Definition: Properties: Consider a symmetric matrix A; Then we have the followings:

Properties of Positive Definite The last two equations imply Results: 1. The largest element in A is on the diagonal and it is positive. 2. A symmetric positive definite matrix does NOT need pivoting, and a special factorization (Cholesky) is available.

Positive Definiteness A positive definite matrix is always Nonsingular

Positive Definiteness Computation Safety : 1. Matrix-A is positive definite, but pivoting is required for safe computation.

Unsymmetric Positive Definite Systems Consider a general matrix A; Symmetric part of A: Skew-symmetric part of A: where Matrix-A is positive definite iff matrix-t is positive definite.

Symmetric Positive Definite Systems If matrix A is symmetric positive definite. A=LU exists and is stable to compute. A=LDL T is also stable and exploits symmetry. A variation of LDL T is often handier. (Cholesky factorization) Cholesky Factorization:

Cholesky Factorization G is the Cholesky factor. Solve triangular systems and is known as the Cholesky factorization Algorithm:

Stability of Cholesky Process In exact arithmetic, a symmetric positive definite matrix has a Cholesky factorization. Challenges: 1. Small Diagonal elements and 2. small minimum eigenvalue of A may jeopardize the factorization process. Hence, LDL T can be used to handle ill-conditioned matrix.

LDL T with Symmetric Pivoting Consider a symmetric matrix A and a permutation P. is not symmetric, but is. where Choose symmetric pivoting matrix P such that α is the largest component of A s diagonal entries.

Why LDL T versus Cholesky 1. LDL T is more efficient in narrow band situations because it avoids square roots. 2. LDL T can also handle symmetric semidefinite and symmetric indefinite matrix factorizations.

Positive Semidefinite Case Positive Semidefinite Matrix: for every vector x. Symmetric Positive Semidefinite Properties:

Symmetric Semidefinite Case Rank estimation by LDL T : Consider a symmetric positive semidefinite matrix A Rank(A) =r and k<=r After k th step of factorization, we have If d k =0; then A k =0 because of symmetric positive definiteness Meaning: If d k =0; then k=r, and rank(a)=k=r Note: In practice, a threshold tolerance for small diagonal entries is needed to identify zero at the diagonal.

Sum of Rank-1 Matrices Consider a symmetric positive definite matrix A LDLT factorization of A is; Rewrite as a sum of rank-1 matrices Note : Relatively cheap alternative to SVD rank-1 expansion for symmetric positive semidefinite matrices.

Tridiagonal System Solving Consider a tridiagonal symmetric positive definite matrix A; LDLT factorization is in the form: From the equation of ; we can get the followings:

Tridiagonal System Solving Algorithm: The solution of the system:

Symmetric Indefinite Systems LDL T Challenges: 1. Without pivoting, no stability 2. Even with pivoting, no stability 3. Pivoting destroys symmetry. Question: Is there any other way to compute LDLT safely while maintaining symmetry?

Stability for Symmetric Indefinite Systems 1. Aasen Method: 2. Diagonal Pivoting Method:

Aasen s Method Method is also known as The Parlett-Reid Algorithm. Consider a symmetric indefinite matrix A; At the 2 nd step, we have Scan the vector [v 3 v 4 v 5 ] T for its largest entry. Determine P 2 such that;