Jae Heon Yun and Yu Du Han

Transcription

1 Bull. Korean Math. Soc. 39 (2002), No. 3, pp MODIFIED INCOMPLETE CHOLESKY FACTORIZATION PRECONDITIONERS FOR A SYMMETRIC POSITIVE DEFINITE MATRIX Jae Heon Yun and Yu Du Han Abstract. We propose variants of the modified incomplete Cholesy factorization preconditioner for a symmetric positive definite (SPD) matrix. Spectral properties of these preconditioners are discussed, and then numerical results of the preconditioned CG (PCG) method using these preconditioners are provided to see the effectiveness of the preconditioners.. Introduction In this paper, we consider a linear system of equations () Ax = b, x, b R n, where A R n n is a large sparse symmetric positive definite (SPD) matrix. Since A is a large sparse matrix, direct methods such as Gaussian elimination become prohibitively expensive because of a lot of fillin elements. As an alternative, the preconditioned CG (PCG) iterative method [2] is widely used for the purpose of finding an approximate solution of the problem (). Given an initial guess x 0, the PCG method computes iteratively new approximations x to the true solution x =A b. The iterate x is accepted as a solution if the residual r = b Ax satisfies r 2 b 2 tol. The convergence rate and robustness of the PCG largely dep on how well the preconditioner approximates A. One of the powerful preconditioning methods in terms of reducing the number of iterations is Received February 5, Revised April 24, Mathematics Subject Classification: 65F0, 65F50. Key words and phrases: modified incomplete Cholesy factorization, P-regular splitting, preconditioner, PCG method. This research was supported by the Chungbu National University Basic Science Research Fund BSRI-0-6.

2 496 Jae Heon Yun and Yu Du Han the incomplete Cholesy factorization method studied by Meijerin and van der Vorst [4] and Yun [8]. It was shown in [4] that for every zero pattern set, there exists an incomplete Cholesy factorization of a symmetric M-matrix. However, the incomplete Cholesy factorization for a symmetric positive definite matrix does not always exist. To this, Robert [7] introduced the modified incomplete Cholesy factorization which exists for any symmetric positive definite matrix. The purpose of this paper is to study variants of the modified incomplete Cholesy factorization for a symmetric positive definite matrix. This paper is organized as follows. In Section 2, we consider some properties of P - regular splittings. In Section 3, we consider variants of the modified incomplete Cholesy factorization. In Section 4, we provide numerical results. In Section 5, some conclusions are drawn. 2. Properties of P -regular splitting For two matrices A = (a ij ) and B = (b ij ), A B denotes a ij b ij for all i and j, and A B denotes a ij b ij for all i and j. We say that a real matrix A is monotone if Ax 0 implies x 0. It is well nown that A is monotone if and only if A 0. A real square matrix A = (a ij ) is called an M-matrix if a ij 0 for i j and it is monotone. A real matrix A is positive definite (positive semi-definite) if Re(x H Ax) > 0 (Re(x H Ax) 0) for every nonzero vector x C n, or equivalently x T Ax > 0 (x T Ax 0) for every nonzero vector x R n, where Re(x H Ax) refers to the real part of x H Ax. The spectral radius ρ(a) of a matrix A is ρ(a) = max{ λ λ σ(a) }, where σ(a) denotes the spectrum of A, that is, the set of eigenvalues of A. A representation A = M N is called a splitting of A when M is nonsingular. A splitting A = M N is a convergent splitting of A if ρ( M N ) <. A splitting A = M N is a P-regular splitting if M + N is positive definite. Obviously, the matrix M + N is positive definite if and only if M T + N is positive definite. Throughout the paper, we use the notation M 0 (M 0) for a matrix to be symmetric positive definite (symmetric positive semi-definite), and M M 2 (M M 2 ) denotes that M M 2 0 (M M 2 0). Theorem 2. ([3]). Let A be a symmetric matrix and let A = M N be a P-regular splitting of A. Then ρ( M N) < if and only if A is positive definite.

3 Modified incomplete Cholesy factorization preconditioners 497 Theorem 2.2 ([5]). Let A 0 and let A = M N = M 2 N 2 be two splittings of A. If 0 N N 2, then ρ( M N ) ρ( M2 N 2) <. If 0 N N 2, then ρ( M N ) < ρ( M2 N 2) <. Theorem 2.3 ([6]). Let A be a symmetric matrix and let A = M N be a splitting of A. If M 0 and ρ( M N ) <, then A is positive definite and A = M N is a P -regular splitting of A. Theorem 2.4. Let A 0 and let A = M N be a splitting of A with N 0. Then ρ( M N ) <, 0 < λ M A, and 0 λ M N <, where λ M A and λ M N denote eigenvalues of M A and M N, respectively. Proof. Suppose that λ is an eigenvalue of M N. Then there exists a nonzero vector x such that M Nx = λx. Thus, Nx = λmx. It follows that x H Nx = λx H Mx = λx H Ax + λx H Nx. Since A 0 and λ, x H Nx = λ λ xh Ax. Since x H Ax > 0 and x H λ Nx 0, λ 0. It follows that 0 λ <. Hence 0 λ M N < is proved. From M A = I M N, λ M A = λ M N. Thus, 0 < λ M A. For the proof of ρ(m N) <, N 0 implies that M = A+N 0. Hence, A = M N is a P -regular splitting of A. From Theorem 2., ρ(m N) < is obtained. Theorem 2.5. Suppose that A 0. Let A = M N be a splitting of A with M symmetric. Then ρ( M N ) < if and only if A = M N is a P -regular splitting. Proof. Suppose that A = M N is a P -regular splitting. Then by Theorem 2., ρ(m N) < is immediately obtained. For the proof of the other direction, let s suppose that ρ(m N) <. From the identity M A = I M N, it can be seen that M A has eigenvalues with positive real parts. Notice that M A is similar to A /2 M A /2. Since A /2 M A /2 is symmetric, M A has real eigenvalues. It follows that M A has positive eigenvalues. Hence, A /2 M A /2 0, from which M 0. From Theorem 2.3, one obtains that A = M N is a P -regular splitting. Theorem 2.6 ([]). Let A 0 and let A = M N be a splitting of A with M symmetric. Let B = i=0 (M N) i M ( ). Then

4 498 Jae Heon Yun and Yu Du Han (a) B is symmetric. (b) For odd, B is positive definite if and only if M is positive definite. (c) For even, B is positive definite if and only if M + N is positive definite. If the B defined in Theorem 2.6 is symmetric positive definite, then it can be used as a preconditioner for the PCG method and it is called -step polynomial preconditioner corresponding to the splitting A = M N. It is easy to see that lim B = A if ρ(m N) <. It is well-nown that the convergence rate of the PCG method with a preconditioner M for solving Ax = b deps on how small λmax(m A) λ min (M A) is, where λ max (M A) and λ min (M A) denote the maximum and minimum eigenvalues of M A, respectively. More specifically, if A = M N is a splitting of A with M being SPD, then the convergence rate of the PCG with the preconditioner M may dep on how small ρ(m N) is. Thus, if ρ(m N) < and M 0 is easily invertible, then the M may be considered as a good preconditioner for the PCG. 3. Modified incomplete Cholesy factorizations Let P n = {(i, j) i j, i, j n}. Then, it was shown in [4] that for every symmetric zero pattern set Z P n (i.e. (i, j) Z implies (j, i) Z), there exists an incomplete Cholesy factorization of a symmetric M-matrix A such that A = U T DU R is a regular splitting of A, where U is an upper triangular M-matrix and D is a diagonal matrix whose ith diagonal element is an inverse of the ith diagonal element of U. It is clear that U T DU in the above factorization is positive definite since D is positive definite. Thus, the U T DU obtained from the incomplete Cholesy factorization of a symmetric M-matrix can be used as a preconditioner for the PCG method. However, next example shows that the incomplete Cholesy factorization for a symmetric positive definite matrix A which is not an M-matrix does not always exist. That is, there exists a diagonal matrix D with a nonpositive diagonal element such that A = U T DU R.

5 Modified incomplete Cholesy factorization preconditioners 499 Example 3.. Let 0 0. A = Since all eigenvalues of A are positive, A 0. It is clear that A is not an M-matrix. For the zero pattern set Z = {(, 3), (2, 4), (3, ), (4, 2)}, the incomplete factorization of A such that A = U T DU R is U T = , D = , R = Since D contains a negative diagonal element 25, U T DU is not positive definite and hence it can not be used as a preconditioner of the PCG iterative method. Also note that A = U T DU R is not a regular splitting of A. The incomplete factorization of A proposed in [6] such that A = U T DU R requires that all diagonal elements of the matrix R are zero. By suppressing this requirement, Robert [0] proposed a modified incomplete Cholesy factorization which always exists for any symmetric positive definite matrix A. Theorem 3.2 ([7]). Let A 0. Then for every symmetric zero pattern set Z P n there exists a modified incomplete Cholesy factorization of A such that A = U T DU R, where U is an upper triangular matrix with positive diagonal elements, D is a diagonal matrix whose ith diagonal element is an inverse of the ith diagonal element of U, and R is positive semi-definite. The modified incomplete Cholesy factorization (MICF) in [7] is described in a theoretical way, so the MICF algorithm which can be easily implemented is provided below. Let a ij denote the (i, j) component of A and let Z be a zero pattern set. Since A is symmetric, only the lower triangular part of A is used and updated. Algorithm 3. (MICF). d = /a For i = 2, n

6 500 Jae Heon Yun and Yu Du Han For j =, i For = i, n a i = a i a j a ij d j For = i +, n if (, i) Z, then a ii = a ii + a i a = a + a i a i = 0 if d i = /a ii From Algorithm 3., the modified incomplete Cholesy factorization of the matrix A given in Example 3. such that A = U T DU R is 0 U T 2 = , D= , R= Notice that the lower triangular matrix U T and the diagonal matrix D are obtained directly from Algorithm 3., while the matrix R is computed from the identity R = U T DU A. Since we are only interested in the matrix U T DU which can be used as a preconditioner of the PCG method, all algorithms in this paper do not contain the computational step for the matrix R. From now on, the U T DU 0 obtained from Algorithm 3. is called the MICF preconditioner. Below we propose a variant of the modified incomplete Cholesy factorization (VMICF) algorithm which is more efficient than Algorithm 3. (see Tables and 3). This variant also yields a splitting A = Û T ˆDÛ ˆR such that Û T ˆDÛ 0 and ˆR 0. From now on, the Û T ˆDÛ 0 is called the VMICF preconditioner. Algorithm 3.2 (VMICF). For i =, n d i = /a ii For j = i +, n d j = a ji d i

7 Modified incomplete Cholesy factorization preconditioners 50 For j = i +, n For = j, n if (, j) Z, then a j = a j d a ji a = a + a j a jj = a jj + a j a j = 0 else a j = a j d a ji if The main difference between Algorithms 3. and 3.2 is as follows. At the ith step, Algorithm 3. updates the ith column using the first (i ) columns and then modifies diagonal elements according to a zero pattern set Z, while Algorithm 3.2 updates the last (n i) columns using the ith column and then modifies diagonal elements according to the zero pattern set Z. Here, we provide an example which shows the difference between Algorithms 3. and 3.2. Example 3.3. Let 4 0 A = Let Z = {(, 2), (2, ), (3, 4), (4, 3)}. Since A is an irreducibly diagonally dominant matrix with positive diagonal elements, A 0. From Algorithm 3., one obtains A = U T DU R with U T = , D = 2 2 3, R =

8 502 Jae Heon Yun and Yu Du Han From Algorithm 3.2, one obtains A = Û T ˆDÛ ˆR with Û T = , ˆD = 2 2, ˆR = The spectral condition numbers of A, U T DU, and Û T ˆDÛ are κ 2 (A) = A 2 A κ 2 (U T DU) = U T DU 2 (U T DU) κ 2 (Û T ˆDÛ) = Û T ˆDÛ 2 (Û T ˆDÛ) It is easy to show that 0 R ˆR. From Theorem 2.2, one obtains ρ((u T DU) R) ρ((û T ˆDÛ) ˆR) <. Note that direct calculations yield ρ((u T DU) R) = /3 ρ((û T ˆDÛ) ˆR) = /2. Suppose that A = U T DU R and A = Û T ˆDÛ ˆR are splittings of A obtained from Algorithms 3. and 3.2, respectively. Since U T DU 0, Û T ˆDÛ 0, R 0 and ˆR 0, from Theorem 2.5 ρ((u T DU) R) < and ρ((û T ˆDÛ) ˆR) <. In addition, Theorem 2.6 implies that for any M = ((U T DU) R) i (U T DU) i=0 ˆM = ((Û T ˆDÛ) ˆR) i (Û T ˆDÛ) i=0 are symmetric positive definite. Hence, the M and ˆM can be used as preconditioners for the PCG method. The M and ˆM are called -step MICF and -step VMICF polynomial preconditioners, respectively. Notice that for =, the M and ˆM reduce to the standard MICF and VMICF preconditioners, respectively. It can be shown that 0 R ˆR and thus ρ((u T DU) R) ρ((û T ˆDÛ) ˆR). From this point of view, the MICF preconditioner obtained from Algorithm 3. may provide better convergence rate of the PCG than the VMICF preconditioner obtained from Algorithm 3.2. Numerical experiments in Section 4 show that the convergence rate of the PCG with MICF preconditioner is as

9 Modified incomplete Cholesy factorization preconditioners 503 good as or slightly better than that with VMICF preconditioner, but the construction time for MICF preconditioner is much larger than that for VMICF preconditioner (see Tables and 3). So, the VMICF is more efficient preconditioner than the MICF on the Cray T3E supercomputer. Next we consider other variants of the modified incomplete Choesy factorization preconditioner. Let A be a symmetric positive definite 2 2 bloc matrix of the form ( ) A C (2) A =, C T A 2 where A and A 2 are square matrices. Since A 0, A 0 and A 2 0. Thus, there exist modified incomplete Cholesy factorizations for A and A 2. Lemma 3.4. Suppose that A is a symmetric positive) definite matrix of the form (2). Then for every nonzero vector x = x T A x + x T 2 A 2 x 2 + 2x T C x 2 > 0 x T A x + x T 2 A 2 x 2 2x T C x 2 > 0. ( x Proof. Since A is a symmetric positive ) definite matrix of the form (2), for any nonzero vector x = ( x x 2 ( ) ( ) ( ) ( x T, x T x 2 )A = ( x T, x T A C x 2 ) > 0. x 2 C T A 2 x 2 Therefore, x T A x 2x T C x 2 + x T 2 A 2x 2 > 0. Since ( ) ( ) x T Ax = (x T, xt 2 ) A C x > 0, C T A 2 x 2 the second inequality is obtained. Theorem 3.5. Suppose that A is a symmetric positive definite matrix of the form (2). Let A = U T D U R and A 2 = U2 T D 2U 2 R 2 be splittings obtained from Algorithms 3. or 3.2. Let ( ) ( ) ( U 0 D 0 U (U T U =, D =, U = D ) ) C. 0 U 2 0 D 2 0 U 2 x 2

10 504 Jae Heon Yun and Yu Du Han Let M = U T DU, M = U T DU, N = M A, and N = M A. Then A = M N and A = M N are P -regular splitting of A. Proof. Note that Let x = M + N = 2M A = ( x x 2 ) be a nonzero vector. Then ( U T D U + R C C T U T 2 D 2U 2 + R 2 x T (M + N)x = x T (U T D U + R )x +x T 2 (U T 2 D 2 U 2 + R 2 )x 2 2x T C x 2 = 2x T R x + 2x T 2 R 2 x 2 + x T A x + x T 2 A 2 x 2 2x T C x 2. Since R and R 2 are positive semi-definite, Lemma 3.4 implies that x T (M + N)x > 0. Hence, A = M N is a P -regular splitting of A. Next we will show that A = M N is a P -regular splitting of A. Note that M + N = 2M A ( ) U T = 2 D U C C T U2 T D 2U 2 + C T (U T D ) T D (U T D A ) C ( ) A + 2R C = C T 2C T (U T D ) T D (U T D. ) C + A 2 + 2R 2 If we let V = (U T D ) C, then ( ) A + 2R C M + N = C T 2V T D. V + A 2 + 2R 2 Thus for each nonzero vector x, one obtains x T (M + N )x = x T (A + 2R )x + 2x T C x 2 + x T 2 (A 2 + 2R 2 )x 2 + 2x T 2 V T D V x 2 = x T A x + 2x T C x 2 + x T 2 A 2 x 2 + 2x T R x + 2x T 2 R 2 x 2 + 2x T 2 V T D V x 2. By assumption, R 0, R 2 0 and D 0. Hence, Lemma 3.4 implies that x T (M + N )x > 0. It follows that A = M N is a P -regular splitting of A. ).

11 Modified incomplete Cholesy factorization preconditioners 505 Since the two splittings A = M N and A = M N introduced in Theorem 3.5 are P -regular splittings, from Theorem 2. ρ(m N) < and ρ(m N ) <. Hence, the M 0 and M 0 can be used as preconditioners for the PCG method. Theorem 2.6 also shows that B = (M N) i M and i=0 ˆB = i=0 (M N ) i M are symmetric positive definite and thus they can be used as -step polynomial preconditioners for the PCG method. Below we provide an efficient algorithm for computing B r which is one of the basic timeconsuming computational ernels of the PCG, where r is a given vector. Algorithm 3.3 (PRESOL()). x 0 = 0 For i =, x i = x i + M (r Ax i ) From Algorithm 3.3, it can be seen that x = B r. Notice that Algorithm 3.3 computes B r without using the matrix N. If we replace ˆB M in Algorithm 3.3 with M, then r is computed. Since U i s and D i s in Theorem 3.5 can be computed indepently for different i, the M and M can be constructed in parallel based on matrix blocs. The idea provided in Theorem 3.5 can be easily exted to a general m m bloc-tridigonal matrix A which is symmetric positive definite. 4. Numerical results The test PDE problem we are considering in this paper is (3) (a(x, y)u x (x, y)) x (b(x, y)u y (x, y)) y + c(x, y)u(x, y) = f(x, y) with a(x, y) > 0, b(x, y) > 0, c(x, y) 0, and (x, y) Ω, where Ω is a square region, and with suitable boundary conditions on Ω which denotes the boundary of Ω. All numerical results have been obtained using the PCG method. The MICF and VMICF preconditioners we have used for numerical experiments were obtained without fill-ins. In all cases, the PCG was started with an initial vector x 0 = 0 and it was stopped when r i 2 b 2 < 0 8, where r i refers to the ith residual b Ax i. All numerical experiments have been carried out using 64-bit arithmetic on the Cray T3E at the KISTI supercomputing center. In Tables to 4, ITER stands for the number of iterations satisfying the stopping criterion mentioned

12 506 Jae Heon Yun and Yu Du Han above, Prec stands for the preconditioner to be used, P-time and I-time stand for the CPU time required for constructing preconditioners and the CPU time required for the PCG with these preconditioners, respectively. All timing results are reported in seconds using one processor of the Cray T3E. For all test problems, only the matrix A, which is constructed from five-point finite difference discretization of the given PDE, is of importance, so the right-hand side vector b is created artificially. Hence, the right-hand side function f in Examples 4. and 4.2 is not relevant. Example 4.. We consider Equation (3) over the square region Ω = (0, ) (0, ) with a(x, y) = b(x, y) = cos x, c(x, y) = 0, and Dirichlet boundary condition u = 0 on Ω. That is, the following PDE problem is considered: { (cos x u) = f in Ω, u = 0 on Ω. We have used two uniform meshes of x = y = 0 and x = y = 20 which lead to two matrices of order n = and n = , where x and y refer to the mesh sizes in the x-direction and y- direction, respectively. We have used both the natural row-wise ordering and the Red-Blac ordering of the mesh grid. The matrix A generated from this discretization is a symmetric M-matrix and hence it is also a SPD matrix. To generate a SPD matrix which is not an M-matrix, all off-diagonal elements of the matrix A are made positive by taing their absolute values. Once the matrix A is constructed, the right-hand side vector b is chosen so that the exact solution is the discretization of 0xy( x)( y)e x4.5. Numerical results for this problem are listed in Tables and 2. Example 4.2. We consider Equation (3) over the square region Ω = (0, ) (0, ) with a(x, y) = b(x, y) =, c(x, y) = 0, and Dirichlet boundary condition u = 0 on Ω. That is, the following PDE problem is considered: { u = f in Ω, u = 0 on Ω. We have used the same uniform meshes as in Example 4.. Once the matrix A is constructed as in Example 4., the right hand side vector b is chosen so that b = A[,,..., ] T. Numerical results for this problem are listed in Tables 3 and 4.

13 Modified incomplete Cholesy factorization preconditioners 507 Table. Numerical results of the PCG for Example 4. Ordering n = n = (Matrix type) Prec ITER P-Time I-Time ITER P-Time I-Time Natural MICF (M-matrix) VMICF Red-Blac MICF (M-matrix) VMICF Natural MICF (SPD matrix) VMICF Red-Blac MICF (SPD matrix) VMICF Table 2. Numerical results of the PCG with -step VMICF polynomial preconditioner for Example 4. Ordering n = n = (Matrix type) ITER P-Time I-Time ITER P-Time I-Time Natural (M-matrix) Natural (SPD matrix) Red-Blac (M-matrix) Red-Blac (SPD matrix) Conclusions The construction time of the MICF preconditioner is much larger than that of the VMICF preconditioner, and the convergence rate of the PCG with the MICF preconditioner is as good as or slightly better than that with the VMICF preconditioner (see Tables and 3). So, the VMICF preconditioner is recommed for use on the Cray T3E.

14 508 Jae Heon Yun and Yu Du Han Table 3. Numerical results of the PCG for Example 4.2 Ordering n = n = (Matrix type) Prec ITER P-Time I-Time ITER P-Time I-Time Natural MICF (M-matrix) VMICF Red-Blac MICF (M-matrix) VMICF Natural MICF (SPD matrix) VMICF Red-Blac MICF (SPD matrix) VMICF Table 4. Numerical results of the PCG with -step VMICF polynomial preconditioner for Example 4.2 Ordering n = n = (Matrix type) ITER P-Time I-Time ITER P-Time I-Time Natural (M-matrix) Natural (SPD matrix) Red-Blac (M-matrix) Red-Blac (SPD matrix) The PCG with -step VMICF polynomial preconditioner reduces the number of iterations as increases (see Tables 2 and 4), but the computing time of the PCG with -step VMICF polynomial preconditioner increases as increases. This is because the reduction in the number of iterations is not enough to balance the increase in the computing time required for PRESOL(). Notice that -step VMICF polynomial preconditioner is the same as the VMICF preconditioner. Hence, -step

15 Modified incomplete Cholesy factorization preconditioners 509 VMICF polynomial preconditioner is recommed on the Cray T3E only for =. That is, the PCG with VMICF preconditioner performs efficiently on the Cray T3E. References [] L. Adams, m-step preconditioned conjugate gradient methods, SIAM J. Sci. Stat. Comput. 6 (985), [2] M. R. Hestenes and E. L. Stiefel, Methods of conjugate gradients for solving linear systems, J. Res. Nat. Bur. Stand. 49 (952), [3] F. John, Advanced Numerical Analysis, Lecture Notes, Department of Mathematics, New Yor University, 956. [4] J. A. Meijerin and H. A. van der Vorst, An iterative solution method for linear systems of which the coefficient matrix is a symmetric M-matrix, Math. Comp. 3 (977), [5] R. Nabben, A note on comparison theorems for splittings and multisplittings of hermitian positive definite matrices, Linear Algebra Appl. 233 (996), [6] J. M. Ortega, Introduction to parallel and vector solution of linear systems, Plenum Press, New Yor, 988. [7] Y. Robert, Regular incomplete factorizations of real positive definite matrices, Linear Algebra Appl. 48 (982), [8] J. H. Yun, Bloc incomplete factorization preconditioners for a symmetric bloctridiagonal M-matrix, J. Comput. Appl. Math. 94 (998), Jae Heon Yun, Department of Mathematics, College of Natural Sciences, Chungbu National University, Cheongju , Korea gmjae@cbucc.chungbu.ac.r Yu Du Han, Department of Mathematics, College of Natural Sciences, Chungbu National University, Cheongju , Korea