On the Preconditioning of the Block Tridiagonal Linear System of Equations

Transcription

1 On the Preconditioning of the Block Tridiagonal Linear System of Equations Davod Khojasteh Salkuyeh Department of Mathematics, University of Mohaghegh Ardabili, PO Box 179, Ardabil, Iran Abstract Two algorithms for computing the inverse factors of general tridiagonal and pentadiagonal matrices are obtained Then, these algorithms are used for computing a block ILU preconditioner for the block tridiagonal linear system of equations Some numerical results are given to show the robustness and efficiency of the preconditioner The performance of the proposed preconditioner is compared with a recently proposed method AMS Subject Classification : 65F10, 65F50 Keywords: Krylov subspace methods, inverse factors, block ILU factorization, block tridiagonal matrices 1 Introduction Consider the linear system of equations Ax = b, (1) where the vectors x, b R n and the matrix A R n n is of the form D 1 C 2 B 2 D 2 C 3 A = (2) B m 1 D m 1 C m B m D m This type of matrices typically arise in the finite difference or finite element discretization of the second order partial differential equations Nowadays, iterative methods based on the Krylov subspaces such as CG [13], GMRES [14], CGS [15] and Bi-CGSTAB [16] are used to solve (1) In order to be effective, these methods are combined with a good preconditioner More precisely, iterative methods usually involve a second matrix that transforms the coefficient matrix into one with a more favorable spectrum The transformation matrix is called a preconditioner If M is a nonsingular matrix which approximates A 1 (M A 1 or MA I, where I is the identity matrix), the transformed linear system MAx = Mb, (3) which is called the left-preconditioned system, will have the same solution as (1) but the convergence rate of iterative methods applied to (3) may be much higher Also, there are right- and splitpreconditioned systems [4, 5, 13] 1

2 There are many ways to make a preconditioner [3, 4, 5, 10, 11, 17] A very good survey of the preconditioners can be found in [4, 5, 13] One way is to compute lower sparse matrices M L and MU T, such that M L L and M U U, where A = LU is the LU factorization of A In this case we have A M = M L M U which is called an incomplete LU factorization (ILU) of A Here, we have A 1 M 1 = M 1 U M 1 L Iterative methods based on the Krylov subspace methods usually involve matrix-vector multiplications Hence, for computing z = M 1 r, one can solve Mz = r for z with backward and forward substitution In this paper we focus our attention to the ILU preconditioning of (1) This paper is organized as follows In section 2, we review the block ILU preconditioner for the block tridiagonal matrices presented by Koulaei and Toutounian in [9] Two new algorithms for computing the approximate inverse factors of tridiagonal and pentadiagonal matrices are presented in section 3 Some numerical experiments are given in section 4 Section 5 is devoted to concluding remarks 2 Koulaei and Toutounian s algorithms for block ILU preconditioning of block tridiagonal matrices Consider the block tridiagonal matrix A blocked in the form (2) Let m i be order of the ith square diagonal block D i and m i=1 m i = n Let also D be the block diagonal matrix consisting of the diagonal blocks D i, L (U) the block strictly lower (upper) triangular matrix consisting of the sub-diagonal (super-diagonal) blocks B i (C i ) Then the matrix A is of the form A = D + L + U Let Σ be the block diagonal matrix with m i m i blocks Σ i satisfying Σ 1 = D 1, Σ i = D i B i Σ 1 i 1 C i, i = 2, 3,, m Then the matrix A may be factorized in the block form [1, 2, 6, 7, 8, 9] A = (Σ + L)Σ 1 (Σ + U) (4) If either A is a symmetric positive definite matrix or a block H-matrix then the factorization of the form (4) exists [1, 9] Since the inverse of a sparse matrix is in general full, the matrices Σ i are in general full, even if B i, D i and C i are sparse Hence, to compute a sparse approximate factorization of the form (4) for the matrix A, it is enough to use sparse approximations of the inverses of Σ i To do this, let Λ i be sparse matrices defined by Now let 1 = D 1, i = D i B i Λ i 1 C i, Λ i 1 1 i 1, i = 2,, m i = L i U i, i = 1,, m, be the LU factorization of blocks i In this case we have A M = ( + L) 1 ( + U) 2

3 In other words M = L 1 V 2 L 2 V m 1 L m 1 V m L m U 1 W 2 U 2 W 3 U m 1 W m U m, (5) where V i = B i U 1 i 1, W i = L 1 i 1 C i, i = 2,, m In this paper we consider an important case that diagonal blocks D i are tridiagonal and B i and C i are diagonal In [9], two recurrence formulas for computing the sparse approximate inverse factors of tridiagonal and pentadiagonal matrices are obtained We recall that the lower and upper unit triangular matrices W T and Z, respectively, is said to be the inverse factors of matrix A if W T AZ = D, where D is a diagonal matrix Then, the blocks i are kept tridiagonal or pentadiagonal and by using these algorithms the inverse factors of blocks i are computed The recurrence formula presented in [9] for the tridiagonal matrices is as follows Let a 1 c 2 b 2 a 2 c 3 T = tridiag(b i, a i, c i+1 ) =, (6) b n 1 a n 1 c n b n a n and W T = Z = 1 y 11 1 y 21 y 22 1, (7) y n 1,1 y n 1,2 y n 1,n z 11 z 12 z 1,n 1 1 z 22 z 2,n 1 1, (8) zn 1,n 1 1 be its inverse factors, ie W T T Z = D = diag(λ 1,, λ n ) Then the algorithm may run as following (KTT: Koulaei and Toutounian s algorithm for Tridiagonal matrices) Algorithm 1 KTT algorithm 1 λ 1 := a 1 2 For k = 1,, n 1, Do: 3 y kk := b k+1 and z kk := c k+1 3

4 4 If k > 1 then 5 For i = k 1, k 2,, 1, Do: 6 y ki := bi+1 λ i y k,i+1 z ik := ci+1 λ i z i+1,k 7 EndDo 8 EndIf 9 +1 := a k+1 b k+1c k+1 10 Enddo The recurrence formula presented in [9] for the pentadiagonal matrices is as follows Let a 1 c 2 f 3 b 2 a 2 c 3 P = pentadiag(e i, b i, a i, c i+1, f i+2 ) = e 3 fn, (9) bn 1 a n 1 c n e n b n a n and W T and Z be its inverse factors defined as (7) and (8), respectively Then the algorithm proposed in [9], for computing inverse factors of P is as Algorithm 2 (KTP: Koulaei and Toutounian s algorithm for Pentadiagonal matrices) Algorithm 2 KTP algorithm 1 λ 1 := a 1, γ 1 := b 2 and ξ 1 := c 2 2 For k = 1,, n 1, Do: 3 y kk := b k+1 and z kk := c k+1 4 If k > 1, then y k,k 1 := 1 1 (e k+1 γ k 1 y kk ) 5 and z k 1,k := 1 1 (f k+1 ξ k 1 z kk ) 6 If k > 2, then 7 For i = k 2,, 1, Do: 8 y ki := 1 λ i (γ i y k,i+1 + e i+2 y k,i+2 ), 9 z ik := 1 λ i (ξ i z i+1,k + f i+2 y i+2,k ) 10 EndDo 11 EndIf 12 If k = 1, then λ 2 := a 2 b2c2 λ 1, 13 Else +1 := a k+1 e k+1f k+1 1 ξ 14 γ k+1 := b k+2 e k k+2 15 Enddo γ kξ k and ξ k+1 := e k+2 f k+2 γ k In this paper we propose another recurrence formulas but more effective than KTT and KTP algorithms for the tridiagonal and pentadiagonal matrices 3 New recurrence formulas for computing inverse factors of tridiagonal and pentadiagonal matrices 4

5 31 Tridiagonal matrices Let T be the tridiagonal matrix defined by Eq (6) and T = LDU be its LDU factorization For the existence of the LDU factorization we refer the reader to reference [12] It can be easily seen that L = (l i,j ) and U T are two lower unit bidiagonal matrices On the other hand, we have L 1 T U 1 = D and as a result W T = L 1 and Z = U 1 are the inverse factors of T Here, we obtain the matrix Z and the matrix W T can be computed in a similar way Let Z = (z 1, z 2,, z n ) Hence, ZU = I n n Or Therefore We have z 1 = e 1, (z 1, z 2,, z n ) 1 u 12 1 u 23 1 u n 1,n 1 = (e 1, e 2,, e n ) e k = u k,k+1 z k + z k+1 z k+1 = e k+1 u k,k+1 z k, k = 1,, n 1 (10) W T T Z = D T Z = W T D Z T T Z = Z T W T D It can be easily seen that Z T W T D and hence Z T T Z are lower triangular matrices such that diag(z T T Z) = D Therefore { zi T di, i = j; T z j = (11) 0, j > i Hence from (10) we have 0 = z T k T z k+1 = z T k T e k+1 u k,k+1 z T k T z k u k,k+1 = zt k T e k+1 zk T T z k By letting v i = z T i T, we have u k+1,k = v ke k+1 v k z k, (12) and z k+1 = e k+1 u k,k+1 z k z T k+1t = e T k+1t u k,k+1 z T k T Lemma 31 Let v k = (v (1) k, v(2) k,, v(n) k ) Then v(k+1) Proof It can be proven by induction on k For k = 1, we have v k+1 = e T k+1t u k,k+1 v k, (13) v 1 = z T 1 T = e T 1 T = (a 1 c 2 0 0) Hence the lemma is true if k = 1 Let the Lemma be true for k Then v k+1 = e T k+1t u k,k+1 v k k = c k+1 and v (j) k = 0, j = k + 2,, n = (0,, 0, b k+1, a k+1, c k+2, 0, 0) u k,k+1 (v (1) k,, v(k) k, c k+1, 0,, 0) = ( u k,k+1 v (1) k,, u k,k+1v (k) k, a k+1 u k,k+1 c k+1, c k+2, 0,, 0) 5

6 This shows that the lemma is true for k + 1 Now from Eq (11) and Lemma 1 we have v k e k+1 = c k+1 and v k z k = d k Therefore, from (9) we conclude that u k+1,k = c k+1 d k Now we are going to find a recurrence formula for computing d k, k = 1,, n First of all we see that d 1 = v 1 z 1 = e T 1 T e 1 = a 1 From Eq (11) we have v k z k+1 = 0 Hence from Eq (10) we deduce d k+1 = v k+1 z k+1 = (e T k+1t u k,k+1 v k )z k+1 = e T k+1t z k+1 = e T k+1t (e k+1 u k,k+1 z k ) = a k+1 u k,k+1 e T k+1t z k It can be easily verified that e T k+1 T z k = b k+1 Therefore d k+1 = a k+1 u k,k+1 b k+1 Now the above discussion can be summarized as algorithm 2 (NT: New algorithm for Tridiagonal matrices) Algorithm 3 NT algorithm 1 z 11 := 1, w 11 := 1, and d 1 := a 1 2 For k = 1,, n 1, Do: 3 u k,k+1 := c k+1 d k and l k+1,k := b k+1 d k 4 z k+1,k+1 := 1 and w k+1,k+1 := 1 5 For i = 1, 2,, k 6 z i,k+1 := u k,k+1 z i,k 7 w i,k+1 := l k+1,k w i,k 8 EndDo 9 d k+1 := a k+1 u k,k+1 b k+1 10 Enddo It is necessary to mention that in this algorithm z k = (z 1,k,, z n,k ) T and w k = (w 1,k,, w n,k ) T are the column vectors of Z and W, respectively The NT algorithm gives not only the inverse factors of T but also its LDU factorization Hence, if the LDU factorization of T is available then the inverse factors of T can be easily computed In the next subsection, we use this fact for computing the inverse factors of a pentadiagonal matrices Both of the NT and KTT algorithms give the same inverse factors of the matrix T But the cost of computing the inverse factors by the NT algorithm is less than that of the KTT algorithm Since in computing the kth (k 2) column of Z (W ) by the NT algorithm, k + 2 flops are required, whereas this number for KTT algorithm is 2k + 1 The numerical experiments in the next section confirm this fact Another advantage of the NT algorithm over KTT algorithm is its parallelism In fact, the entries of each column or row of the inverse factors can be computed by the NT algorithm from the previous column or row simultaneously But this is not correct for KTT algorithm Since, in the KTT algorithm, each entry of either column or row of the inverse factors should be computed from 6

7 previous entry of the same column or row In fact, without completing the computation of a column or row, the computation of the next column or row may not be started 32 Pentadiagonal matrices Let us consider the pentadiagonal matrix (9) and P = LDU be its LDU factorization It can be verified that L and U T are lower unit triangular matrices of bandwidth 3 Let 1 v 2 1 ṽ 3 v 3 L = ṽ 4 vn 1 1 ṽ n v n 1, U = 1 w 2 w 3 1 w 3 w 4 wn 1 w n 1 w n 1 and D = diag(λ 1, λ 2,, λ n ) It can be easily verified that the entries of L, U and D can be computed via Algorithm 4 Algorithm 4 LDU factorization of pentadiagonal matrices 1 v 2 := b2 λ 1, w 2 := c2 λ 1 2 λ 2 := a 2 b 2 w 2 (:= a 2 c 2 v 2 ) 3 For k = 3,, n 4 v k := 1 1 (b k e k w k 1 ) and ṽ k := 5 w k := 1 1 (c k f k v k 1 ) and w k := 6 := a k 1 v k w k 1 2 e k f k 7 Enddo e k 2 f k 2 Similar to the method used in the previous subsection, let W T and Z be the inverse factors of P, ie, W T P Z = D = diag(λ 1,, λ n ) In this case we have Z = U 1 Hence ZU = I, where I is the identity matrix of order n By equating the kth (k = 1,, n) column of two sides of ZU = I we obtain z 1 = e 1, z 2 = e 2 w 2 z 1, z k = e k w k z k 1 w k z k 2, k = 3,, n, where e j is the jth column I In the same manner, if W = (y 1,, y n ), then we deduce that y 1 = e 1, y 2 = e 2 v 2 y 1, y k = e k v k y k 1 ṽ k y k 2, k = 3,, n Therefore by the above discussion we can state a new algorithm for computing the inverse factors of pentadiagonal matrices as follows (NP: New algorithm for Pentadiagonal matrices), 7

8 Algorithm 5 NP algorithm 1 Let W = (y i,j ) and Z = (z i,j ) Set W = Z = I 2 λ 1 := a 1, v 2 := b2 λ 1, w 2 := c2 λ 1 3 λ 2 := a 2 c 2 v 2 4 z 12 = w 2 and y 12 = v 2 5 For k = 3,, n, Do: 6 v k := 1 1 (b k e k w k 1 ) and ṽ k := e k 2 f k 2 7 w k := 1 1 (c k f k v k 1 ) and w k := 8 z k 1,k = w k and y k 1,k = v k 9 z k 2,k = w k z k 2,k 1 w k and y k 2,k = v k y k 2,k 1 ṽ k 10 For i = k 3, k 2,, 1 Do: 11 z i,k = w k z i,k 1 w k z i,k 2 12 y i,k = v k y i,k 1 ṽ k y i,k 2 13 EndDo 14 := a k 1 v k w k 1 2 e k f k 15 Enddo In the NP algorithm, for computing k th (k 3) column of Z (W ) we need 3k + 3 flops, whereas this number for the KTP algorithm is 4k+5 Another advantage of NP algorithm over KTP algorithm, as we mentioned for NT algorithm, is its potential parallelism 4 Numerical experiments All the numerical experiments presented in this section were computed in double precision with some MATLAB codes on a personal computer Pentium 3, CPU 797 MHz In all the experiments, vector b = A(1, 1,, 1) T was taken to be the right-hand side of the linear system and a null vector as an initial guess The stopping criterion used was b Ax i 2 b 2 < 10 7 The used iterative methods were CG (in the SPD case), CGS, Bi-CGSTAB and GMRES(10) algorithms For the first set of the numerical experiments, let A 1 = tridiag( 271, 2, 1), and A 2 = pentadiag( 1, 1, 25, 1, 1) We used the Bi-CGSTAB algorithm in conjunction with the preconditioners obtained by KTT and NT algorithms for A 1 x = b and KTP and NP algorithms for A 2 x = b for different values of n In both cases the preconditioner was taken to be M = ZD 1 W T where Z and W T are approximations of the inverse factors Z and W, respectively, with six diagonals and applied as a left preconditioner The numerical results were given in tables 1 and 2 In this table (and the next tables) P-time, Ittime, T-time and P-Its stand for the CPU time for constructing the preconditioner, the required time for the convergence, T-time=P-time+It-time and the number of the iterations for the convergence, respectively It is mentioned that the timings are in seconds Meanwhile, if the number of iterations 8

9 Table 1: Numerical results of the left-preconditioned Bi-CGSTAB algorithm in conjunction with KTT and NT algorithms for the matrix A 1 = tridiag( 271, 2, 1) NT Alg KTT Alg n P-time It-time T-time P-Its P-time It-time T-time P-Its Table 2: Numerical results of the left-preconditioned Bi-CGSTAB algorithm in conjunction with KTP and NP algorithms for the matrix A 2 = pentatridiag( 1, 1, 25, 1, 1) NP Alg KTP Alg n P-time It-time T-time P-Its P-time It-time T-time P-Its of an iterative method for a problem with two different preconditioners are the same then we recorded a CPU time for both of them Some observations can be posed here Table 1, shows that the CPU time for constructing the preconditioner by using NT algorithm is always less than that of the KTT algorithm There is not any significant difference between the number of iterations of two methods The small differences between the number of iterations and consequently the total CPU time are due to different error propagations which is different for two algorithms Numerical results presented in Table 2 show that the NP algorithm gives better results than the KTP algorithm For the second set of our numerical experiments we consider the equation u = f, in Ω = (0, 1) (0, 1) Discretization of this equation on an (m + 1) (m + 1) grid, by using the second order centered differences for the Laplacian, gives a linear system of equations of order n = m 2 with n unknowns where 9

10 Table 3: Numerical results for the second set of experiments NT and NP algorithms KTT and KTP algorithms n p P-time It-time T-time P-Its P-time It-time T-time P-Its its coefficient matrix is SPD [12] and of the form (2) with B i = C i = I and D i = tridiag( 1, 4, 1) For solving the linear system of equations with the SPD coefficient matrix by the iterative methods, the CG algorithm is the method of choice Hence the left-preconditioned CG algorithm was used with the preconditioner computed by (5) and the numerical results were given in Table 3 Since the coefficient matrix is symmetric, only one of the inverse factors was computed The preconditioners were constructed by keeping blocks i tridiagonal or pentadiagonal For keeping i tridiagonal, we used KTT and NT algorithms In the same way, for keeping i pentadiagonal, we used KTP and NP algorithms It is necessary to mention that for keeping blocks i tridiagonal it is enough to keep Λ i 1 tridiagonal To do this we first compute tridiagonal approximate inverse factor Z i 1 of i 1 by using KTT or NT algorithms, ie, i 1 Z i 1 T D Z i 1 i 1 Then we set Λ i 1 = Z i 1 D 1 Z i 1 i 1 T Obviously, block Λ i 1 is tridiagonal In the same manner by using KTP and NP algorithms one can keep blocks i pentadiagonal Numerical results presented in Table 3 show that the NT and NP algorithms in general are more effective than the KTT and KTP algorithms, respectively Note that p = 1 stands for NT and KTT and p = 2 for NP and KTP algorithms Our third set of test matrices used arise from the centered difference discretization of u + 2δ 1 u x + 2δ 2 u y δ 3 u = f, in Ω = [0, 1] [0, 1], where δ 1, δ 2 and δ 3 are constants, on a uniform (m + 1) (m + 1) grid This kind of discretization gives a linear system of equations of n = m 2 unknowns u ij = u(ih, jh), (1 + δ 2 h)u i,j 1 (1 + δ 1 h)u i 1,j + (4 δ 3 h 2 )u ij (1 δ 1 h)u i+1,j (1 δ 2 h)u i,j+1 = h 2 f ij, where f ij = f(ih, jh) and h = 1/(m + 1) In our test we let δ 1 = 2, δ 2 = 4 and δ 3 = 0 In this case it can be easily verified that the coefficient matrix is diagonally dominant We have used the left-preconditioned GMRES(10), Bi-CBSTAB and CGS for solving the preconditioned linear system for m = 100, 200 and 300 It is mentioned that for constructing the preconditioner of the form (5), blocks i were kept tridiagonal or pentadiagonal as we did in the second set of numerical experiments Numerical results were given in Table 4 In column 3 and 4 of this table the numerical results of the iterative methods without preconditioning were shown As the numerical experiments in Table 4 show, the CPU time for constructing the preconditioner by NT and NP algorithms are always less than that of the KTT and KTP algorithms, respectively We also see that the number of iterations of the new methods and the previous ones are comparable 10

11 Table 4: Numerical results for the third set of experiments Unp NT and NP algorithms KTT and KTP algorithms n Method Its Time p P-time It-time T-time P-Its P-time It-time T-time P-Its GMRES(10) Bi-CGSTAB CGS GMRES(10) Bi-CGSTAB CGS GMRES(10) Bi-CGSTAB CGS Conclusion In this paper, we have presented two recurrence formulas for computing the inverse factors of general tridiagonal and pentadiagonal matrices Then by using these algorithms we constructed an ILU preconditioner for the block tridiagonal matrices Numerical experiments on some test matrices were given to compare our approaches with Koulaei and Toutounian s algorithms Numerical results show that both of the methods are robust But the CPU time of the phase of constructing the preconditioner of the new methods are always less than the method of [9] Meanwhile, the new methods are suitable for parallel computers Acknowledgements The author would like to thank reviewers for their valuable comments that improved the presentation of the paper References [1] O Axelsson, Iterative solution methods, Cambridge University Press, Cambridge, 1996 [2] O Axelsson and B Polman, On approximate factorization methods for block matrices suitable for vector and parallel processors, Numerical Linear Algebra with Applications, 77(1986), 3-26 [3] M Benzi, Preconditioning techniques for large linear systems: A survey, J of Computational Physics, 182 (2002) [4] M Benzi, M Tuma A comparative study of sparse approximate inverse preconditioners, Applied Numerical Mathematics, 30 (1999) [5] M Benzi, M Tuma, A sparse approximate inverse preconditioner for nonsymmetric linear systems, SIAM J Sci Comput, 19 (1998)

12 [6] T F Chan and P S Vassilevski, A framework for block ILU factorizations using block-size reduction, Mathematics of Computation, 64 (1995) [7] P Concus, G H Golub and G Meurant, Block preconditioning for the conjugate gradient, SIAM J Sci Stat Comput, 6 (1985) [8] D K Salkuyeh and F Toutounian, BILUS: A block version of ILUS factorization, J Appl Math & Computing, 15 (2004) [9] M H Koulaei and F Toutounian, On computing of block ILU preconditioner for block trdiagonal systems, Journal of Computational and Applied Mathematics, 202 (2007) [10] L Y Kolotilina and A Y Yeremin, Factorized sparse approximate inverse preconditioning I Theory, SIAM J Matrix Anal Apll, 14 (1993) [11] L Y Kolotilina and A Y Yeremin, Factorized sparse approximate inverse preconditioning II: Solution of 3D FE systems on massively parallel computers, Int J High Speed Comput, 7 (1995) [12] CD Meyer, Matrix analysis and applied linear algebra, SIAM, 2004 [13] Y Saad, Iterative Methods for Sparse linear Systems, PWS press, New York, 1995 [14] Y Saad and M H Schultz, GMRES: A generalized minimal residual algorithm for nonsymmetric linear systems, SIAM J Sci Statist Comput, 7(1986) [15] P Sonneveld, CGS, a fast Laczos-type solver for nonsymmetric linear systems, SIAM J Sci Statist Comput, 14 (1993) [16] H A van der Vorst, Bi-CGSTAB: a fast and smoothly converging variant of Bi-CG for the solution of nonsymmetric linear systems, SIAM J Sci Statist Comput, 12 (1992) [17] J Zhang, A sparse approximate inverse technique for parallel preconditioning of general sparse matrices, Appl Math Comput, 130 (2002)