Variants of BiCGSafe method using shadow three-term recurrence

Transcription

1 Variants of BiCGSafe method using shadow three-term recurrence Seiji Fujino, Takashi Sekimoto Research Institute for Information Technology, Kyushu University Ryoyu Systems Co. Ltd. (Kyushu University) 1 Introduction After appearance of CGS[10] and BiCGStab[12] methods, a number of iterative methods based on Lanczos polynomial added with auxiliary polynomial were proposed independently [9] [14] [13]. Then strategy of combination of two polynomials was generalized as a form of product of two polynomials in 1997[15]. However, the optimization of product of polynomials remains as an open problem. The first solution among naive realization of product-type iterative methods was made partly by Fujino et al. in 2005[4] because of adoption of associate residual in place of residual for decision of undetermined two parameters. They found out a clue in the ordering of developing of polynomials. As a result, they succeeded fairly in reduction of instability of convergence. They referred BiCGSafe method in view of safety convergence. With the same strategy, i.e., adoption of associate residual, variants of GPBiCG method were produced such as GPBiCG AR in 2009 [6]. On the other hand, different approach exists also, e.g., an approach of two-term recurrence proposed by prof. Rutishauser in 1959 [5][8]. Recently profs. Abe and Sleijpen [1] applied it to improvement of some variants of GPBiCG method. However, some variants succeeded in, some variants may failed out in view of convergence rate and stability of convergence. In this paper, we derive variants of BiCGSafe method [4] by means of considering on ordering of Lanczos and stability polynomials which construct algorithm of product-type BiCG method [3]. This paper is organized as follows. In section 2 we describe derivation of variants of BiCG method. In section 3, algorithm of revised variant of BiCGSafe method is derived using shadow three-term recurrence. In section 4, safety convergence of variants of BiCGSafe method will be demonstrated by numerical experiments, and in section 5 finally we will draw some conclusions. 2 Derivation of variants of BiCGSafe method We will focus on the iteration solution of a linear system of equations Ax = b (2.1) in which A is a non-singular real n n matrix and b a given real n-vector. Starting from some initial guess x 0 for the solution, BiCG(Bi-Conjugate Gradient) method[3] generates a sequence x k with the property that the kth residual r k := b Ax k lies in the Krylov subspace generated by A from r 0, i.e., The sequence of residual polynomials R k are defined by r k K k+1 := span(r 0, Ar 0,, A k r 0 ). (2.2) r k := R k (A)r 0 (2.3) and polynomials R k (λ) are refered to as the so-called Lanczos polynomials. In the standard treatment of the BiCG method, second polynomials P k play a role, and are defined by p k := P k (A)r 0. (2.4) We note that the basic recurrence relations between R k (λ) and P k (λ) hold as follows: R 0 (λ) = 1, P 0 (λ) = 1, (2.5) R k+1 (λ) = R k (λ) α k λp k (λ), P k+1 (λ) = R k+1 (λ) + β k P k (λ), k = 1, 2,.... (2.6) Then we can introduce the three-term recurrence relations for Lanczos polynomials R k (λ) only by eliminating P k (λ) from (2.5) and (2.6) as follows: R 0 (λ) = 1, R 1 (λ) = (1 α 0 λ)r 0 (λ), (2.7) R k+1 (λ) = (1 + β k 1 α k 1 α k α k λ)r k (λ) β k 1 α k 1 α k R k 1 (λ), k = 1, 2,.... (2.8) 1

2 Prof. Zhang [15] discovered that an often excellent convergence property can be gained by choosing for stability polynomials H k (λ) that are built up in the three-term recurrence form as polynomial R k (λ) in (2.7) and (2.8) by adding suitable undetermined parameters ζ k and η k as follows: H 0 (λ) = 1, H 1 (λ) = (1 ζ 0 λ)h 0 (λ), (2.9) H k+1 (λ) = (1 + η k ζ k λ)h k (λ) η k H k 1 (λ), k = 1, 2,.... (2.10) Moreover we define polynomial G k (λ) as below. G k (λ) = (H k+1 (λ) H k (λ))/λ. (2.11) By reconstruction of (2.8) using the stability polynomials H k (λ) and G k (λ), we have the following coupled twoterm recursion of the form as H 0 (λ) = 1, G 0 (λ) = ζ 0, (2.12) H k (λ) = H k 1 (λ) λg k 1 (λ), (2.13) G k (λ) = ζ k H k (λ) + η k G k 1 (λ), k = 1, 2,.... (2.14) 3 Algorithm of revised variants of BiCGSafe method In this section, we consider on the following alternative recurrence [8] with polynomials G k+1 (λ) and H k+1 (λ) in place of polynomials G k (λ) and H k+1 (λ). G 0 (λ) = 0, H 0 (λ) = 1, (3.1) G k+1 (λ) = ζ k λh k (λ) + η k Gk (λ), (3.2) H k+1 (λ) = H k (λ) G k+1 (λ), k = 1, 2,.... (3.3) Here, parameters ζ k, η k are undetermined scalars. Polynomial Gk+1 (λ) satisfies relationship of G k+1 (λ) = λg k (λ). When λ is very close to zero, computation of G k (λ) = (H k+1 (λ) H k (λ))/λ causes more roundoff errors than that of G k (λ) as below. G k (λ) = λg k (λ) = λ{ H k+1(λ) H k (λ) } λ = (H k+1 (λ) H k (λ)) (3.4) Using eqns. (3.2) and (3.3), the following recurrence (3.5) holds. We refer this recurrence as shadow three-term recurrence. 3.1 Ordering of developping of two polynomials H k+1 (λ) = H k (λ) ζ k λh k (λ) η k Gk (λ). (3.5) In this subsection, we consider on developing polynomial of the residual H k+1 (λ)r k+1 (λ) by means of the alternative polynomials of BiCG method, the alternative recurrences (3.1)-(3.3) by Rutishauser and the above shadow three-term recurrence (3.5). We derive the algorithm of variants of BiCGSafe method by developing firstly Lancoz polynomial R k+1 (A) in the residual vector r k := H k+1 (A)R k+1 (A)r 0 as well as the algorithm of BiCGSafe method itself. H k+1 (λ)r k+1 (λ) = H k+1 (λ)(r k (λ) α k λp k (λ)) = (H k (λ) ζ k λh k (λ) η k Gk (λ))r k (λ) α k λ(h k (λ) G k+1 (λ))p k (λ) = H k (λ)r k (λ) α k λh k (λ)p k (λ) (ζ k λh k (λ)r k (λ) + η k Gk (λ)r k (λ) α k λ G k+1 (λ)p k (λ)) (3.6) 2

3 Moreover we can develop some recurrences in eqn. (3.6) as below. H k (λ)p k (λ) = H k (λ)(r k (λ) + β k 1 P k 1 (λ)) = H k (λ)r k (λ) + β k 1 (H k 1 (λ)p k 1 (λ) G k (λ)p k 1 (λ)), (3.7) λh k (λ)p k (λ) = λh k (λ)r k (λ) + β k 1 (λh k 1 (λ)p k 1 (λ) λ G k (λ)p k 1 (λ)), (3.8) G k+1 (λ)p k (λ) = (ζ k λh k (λ) + η k Gk (λ))p k (λ) (3.9) = ζ k λh k (λ)p k (λ) + η k ( G k (λ)r k (λ) + β k 1 Gk (λ)p k 1 (λ)), (3.10) G k+1 (λ)r k+1 (λ) = G k+1 (λ)(r k (λ) α k P k (λ)) Here we introduce three auxiliary vectors as below. = ζ k λh k (λ)r k (λ) + η k Gk (λ)r k (λ) α k λ G k+1 (λ)p k (λ). (3.11) p k := H k (A)P k (A)r 0, (3.12) u k := G k+1 (A)P k (A)r 0, (3.13) w k+1 := G k+1 (A)R k+1 (A)r 0. (3.14) We gain the following vectors by substituting these auxiliary vectors into eqns. (3.6)-(3.11). r k+1 = r k α k Ap k w k+1, (3.15) p k = r k β k 1 (p k 1 u k 1 ), (3.16) Ap k = Ar k β k 1 (Ap k 1 Au k 1 ), (3.17) u k = ζ k Ap k + η k (w k + β k u k 1 ), (3.18) w k+1 = ζ k Ar k + η k w k α k Au k. (3.19) Concerning update formula of the approximate solution vector x k+1, we utilize the next recurrence from eqn. (3.15). b Ax k+1 = r k+1 = r k α k Ap k w k+1 = b Ax k α k Ap k w k+1. (3.20) Here we introduce an additional auxiliary vector z k defined as w k+1 = Az k, i.e., z k = A 1 Gk+1 (A)R k+1 (A)r 0. Then, from the following relationship, λ 1 Gk+1 (λ)r k+1 (λ) = ζ k H k (λ)r k (λ) + η k λ 1 Gk (λ)r k (λ) α k Gk+1 (λ)p k (λ), (3.21) we can derive the recurrence of the auxiliary vector z k. Accordingly we get the next recurrence of the solution vector x k+1. z k = ζ k r k + η k z k 1 α k u k. (3.22) x k+1 = x k + α k p k + z k. (3.23) As well as BiCGSafe method, we decide parameters ζ k, η k by introduction of associate residual of a r k := H k+1 (A)R k (A)r 0. By using the following relationship, H k+1 (λ)r k (λ) = H k (λ)r k (λ) ζ k λh k (λ)r k (λ) η k Gk (λ)r k (λ) (3.24) we can derive recurrence of associate residual a r k as, and decide parameters ζ k, η k from 2-norm of a r k. a r k = r k ζ k Ar k η k w k (3.25) a r k 2 = r k ζ k Ar k η k w k 2 (3.26) 3

4 From the above derivation, we can gain the algorithm. We refer to this as BiCGSafe variant 1 (abberivated as BiCGSafe var 1). Moreover, in developping of polynomial G k+1 (λ)p k (λ) of eqn. (3.9), the following developping can be gained. G k+1 (λ)p k (λ) = G k+1 (λ)(r k (λ) β k 1 P k 1 (λ)) = G k+1 (λ)r k (λ) β k 1 G k+1 (λ)p k 1 (λ) = G k+1 (λ)r k (λ) β k 1 (ζ k λh k (λ)p k 1 (λ) + η k Gk (λ)p k 1 (λ)) (3.27) As well as the algorithm of BiCGSafe var 1 method, we can gain the algorithm of BiCGSafe variant 2 (abberivated as BiCGSafe var 2) method. We show difference between BiCGSafe var 1 method and BiCGSafe var 2 method in the update formula for residual vector r k+1 as eqn. (3.43) and eqn. (3.44). The underlined update procedures differ from that of the algorithm of BiCGSafe method. 4 Numerical experiments Algorithms of BiCGSafe var 1(var 2) methods x 0 is an initial guess, r 0 = b Ax 0, Choose r 0 such that (r 0, r 0 ) 0, β 1 = 0, for k = 0, 1, until r k ε r 0 p k = r k + β k 1 (p k 1 u k 1 ), (3.28) Compute Ar k, (3.29) Ap k = Ar k + β k 1 t k, (3.30) α k = (r 0, r k )/(r 0, Ap k ), (3.31) a k = r k, b k = y k, c k = Ar k, (3.32) ζ k = (b k, b k )(c k, a k ) (b k, a k )(c k, b k ) (c k, c k )(b k, b k ) (b k, c k )(c k, b k ), (3.33) η k = (c k, c k )(b k, a k ) (b k, c k )(c k, a k ) (c k, c k )(b k, b k ) (b k, c k )(c k, b k ), (3.34) (if k = 0, then ζ k = (c k, a k )/(c k, c k ), η k = 0) (3.35) q k = ζ k Ar k + η k y k, (3.36) u k = q k + β k 1 (ζ k t k + η k u k 1 ), (3.37) z k = ζ k r k + η k z k 1 α k u k, (3.38) Compute Au k, (3.39) y k+1 = q k α k Au k, (3.40) t k+1 = Ap k Au k, (3.41) x k+1 = x k + α k p k + z k, (3.42) r k+1 = r k α k Ap k y k+1, (3.43) (= r k α k t k q k ), (3.44) β k = α k (r 0, r k+1 ) ζ k (r 0, r k), (3.45) end for All computations were done in double precision floating point arithmetic, and performed on Dell PowerEdge R210 II(CPU: Intel Xeon E3-1220, clock: 3.1GHz, memory: 8Gbytes, OS: Scientific Linux 6.0). Compiler of Intel Fortran Compiler version 11.0 is used. All codes were compiled with the -O3 optimization option. The righthand side b was imposed from the physical load conditions. The stopping criterion for successful convergence of the iterative methods is less than of the relative residual 2-norm r k+1 2 / b Ax 0 2. In all cases the iteration was started with the initial guess solutions x 0 = 0. The maximum number of iterations is fixed as Matrices are normalized with diagonal scaling. The initial shadow residual r 0 is equal to the initial residual r 0 (= b Ax 0 ). We show specifications of test matrices in Table1. All matrices are taken from Florida sparse matrix collection[11]. 4

5 matrix n total ave. nnz nnz af , , air-cfl5 1,536,000 19,435, airfoil 2d 14, , atmosmodd 1,270,432 8,814, bcircuit 68, , big 13,209 91, epb1 14,734 95, Table 1: Specifications of test matrices. matrix n total ave. nnz nnz epb2 25, , epb3 84, , Freescale1 3,428,755 17,052, poisson3db 85,623 2,374, raefsky3 21,200 1,488, sme3da 12, , sme3db 29,067 2,081, matrix n total ave. nnz nnz sme3dc 42,930 3,148, visco- 4,326 61, plastic1 wang3 26, , wang4 26, , water tank 60,740 2,035, xenon2 157,464 3,866, We examined performance of GPBiCG, GPBiCG v1 (Abe-Sleijpen GPBiCG variant-1), GPBiCG v2 (Abe- Sleijpen GPBiCG variant-2), BiCGSafe, BiCGSafe var 1 and BiCGSafe var 2 methods with Eisenstat SSOR(m) preconditioning [2] [7]. Table 2 presents the numerical results of GPBiCG, GPBiCG v1, GPBiCG v2, BiCGSafe, BiCGSafe var 1 and BiCGSafe var 2 methods. pre-t. means computation time of making preconditioner, itr-t. means iteration time and tot-t. means total time in seconds. ratio means the ratio of computation time of GPBiCG method to that of each iterative method. max denotes non-convergence until iterations reach at the maximum iteration counts. TRR (True Relative Residual) means a value of b Ax k+1 2 / b Ax 0 2 for the approximate solution x k+1. The bold figures implies the fastest case for each matrix. We can see the following facts from Table 2. BiCGSafe var 1 and BiCGSafe var 2 methods outperform among the tested iterative methods in view of both computation time for successful convergence. Performance of GPBiCG v1 and GPBiCG v2 methods is poor. TRRs of BiCGSafe var 1 and BiCGSafe var 2 methods are good. We exhibit comparison of computation time in seconds of iterative methods with E-SSOR(m) preconditioning in Table 3. spurious means that TRR of the approximate solution is more than It can be seen that Performance of BiCGSafe, BiCGSafe var 1 and BiCGSafe var 2 methods are very efficient. 5 Conclusions We derived algorithms of two variants of BiCGSafe method, i.e., BiCGSafe var 1 and BiCGSafe var 2 methods. Moreover we examined performance and robustness of convergence of these iterative methods through numerical experiments. As a result, it turned out that BiCGSafe var 1 and BiCGSafe var 2 methods work well from the viewpoint of convergence rate and accuracy of the approximate solution. References [1] Abe, K., Sleijpen, G.L.G., Solving linear equations with a STABilized GPBiCG method, Proc. of 14th Kansetouchi workshop of JSIAM, pp.29-34, Okayama University of Science, January, [2] Eisenstat, S.C., Efficient implementation of a class of preconditioned conjugate gradient methods, SIAM J. Sci. Stat. Compute., 2-1, 1/4(1981). [3] Fletcher, R., Conjugate Gradient preconditioning for indefinite systems, Lecture Notes in Math., 506, pp , [4] Fujino, S., Fujiwara, M., Yoshida, M., A proposal of preconditioned BiCGSafe method with safe convergence, Proc. of The 17th IMACS World Congress on Scientific Computation, Applied Math. and Simul., CD-ROM, Paris, [5] Jea, K.C., Young, D.M., Generalized conjugate gradient acceleration of nonsymmetrizable iterative methods, Linear Algebra Appl., Vol.34, pp , 1980 [6] Moethuthu, Fujino, S., Stability of GPBiCG AR method based on minimization of associate residual, Journal of ASCM, pp ,

6 Table 2: Performance of some GPBiCG type and BiCGSafe type methods with E-SSOR(m) preconditioning. (a)matrix: bcircuit method ω m itr. pre-t. itr-t. tot-t. log 10 (TRR) ratio GPBiCG GPBiCG v GPBiCG v BiCGSafe BiCGSafe var BiCGSafe var (b)matrix: sme3da method ω m itr. pre-t. itr-t. tot-t. log 10 (TRR) ratio GPBiCG (-7.32) spurious GPBiCG v (-7.40) spurious GPBiCG v (-6.59) spurious BiCGSafe BiCGSafe var BiCGSafe var (c)matrix: sme3dc method ω m itr. pre-t. itr-t. tot-t. log 10 (TRR) ratio GPBiCG GPBiCG v (-7.20) spurious GPBiCG v (-7.02) spurious BiCGSafe BiCGSafe var BiCGSafe var Table 3: Comparison of computation time in seconds of iterative methods with E-SSOR(m) preconditioning. method max spurious conv. GPBiCG v GPBiCG v BiCGSafe BiCGSafe var BiCGSafe var [7] Murakami, K., Fujino, S., Onoue, Y., and Taira, K., Consideration on Eisenstat preconditioning from the viewpoint of memory access, Transaction of JSST, Vol.3, No.2, pp.36-47, [8] Rutishauser, H., Theory of gradient method, in: Refined Iterative Methods for Computation of the Solution and the Eigenvalues of Self-Adjoint Value Problems, in: Mitt. Inst. Angew. Math. ETH Zürich, Birkhäuser, pp.24-49, [9] Saad, Y., Iterative methods for sparse linear systems, 2nd ed., SIAM, Philadelphia, [10] Sonneveld, P., CGS: a fast Lanczos-type solver for nonsymmetric linear systems, SIAM J. Sci. Statis. Comput., Vol.10, pp.36-52, [11] University of Florida Sparse Matrix Collection: matrices/index.html [12] van der Vorst, H.A., Bi-CGSTAB: A fast and smoothly converging variant of Bi-CG for the solution of nonsymmetric linear systems, SIAM J. Sci. Comput., Vol.13, pp , [13] van der Vorst, H.A., Iterative Krylov Methods for Large Linear Systems, Cambdrige University Press, Cambdrige, [14] Weiss, R., Parameter-free iterative linear solvers, Akademie Verlag, Berlin, [15] Zhang, S.-L., GPBi-CG: Generalized product-type methods preconditionings based on Bi-CG for solving nonsymmetric linear systems, SIAM J. Sci. Comput., Vol.18, pp ,