arxiv: v2 [math.na] 1 Sep 2016

Transcription

1 The structure of the polynomials in preconditioned BiCG algorithms and the switching direction of preconditioned systems Shoji Itoh and Masaai Sugihara arxiv: v2 [math.na] 1 Sep 216 Abstract We present a theorem that defines the direction of a preconditioned system for the biconjugate gradient BiCG method and we extend it to preconditioned bi-lanczos-type algorithms. We show that the direction of a preconditioned system is switched by construction and by the settings of the initial shadow residual vector. We analyze and compare the polynomial structures of four preconditioned BiCG algorithms. 1 Introduction The bi-lanczos-type methods are based on the bi-conjugate gradient BiCG method [3 7] and solve the system of linear equations 1.1 Ax b where A is a large sparse coefficient matrix of size n n x is the solution vector and b is the right-hand side RHS vector. Bi-Lanczos-type methods are a ind of Krylov subspace method and they assume the existence of a dual system: 1.2 A T x b ; 1.2 will be referred to as the shadow system. In general the degree of the Krylov subspace generated by A and r is displayed as K Ar span { r Ar A 2 r A r } where r is the initial residual vector r b Ax for an initial guess to the solution x. The Krylov subspace K Ar generated by the -th iteration forms the structure of 1.3 x x +K Ar where x is the approximate solution vector or simply the solution vector. The preconditioned bi-lanczos-type algorithms have not been widely discussed in the literature. In general with a preconditioned Krylov subspace method there are some different algorithms depending on the preconditioning conversion. The structure of the approximate solution formed by the Krylov subspace and the performance of a given algorithm may differ substantially from those of other algorithms [5 6]. In particular preconditioned bi-lanczos-type algorithms construct dual systems and so their analysis is more complex. The conjugate gradient squared CGS method [11] is one of the bi-lanczos-type methods and an improved preconditioned CGS improved PCGS algorithm has been proposed [5]. In a previous study [6] we compared the structures of the vectors and Krylov subspaces of four PCGS algorithms including the improved PCGS. However it is also important to analyze the structures on the polynomials of the vectors in such bi-lanczos-type algorithms and therefore in this paper we analyze the structures on the polynomials of the preconditioned BiCG PBiCG algorithms that correspond to those analyzed in our previous study [6]. Furthermore in [6] we also discussed the construction of the initial shadow residual vector ISRV in terms of the direction of the preconditioned system; we further analyze this topic in this paper. Division of Science School of Science and Engineering Toyo Deni University. itosho@acm.org. Department of Physics and Mathematics College of Science and Engineering Aoyama Gauin University. 1

2 In this paper when we refer to a preconditioned algorithm we mean one involving a preconditioning operator M or a preconditioning matrix and by preconditioned system we mean one that has been converted by some operators based on M. These terms never indicate the algorithm for the preconditioning operation itself such as incomplete LU decomposition or the approximate inverse. For example under a preconditioned system the original linear system 1.1 becomes à x b à M L AM R x M Rx b M L b with the preconditioner M M L M R M A. In this paper the matrix and the vector in the preconditioned system are indicated by a tilde. However the conversions in 1.4 and 1.5 are not implemented directly; rather we construct the preconditioned algorithm that is equivalent to solving 1.4. This paper is organized as follows. In section 2 we analyze various PBiCG algorithms in terms on their polynomial structures and we clarify the details of the PCGS algorithms discussed in [6]. In section 3 we analyze the mechanism that switches the direction of a preconditioned system for the BiCG method and we provide the details for some instances that show that depending on the construction and setting of the ISRV the BiCG method may be transformed to another method or the direction of the preconditioned system may not be determined. In section 4 we present some numerical results that verify the equivalence of the PBiCG and PCGS methods the properties of each of the four PBiCG algorithms discussed in section 2 the switching of the direction of a preconditioned system for the BiCG method and the resulting basic properties as discussed in section 3. Our conclusions are presented in section 5. 2 Analysis of various preconditioned BiCG algorithms In this section we consider four different PBiCG algorithms these PBiCG algorithms correspond to four PCGS algorithms as shown in Figure 1; these are the same ones discussed in [6]. Algorithm 1 can be used to derive these four PBiCG algorithms. Figure 1: Relations between the four different PCGS algorithms[6]. : Splitting left vector to right members preconditioner and vector : Substituting left vector for right members. Algorithm 1. BiCG method under preconditioned system: x is an initial guess r b à x set β PBiCG r r e.g. r r For 12 until convergence Do: p r +β PBiCG p p r +βpbicg p r α PBiCG r p à p 2

3 x +1 x +α PBiCG p r +1 r α PBiCG à p r +1 r αpbicg à T p β PBiCG r +1 r +1 r r End Do Any preconditioned algorithm can be derived by substituting the matrix with the preconditioner for the matrix with the tilde and the vectors with the preconditioner for the vectors with the tilde. Obviously Algorithm 1 without the preconditioning conversion is the same as the BiCG method. If à is a symmetric positive definite SPD matrix and r r then Algorithm 1 is mathematically equivalent to the conjugate gradient CG method [4] under a preconditioned system. We present the following general definition; however the PBiCG will also require Theorem 3 which will be presented in section 3. Definition 1 For the system and solution 1.4' à x b 1.5' à M L AM R x M Rx b M L b we define the direction of a preconditioned system of linear equations as follows: The two-sided preconditioned system: Equation 1.5'; The right-preconditioned system: M L I and M R M in 1.5'; The left-preconditioned system: M L M and M R I in 1.5' where M is the preconditioner M M L M R M A and I is the identity matrix. Other vectors in the solving method are not preconditioned. The initial guess is given as x and x M R x. The recurrence relations of the BiCG under a preconditioned system are R λ 1 P λ 1 R λ R λ α PBiCG λp λ P λ R λ+β PBiCG P λ. R λ is the degree of the residual polynomial and P λ is the degree of the probing direction polynomial that is r R à r p P à r. Further for the shadow under the preconditioned system ÃT x b we have 2.6 r R ÃT r 2.7 p P ÃT r. Theorem 1 Lanczos [7] Fletcher [3] Itoh and Sugihara [5] The BiCG method under a preconditioned system satisfies the following conditions: 2.8 r i r j i j biorthogonality 2.9 p i à p j i j biconjugacy. 3

4 Figure 2: Relations between the four PBiCG algorithms that correspond to the respective PCGS algorithms shown in Figure 1. : Splitting left vector to right members preconditioner and vector : Substituting left vector for right members. Proposition 1 The direction of a preconditioned system is determined by the operations of α and β in each PBiCG algorithm. These intrinsic operations are based on biorthogonality and biconjugacy. Theorem 2 There exists a PBiCG algorithm that corresponds to the preconditioning conversion defined by any given PCGS and the values of α and β will be equivalent to those of the PCGS. Proof See [5]. In particular Reference [5] explains the relations between α PBiCG and β PBiCG of the standard PBiCG and α PCGS and β PCGS of the improved PCGS. In this paper we consider four PBiCG algorithms shown in Figure 2 and these correspond to the four PCGS algorithms shown in Figure PBiCG corresponding to conventional PCGS of the right system The PBiCG algorithm corresponding to the conventional PCGS the right-preconditioned system is derived by applying the following preconditioning conversion 1 to Algorithm 1: 2.1 Ã M L AM R x M R x b M L b r M L r p M L p r MT Lr p MT Lp. Algorithm 2 is presented below. Algorithm 2. PBiCG algorithm corresponding to the conventional PCGS: x is an initial guess r b Ax set β r r r r e.g. r r For 12 until convergence Do: p r +β p p r +β p r r α p AM p x +1 x +α M p r +1 r α AM p 1 In this case the shadow vectors of r and p are converted to MT L r and MT L p but there is no problem with displaying ML Tr and MT L p in the notation of the algorithm. However these internal structures are r M T r and p M T p. The details of this notation will be discussed in sections 2.5 and 3. The same applies to

5 r +1 r α M T A T p r β +1 r +1 r r End Do The stopping criterion is r +1 b ε. This algorithm can also be derived by the following conversion: à AM x Mx b b r r p p r r p p. This is the same as using M L I and M R M in 2.1. Note that this is the same as preconditioning to obtain à x and b but not converting the other vectors; thus it is the rightpreconditioned system. Now we convert à and r using 2.1 in order to obtain the polynomial representations of 2.4 and 2.5 as r and p respectively: r p R R à r M L RR AM r P R à r M L PR AM r. We have denoted these polynomials with a superscript R 2 to indicate that Algorithm 2 which corresponds to the conventional PCGS method is a right-preconditioned system [6]. The ISRV is set as r r in this algorithm. Furthermore we use 2.1 to convert r and p : r R R AM r p P R AM r. The shadow system is also treated in a similar manner using 2.1: Finally we have r R R ÃT r MT L RR M T A T r p P R ÃT r MT LP R M T A T r. r R R M T A T r p P R M T A T r. We note that and 2.16 can also be obtained using The structures of biorthogonality 2.8 and biconjugacy 2.9 are as follows: 2.17 r i r j MLr T im L r j r ir j R i R M T A T r Rj R AM r 2.18 p i à p j ML T p i M L AM R M L p j p i AM p j Pi R M T A T r AM Pj R AM r. In Algorithm 2 the structures of r RR M T A T r and p PR M T A T r are fixed and their coefficient matrices are fixed as M T A T because the ISRV is r and RR M T A T r cannot be transformed into M T R RAT M T r. Therefore the coefficient matrix of their linear system is AM so Mx Mx +K RAM r is structured and Algorithm 2 is confirmed to correspond to the right-preconditioned system. 2 In a similar manner we will use L to indicate left-preconditioned system and W to indicated two-sided preconditioned system see section 3. 5

6 2.2 PBiCG corresponding to the left system PCGS Left-PBiCG The left-pbicg algorithm corresponding to the left-pcgs can be derived by using the following preconditioning conversion 3 in Algorithm 1: 2.19 Ã M A x x b M b r r + p p + r r p p. Algorithm 3. PBiCG algorithm corresponding to left-pcgs: x is an initial guess r + M b Ax set β r r r r+ e.g. r r+ For 12 until convergence Do: End Do In this algorithm the stopping criterion is p + r+ +β p + p r +β p r r+ α p M Ap + x +1 x +α p + r + +1 r+ α M Ap + r +1 r α A T M T p r +1 r+ +1 β r r+ 2.2 r + +1 M b ε. The polynomials of the linear system are converted as follows: 2.21 r R L Ã r R L M Ar p P L Ã r P L M Ar + and In the shadow system we have and r + R L M Ar + p + P L M Ar +. r RL ÃT r RL A T M T r p PL ÃT r PL AT M T r r R L A T M T r p P L AT M T r. 3 The notation r + is important and will be discussed in section 2.5 but there is no problem with displaying r in the notation of the algorithm. However its internal structure is r + M r. Note that this is also true for p +. 6

7 The structures of biorthogonality and biconjugacy are as follows: 2.23 r i r j r i r+ j Ri L AT M T r RL j M Ar p i à p j p i M Ap + j Pi L AT M T r M APj L M Ar +. In Algorithm 3 the structures of r + RL M Ar + and p+ PL M Ar + are fixed and their coefficient matrices are fixed as M A because the initial residual vector is r +. Therefore x x +K LM Ar + is structured and Algorithm 3 is confirmed to be the left-preconditioned system. This ISRV is set as r r+. For reference this algorithm can also be derived by the following conversion: 2.25 à M L AM R x M R x b M L b r M R r + p M Rp + r M T R r p M T R p. If M L M and M R I are set then this is the same as Standard PBiCG This is the most general algorithm for the PBiCG and it corresponds to the PCGS algorithm labeled Improved1 in [6]. This algorithm is derived from the following preconditioning conversion applied to Algorithm 1: 2.26 à M L AM R x M R x b M L b r M L r p M R p + r M T R r p MT L p. Algorithm 4. Standard PBiCG algorithm: x is an initial guess r b Ax set β r r r M r e.g. r M r For 12 until convergence Do: p + M r +β p + p M T r +β p r M r α p Ap + x +1 x +α p + r +1 r α Ap + r +1 r α A T p r +1 M r +1 β r M r End Do In this algorithm the stopping criterion is Although sometimes the ISRV is set such that r r e.g. r r in many cases we will assume r r e.g. r M r since r r M T R r M L r r M r from 2.26; see section 3. The polynomials of the linear system are converted as r p R L à r M L RL AM r P L à r M L PL AM r 7

8 and r R L AM r MR L M AM r p + M P L AM r P L M AM r. In the shadow system we have and r p R L ÃT r M T R RL AT M T r P L ÃT r M T R PL A T M T r r R L AT M T r p M T P L A T M T r. The structures of biorthogonality and biconjugacy are as follows: 2.31 r i r j M T R r i M L r j M T r i r j r i M r j R i L AT M T r M Rj L AM r 2.32 p i à p j ML T p i M L AM R M Rp + j p i Ap+ j M T Pi L A T M T r AM Pj L AM r. Remar 1 In Algorithm 4 the biorthogonal and biconjugate structures are not immediately apparent when either M operates on the linear system or M T operates on the shadow system. However Algorithm 4 can be reduced to Algorithm 3 of the left system by using r + M r and p M T p ; therefore Algorithm 4 is coordinative to the left system. The structure of the solution vector for each Krylov subspace is x x +K LM Ar + x x +M K LAM r ; this is obtained by splitting r +. These structures are verified theoretically in section 3 and numerically in section 4. Remar 2 We explicitly provided the equations for the right endpoints of 2.29 and 2.3. These are the final structures for the setting of r M r see Example 2 in the Appendix A. 2.4 PBiCG corresponding to Improved2 The PBiCG algorithm corresponding to the Improved2 PCGS algorithm in [6] Improved2 is derived from applying the following preconditioning conversion to Algorithm 1: 2.33 à M L AM R x M R x b M L b r M L r p M L p r M T R r p M T R p. This is different from the conversion applied to p and p in 2.26 for Algorithm 4. Algorithm 5. PBiCG algorithm corresponding to Improved2: x is an initial guess r b Ax set β r r r M r e.g. r M r For 12 until convergence Do: p r +β p M p M r +β M p p r +β p 8

9 M T r r r α M M r T p AM p M T p AM p x +1 x +α M p r +1 r α AM p r +1 r α A T M T p M T r +1 r +1 r +1 β M r +1 M T r r r M r End Do In this algorithm 4 the stopping criterion is The structure of the biorthogonality is the same as that of 2.31 for Algorithm 4 because the same preconditioning conversion is used for r and r. The probing direction polynomials of the linear and shadow systems are converted as and p p P L à r M L PL AM r P L ÃT r M T R PL A T M T r p P L AM r MP L M AM r p P L AT M T r. The structure of the biconjugacy is 2.4 p i à p j M T R p i M L AM R M L p j M T p i AM p j M T Pi L AT M T r AM Pj L AM r. This structure is the same as that of 2.32 for Algorithm 4. This ISRV is set as r M r. Remar 3 As before in Algorithm 5 the biorthogonal and biconjugate structures are not immediately apparent when either M operates on the linear system or M T operates on the shadow system. However the structure of the solution vector for each Krylov subspace is again x x +M K LAM r because Algorithm 5 is equivalent to Algorithm 4 on the α and β the residual and shadow residual vectors respectively. These properties are verified theoretically in section 3 and numerically in section 4. Remar 4 We explicitly provided 2.38 for the right endpoint. This is the final structure obtained for r M r see Remar Characteristic features of the four PBiCG algorithms In this section we present the characteristics of each of the PBiCG algorithms. These include the construction of the ISRV the biorthogonal and biconjugate structures of the α and β and the structures of the solution vector for each Krylov subspace. In the following equations the underlined inner products are the typical descriptions on α and β. 4 Practically 2.35 is implemented as p + M p therefore 2.34 is needless and its preconditioning operations in the iterated part are just M T p and M r. 9

10 PBiCG corresponding to the conventional PCGS Algorithm 2: r r r r R R M T A T r RR AM r r r p à p P R M T A T r AM P R AM r p AM p Mx Mx +K R AM r. Left-PBiCG Algorithm 3: r r + r r R L AT M T r RL M Ar + r r+ p à p P L AT M T r M A P L M Ar + x x +K L M Ar +. Standard PBiCG Algorithm 4: p M Ap + r M r r r R L AT M T r M R L AM r r M r p à p M T P L AT M T r AM P L AM r p Ap+ x x +M K L AM r. PBiCG corresponding to Improved2 Algorithm 5: r M r r r M T R L AT M T r RL AM r M T r r r M r p à p M T P L AT M T r AM P L AM r M T p AM p p M AM p x x +M K L AM r. Although superficially it appears that the solution vector has the same recurrence relation in both Algorithm 2 and Algorithm 5 x +1 x + α M p they belong to different systems because in Algorithm 2 we have α R and p P RAM r p R whereas in Algorithm 5 we have α L and p PL AM r p L. We have the following proposition about the direction of the preconditioning conversion 5. Proposition 2 Congruency There is congruence to a PBiCG algorithm in the direction of the preconditioning conversion. Proof We have already shown the following instances: the PBiCG of the right system Algorithm 2 can be derived from the two-sided conversion 2.1; ifm L I andm R M the conversion of 2.1 is reduced to that of 2.12 then Algorithm 2 is derived. Still if M L M M R I then Algorithm 2 can be derived. Each of the other preconditioned algorithms Algorithm 3 4 and 5 has the same relationship to its corresponding preconditioning conversion. 5 Although this property has been repeatedly discussed in the literature it should be considered when evaluating the direction of a preconditioned system. 1

11 Proposition 3 In the biorthogonal structure r r in the iterated part of each PBiCG algorithm there exists a single preconditioning operator between r basic form of the residual vector and r basic form of the shadow residual vector such that M operates on r or M T operates on r. Here the basic form of the residual vector of a linear system includes its polynomial structure of r R AM r and the basic form of the shadow residual vector includes its polynomial structure of r R A T M T r ; these vectors and polynomials are not considered when setting the ISRV. Proof 1 From the viewpoint of the matrix and vector structure of each algorithm: We split r M T r and r+ M r in Algorithms 2 to 5 then we set r r r r M T r r r r r r+ r M r r M r r r r r M T r r r M r. The underlined inner products are the typical descriptions for the various PBiCG. In addition for the two-sided conversion we obtain r r M T R r M L r M T r r r M r. Proof 2 From the viewpoint of the polynomial of the residual vector: We split r M T r and r+ M r in Algorithms 2 to 5 then we set r r R M T A T r R AM r M T R A T M T r R AM r r r r r r r R A T M T r R M Ar + R A T M T r M R AM r R A T M T r M R AM r M T R A T M T r R AM r. The underlined inner products are the structures of the polynomial corresponding to the residual vectors in each PBiCG. In addition for the two-sided conversion we obtain r r R M T R AT M T L M T R r R M L AM R M L r M T R R A T M T r M L R AM r M T R A T M T r R AM r R A T M T r M R AM r. Corollary 1 In the biconjugate structure p à p in the iterated part of each PBiCG algorithm there exists a single preconditioning operator between A coefficient matrix and p basic form of the shadow probing direction vector such that M operates on A or M T operates on p ; furthermore there exists a single preconditioning operator between A and p basic form of the probing direction vector. 11

12 Here the basic form of the probing direction vector of a linear system includes the polynomial structure of p P AM p and the basic form of the shadow probing direction vector includes the polynomial structure of p P A T M T p. These vectors and polynomials are not considered when setting the ISRV. 3 Switching the direction of the preconditioned system for the BiCG method From the analyses presented in the previous sections and in [6] we now that the intrinsic biorthogonal and biconjugate structures of the preconditioned system are the same for each of the four PBiCG algorithms and their corresponding PCGS algorithms and this is independent of the setting of the ISRV. We now consider the other factor that can switch the direction of the preconditioning: the construction and setting of the ISRV. As stated above if the coefficient matrix A is SPD and r r then the BiCG method is mathematically equivalent to the CG method. However the BiCG method solves systems of linear equations that correspond to a nonsymmetric coefficient matrix and the ISRV r is usually regarded as arbitrary providing that r r. On the other hand we may construct an arbitrary vector r Ur such that r Ur. Here the matrix U is unprescribed. Obviously Ur can generate random vectors. However by the appropriate construction of U the BiCG can be reduced to the other method [1 2]. We show this result as the following proposition. Proposition 4 If we let U A T when r Ur in the BiCG method then we obtain the biconjugate residual BiCR method [9 1] 6. Theorem 3 The direction of a preconditioned system for the BiCG method is switched by the construction and setting of the ISRV. Proof It is sufficient to prove the following cases regarding the biorthogonality. The biconjugacy can be proven in a similar manner. Case I: If r r then r r. We mention the following special case for future reference. Case II: If r Ũ r then r Ũ r Ũ : preconditioned system of U. I. The case of r Ũ I such that Ũ r r r : With the equation r r we may construct the following three items. Each item has two verifications the first one directly applies the ISRV to the polynomials of the preconditioned system the second applies the ISRV to the polynomials of the standard PBiCG which have the same form in all items. The double-underlined equations show the construction of the ISRV that is specialized for switching the direction of a preconditioned system; when right-hand side is given we term setting the direction is fixed. 1 The left-preconditioned system ISRV1: r M r : If r M T R r r M L r r r M T R r M L r r M r then r M r. This is equivalent to r r r M r. r r R ÃT r RÃ r R A T M T r R M AM r 6 A series of product-type methods based on the BiCR have been proposed by Sogabe et al. [1]; these methods are based on an idea presented in [13]. The BiCR method was described in [9] in a discussion of the product-type methods based on it. Other product-type methods based on the BiCR have been proposed [1 2]; their derivation is different from that in [1] and these methods can be implemented more easily than that of [1]. Note that the latter method can only be implemented to multiply r by A T as the ISRV that is U A T. However References [1 2] describe setting the ISRV to A T r if U A T at r Ur ; these BiCR-type methods are then reduced to BiCG-type methods also see Remar 6. 12

13 R L AT M T M r R L M AM r. In the standard PBiCG with r M r that constructs the left system r r R A T M T r M R AM r R LAT M T M r R LM AM r. 2 The right-preconditioned system ISRV2: r MT r : If r M T R r r M L r r r M T R r M L r M T r r then M T r r r or r MT r. This is equivalent to r M T r r r r. r r R ÃT r RÃ r R M T A T M T r R AM r R M T A T r R AM r R R M T A T r R R AM r. In the standard PBiCG with r MT r that constructs the right system r r R A T M T r M R AM r M T R RAT M T M T r R RAM r R R M T A T r R R AM r. 3 The two-sided preconditioned system ISRV3: r MT R M L r : If r M T R r r M L r r r M T R r M L r then M T R r M L r or r MT R M L r. This is obviously equivalent to r M T R r r M L r. r r R ÃT r RÃ r R M T R AT M T L M T R r R M L AM R M L r R W M T R AT M T L M L r R W M L AM R M L r. In the standard PBiCG with r MT R M L r that constructs the two-sided system r r R A T M T r M R AM r R WAT M T MR TM L r M R M L RW AM r MR TRW M T R AT M T L M L r M R RW M L AM R M L r R WM T R AT M T L M L r R WM L AM R M L r. II. The case of an arbitrary r Ũ such that Ũ r : If r M T R r r M L r r Ũ r M T R r ŨM L r then M T R r ŨM L r or r MT RŨM L r. Here if Ũ I then the two-sided system is constructed because r MT R M L r ISRV3; if Ũ M T R M R then the left system is constructed because r M r ISRV1; and if Ũ M T R MT M L ML TM L then the right system is constructed because r MT r ISRV2. 13

14 In the next section Theorem 3 will be verified numerically. Here we note the following remars; further information can be found in Appendix A. Remar 5 For any items for Case I in the final structure the coefficient matrix in the residual polynomial is the same as the direction of the preconditioned system; further the initial residual vector of the linear system is the same as that of the shadow system; that is r r. Specifically in the left system ISRV1 the final structure is R L A T M T M r R LM AM r ; in the right system ISRV2 the final structure is R R M T A T r R RAM r ; and in the two-sided system ISRV3 the final structure is R W M T R AT M T L M L r R WM L AM R M L r. Note that here Proposition 3 is satisfied and Remar 1 section 2.3 and Remar 3 section 2.4 become apparent. Remar 6 From Proposition 4 and Case II in the proof of Theorem 3 if either U or Ũ is arbitrarily chosen then the appropriate method for solving and the direction of the preconditioned system may be indeterminable. Even if r is replaced by an arbitrary vector s then Case II is still proven without loss of generality because s Ur. Conversely if U or Ũ is defined adequately as in Case I for the PBiCG then the appropriate solving method and the direction of the preconditioned system can be determined. Remar 7 As mentioned in section 2.3 there are instances in which r r e.g. r r at the initial part of the standard PBiCG. However in this case à in Algorithm 1 must be SPD with the modification r r e.g. r r. The reason for this is as follows. Let A be SPD with the preconditioner M CC T M A then the two-sided preconditioning requires à C AC T in order to ensure it is still SPD and ISRV3 is constructed as r CC r r. Remar 8 The definition of the direction of a preconditioned system for the BiCG method requires Theorem 3 in addition to Definition 1. 4 Numerical experiments In section 4.1 by comparing the value of α and β for each of the four PBiCG algorithms presented in section 2 and their corresponding PCGS algorithms [6] we verify that the behavior of the right system is different from that of the other preconditioned systems i.e. the left-preconditioned algorithms and the improved preconditioned algorithms. Next the switching of the direction of the preconditioned system by the construction and setting of the ISRV Theorem 3 is verified in section Behavior of α and β in the four PBiCG methods and their corresponding PCGS methods The test problems were generated by using real nonsymmetric matrices obtained from the Matrix Maret [8]sherman4 and watt 1. The RHS vector b of 1.1 was generated by setting all elements of the exact solution vector x exact to 1.. The initial solution was x. The numerical experiments were executed on a Dell Precision T74 Intel Xeon E GHz CPU 16 GB RAM running the Cent OS ernel and Matlab 7.8. R29a. In all tests ILU was adopted as the preconditioning operation and the value zero was set to mean the fill-in level. The ISRVs were r r in the PBiCG corresponding to the conventional 14

15 PCGS Algorithm 2 and the conventional PCGS they were r r+ in the left-pbicg Algorithm 3 and the left-pcgs and they were r M r in the standard PBiCG Algorithm 4 the PBiCG corresponding to Improved2 Algorithm 5 Improved1 PCGS and Improved2 PCGS. We plotted the values of α and β for each of the four PBiCG algorithms presented in section 2 and for each of their corresponding PCGS algorithms [6]; these are shown in Figures 3 to 1. The labels in the graphs are as follows: PBiCG Right Algorithm 2 means the PBiCG corresponding to the conventional PCGS that is the right-preconditioned system. PBiCG Left Algorithm 3 means the PBiCG of the left-preconditioned system. PBiCG Std Algorithm 4 means the PBiCG of the standard preconditioned BiCG that is the PBiCG corresponding to Improved1. PBiCG Impr2 Algorithm 5 means the PBiCG corresponding to the Improved2 PCGS. PCGS Conv means the PCGS of the conventional preconditioning conversion. PCGS Left means the PCGS of the left-preconditioned system. PCGS Impr1 means the PCGS of Improved1. PCGS Impr2 means the PCGS of Improved2. Figures 3 and 7 show the behavior of α for the right-pbicg and the left-pbicg and their corresponding PCGS algorithms. Figures 4 and 8 show the behavior of α for the left-pbicg the standard PBiCG the Improved2 PBiCG the PBiCG corresponding to the Improved2 PCGS and the corresponding PCGS algorithms. From these results we now that for each of the four PBiCGs the value of α is the same as that in their respective PCGS but the values for the right-pbicg and for the conventional PCGS are different from the others. A comparison of these results on β can be seen in Figures and 1. In these graphs the behaviors of α and β are the same for each PBiCG algorithm and its corresponding PCGS algorithm; that is we numerically verified the correspondence between the PBiCG algorithms in Figure 2 in section 2 and the PCGS algorithms in Figure 1 also see [6]. We also verified that the standard PBiCG Algorithm 4 is coordinative to the left-pbicg Algorithm 3; that is α and β are equivalent although the residual vector is not r + M r where r is the standard PBiCG and r + is the left-pbicg. We also verified the difference between the rightpreconditioned system and the left-preconditioned system including the standard PBiCG because the behavior of α and β in the conventional PCGS and its corresponding PBiCG are different from the behaviors seen in the other algorithms. 4.2 Behavior of the left- right- and two-sided PBiCG and standard PBiCG when switched by the ISRV For the experiments described in this subsection the experimental environment was the same as that described in section 4.1 but the ISRVs of the PBiCG method were different. We will verify Theorem 3 by using the BiCG under the preconditioned system Algorithm 1 and the standard PBiCG Algorithm 4 with three different ISRVs. Here Algorithm 1 is based on Definition 1 and Algorithm 1 is used to construct the left-preconditioned system with M L M and M R I PrecDirl-BiCG; it is used to construct the right-preconditioned system with M L I and M R M PrecDirr-BiCG; and it is used to construct the two-sided preconditioned system PrecDirw-BiCG for the above algorithms; the ISRV was uniformly set to r r. The algorithm relative residual 2-norm was adjusted as following: M r +1 2 / b 2 for the left system r +1 2 / b 2 for the right system and M L r +1 2 / b 2 for the two-sided system. On the other hand as shown in Theorem 3 Algorithm 4 was used to construct the left-preconditioned system with r M r ISRV1-PBiCG the right-preconditioned system with r MT r ISRV2-PBiCG and the twosided preconditioned system withr MT R M L r ISRV3-PBiCG these algorithm relative residual 2-norm were all r +1 2 / b 2. Figures 11 and 12 illustrate the equivalence of the direction of a preconditioned system obtained by Algorithm 1 based on Definition 1 and the direction switching due to the ISRV when using Algorithm 4; this occurs because the left-preconditioned system PrecDirl-BiCG has the same behavior as that of the standard PBiCG with ISRV1 the right-preconditioned system PrecDirr-BiCG has the same behavior as that of the standard PBiCG with ISRV2 and the two-sided preconditioned system PrecDirw-BiCG has the same behavior as that of the standard PBiCG with ISRV3. 15

16 sherman4 PBiCG_Right PBiCG_Left PCGS_Conv PCGS_Left Alpha log Iteration Number Figure 3: Values of α for the right- and left-pbicg and those of the corresponding PCGS methods sherman4. Alpha log sherman4 PBiCG_Left PBiCG_Std PBiCG_Impr2 PCGS_Left PCGS_Impr1 PCGS_Impr Iteration Number Figure 4: Value of α for the left- and standard PBiCG and the Improved2 PBiCG and that of their corresponding PCGS methods sherman4. 2 sherman4 1 Beta log1 2 3 PBiCG_Right PBiCG_Left PCGS_Conv PCGS_Left Iteration Number Figure 5: Value of β for the right- and left-pbicg and that of the corresponding PCGS method sherman4. 2 sherman4 1 Beta log1 2 3 PBiCG_Left PBiCG_Std PBiCG_Impr2 PCGS_Left PCGS_Impr1 PCGS_Impr Iteration Number Figure 6: Value of β for the left- and standard PBiCG and the Improved2 PBiCG and that of the corresponding PCGS methods sherman4. 16

17 3 watt Alpha log1 2 3 PBiCG_Right PBiCG_Left PCGS_Conv PCGS_Left Iteration Number Figure 7: Value of α for the right- and left-pbicg and that of the corresponding PCGS method watt 1. 3 watt Alpha log1 2 3 PBiCG_Left PBiCG_Std PBiCG_Impr2 PCGS_Left PCGS_Impr1 PCGS_Impr Iteration Number Figure 8: Value of α for the left- and standard PBiCG and the Improved2 PBiCG and that of the corresponding PCGS method watt 1. 5 watt 1 5 Beta log1 5 2 PBiCG_Right PBiCG_Left PCGS_Conv PCGS_Left Iteration Number Figure 9: Value of β for the right- and left-pbicg and that of the corresponding PCGS method watt 1. 5 watt 1 5 Beta log1 5 2 PBiCG_Left PBiCG_Std PBiCG_Impr2 PCGS_Left PCGS_Impr1 PCGS_Impr Iteration Number Figure 1: Value of β for the left- and standard PBiCG and the Improved2 PBiCG and that of the corresponding PCGS method watt 1. 17

18 Algorithm relative residual 2 norm log sherman4 PrecDirl BiCG PrecDirr BiCG PrecDirw BiCG ISRV1 PBiCG ISRV2 PBiCG ISRV3 PBiCG Iteration Number Figure 11: Behavior of the algorithm relative residual 2-norm for the left- right- and two-sided PBiCG and the standard PBiCG with three different settings for the ISRV sherman4. Algorithm relative residual 2 norm log watt 1 PrecDirl BiCG PrecDirr BiCG PrecDirw BiCG ISRV1 PBiCG ISRV2 PBiCG ISRV3 PBiCG Iteration Number Figure 12: Behavior of the algorithm relative residual 2-norm for the left- right- and two-sided PBiCG and the standard PBiCG with three different settings for the ISRV watt 1. 5 Conclusions In this paper we analyzed four different preconditioned BiCG PBiCG algorithms from the viewpoint of their polynomial structure. These PBiCG algorithms correspond to the four PCGS algorithms considered in [6]. We have shown the mechanism that determines the direction of such a preconditioned system; that is the direction is determined by α and β which are constructed by biorthogonal and biconjugate operations. However the biorthogonal and biconjugate structures of the polynomials of the four PBiCG methods are all the same. Therefore we have identified that the final factor that can switch the direction of such a preconditioned system is the construction and setting of the ISRV. In particular we have shown that the direction of the preconditioned system has never been fixed without using the relation r r. Furthermore we have shown an additional theorem regarding the definition of the direction of a preconditioned system for a BiCG method for solving linear equations. In other words the construction and setting of the ISRV affect not only the shadow system but also the linear system on the direction of the preconditioned system due to the inner product of α and β. These properties of PBiCG methods are commonly discussed in the literature of preconditioned bi-lanczos-type algorithms for example preconditioned CGS PCGS and preconditioned BiCG stabilized PBiCGStab algorithms [12]. PCGS algorithms are congruent to the direction of the preconditioning conversion and this has already been analyzed [6]; PBiCGStab algorithms are not congruent and they will be analyzed as an area of future wor. 18

19 References [1] K. Abe and G. L. G. Sleijpen BiCR variants of the hybrid BiCG methods for solving linear systems with nonsymmetric matrices J. Comput. Appl. Math. 234 pp [2] K. Abe T. Sogabe S. Fujino and S.-L. Zhang A product-type Krylov subspace method based on conjugate residual method for nonsymmetric coefficient matrices in IPSJ Transactions on Advanced Computing Systems ACS pp in Japanese. [3] R. Fletcher Conjugate gradient methods for indefinite systems in Numerical Analysis: Proceedings of the Dundee Conference on Numerical Analysis 1975 G. Watson ed. Lecture Notes in Math. 56 Springer New Yor pp [4] M. R. Hestenes and E. Stiefel Methods of conjugate gradients for solving linear systems J. Res. Nat. Bur. Standards 49 pp [5] S. Itoh and M. Sugihara Formulation of a preconditioned algorithm for the conjugate gradient squared method in accordance with its logical structure Appl. Math. 6 pp [6] S. Itoh and M. Sugihara The structure of the Krylov subspace in various preconditioned CGS algorithms arxiv: [math.na] 216. [7] C. Lanczos Solution of systems of linear equations by minimized iterations J. Res. Nat. Bur. of Standards 49 pp [8] Matrix Maret [9] T. Sogabe M. Sugihara and S.-L. Zhang An extension of the conjugate residual method to nonsymmetric linear systems J. Comput. Appl. Math. 226 pp [1] T. Sogabe and S.-L. Zhang Product-type method of Bi-CR RIMS Koyurou Kyoto University Vol pp in Japanese [11] P. Sonneveld CGS A fast Lanczos-type solver for nonsymmetric linear systems SIAM J. Sci. Stat. Comput. 1 pp [12] H. A. Van der Vorst Bi-CGSTAB: A fast and smoothly converging variant of Bi-CG for the solution of nonsymmetric linear systems SIAM J. Sci. Stat. Comput. 13 pp [13] S.-L. Zhang GPBi-CG: Generalized product-type methods based on Bi-CG for solving nonsymmetric linear systems SIAM J. Sci. Comput. 18 pp A Stepwise analysis of the polynomials of the standard PBiCG Here we present detailed examples of the polynomials of the standard PBiCG Algorithm 4 when using ISRV3 r MT R M L r Example 1 and the ISRV1 r M r Example 2; we perform a stepwise analysis by using the recurrence relations 2.1 to 2.3 in section 2. We will use the following notation: à w M L AM R means the two-sided preconditioning à l M A means the left preconditioning and Ãr AM means the right preconditioning. The initial values of the polynomials in the preconditioned system are as follows: A.1 A.2 P Ãw P Ãl P Ãr I R Ãw R Ãl R Ãr I. Example 1. Details of standard PBiCG algorithm with ISRV3: x is an initial guess r b Ax set β r r M T R r M L r e.g. r MT R M L r A.3 : p + M r M R M L r 19

20 A.4 A.5 A.6 A.7 A.8 A.9 A.1 A.11 A.12 A.13 p M T r M T MR T M L r M T L M L r r M r M T α p Ap + R M L r M r M T L M L r AM r M L r M L r M L r M L AM R M L r α W x 1 x +α W p+ x +α W M r x +α W M R M L r r 1 r α W Ap+ M L I α W M L AM R M L r M L R W 1 ÃwM L r MI α W M AM r MR W 1 ÃlM r r α W AM r I α W AM r R W 1 Ãrr r 1 r αw A T p M T RM L r α W A T M T L M L r M T RI α W M T R AT M T L M L r I α W A T M T M T RM L r R W 1 A T M T M T RM L r M T RR W 1 ÃT wm L r M T RR W 1 ÃT wm L r M T I α W M T A T M T L M L r M T R1 W M T A T M T L M L r MRR T 1 W ÃT wm L r r 1 M r 1 MR T β r RW 1 ÃT w M L r M M L R W 1 ÃwML r M r R ÃT wmr TM L r M R Ãwr R1 WÃT w M L r R1 W ÃwML r β R ÃT w M L r R ÃwM L r W. The double-underlined equations show the important polynomial structures. By way of contrast neither A.3 nor A.4 is double underlined and their polynomials are not displayed; this is because they are the identity matrix as indicated in A.1 and A.2. In the above description we will focus on M L r in the final structure of each equation. Because M L r is the initial residual vector of the two-sided preconditioned system details of its properties can be found in Theorem 3 and Remar 5 in section 3. However at steps A.3 and A.4 the intrinsic structure ofp + and p does not play a role in determining the direction of the preconditioned system because neither vector has parameter α or β. The direction of preconditioned system is thus fixed as the two-sided system whenα is calculated in A.5. The approximate solution vector x 1 is calculated under the two-sided system in A.6 because A.6 has α W. The intrinsic structure of the residual vector r 1 may be that of A.7 to A.9 that is twosided left or right respectively 7. However the direction of the preconditioned system has been already fixed in A.5 the operation on α therefore the intrinsic structure of r 1 is fixed as r 1 M L I α W M L AM R M L r M L R1 W ÃwML r. Furthermore this initial residual vector part is M L r. On the other hand the intrinsic structure of the residual vector r 1 may be created by A.1 to A.12 but these all reduce to the same structure r 1 MT R I αw M T R AT M T L M L r MR TRW 1 ÃT wm L r. The reason for this is that the direction of the preconditioned system has been already fixed asα W the same as forr 1. Furthermore the part of M L r and the shadow system with the transpose matrices may not be compatible 8. When β operates in the denominator R ÃT w does not fix the direction of the preconditioned system because of A.2. 7 For the same reason p + of A.3 and x 1 of A.6 may be two-sided left or right. 8 For the same reason as for p of A.4 the part of M L r and the shadow system with the transpose matrices may not be compatible. 2

21 The subsequent iterated operations are as follows: For 123 Do : End Do p + p α W x +1 r +1 r +1 M r +β W p+ M R PW p Ap + ÃwM L r M T r +βw p M T L PW ÃT wm L r r M r R WÃT wm L r R W x +α W p+ x +α W M R PW P W ÃT wm L r M L AM R PW r α W Ap+ M LR+1 W ÃwML r ÃwM L r ÃwML r ÃwM L r r αw AT p MT R RW +1 ÃT w M L r r +1 β W M r +1 R+1 W r ÃT wm L r R W +1 ÃwML r M r R WÃT wm L r R W ÃwML r Next we will also briefly describe the polynomials of the standard PBiCG Algorithm 4 with ISRV1 r M r. The initial values of the polynomials under the left-preconditioned system are P LÃl R LÃl I. Refer to Example 1 for a detailed description. Example 2. Polynomial description of the standard PBiCG algorithm with ISRV1: x is an initial guess r b Ax set β L r r M T R r M L r e.g. r M r For 123 Do: p + M r +β L p + PL ÃlM r p M T r +βl p M T P L ÃT l M r r M r R LÃT l M r R LÃlM r α L p Ap + P L ÃTl M r M AP L ÃlM r x +1 x +α L p+ x +α L PL ÃlM r A.14 r +1 r α L Ap + MRL +1ÃlM r r +1 r αl AT p RL +1 ÃT l M r r +1 M r +1 R+1 L ÃT l M r R+1 L ÃlM r β L r M r R L ÃTl M r R L ÃlM r End Do For the polynomial structures of A.14 refer to Remar 2 in section