Peter Deuhard. for Symmetric Indenite Linear Systems

Peter Deuhard A Study of Lanczos{Type Iterations for Symmetric Indenite Linear Systems Preprint SC 93{6 (March 993)

Contents 0. Introduction. Basic Recursive Structure 2. Algorithm Design Principles 7 2. Iterative Error Minimization : : : : : : : : : : : : : : : : : : : : 8 2.2 Truncated Coecient Approximation : : : : : : : : : : : : : : : 9 2.3 Iterative Error Reduction : : : : : : : : : : : : : : : : : : : : : : 3. Compact Iteration Schemes 2 3. Residual Projection : : : : : : : : : : : : : : : : : : : : : : : : : 2 3.2 Residual Orthogonality : : : : : : : : : : : : : : : : : : : : : : : 3 3.3 Blended Iteration Schemes : : : : : : : : : : : : : : : : : : : : : 7 Abstract The Lanczos iteration for symmetric indenite linear systems seems to be well{known for quite a while. However, in order to modify it with the aim of improved performance, the present paper studies certain aspects in terms of an adjoint scalar three{term recurrence. Thus, at least a dierent view is opened. Moreover, an alternative 3n{implementation in terms of the Euclidean orthogonal basis has been found that easily permits generalizations. The study is understood as a start{o for further numerical investigations and experiments.

0. Introduction This paper deals with the numerical solution of linear n{systems Ax = b ; x; b 2 R where A is a symmetric (n; n){matrix, in general indenite. For suciently large systems, an iterative method will be the method of choice. Since A is symmetric, an orthogonal basis of R n can be constructed via three{term recurrences an observation rst exploited by C. Lanczos [7]. As a consequence, compact algorithms with xed array storage q n, q = O(), can be designed. However, utmost care must be taken to achieve numerical stability. Even the algorithm originally proposed by Lanczos turned out to be unstable. The most successful candidate among the stable algorithms is the conjugate gradient method, which, for positive denite A, minimizes the energy norm of the error in each step a concept, however, which does not carry over to the general indenite case. An extension to the indenite case, aiming at minimization of the Euclidean norm of the error, has been proposed by Fridman [5]. Unfortunately, this algorithm is also numerically unstable. A numerically stable alternative using Givens rotations has been designed by Paige and Saunders [8] in a classical numerical linear algebra framework. The purpose of the present paper is to revisit the topic mainly from the side of its recursive structure without xing the choice of norm too early. Thus the iterative features of a whole class of algorithms can be viewed in a more general framework such as compact storage realization, iterative convergence and numerically stable implementation. In Section, a few simple technical results are exposed, which may nevertheless not be folklore. On this basis, construction principles for the design of algorithms are discussed such as approximate Krylov subspace error minimization, error reduction or residual orthogonality. Compact implementations and their numerical stability properties are discussed in Section 3 together with a 3n{implementation that permits blending of the Lanczos iteration with other possible iterations.. Basic Recursive Structure Given an initial guess x 0 of the unknown solution x, let x := x? x 0 denote the correction to be computed. Given additionally the initial residual r 0 := b? Ax 0 ;

then the symmetric indenite linear system to be solved arises as Ax = r 0 ; A T = A : (.) We assume that A is nonsingular (and will test for this hypothesis in the course of computation by means of the algorithms to be studied). For a given inner product h; i, which will be xed later, let fv j g dene an orthogonal basis such that hv i ; v j i = j ij : (.2) We dispose about normalization such that v j = p j (A)r 0 = A j? r 0 + : : : in terms of a sequence of polynomials fp j g with leading coecient. Then the sequence fv j g can be computed recursively via Schmidt orthogonalization v = r 0 ; v 2 = Av? v v k+ = Av k? k v k? 2 k v k? k = 2; : : : ; n? (.3) where k = hv k ; Av k i= k ; 2 k = k = k? : The fv ; : : : ; v k g span a Krylov subspace Let K k (A; r 0 ) := fr 0 ; Ar 0 ; : : : ; A k? r 0 g : v j := v j = p j j = : : : : ; n denote the associated orthonormal basis. In order to introduce matrix notation, let V k := [v ; : : : ; v k ] denote orthogonal (n; k){matrices such that V T k V k = I k k = ; : : : ; n and V n V T n = I n : In this notation, (.3) can be written as AV k = V k T k + k+ v k+ e T k (.4) in terms of the symmetric tridiagonal (k; k){matrices T k = V T k AV k ; T k = 2 6 4 2. 2........... k k k 3 7 5 (.5) 2

With the decomposition (.4) for k = n, the original system (.) to be solved now arises as T n n = kr 0 ke ; n := V T nx : (.6) In componentwise notation we thus obtain the solution in the form j := hv j ; xi j = ; : : : ; n (.7) x = nx j v j : (.8) As for the formal solution of (.6), we apply rational Cholesky decomposition with B k = 2 6 4 T k = B k D k B T k k = ; : : : ; n (.9) b 2......... Comparing entries of B k, D k b k 3 7 5 ; D k = diag(d ; : : : ; d k ) : and T k then yields the forward recursions d = ; d j+ = j+? 2 j+ =d j j = ; : : : ; n? (.0) and b j+ = j+ =d j j = ; : : : ; n? : By Sylvester's theorem of inertia [6], the sign distribution of the sequence fd j g equals the one of the eigenvalues of A. In particular, since A is assumed to be nonsingular, none of the d ; : : : ; d n will vanish. On the other hand, if any of the d j vanishes in the course of computation, then A must be singular. For this reason, utmost care must be taken, whenever these d j are used in actual computation, to treat the \nearly singular" case. Upon rechanging normalization from v j to v j, the coecients j will change to j = hv j; xi hv j ; v j i = hv j; xi= j j = : : : : ; n in terms of the expansion x = nx j v j : (.) 3

Accordingly, the tridiagonal system arises as T n n = e (.2) with T n := 2 6 4 2 2............ 2 n n In order to construct an iterative method, dene 3 7 5 : x k := x k? x 0 ; r k := b? Ax k ; k = 0; : : : ; n ; (.3) which implies the iterative updates Note that x k = x k? + k v k ; r k = r k?? k Av k k = ; : : : ; n : (.4) x n = x ; r n = 0 : (.5) Obviously, once the coecients f j g can be computed recursively, the updates (.4) can be performed easily. Lemma Let r 0 6= 0. In the notation just introduced, the coecients f j g satisfy the (adjoint) three{term recurrence j? + j j + 2 j+ j+ = 0 (.6) with 0 =?. At most one component q of f j g can vanish. Moreover, the iterative residuals can be represented as r k = k+ 2 k+ v k? k v k+ k = 0; : : : ; n? : (.7) Proof. In order to show (.6) above, just recall (.2) T n n = e = (; 0; : : : ; 0) T : Herein, rows 2 up to (n? ) anyway represent (.6). Row is included by dening 0 =?. Row n is included, since n+ = hv n+ ; v n+ i = 0, which implies 2 = n+ n+= n = 0, so that the last row degenerates. With the homogeneous three{term recurrence (.6) valid, no more than one single component q is allowed to vanish otherwise all components must vanish, which is a contradiction to the condition r 0 6= 0. 4

Finally, the representation for the residuals is easily obtained: r k = r 0? Ax k = v? P kp j Av j =? k? ( j? + j j + j+ 2 j+ )v j? ( k+ + k k )v k? k v k+ = k+ 2 k+ v k? k v k+ : With 0 =? and the recurrence (.6) given, a forward recursion for x k, k = ; 2; : : : can be principally constructed, once the coecient = hr 0; xi hr 0 ; r 0 i hv ; xi (.8) is computationally available. For the Euclidean inner product, (.8) is just the denition, which cannot be evaluated, unless the solution x is already known. If, however, A is not only symmetric but also positive denite, then h; i can be specied as the well{known energy product As a consequence, one then obtains hu; vi A := hu; Avi : (.9) = hr 0; xi A hr 0 ; r 0 i A = hv ; v i hv ; Av i = hv ; v 2 + v i = : (.20) This property leads to the conjugate gradient method. Assume, however, that the inner product is specied such that cannot be directly evaluated. Then the dependence of the iterative solutions x k on is of interest. Lemma 2 Let f j g, f j g denote two solutions of the adjoint three{term recurrence (.6) with starting values Then the Krylov subspace solution x k or, equivalently 0 =? ; = 0 and 0 = 0 ; = : (.2) x k = can be represented as j v j ; j = j + j (.22) x k = y k + z k 5

in term of y k = j v j ; z k = j v j : The sequences f j g, f j g or, equivalently, the corrections y k, z k, k = ; : : : ; n are linearly independent. Moreover, one has In particular, the {sequence can be obtained from withfd k g as dened in (.0). j 6= 0 ; j 6= 0 j = 2; : : : ; n (.23) k+ =? k k+ d k k ; k = ; : : : ; n? ; (.24) Proof. The adjoint three{term recurrence for the coecients j has two linearly independent solutions. With 0 =? given and to be used as a parameter, the selection of the solutions f j g, f j g by 0 =?, = 0 and 0 = 0, = is natural. Both sequences already have one vanishing element, which implies that a further element cannot vanish, unless the whole sequence vanishes which would be a contradiction. As for the linear independence, let D(k; k + ) := k k+? k k+ (.25) denote the associated Casorati determinant compare e.g. [2] or [3]. Then insertion of (.6) yields 2 k+ D(k; k + ) =? k( k? + k k ) + k ( k? + k k ) = D(k? ; k) : Hence, with 2 k+ = k+ = k, the invariance relation k+ D(k; k + ) = : : : = D(0; ) =? (.26) is directly obtained. As a consequence D(k; k + ) 6= 0 k = 0; : : : ; n? : which implies linear independence of the sequences f j g, f j g and the corrections y k, z k for all k = ; : : : ; n. Finally, with (.6) for f j g, i.e. j? + j j + 2 j+ j+ = 0 j = ; : : : ; n? and j 6= 0 for j = ; : : : ; n, we obtain j? j + j + j+ j+ j j = 0 ; j = ; : : : ; n? : 6

Upon comparing this continued fraction representation with the rational recursion (.0), one immediately observes equality of these recursions for the choice d j =? j+ j+ j j and, in addition, consistence of the starting values, since = d =? 2 2 2 = 0 + : 2. Algorithm Design Principles The problem stated in the preceding section is the ecient recursive computation of the Krylov subspace solutions for increasing index k with the tridiagonal matrices wherein T k k = e ; x k = V k k (2.) T k = 2 6 4 2 2............ 2 k k 3 7 5 j := hv j ; v j i ; j := hv j ; Av j i= j : For A symmetric and positive denite, which implies the same for T k, the corrections x k are just those obtained from the conjugate gradient method. Following [8], this can be seen (in our notation) starting from (.6) and (.26): x k = V k k = V k T? k kr 0ke = V k B?T k D? k B? k kr 0ke : At this point, a splitting into pure forward recursions is done by introducing a new basis fp j g by V k B?T k =: P k = [p ; : : : ; p k ] : With b j = j =d j?, j = 2; : : : ; n, this can be written as v = p = r 0 =kr 0 k ; v j = j d j? p j? + p j j = 2; : : : ; n (2.2) 7

For this basis, we obtain P T k AP k = B? k V T k AV kb?t k = D k ; (2.3) which means that the fp j g are A{orthogonal and So the energy product h; i A d j = hp j ; Ap j i > 0 ; j = ; : : : ; n : (2.4) = h; Ai is the appropriate inner product. Then the associated coecients, say e k = ( e ; : : : ; e k ), can be obtained from B k D k e k = kr 0 ke (2.5) by pure forward recursion. For A symmetric indenite, however, we still have d j 6= 0 for all j, but an energy product can no longer be dened. Rather, the Euclidean inner product or other suitable choices need to be investigated. For this reason, various principles for constructing such algorithms are now discussed in the light of the results of Section. 2. Iterative Error Minimization For a given iterate x k = x 0 + x k, let " k := hx k? x; x k? xi =2 = kx k? xk (2.6) denote the error in the norm induced by the given inner product h; i. Let this inner product be such that can be actually evaluated. Then (2.) leads to the well{known Galerkin condition hx k ; x k? xi = 0 ; (2.7) which then directly induces the minimization property and the associated reduction property kx k? xk = min ky? xk (2.8) y2kk " 2 k+ = "2 k? kx k+? x k k 2 " 2 k : (2.9) As a consequence of (2.8), the ultimate solution is obtained at the nal step, i.e. x n = x : (2.0) For A positive denite, this situation can be constructed by choosing the energy product h; i := h; Ai (2.) 8

as inner product. As a consequence, q 2 (A) with 2 (A) the spectral condition number of A governs the convergence behavior of the iterates. Unfortunately, for A indenite, (2.) would no longer dene an inner product. Therefore, even though the formal extension of the cg{method can be dened, its convergence properties are no longer satisfactory and, in addition, its traditional implementation may just fail at intermediate steps. In this situation, Fridman [5] had suggested to modify the Krylov subspaces by choosing which then implies that v = Ar 0 (2.2) = hv ; xi= = hr 0 ; Axi= = : (2.3) Upon looking back to the proof of Lemma, it can be seen that the above choice means a switch to the normal equations A 2 x = Ar 0 (2.4) and, as a consequence, to Krylov subspaces of the form K k (A 2 ; Ar 0 ). This is not what is wanted here. Hence, other possibilities need to be discussed. 2.2 Truncated Coecient Approximation The short{hand notation (2.) hides the fact that the components of the vectors k change fully with each increase of k. In componentwise notation, we have to write x k = k j v j : (2.5) At this point, Paige and Saunders [8] decided to exploit the tridiagonal matrices T k by Givens transformations from the right thus reducing the entries to be changed. Here, however, we will resort to Lemma 2 above. From this, the coecients k j can be seen to have a fairly simple structure. Lemma 3 The coecients k j in terms f j g, f j g as dened in Lemma 2 with as dened in (2.5) may be expressed by k j = j + k j j = ; : : : ; n (2.6) k =? k+ k+ k = ; : : : ; n ; n : (2.7) The associated residuals r k are mutually orthogonal with respect to h; i and can be represented as r k =? b k v k+ k = 0; : : : ; n? : (2.8) with b k := k k =? k = 0; : : : ; n? : (2.9) k+ k+ 9

Proof. Let k := ( k ; : : : ; k k )T with k j as in (2.5). Then x k can be dened via T k k = e : (2.20) In view of Lemma 2, k is split according to k j = k j + k k j j = 0; : : : ; k ; k = ; : : : ; n ; which is just (2.6). Let k := ( ; : : : ; k ) T, k := ( ; : : : : k ), then the equivalent equations to (2.20) are easily veried to be T k k = e? 2 k+ k+e k (2.2) T k k =? 2 k+ k+e k : (2.22) Now with (2.6) and the combination (2.20), (2.2), and (2.22) we have O = T k ( k + k k )? T k k = 2 k+( k+ + k k+)e k : From this, (2.7) follows immediately. For k = n, we have n =, since x n = x. Finally, the residual representation (.7) can be employed to obtain r k = r 0? Ax k = r 0? A(y k + k z k) = 2 k+( k+ + k k+)v k? ( k + k k)v k+ : In the notation above this boils down to r k =? b k v k+ ; b k = ( k + k k) : Insertion of (2.7) and use of the Casorati determinant relation (.26) then yields b k = ( k k+? k k+ ) = D(k; k + )= k+ k+ = D(0; )=( k+ k+ ) =? =( k+ k+ ) : This completes the proof of the lemma. Note that (2.6) is a representation both for the cg{iterates (including the indenite case) and the iterates obtained from SYMMLQ due to [8]. The residual orthogonality is well{known but usually proved for the spd{case only. In Section 3.2 below, we will give an ecient algorithm to compute the iterates based on the orthogonal residuals. Unfortunately, these iterates do not reduce any error norm, if A is indenite. For this reason, a further design principle is investigated next. 0

2.3 Iterative Error Reduction As was shown in Section 2.2 before, the fact that k 6= for k < n leads to the consequence that the error x k? x is not reduced systematically in the norm k k = h; i =2. It might therefore be of interest to study a correction of the type (2.5), but with k j := j + k j j = ; : : : ; n (2.23) for arbitrary choice of f k g. The question of interest will then be: can the coecients k be determined in such a way that at least (2.7) and (2.9) can be saved, if not the full minimization property (2.8)? Lemma 4 Consider general iterates x k = x 0 + x k with x k dened by (2.5) and (2.23). Let k k be the norm induced by the specied inner product h; i. Then the choice k =? hy k; z k i (2.24) hz k ; z k i guarantees that " 2 k+ = " 2 k? kx k+? x k k 2 " 2 k : The denominator in (2.24) never vanishes. Proof. Starting from (2.23), we obtain the error expansion P P x k? x = x k? x = k ( j + k j )v j? n = np ( j + j )v j + ( k? ) k P j=k+ ( j + j )v j j v j Hence, with (2.5) and the orthogonality of the fv j g hx k? x; x k i = ( k? ) j ( j + k j ) j : There are two choices that make this inner product vanish: either k =, the case already discussed in Section 2., or j j j + k 2 k j = 0 : (2.25) Upon introducing the denitions of y k, z k from Lemma 2, the result (2.24) is directly conrmed. Finally note that hz ; z i = 2 = > 0 ;

which implies hz k ; z k i > 0 8k : For actual computation, (2.25) will be used to dene f k g recursively := 0 ; := ; = 0 k = ; 2; : : : : k+ = k + k+ k+ k+ (2.26) k+ = k + 2 k+ k+ k+ =? k+ = k+ Finally, note that this choice of k implies the following orthogonal projection property! x k = y k? hy k; z k i hz k ; z k i z k = I? z T kzk y z T k (2.27) k z k written, for ease of notation, in terms of the Euclidean inner product. 3. Compact Iteration Schemes In the positive denite case, an ecient compact iteration scheme with essentially 3n array storage is the well{known conjugate gradient method. The aim of this section is to construct a comparable scheme for the indenite case. One trick in the cg{method certainly is the introduction of the residuals r k := b? Ax k = r 0? Ax k (3.) as the actual variables in order to avoid computational use of the ill{conditioned n{system of three{term recurrences (.3) for the basis elements fv j g. This trick is now analyzed for the indenite case. 3. Residual Projection The result (.9) of Lemma readily leads to two dierent ways of representing the coecients f j g. First, we have Second, we obtain hv k ; r k i = k+ 2 k+ k = k+ k+ ; k = ; 2; : : : ; n? (3.2) hv k+ ; r k i =? k k+ ; k = 0; : : : ; n? : For k = 0, the latter representation leads to 0 =?, as required. For k > 0, however, this representation is just an identity, since the actual computation 2

of r k involves x k, which, in turn, requires the knowledge of k. Therefore, only (3.2) remains useful for actual computation. Since (3.2) starts with 2, any approach based on this formula will only be executable, if is known in agreement with the structural results of Lemma. Suppose therefore that is available, then (.9) inspires the following outline of an algorithm: k = 0 : v := r 0 ; x := x 0 + v ; = hv ; v i k = ; : : : ; n? : r k := r k?? k Av k [ k+ ] := hv k ; r k i if k 6= 0 then v k+ := [ k+ ] v k k? r k = k (3.3) k+ := hv k+ ; v k+ i ; k+ := [ k+ ]= k+ The critical step in this algorithm is certainly (3.3) for k = 0. Fortunately, vanishing components of f j g can only occur once see Lemma. So a type of \look{ahead strategy" can be derived, in principle. For details of such strategies, see e.g. [4] or []. Unfortunately, however, \nearly vanishing" components of f j g cannot generally be excluded with the consequence that a look{ ahead strategy of variable length must be developed for emergency situations. Moreover, upon inserting (3.2) into (3.3), any new basis element is seen to come from a residual projection, since!! v k+ = hv k; r k i hv k ; v k i v k? r k =? I? v kv T k v T v r k : (3.4) k k (Once more, for ease of writing, the Euclidean inner product has been used.) As a consequence, either numerical instability or frequent restart under a condition such as k+ < " (3.5) will occur. For this reason, this type of algorithm is not pursued further here. 3.2 Residual Orthogonality Upon recalling the general iteration for x k = k j v j ; k j = j + k j j = ; : : : ; k ; (3.6) 3

an alternative simple idea to exploit the residuals as actual variables can be derived. Lemma 3 above shows that for b k := k =? k+ k+ (3.7) the iterates bx k := x 0 + x k give rise to orthogonal residuals of the form br k =? b k v k+ k = 0; : : : ; n? ; with b k :=? =( k+ k+ ) k = 0; : : : ; n? : For actual computation of the f b k g note that, with (.24) b k b k? = k k k+ k+ =? d k k = ; : : : ; n : (3.8) This recursion is started with b 0 =?, which implies b = =. In a similar way, the f b k g can be obtained recursively starting from Now, dene b := b and Then, with (.26), we have b =? 0 + 0 + = : (3.9) b k := b k? b k? ; k = 2; : : : ; n : (3.0) b k =? k+ k+ + k k = D(k; k + ) =? k k+ k k+ k+ = b k k : (3.) Hence, for k = ; : : : ; n? : b k+ = k k+ k+ = k+ b k k+ k+2 k+2 k which, by insertion of (.24), yields k k k+ k+ k+ k+ k+2 k+2 ; b k+ = b k k+ =( k d k d k+ ) k = ; : : : ; n? : (3.2) The orthogonality of the residuals can be exploited to derive an alternative recursive scheme in terms of the basis fv j g. The derivation requires some rather straightforward calculations, which we summarize in the following lemma. 4

Lemma 5 Let fbx k g denote the extended cg{iterates, fbr k g the associated residuals, and dbx k := bx k? bx k?, the iterative corrections. Moreover, dene the expressions b k := hbr k? ; br k? i ; b dk := d k =b k ; b k = hbr k? ; Abr k? i=b k : (3.3) Then the iterates can be computed by the compact scheme (k = ; : : : ; n? ) as follows: wherein br k := br k?? Adbx k dbx k+ := The iteration is started with b d k bd k+ dbx k + bx k+ := bx k + dbx k+ ; bd k = b k+ b dk+ br k (3.4) (?) k?j b j =b j : (3.5) br 0 := b? Ax 0 ; dbx = br 0 ; bx = x 0 + dbx : Proof. For general iterates fx k g we obtain dx k+ = x k+? x k = k+ X ( j + k+ j )v j? X k+ = ( k+ + k k+ )v k+ + ( k+? k ) j v j ( j + k j )v j = ( k+ + k k+ )v k+ + ( k+? k )z k+ : (3.6) The special iterates fbx k g are characterized by k = b k, which yields From this, we directly calculate dbx k+ = b k+ z k+ : dbx k+ = b k+ (z k + k+ v k+ ) = b k+ b k dbx k? b k+ k+ b k br k : Insertion of (3.), (3.2) and (.24) yields b k+ k+ b k = b k b k k+ k+ k d k d k+ = k+ k+ k k d k =? : d k+ d k+ 5

Once more with (3.2), we thus have dbx k+ = k+ k d k d k+ dbx k + d k+ br k : At this point, the continued fraction representation (.0) can be used, in principle, which reads Note that due to (2.8) d = ; d k+ = k+? k+ k d k k = ; : : : ; n? : b k+ = b 2 k k+ ; b k+ = k+ : Upon introducing b d k, we arrive at the linear recursion bd k = b k b k? b d k? ; (3.7) which can be solved in closed form to conrm (3.5). Finally, with (3.8) it is seen that k+ = b b 2 k+k+ = b k+d k = d b k = d b k+ : k d k d k+ b k d k+ b 2 k b k d k d k+ Remark. For the sake of clarity, the connection of the recursive scheme (3.4) with the usual cg{scheme is described. For this purpose, just combine (2.2), which denes the basis fp j g from the basis fv j g, with the residual representation (.9). By means of the simple scaling transformation one arrives at p k :=? b k? p k p k ; (3.8) p = br 0 ; p k+ = br k + b k b k+ p k ; k = ; : : : ; n? ; (3.9) which is the usual formula see e.g. [6] or [3]. In the standard cg{scheme, the fd k g are computed via (2.4), whereas here formula (3.5) is employed. In order to actually evaluate f b d k g via (3.5), dene S k := (?) j? b j =: S + k? S? k (3.20) 6

wherein S +, k S? k sum so that represent all positive or all negative terms, respectively, in the bd k = (?) k? (S + k? S? k ) ; S + k ; S? k > 0 : (3.2) The relative condition number of this summation is then k := S+ k S + k + S? k? S? k : (3.22) Restart of the iteration (3.4) will therefore be necessary whenever the requirement " k < (3.23) is violated for some suitably chosen default value ". 3.3 Blended Iteration Schemes The above considerations led to an alternative compact scheme for the cg{ iteration, especially designed for the symmetric indenite case. It can, however, also just be used to compute the necessary intermediate terms for any other iteration characterized by x k = ( j + k j )v j with k 6= b k. For the associated general iteration recall from (3.6) that dx k+ = ( k+ + k k+ )v k+ + k+ z k+ with k+ := k+? k. In the frame of Lemma 5 just observe that v k+ =? b br k ; z k+ = dbx k+ : (3.24) k b k+ Therefore, we must just replace the bx k {update by an x k {update of the form x k+ = x k + k+ b k+ dbx k+? ( k+ + k k+) b k br k : (3.25) For computational realization, note that (.24) is equivalent to and (2.9) can be used to derive k+ =? k =( b d k b k+ ) (3.26) b k =? k+ b k+ =b = k =( b d k b ) : (3.27) 7

Moreover, b k+ can be expressed by b k+ = =(b b dk+ ) : (3.28) Finally, the factor before br k should be evaluated in the form k+ + k k+ b k = ( k? b k )=( b k k+ ) ; which then assures that x k = bx k arises also numerically for k = b k and k+ = b k+. 8

References [] R. Bank, T. Chan: A Composite Step Bi{Conjugate Gradient Method for Nonsymmetric Linear Systems. Lecture given at Oberwolfach (July 992). [2] L. Brand: Dierence and Dierential Equations. New York: Willey (986). [3] P. Deuhard, A. Hohmann: Numerische Mathematik. Eine algorithmisch orientierte Einfurung. Kap. 6: Drei-Term-Rekursionen. Verlag de Gruyter Berlin, New York (99). [4] R.W. Freund, M.H. Gutknecht, N.M. Nachtigal: An implementation of the look{ahead Lanczos algorithm for non{hermitian matrices. I. Tech. Rep. 90.45, RIACS, NASA Ames Research Center (990). [5] V.M. Fridman: The Method of Minimum Iteration with Minimum Errors for a System of Linear Algebraic Equations with a Symmetrical Matrix. Zh. vych. math. 2, pp. 34{342 (962). [6] G.H. Golub, C.F. van Loan: Matrix Computations. The Johns Hopkins University Press (989). [7] C. Lanczos: Solution of systems of linear equations by minimized iterations. J. Res. NBS 49, pp. 33{53 (952). [8] C.C. Paige, M.A. Saunders: Solution of Sparse Indenite Systems of Linear Equations. SIAM J. Numer. Anal. 2, pp. 67{629 (975). 9