Methods for Large Unsymmetric Linear. Systems. Zhongxiao Jia y. Abstract. The convergence problem of Krylov subspace methods, e.g.

Transcription

1 The Convergence of Krylov Subspace Methods for Large Unsyetric Linear Systes Zhongxiao Jia y Abstract The convergence proble of Krylov subspace ethods, e.g. OM, MRES, CR and any others, for solving large unsyetric linear systes has been intensively investigated. There are any results in the literature, ainly for the case where the coecient atrix A is diagonalizable and its spectru lies in the open right (left) half plane. In this paper, we focus on a convergence analysis of those Krylov subspace ethods which give rise to a siilar ini proble in the case where the coecient atrix A is defective. When the spectru of A either lies in the open right (left) half plane or is on the real axis, we establish the related theoretical error bounds and reveal soe intrinsic relationships between the convergence speed and the spectru of A. The results show that these ethods are likely to converge slowly whenever one of three cases occurs: A is defective, the distribution of its spectru is not favorable, or the Jordan basis of A is ill-conditioned. Keywords. unsyetric linear systes, Krylov subspace, the Chebyshev polynoials, defective, derivatives. AMS(MOS) subject classications. 6510, 49K35 1 Introduction In recent years, considerable eorts have been devoted to solving large unsyetric linear syste Ax = b; (1) This work was supported by the raduiertenkolleg at the University of Bielefeld y akultat fur Matheatik, Universitat Bielefeld, eail: jia@atheatik.uni-bielefeld.de, Postfach , Bielefeld, erany 1

2 where A 2 C NN is large, unsyetric (non-heritian) and nonsingular, and b an N1 colun vector. A ajor class of ethods for solving (1) is Krylov subspace type or conjugate gradient type ethods, and so far any researchers have ade contributions to the [2, 9, 8, 6, 11, 22, 23, 24, 25, 31, 32]; see the survey papers [26, 12]. There are soe very successful schees aong the, e.g. CR or ORTHOMIN [8, 9, 31], OM or Arnoldi's ethod [22], MRES [25] and QMR [11], to nae only a few. We can observe that as far as the convergence analysis is concerned, any of the Krylov subspace ethods including those naed above and soe others, e.g. the ethod presented in [6], all give rise to a siilar ini proble. The only dierence is that soe of the ethods require that the spectru of the coecient atrix A is restricted to the open right (left) half plane, e.g. CR, while soe of the do not, e.g. OM and MRES. Until now, several authors, e.g. Manteuel [18, 19], Eisenstat et al. [8], Saad et al. [22, 25] and reund et al. [11], have derived a nuber of results, ainly when A is diagonalizable and its spectru lies in the open right (left) half plane. This paper focuses on the case where A is defective and its spectru either lies in the open right (left) half plane or is on the real axis, and studies the associated convergence proble of these ethods. We establish the related theoretical error bounds and reveal soe intrinsic relationships between the convergence speed and the spectru of A. Here I want to point out that Trefethen [28, 20] has introduced soe concepts of -pseudo-spectra to deal with non-noral atrices, which are very useful in aking a convergence analysis. Indeed, in the presence of round errors, an original defective atrix will becoe a diagonalizable one with siple eigenvalues. However, the eigenvector atrix will be very ill-conditioned in that case. Nuerical experients have shown that one could not see the dierence in convergence behaviour for atrices with siple eigenvalues, but ill-conditioned eigenvectors for soe alost ultiple eigenvalues, and atrices with those alost ultiple eigenvalues replaced by Jordan blocks of appropriate size, e.g. [30]. Therefore, fro both atheatical and nuerical points of view, it is worthwhile to consider the Jordan block case in a convergence analysis, and our analysis ay help to understand phenoena that are observed in actual oating-point coputation. In Section 2 we introduce the notation used; in Section 3, we briey review OM and MRES fro a theoretical point of view; taking OM and MRES as exaples, we devote Section 4 to a convergence analysis of these ethods. 2 Notation This part is ainly taken fro [13, 19]. Assue that A 2 C NN ; N 1, is unsyetric (non-heritian). Let Q denote the set of all polynoials of degree not exceeding and the Krylov subspace K (r (0) ; A) = spanfr (0) ; Ar (0) ; : : : ; A?1 r (0) g, and let be the orthogonal projector on K (r (0) ; A). Let k k denote the 2-nor, the superscript \H" the conjugate 2

3 transpose, (S) = kskks?1 k and R the set of real nubers. According to the above assuptions, there exists a nonsingular atrix S such that A = S?1 JS, where J = diag(j 1 ; J 2 ; : : : ; J M ) with J i = 0 i 1 i i 1 C A being the d i d i Jordan block. In this case, for any p 2 Q we have p(a) = p(s?1 JS) = S?1 p(j)s = S?1 diag(p(j 1 ); p(j 2 ); : : : ; p(j M ))S with p(j i ) = 0 p( i ) p 0 ( i ). p( i )..... p (d i?1) (i) (di?1)! p 0 ( i ) p( i ) 1 C A : 3 OM and MRES In order to give the convergence proble and the notation to be used in the convergence analysis, in this section we briey review two typical and well known Krylov subspace ethods, OM and MRES. Let x be the solution of (1), x (0) an initial guess, and r (0) = b? Ax (0) the initial residual, and let di(k (r (0) ; A)) =. Then setting x = x (0) +z, we have the following linear syste equivalent to (1): Az = r (0) : (2) The ull Orthogonalization Method (OM) seeks an approxiate solution of the = x (0) + z for x where r = b?ax solution of the for x satisfying the orthogonal projection ( z = r (0)?Az = x (0) + z 2 K (r (0) ; A); r? K (r (0) ; A); (3), while MRES consists in nding an approxiate satisfying the orthogonal projection ( z 2 K (r (0) ; A); r? AK (r (0) ; A); (4) 3

4 where r that = b? Ax = r (0)? Az, or equivalently nding z 2 K (r (0) ; A) such kr k = kr(0)? Az k = in kr (0)? Azk: (5) z2k (r (0) ;A) Let A = A and note that r (0) = r (0). Then (3) is equivalent to solving the following linear syste in K (r (0) ; A): A z = r (0) : (6) Let z = A?1 r (0) be the exact solution of Az = r (0). Then we have x = x (0) + z. In ters of the distance k(i? )z k between z and K (r (0) ; A), Saad [22, 23] has established the following results. Proposition 1 Let = k A(I? )k. Then the residual of z for (6) satises kr (0)? A z k k(i? )z k: (7) Corollary 1 Let be dened as above and = k(a j K(r (0) ;A))?1 k. Then q kz? z k k(i? )z k; (8) where A j K(r (0) ;A) is the restriction of A to K (r (0) ; A). The corollary shows that if (6) is not ill-conditioned, the error kz? z k will be of the sae order as k(i? )z k. Brown [4] has studied the intiate connections between OM and MRES. Here we only point out that in the context of general unsyetric atrices, unlike syetric positive atrices, we cannot guarantee that A j K(r (0) ;A) is invertible for any A. In what follows, we always assue that (A j K(r (0) ;A))?1 exists for all N. We can present soe algoriths based on Krylov subspace ethods described above. or ore details, refer to [22, 25]. 4 Convergence Analysis Taking OM and MRES as exaples, in this section we consider the dicult proble of analyzing how these ethods converge as increases. Saad [22, 25] has given the following Proposition 2 The distance k(i? )z k between z and K (r (0) ; A) and kr k satisfy respectively. k(i? )z k = in kp(a)z k; (9) p2q;p(0)=1 kr k = in kp(a)r (0) k; (10) p2q;p(0)=1 4

5 In ters of Proposition 2, it is easy to prove that these ethods give rise to the following siilar ini proble. Theore 1 With the notation in Section 2, we have k(i? )z k (S) kz k; (11) kr k (S) kr (0) k; (12) where = in p2q;p(0)=1 i=1;2;:::;m kp(j i )k: (13) Proof. It is enough for us to look at in p2q;p(0)=1 kp(a)z k. By Proposition 2, we have k(i? )z k in kp(a)kkz k p2q;p(0)=1 = in ks?1 p(j)skkz k p2q;p(0)=1 (S) in kp(j)kkz k p2q;p(0)=1 = (S) in kp(j i )kkz k: p2q;p(0)=1 i=1;2;:::;m Thus, the assertions hold. 2 Therefore, the convergence proble of these ethods aounts to estiating, provided that the Jordan basis of A is not too ill-conditioned. Reark. If (S) is very large, naely, the Jordan basis of A is very ill-conditioned, these ethods ay converge very slowly even though tends to zero quite rapidly. We can thus expect that all their restarted versions ay not converge at all unless the steps per restarting are taken suciently large. When A is diagonalizable, there are any upper bounds for, e.g. [14, 17, 1, 18, 19, 32, 22, 7, 3]. or large unsyetric eigenprobles, a uch ore coplicated analog of has been established in [16]. ro Theore 1, we now want to nd a polynoial p 2 Q such that it satises the condition p(0) = 1, and its values of function and the jth order derivatives, j < d i ; i = 1; 2; : : : ; M, are as sall as possible on the spectru of A. To this end, we need a lea. 5

6 Lea 1 Let T (z) be the rst kind Chebyshev polynoial of degree in the coplex plane [21]. Then 1. If z 2 [?1; 1] and 0 j, then j T (j) (j) (z) jj T (1) j= 2 ( 2? 1) ( 2? (j? 1) 2 ) (2j? 1) Here for j 1, C(; j) = (1? 1 1. )(1? (j?1) )(1? ) 2 2 = 2j C(; j): (14) 135(2j?1) is decreasing in j, and C(; 0) = 2. If z 6= 1, 0 j and for Re(z) 0 B (z; j) is dened by T (j) (z) = j (z + p z 2? 1) (z 2? 1) j B (z; j); (15) then B (z; j) is uniforly bounded for, and is at ost of order O(z j ) if z 62 [?1; 1]. If z > 1 is xed and 0 j, T (j) (z) is increasing in j. 3. Assue E(0; 1; a) to be an ellipse with the center at the origin, the foci distance 1 and the ain seiaxis a. Then 2j C(; j) < j T (j) (z) j z2e(0;1;a) = j (a + p a 2? 1) (a 2? 1) j B (a; j): (16) Proof. Part 1. See Rivlin [21, p. 33]. Part 2. According to one of the denitions of T (z), for Re(z) 0 we can write T (z) in the for T (z) = 1 2 [(z + p z 2? 1) + (z + p z 2? 1)? ] (17) = 1 2 (z + p z 2? 1) Q (z); (18) where Q (z) = 1 + (z + p z 2? 1)?2 (If Re(z) < 0, T (z) = 1 2 (z? p z 2? 1) Q (z), where Q (z) = 1 + (z? p z 2? 1)?2 ). Then we have li (z)!1 = 1; for z 62 [?1; 1]; j Q (z) j 2; for z 2 [?1; 1]: It is clear that both (z + p z 2? 1) and Q (z) are analytic in the coplex plane excluding the points 1 though T (z) is analytic in the whole coplex plane. 6

7 Now fro (18), we get T 0 (z) = 1 2 (z + p z 2? 1) p Q (z) + 1 z2? 1 2 (z + p z 2? 1) Q 0 (z) = (z + p z 2? 1) ( 1 p z2? 1Q (z) + 1 z 2? (z2? 1)Q 0 (z)) = (z + p z 2? 1) B (z; 1): (z 2? 1) Obviously, fro the above analysis, for z 62 [?1; 1] we can see that B (z; 0) = 1Q 2 (z) contains the factor (z + p z 2? 1)?2 and is of order O(1). It follows that B (z; 1) also contains this factor. Therefore, B (z; 1) is uniforly bounded for all, and B (z; 1) = O(z)O(1) + O(z 2 )O(z?1 ) = O(z). So the assertion holds for j = 1. By induction, suppose for k = j that the assertion is valid, and B (z; j) contains the factor (z + p z 2? 1)?2. Then T (j+1) (z) = j (z + p z 2? 1) ( p z 2? 1) 2j?1? 2j(z + p z 2? 1) z(z 2? 1) j?1 (z 2? 1) 2j B(z; j) + j (z + p z 2? 1) B (z 2? 1)j 0 (z; j) = j+1 (z + p z 2? 1) ( pz 2? 1B(z; j)? 2j (z 2? 1)j+1 zb (z; j) + 1 (z2? 1)B 0 (z; j)) = j+1 (z + p z 2? 1) B(z; j + 1): (z 2? 1)j+1 Since B (z; j) contains the factor (z + p z 2? 1)?2, it is easily seen that B (z; j + 1) is uniforly bounded for all, and for z 62 [?1; 1] B (z; j + 1) = O(z)(z j ) + O(z)O(z j ) + O(z 2 )O(z j?1 ) = O(z j+1 ): In ters of a result of [21, p. 51]?j X T (j) (z) = where A lj 0; 0 j, we obtain l=0 X?j T (j+1) (z)? T (j)(z) = l=0 A lj T l (z); 1 j ; (19) A lj (T 0 l (z)? T l(z)); 1 j + 1 : It is known that the left side of last relation is nonnegative since it is easy to show that T 0 l (z)? T l (z) > 0 for z > 1, so the assertion is proved. 7

8 Part 3. to be the boundary of E(0; 1; a). Then, by the iu odulus principle and (19), we can get j T (j) (z) j = j T (j) (z) j z2e(0;1;a) z2@e X?j = j z2@e l=0 A lj T l (z) j : Since A lj 0 and T l (z), 0 l?j, achieves the iu at the point a, it follows iediately that j T (j) (z) j = T (j) (a) z2e(0;1;a) = j (a + p a 2? 1) (a 2? 1) j B (a; j): On the other hand, since the points 1 are the interior points of E(0; 1; a), we have Thus, the assertion (16) holds. 2 Reark. 2j C(; j) = j T (j) (1) j< j T (j) (z) j z2e(0;1;a) = j (a + p a 2? 1) (a 2? 1) j B (a; j): By a continuity arguent and coparing Part 2 with Part 1 of Lea 1, for z belonging to a neighborhood of the point 1 or?1, and 0 j, we then have B (z; j) (z 2? 1) j = O(j ) for! 1: (20) This result corrects an error in [18], and Lea 1 also appears in [16]. 4.1 The case of atrices with real spectru Assue that the eigenvalues of A are ordered as follows: 0 <j 1 jj 2 j j M j : (21) irst we consider the case where the spectru of A is real positive. Theore 2 Assue the spectru of A to be real positive and the ordering rule as (21), and let d = fd i g; = M ; = : i=1;2;:::;m M? 1 M? 1 8

9 Then ( d 2(d?1) =T (); if 1, d 2(d?1) =T (); if > 1; where T (x) is the rst kind Chebyshev polynoial of degree. (22) Proof. Write J i (x) = 0 x 1 x x 1 C A ; i = 1; 2; : : : ; M; didi Then by Theore 1, we have I A = [ 1 ; M ]: in p2q;p(0)=1 i=1;2;:::;m;x2ia kp(j i (x))k: (23) ro approxiation theory [5], we know that the optial polynoial p o 2 Q satisfying (23) exists. urtherore it is well known [21] that for the ini proble the optial polynoial is in j p(x) j; p2q;p(0)=1 x2ia p o (x) = T ( M + 1? 2x )=T (): M? 1 Therefore, for (23) we can take an approxiate optial polynoial In view of [15, p. 547], we get p a (x) = p o (x): Now due to kp a (J i (x))k i=1;2;:::;m;x2ia d i i=1;2;:::;m d 0jd?1;x2IA 0jdi?1;x2IA j p(j) a (x) j : j! M + 1? 2x M? 1 2 [?1; 1]; j p(j) a (x) j! j 9

10 so by Part 1 of Lea 1 we have d 0jd?1 1 j! 2j 2j C(; j)=t (): Since, fro Part 1 of Lea 1, C(; j) is a decreasing function in j and C(; 0) = 1, we obtain ( d 2(d?1) =T (); if 1; d 2(d?1) 2(d?1) =T (); if > 1; which shows that (22) holds. 2 ro atheatical analysis, the above estiates converge to zero as increases since T?1 () 2( + p 2? 1)? for large. Moreover, they usually converge to zero ore rapidly whenever 1 is not well isolated fro M since + p 2? 1 is not close to 1; otherwise, they ight converge very slowly when 1 is well isolated fro M since + p 2? 1 is close to 1 at this point. It can be seen fro Theore 2 and its proof that the defectiveness of A aects the convergence speed of these ethods. In general, the higher the degree of nonlinear eleentary divisors is, the ore slowly these ethods are likely to converge. Now, we consider the case where the spectru of A is on the real axis. Theore 3 Assue the spectru of A to be real and the ordering rule as (21). Let d as before and = j M j j 1 j ; = 2 2? : M 2 1 Then ( d[ 2 ]2(d?1) =T [ ( 2 +1 ); if 1; ] 2 2?1 d([ 2 ])2(d?1) =T [ ( 2 +1 ); if > 1; (24) ] 2 2?1 where [ 2 ] denotes the integer part of 2, and T [ 2 ] (x) is the rst kind Chebyshev polynoial of degree [ 2 ]. Proof. Let the notation J i (x); i = 1; 2; : : : ; M as before, and write D A = [ 2 1; 2 M]. Then according to Theore 1, we have in p2q;p(0)=1 kp(j i(x))k: i=1;2;:::;m;j 1 jjxjjmj ro well known results in approxiation theory [5], the optial polynoial p o 2 Q satisfying the above ini proble exists; furtherore, it can be veried that p o is an even function using the syetry of variable x, i.e. p o (x) = q o (x 2 ),q o 2 Q [. It ] 2 thus follows that in kq(j i (x))k: (25) q2q [ ];q(0)=1 i=1;2;:::;m;x2da 2 10

11 It is well known [21] that for the ini proble the optial polynoial is in q2q ;q(0)=1 x2da 2 j q(x) j; q (x) = T [ ( 2 + M 2? 2x 1 )=T ] 2 2 M? 2 [ ( ] 2 1 2? 1 ): We can thus choose an approxiate optial polynoial q a (x) = q (x) for (25). With this choice, we then get kq a (J i (x))k i=1;2;:::;m;x2da i=1;2;:::;m d i d 0jd?1;x2DA 0jdi?1;x2DA j q(j) a (x) j : j! j q(j) a (x) j j! Since 2 M + 2 1? 2x 2 M? 2 1 and due to Part 1 of Lea 1, we obtain 2 [?1; 1] d 0jd?1 1 j! 2j [ 2 ]2j C([ 2 ]; j)=t [ 2 ] ( ? 1 ); fro which it follows that our assertion holds. 2 As in Theore 2, the defectiveness of A ight ake the convergence speed of these ethods becoe slow. Usually, the higher the degree of nonlinear eleentary divisors is, the ore slowly these ethods are likely to converge. Another iportant point is that the distribution of the spectru of A has a strong eect on the convergence speed. enerally speaking, the bigger the dierence between j 1 j and j M j, the ore slowly these ethods ay converge; otherwise, they converge relatively rapidly. 4.2 The case of atrices whose spectru A lies in the open right (left) half plane We consider upper bounds for in which the spectru of A is contained in an ellipse E which is syetric with respect to the real axis and whose ajor axis is either the real axis or parallel to the iaginary axis. 11

12 Theore 4 Assue that the spectru of A lies in an ellipse E with the center c, the foci c + e; c? e and the ajor seiaxis a. Let E be syetric with respect to the real axis and its ajor axis either the real axis or parallel to the iaginary axis, and let E exclude the origin. Dene d as before, and = a=e: Then, if j e j 1, d j e j?d+1 B p (; d? 1) ( + d?1 2? 1) ; (26) ( 2? 1) d?1 j T (c=e) j if j e j> 1, d B p (; d? 1) ( + 2? 1) d?1 ; (27) ( 2? 1) d?1 j T (c=e) j where T (z) is the rst kind Chebyshev polynoial of degree. Proof. Let us dene J i (z); i = 1; 2; : : : ; M analogously. Then in p2q;p(0)=1 i=1;2;:::;m;z2e kp(j i (z))k: (28) It is dicult for us to nd the optial polynoial satisfying the above ini proble, so we can only seek an approxiate optial one. or the ini proble in j p(z) j; p2q;p(0)=1 z2e T ( c? z e )=T (c=e) often turns out to be optial though it is not always the case [10]. Based on this assertion, we take for (28) p a (z) = T ( c? z e )=T (c=e): Since (c? z)=e belongs to an ellipse with the center at zero, the foci distance one and the ajor seiaxis = a=e, we can obtain in ters of Theore 1 and Part 3 of Lea 1 kp a (J i (z))k i=1;2;:::;m;z2e d j i=1;2;:::;m d z2e;0jd?1 z2e;0jdj?1 j p(j) a (z) j j! j p(j) a (z) j j! p 2? 1) ; 1 = d 0jd?1 j! j e j?j B (; j) ( + j ( 2? 1) j j T (c=e) j 12

13 fro which and Part 2 of Lea 1 it follows that our assertions hold. 2 Obviously, it follows fro (26) and (27) that converges to zero as goes to innity. It is easy to see how rapidly these ethods converge depends strongly on the factors q B (; d? 1) d?1 ( 2? 1) ; d?1 1 = j c=e j + j c=e j2?1 + p = j c j +q j c j 2?e 2 2? 1 a + p a 2? e 2 if the ajor axis of E is the real axis, or q B (; d? 1) d?1 ( 2? 1) ; d?1 2 = j c=e j + j c=e j p 2? 1 = j c j +q j c j 2 + j e j 2 j a j + q j a j 2? j e j 2 if the ajor axis of E is parallel to the iaginary axis. enerally speaking, by Part 3 of Lea 1, we ay clai that these ethods are likely to converge ore slowly in the presence of coplex eigenvalues. If the ajor axis of E is the real axis, we ay assert that the convergence speed is likely to be ore rapid, provided that the spectru is close to the real line since 1 will be larger in this case; if E has nearly a circular shape, the convergence speed is likely to be slower since both e and 1 are sall; if the ajor axis of E is parallel to the iaginary axis, we ay assert that the convergence speed is likely to be slower whenever the spectru is alost purely iaginary since 2 will be close to one at this oent. Also, the higher the degree of nonlinear eleentary divisors is, the slower the convergence speed is likely to be. It can be seen fro Theore 4 that, if e = 0, our results (26) and (27) have no eaning. In fact, if e = 0, E will degenerate to a disk D with the center c and the radius a. In this case, we can establish the following result. Theore 5 Assue that the spectru of A lies in the open right (left) half plane, and D contains the spectru of A but not the origin. Let d as before. Then once is bigger than certain value depending on a, dc(; d) d?1 a?d+1 = j c j ; (29) where c(; d) is a constant less than unity, depending on and d. Proof. As before, we dene J i (z); i = 1; 2; : : : ; M. Then, in ters of Theore 1, we get in kp(j i (z))k: (30) or the ini proble p2q;p(0)=1 i=1;2;:::;m;z2d in j p(z) j; p2q;p(0)=1 z2d 13

14 it is known fro [29] that the optial polynoial is ( c? z ) c whose iu over the disk D is equal to ( a jcj ) and tends to zero as increases since a jcj < 1. Based on this assertion, we choose for (30) Therefore, p a (z) = ( c? z ) : c i=1;2;:::;m;z2d kp a(j i (z))k d i i=1;2;:::;m d 0jd?1;z2D It is easy to prove that for 0 i d? 1 z2d Thus, for 0 j < i, we have z2d j p (i) a (z) j z2d j p (j) a (z) j 0jdi?1;z2D j p(j) a (z) j! j : j p(j) (z) a j! j p (i) a (z) j= (? 1) (? i + 1)a?i = j c j : = (? 1) (? i + 1)a?i (? 1) (? j + 1)a?j = (? j)(? j? 1) (? i + 1)a j?i : (31) It is then obvious that (31) is bigger than unity for any d? 1 if a 1. Therefore, in this case, we have j p (j)(z) j = (? 1) (? d + a 2)a?d+1 = j c j 0jd?1;z2D where c(; d) = (1? 1 = c(; d) d?1 a?d+1 = j c j ; d?2 ) (1? ). It then follows that dc(; d) d?1 a?d+1 = j c j ; which is just (29). If a > 1, it is easy to verify that (31) is bigger than unity once achieves certain value depending a. Hence, (31) still holds if is bigger than soe value depending on a. 2 It is easily seen fro (29) that converges to zero as increases. urther analysis shows that the defectiveness of A decreases the estiated convergence speed, and the better the spectru of A is isolated fro the origin, the ore rapidly these ethods ay converge. 14 j

15 5 Conclusion The goal of this paper is to prove how the Krylov subspace ethods, which give rise to a siilar ini proble concerning the convergence analysis, behave as increases. In the paper, we have established the error bounds when A is defective and the spectru of A either lies in the open right (left) half plane or is on the real axis. The results show that these ethods are likely to converge slowly whenever one of three cases occurs: The atrix in question is defective, the distribution of its spectru not favorable and the Jordan basis of the atrix in question ill-conditioned, such that solving (1) ay becoe dicult. inally, we point out that although the upper bounds for are likely to be overestiates, they are adequate for the purpose of convergence analysis. Acknowledgeent. I thank Professor L. Elsner for helpful discussions on Lea 1 and other suggestions and coents, which ade e iprove the presentation of this paper. References [1] B. Atlesta, Tschebysche-polynoials for sets consisting of two disjoint intervals with applications to convergence estiates for the conjugate gradient ethod, Technical Report R (1977). [2] O. Axelsson, Conjugate gradient type ethods for unsyetric and inconsistent systes of linear equations, Linear Algebra Appl., 29 (1980) 1{16. [3] O. Axelsson and. Lindskog, On the rate of convergence of the preconditioned conjugate gradient ethod, Nuer. Math., 48 (1986) 499{523. [4] P. Brown, A theoretical coparison of the Arnoldi and MRES algoriths, SIAM J. Sci. Statist. Coput., 12 (1991) 58{78. [5] W. E. Cheney, Introduction to Approxiation Theory, Mcraw-Hill, New York (1966). [6] A. T. Chronopoulos, s-step iterative ethods for (non)syetric (in)denite linear systes, SIAM J. Nuer. Anal., 28 (1991) 1776{1789. [7] C. de Boor and J. R. Rice, Extreal polynoials with application to Richardson iteration for indenite linear systes, SIAM J. Sci. Statist. Coput., 3 (1982) 47{57. [8] S. C. Eisenstat, H. C. Elan and M. H. Schultz, Variational iterative ethods for nonsyetric systes of linear equations, SIAM J. Nuer. Anal., 20 (1983) 345{357. [9] H. C. Elan, Iterative ethods for large sparse nonsyetric systes of linear equations, Ph. D. Thesis, Yale University (1982). 15

16 [10] B. ischer and R. W. reund, Chebyshev polynoials are not always optial, J. Approx. Theory, 65 (1991) 261{272. [11] R. W. reund and N. M. Nachtigal, QMR: a Quasi-inial ethod for non-heritian linear systes, Nuer. Math., 60 (1991) 291{313. [12] R. W. reund,. H. olub and N. M. Nachtigal, Iterative solution of linear systes, Acta Nuerica (1992) 1{44. [13]. R. antacher, The Theory of Matrices, Vol. 2, Chelsea Publishing Copany, New York (1959). [14]. H. olub and R. S. Varga, Chebyshev sei-iterative ethods, successive overrelaxation iterative ethods, and second order Richardson iterative ethods, Nuer. Math., 3 (1961) 147{168. [15]. H. olub and C.. Van Loan, Matrix Coputations, The John Hopkins University Press, 2nd Edition, Baltiore (1989). [16] Z. Jia, The convergence of generalized Lanczos ethods for large unsyetric eigenprobles, subitted for publication, [17] V. I. Lebedev, Iterative ethods for solving operator equations with a spectru contained in several intervals, USSR Coput. Math. Math. Phys., 9 (1969) 17{24. [18] T. A. Manteuel, An iterative ethod for solving nonsyetric linear systes with dynaic estiation of paraeters, Digital Coputer Laboratory Reports, Rep. UIUCDS- R , University of Ill., (1975). [19] T. A. Manteuel, The Tchebychev iteration for nonsyetric linear systes, Nuer. Math., 28 (1977) 307{327. [20] N. W. Nachtigal, S. C. Reddy and L. N. Trefethen, How fast are nonsyetric atrix iterations? SIAM J. Matrix Anal. Appl., 13 (1992), pp. 778{795. [21] T. R. Rivlin, The Chebyshev Polynoials, J. Wiley and Sons Inc., New York (1974). [22] Y. Saad, Krylov subspace ethods for solving large unsyetric linear systes, Math. Coput., 37 (1981) 105{126. [23] Y. Saad, The Lanczos biorthogonalization algorith and other oblique projection ethods for solving unsyetric systes, SIAM J. Nuer. Anal., 19 (1982) 485{506. [24] Y. Saad, Practical use of soe Krylov subspace ethods for solving indenite and nonsyetric linear systes, SIAM J. Sci. Stat. Coput., 5 (1984) 203{228. [25] Y. Saad and M. H. Schultz, MRES: a generalized inial residual algorith for solving nonsyetric linear syste, SIAM J. Sci. Stat. Coput., 7 (1986) 856{869. [26] Y. Saad, Krylov subspace ethods on supercoputers, SIAM J. Sci. Statist. Coput., 10 (1989) 1200{

17 [27] J. Stoer and R. W. reund, On the solution of large indenite systes of linear equations by conjugate gradient algoriths, in Coputing Methods in Applied Science and Engineering V, eds. R. lowinski and J. L. Lions, North-Holland, Asterda (1982) 35{53. [28] L. N. Trefethen, Pseudospectra of Matrices, in D.. riths and. A. Watson, Nuerical Analysis 1991, Longan, [29] R. S. Varga, A coparison of the successive overrelaxation ethod and seiiterative ethod using Chebyshev polynoials, J. Soc. Indust. Appl. Math., 5 (1957) 39{46. [30] H. A. Van der Vorst and C. Vuik, The superlinear convergence behaviour of MRES, J. Coput. Appl. Math., 48 (1993), pp. 327{341. [31] P. K. W. Vinsoe, Orthoin, an iterative ethod for solving sparse sets of siultaneous linear equations, in Proc. ourth SPE Syposiu on Reservoir Siulation, Los Angeles (1976) 149{160. [32] O. Widlund, A Lanczos ethod for a class of nonsyetric systes of linear equations, SIAM J. Nuer. Anal., 15 (1978) 801{