SCHWARZ ITERATIONS FOR SYMMETRIC POSITIVE SEMIDEFINITE PROBLEMS

Transcription

1 SCHWARZ ITERATIONS FOR SYMMETRIC POSITIVE SEMIDEFINITE PROBLEMS REINHARD NABBEN AND DANIEL B. SZYLD Abstract. Convergence propertes of addtve and multplcatve Schwarz teratons for solvng lnear systems of equatons wth a symmetrc postve semdefnte matrx are analyzed. The analyss presented apples to matrces whose prncpal submatrces are nonsngular,.e., postve defnte. These matrces appear n dscretzatons of some ellptc partal dfferental equatons, e.g., those wth Neumann or perodc boundary condtons. Key words. Lnear systems, addtve Schwarz, multplcatve Schwarz, doman decomposton methods, symmetrc postve semdefnte systems, sngular matrces, comparson theorems, overlap, coarse grd correcton. AMS subject classfcatons. 65F10, 65F35, 65M Introducton. Doman decomposton methods, ncludng addtve and multplcatve Schwarz, are wdely used for the numercal soluton of partal dfferental equatons; see, e.g., [38], [41], [44]. Advantages of these methods nclude enhancement of parallelsm and a localzed treatment. One can fnd algebrac descrptons of them e.g., n [14], [20], [47], especally for symmetrc postve defnte problems. In ths paper, we adopt the algebrac representaton of addtve and multplcatve Schwarz developed n a seres of papers [1], [18], [19], [34], [35], where analyss of convergence and propertes for several varants of the methods are provded, both for symmetrc postve defnte and for nonsngular M-matrces. Recently, convergence propertes were studed for sngular systems arsng n the soluton of Markov chans,.e., sngular M-matrces wth all prncpal submatrces beng nonsngular [7], [32]. In partcular ths theory apples to sngular matrces wth a one-dmensonal nullspace, and to those representng rreducble Markov chans; see, e.g., [42]. We also menton the recent work on multplcatve Schwarz teratons for postve semdefnte operators [26], [28]. In ths paper, we extend the theory to the symmetrc postve semdefnte case, wth partcular emphass on the sngular case (the analyss of the symmetrc postve defnte case s known; see, e.g., [1], [21, Ch. 11], [41], [44]). We study n partcular the case when all prncpal submatrces are nonsngular,.e., postve defnte. Ths stuaton arses n practce, e.g., n the dscretzaton of certan ellptc dfferental equatons such as u + u = f wth Neumann or perodc boundary condtons; see, e.g., [5]. We show that n ths case, the addtve and multplcatve Schwarz teratons are convergent and we characterze the convergence factor γ for such methods (sectons 4 and 5). We use the theory of matrx splttngs (see secton 3) to obtan these convergence propertes. We remark that we do not use splttngs to produce new statonary teratve methods. What we do s recast the Schwarz teraton matrces as comng from specfc splttngs, and use ths setup as analytcal tools to obtan Frst submtted 3 November Ths verson 5 June 2006 Insttute für Mathematk, Technsche Unverstät Berln, D Berln, Germany (nabben@math.tu-berln.de). Department of Mathematcs, Temple Unversty (038-16), 1805 N. Broad Street, Phladelpha, Pennsylvana , USA (szyld@temple.edu). Supported n part by the U.S. Natonal Scence Foundaton under grant DMS , and by the U.S. Department of Energy under grant DE- FG02-05ER

2 2 Schwarz for semdefnte systems convergence results. The convergence theory we develop mples that the correspondng precondtoned matrces have zero as an solated pont n the spectrum. The rest of the spectrum s contaned n a crcle centered at one wth radus γ < 1. When consderng addtve and multplcatve Schwarz precondtoners for sngular systems, one needs to use Krylov subspace methods whch are sometmes talored for ths case; see, e.g., [17], [23], [39] and the references gven theren. We beleve that our purely algebrac approach s much smpler than that of [26], [28], and n addton, t can be appled to problems whch may not have a varatonal formulaton. Of course our approach s only vald for the fnte dmensonal case. We also consder the case of nexact local solvers (secton 6), and the nfluence of the amount of overlap and the number of blocks n the convergence rate (sectons 7 and 8). Fnally, we study the convergence of two-level methods,.e., methods where a coarse grd correcton s consdered as well (secton 9). 2. The algebrac representaton and notaton. We frst brefly descrbe the addtve and multplcatve Schwarz methods and gve some auxlary results. Addtonal notaton and background s also gven n the next secton. Let R(A) be the range of A. Consder the lnear system n R n of the form Ax = b, b R(A). (2.1) In ths paper we consder the case where A s symmetrc postve semdefnte, and we denote ths by A O. We assume that every prncpal submatrx of A s nonsngular,.e., a symmetrc postve defnte matrx, and f A s such a submatrx, we denote ths by A O. Ths stuaton occurs, for nstance, when the null space of A, N(A), s undmensonal and any generator of t has no zero entres, cf. [5]. We consder p subspaces V, wth dmv = n, = 1,..., p, whch are spanned by columns of the dentty I over R n and such that n V = R n =: V. (2.2) Note that the subspaces V may overlap. Between the subspaces V and the space V we consder the followng mappngs R : V V, R T : V V, where rank(r T) = n. R s called the restrcton operator whle R T prolongaton operator. We also use the matrces P = R T A R A = R T (R AR T ) R A, s called the where A := R AR T s a permutaton of a prncpal submatrx of A, whch because of our assumpton s nonsngular. Note that P s a projecton. Wth these projectons the damped addtve Schwarz method used as an teratve method to solve (2.1) can be descrbed as x k+1 = x k + = (I R T A R (b Ax k ) (2.3) R T A R A)x k + ( R T A R )b

3 Renhard Nabben and Danel B. Szyld 3 where 0 < 1 s a dampng parameter; see [8], [11], [12], [13], [20], [21, Ch. 11], [41], [44]. The teraton matrx s then gven by or, usng the notaton T AS, = I M R T A R A = I AS = then, the teraton matrx (2.4) can be wrtten as T AS, = I M AS A. P, (2.4) R T A R, (2.5) Later on, n Theorem 4.2, we show that the matrx on the rght hand sde n (2.5) s nonsngular, and therefore t makes sense to denote t as M AS. Furthermore, for each > 0 one can defne a splttng of A for whch the teraton matrx s precsely (2.4). One such splttng s A = 1 M AS ( 1 M AS A). When A s sngular, such splttng however s not unque; see [2]. Very often n practce the addtve Schwarz method s used for precondtonng a Krylov subspace method. In the symmetrc cases consdered here the method of choce s the Conjugate Gradent method; for a study of ths method for sngular systems, see [23]. Whle the matrx A may be sngular, the precondtonng matrx M s usually assumed to be symmetrc postve defnte. The addtve Schwarz precondtoner s and the precondtoned matrx s then M AS AS A = P = I T AS,1. M The multplcatve Schwarz method can be wrtten as the teraton wth the teraton matrx x k+1 = T MS x k + c, k = 0, 1,..., (2.6) T MS = (I P p )(I P p ) (I P 1 ) = 1 (I P ), (2.7) and a certan vector c. The correspondng precondtoned matrx n ths case s I T MS. Remark 2.1. Observe that for any vector y N(A),.e., such that Ay = 0, one has Ty = y for both teraton matrces T = T AS, of (2.4), or T = T MS of (2.7). Ths mples n partcular that we need to requre n our teratons such as (2.3), that x 0 / N(A). We outlne our strategy to prove the convergence of the teratons (2.3) and (2.6). We need to show that the powers of the teraton matrces (2.4) and (2.7) converge to a lmt; see defnton 3.1 below. One suffcent condton for ths to hold s that there s a splttng of A of the form A = M N wth M nonsngular such that M N s the teraton matrx, and show that ths splttng s P-regular (see defnton 3.3 below), =p

4 4 Schwarz for semdefnte systems whch mples convergence; see Theorem 3.2 below. We also use certan comparson theorems to relate the convergence of dfferent versons of these teratons. We present a context for these analytcal tools n secton 3. In the rest of ths secton, we repeat the algebrac characterzaton of the Schwarz methods used, e.g., n [1], whch s the bass to produce the above mentoned splttngs. As already mentoned, we assume that the rows of R are rows of the n n dentty matrx I, e.g., of the form R = Ths restrcton operator s often called a Boolean gather operator, whle ts transpose R T s called a Boolean scatter operator. Formally, such a matrx R can be expressed as. R = [I O]π (2.8) wth I the dentty on R n and π a permutaton matrx on R n. Then A s a symmetrc permutaton of an n n prncpal submatrx of A. In fact, we can wrte [ ] π Aπ T A K = K T, (2.9) A where A s the prncpal submatrx of A complementary to A,.e. wth I the dentty on R n n. For each = 1,...,p, we defne A = [O I ] π A π T [O I ] T E := R T R R n n. (2.10) These dagonal matrces have ones on the dagonal n every row where R T has nonzeros. We further need sets S defned by Then S := {j {1,...,n} : (E ) j,j = 1}. p S = S = {1, 2,...,n}, (2.11).e., each ndex s n at least one set S. Ths s equvalent to sayng that E I, wth equalty f and only f there s no overlap. In other words, n the case of overlappng subspaces, we have here that each dagonal entry of E s greater than or equal to one, whch mples nonsngularty. Only n the rows correspondng to overlap ths matrx has an entry dfferent from one.

5 Renhard Nabben and Danel B. Szyld 5 For each = 1,...,p, we construct a second set of matrces M R n n assocated wth R from (2.8) as follows [ ] M = π T A O π O D, (2.12) where under our assumptons on A O, we have that D = dag(a ) O, and thus M s nvertble. Wth the defntons (2.10) and (2.12) we obtan the followng equalty whch we wll use throughout the paper E M A = R T A R A = P, = 1,...,p. (2.13) 3. Convergent matrces, splttngs and comparson theorems. In ths secton we present some more defntons and results whch we use n the rest of the paper. Defnton 3.1. A matrx T s called convergent f lm k T k exsts. Ths s equvalent to the followng three condtons: 1) ρ(t) 1; 2) rank(i T) = rank(i T) 2 ; 3) If λ = 1 for an egenvalue λ of T, then λ = 1. Condton 2 states that the ndex of the matrx I T s one, or n ths case that nd 1 T = 1 [3]. Several equvalent condtons can be found n [43]. One of them s the followng: nd 1 T = 1 R(I T) N(I T) = {0}, (3.1).e., that the ntersecton of the range and the null space of I T s trval. If ρ(t) = 1 for a convergent matrx then the asymptotc rate of convergence s gven by γ(t) := max{ λ : λ σ(t), λ < 1}. (3.2) When A s sngular, and we have a nonsngular matrx M, and a convergent matrx T such that A = M(I T), then P = lm k T k s a projecton onto N(A) = N(I T). In fact P = I (I T)(I T) D, where (I T) D denote the Drazn nverse of (I T). Furthermore, f we let c = M b, and consder the teraton x k+1 = Tx k +c, x 0 / N(A), cf. (2.3), then lm k x k = (I T) D c + (I P)x 0 ; see, e.g., [3, Ch. 7.6]. A useful result n the analyss of convergent teraton matrces s the followng, due to Keller [24]. Theorem 3.2. Let A be symmetrc and let M be nonsngular such that M + M T A s postve defnte. Then T = I M A s convergent f and only f A s postve semdefnte. Note than when M s symmetrc ths Theorem says that f 2M A O, then T s convergent f and only f A O. Defnton 3.3. A splttng A = M N s called P-regular f M +M T A O [36], and strong P-regular f n addton N O [33]. Wth ths defnton, Theorem 3.2 ndcates that a suffcent condton for convergence of T s that A = M N s a P-regular splttng of a postve semdefnte matrx. Weaker suffcent condtons, and also necessary condtons, not requrng the nonsngularty of M, can be found n the recent paper [27].

6 6 Schwarz for semdefnte systems The followng result s a new suffcent condton for convergence, whch we use later n the paper. Lemma 3.4. Let A be symmetrc postve semdefnte and let A = M N wth M symmetrc postve defnte. If A 1 2 M A 1 2 2I, then T = I M A s convergent and A = M N s a P-regular splttng. Proof. We have A 1 2 M A 1 2 2I. Thus Snce σ(a 1 2 M A 1 2 ) [0, 2). σ(a 1 2 M A 1 2 ) = σ(m A) = σ(am ) = σ(am 2 M 1 2 ) = σ(m 1 2 AM 1 2 ), we have that Hence, and therefore, 2I M 1 2 AM M 1 2 (2I M 1 2 AM 1 2 )M M A 0,.e., we have a P-regular splttng. Usng Theorem 3.2 we obtan that T = I M A s convergent. The use of P-regular splttngs as suffcent condtons for convergence of classcal statonary teratve methods for symmetrc matrces, mmcs the use of regular or weak regular splttngs as suffcent condtons for the convergence of classcal statonary teratve methods for monotone matrces; see, e.g., the classc books [3], [37], [45]. In ths case, the rate of convergence of the teratve method s gven by the spectral radus of the teraton matrx. Thus, the rate of convergence of two teratve methods for monotone matrces can be compared by lookng at the correspondng spectral rad. Many comparson theorems usng dfferent hypothess on the splttngs have appeared n the lterature; see, e.g., [9], [10], [16], [29], [33], [45], [46], and other references theren. When the teraton matrces have spectral radus equal to one, as s usually the case for sngular lnear systems, the convergence rate s gven by (3.2). Comparson theorems for these can be found n [30], [31]. Here we present a new comparson theorem, whch we use n our context. We frst present the followng result due to Weyl; see [22, Theorem 4.3.7]. Let M O, and denote ts egenvalues by λ 1 (M) λ 2 (M),...,λ n (M) 0. Proposton 3.5. Let M 1 and M 2 be two symmetrc postve semdefnte matrces. If M 1 M 2 then λ (M 1 ) λ (M 2 ) for all. Of course, ths Proposton s vald when M s postve defnte as well. Theorem 3.6. Let A be symmetrc postve semdefnte. Let M 1 and M 2 be symmetrc postve defnte and let N 1 := M 1 A and N 2 := M 2 A. If M1 M2

7 Renhard Nabben and Danel B. Szyld 7 Then λ (M 1 N 1) λ (M 2 N 2) for all. If addtonally N 1 and N 2 are postve semdefnte then Proof. We frst note that γ(m 1 N 1) γ(m 2 N 2). σ(m k A) = σ(mk A A 2 ) = σ(a Wth Proposton 3.5 we obtan for each that Snce M k 2 M k A 1 2 ), k = 1, 2. λ (M1 A) = λ (A 1 2 M 1 A 1 2 ) λ (A 1 2 M 2 A 1 2 ) = λ (M2 A). (3.3) N k = I M k A, k = 1, 2, (3.3) ndcates that for each, λ (M 1 N 1) λ (M 2 N 2). If N 1 and N 2 are postve semdefnte then all egenvalues of M 1 N 1 and M 2 N 2 are nonnegatve, and therefore γ(m 1 N 1) γ(m 2 N 2). 4. Convergence of Addtve Schwarz. We begn wth an auxlary result, the proof of whch follows by a straghtforward calculaton. Lemma 4.1. Let A be symmetrc postve semdefnte. Then A 1 2 R T (R AR T ) R A 1 2 s an orthogonal projecton. Thus, I A 1 2 R T (R AR T ) R A 1 2 s also an orthogonal projecton and as a consequence and A 1 2 R T (R AR T ) R A 1 2 I, (4.1) σ(a 1 2 R T (R AR T ) R A 1 2 ) = {0, 1}. Theorem 4.2. Let A be symmetrc postve semdefnte such that each prncpal submatrx s postve defnte. Let b R(A) and x 0 / N(A). If 0 < < 2/p, then the addtve Schwarz teraton defned by (2.4) s convergent and the splttng defned by M = 1 M AS s P-regular. Proof. Frst, as s done n [21] for the nonsngular case, we prove that the matrx R T (R AR T ) R s nonsngular. To that end, let the vector x be such that R T (R AR T ) R x = 0.

8 8 Schwarz for semdefnte systems Hence and thus x T R T (R AR T ) R x = 0, (A 2 R x) T A 2 R x = A 1 2 R x 2 2 = 0, whch mples R x = 0 for = 1,...,p. By our assumpton (2.2) ths mples that x = 0. Usng Lemma 4.1 we have that (4.1) holds. Summng up, we have A 1 2 ( R T (R AR T ) R )A 1 2 pi, (4.2) and snce < 2/p, we have A 1 2 M AS A 1 2 2I. We can now use Lemma 3.4, and ths completes the proof. As s done n [21, Ch ] n the symmetrc postve defnte case, a careful look at the sum n (4.2) ndcates that we can replace the number of subdomans p wth the number of colors q of the graph of A. Thus A 2M 1 AS A 1 2 qi, and f < 2/q, we have convergence. Remark 4.3. If we further restrct the value of the dampng parameter to < 1/p (or < 1/q), we have that the splttng defned by 1 M AS s strong P-regular. Ths follows snce n ths case A 1 2 M AS A 1 2 I, whch mples 1 M AS A. We note that the result n Theorem 4.2 apples n partcular to the symmetrc postve defnte case. Thus, n our formulaton we have doubled the nterval of admssble dampng factors for convergence of the damped addtve Schwarz method, snce the usual restrcton s that < 1/q; see [18], [21, Ch ]. We menton also that smple examples show that ths method may not be convergent for = 1. From Theorem 4.2 t follows that the only egenvalue of T n the unt crcle s λ = 1, and snce we showed that M AS s nonsngular, the correspondng egenvector s a generator of the one-dmensonal N(A). It follows then (see, e.g., [22, 4.2]), that the convergence factor (3.2) of the Addtve Schwarz teraton can be characterzed as γ(t AS, ) = max z T T AS, z z N(A) z T z=1 ( = max z N(A) (z,z)=1 ( = 1 1 mn z N(A) (z,z)=1 (R T A (R T A R z, Az) R z, Az) ) ). (4.3) We note that on the subspace N(A), the matrx A s postve defnte. Let us call  = A N(A), and we can thus replace A wth  n (4.3). Furthermore snce Â1/2 s nvertble, we can wrte w = Â1/2 z, and wrte (4.3) as γ(t AS, ) = 1 mn  /2 w N(A) (w,â w)=1 w T  /2 R T A R  1/2 w. (4.4)

9 Renhard Nabben and Danel B. Szyld 9 We pont out that the characterzaton (4.4) s also vald for the case of A symmetrc postve defnte, n whch case we have  = A. 5. Convergence of Multplcatve Schwarz. We begn wth an mportant auxlary result. Lemma 5.1. Let A be a symmetrc postve semdefnte matrx such that each prncpal submatrx s postve defnte. Let x, y R n, such that y = (I E M A)x, (5.1) where E s defned n (2.10) and M n (2.12). Then the followng holds: y T Ay x T Ax = (y x) T E AE (y x) 0. (5.2) Proof. Consder x = π T(xT 1, x T 2 ) T and y = π T(yT 1, y2 T ) T, wth x 1, y 1 R n. Further, from (2.10) and (2.8) we have that [ ] E = π T I O π O O. (5.3) Consder now (5.1), whence we mmedately have that and usng (2.12) and (2.9), we also get y 2 = x 2, (5.4) A y 1 = A 12 x 2, (5.5) where here we use the notaton A 12 = K, and smlarly A 21 = K T these denttes we wrte y T Ay x T Ax = (y T 1, y T 2 )π Aπ T (y T 1, y T 2 ) T (x T 1, x T 2 )π Aπ T (x T 1, x T 2 ) T = A T 12. Usng = y T 1 A y 1 + y T 2 A 21 y 1 + y T 1 A 12 y 2 x T 1 A x 1 x T 2 A 21 x 1 x T 1 A 12 x 2 = x T 2 A 21 (y 1 x 1 ) + (y T 1 x T 1 )A 12 x 2 + y T 1 A y 1 x T 1 A x 1 = y T 1 A (y 1 x 1 ) (y T 1 x T 1 )A y 1 + y T 1 A y 1 x T 1 A x 1 = (y T 1 x T 1 )A (y 1 x 1 ) = (y x) T E AE (y x), where the last equalty follows from the dentty [ E AE = π T A O O O ] π. Snce A O, E AE s semdefnte as well, and the rght hand sde of (5.2) s nonpostve. Theorem 5.2. Let A be a symmetrc postve semdefnte matrx such that each prncpal submatrx s postve defnte. Let b R(A) and x 0 / N(A). Then the multplcatve Schwarz teraton defned by (2.6) s convergent. Proof. We need to prove that the teraton matrx T = T MS s convergent,.e. we need to prove condtons 1), 2) and 3) of Defnton ): Startng wth z = x (1) / N(A) let x (+1) = (I P )x (). Thus x (p+1) = Tx (1). Usng (5.2) repeatedly, and cancelng terms, we obtan z T T T ATz z T Az = (x (+1) x () ) T E AE (x (+1) x () ) = ((x (+1) x () ) T E )E AE (E (x (+1) x () )). (5.6)

10 10 Schwarz for semdefnte systems Snce E AE s postve defnte t follows that the rght hand sde of (5.6) s nonpostve. However, the rght hand sde s zero f and only f E (x (+1) x () ) = 0 for all, = 1,...,p. The other n n components of x (+1) x () are also zero usng the same argument as n Lemma 5.1 to obtan (5.4). But ths mples x (p+1) = x (+1) = x () = x (1), = 1,...,p. Thus x (1) must be a common fxed pont of (I P ) for all = 1,..., p. However, the fxed ponts of the projectons (I P ) are just the vectors z R n wth E z = 0. Snce E I there s no such common nonzero fxed pont. Hence the rght hand sde of (5.6) must be negatve, and we obtan z T T T ATz z T Az < 0. Thus we have that for all λ σ(t) wth correspondng egenvector y / N(A) Hence λ 2 1 < 0. Thus λ 2 y T Ay y T Ay < 0. (5.7) λ < 1. If λ σ(t) but the correspondng egenvector y N(A), we easly obtan from the defnton of T that λ = 1. Hence, ρ(t) 1. 2): By (3.1), t suffces to prove that N(I T) R(I T) = {0}. Here we have that N(A) = N(I T). Ths holds snce y / N(A) mples Ty y by part 1),.e., y / N(I T). On the other hand y N(A) mples y N(I T), usng the defnton of T, cf. Remark 2.1. Hence, we need to prove that N(A) R(I T) = {0}. (5.8) Let x N(A) R(I T). Then there exsts a y wth (I T)y = x,.e., y = Ty+x. Snce x N(A) we obtan Usng y = Ty + x we get A(I T)y = Ax = 0, and thus y T Ay y T ATy = 0. y T Ay y T T T ATy + x T ATy = y T Ay y T T T ATy = 0. Part 1) of ths proof now mples y N(A), cf. (5.7). Therefore, by Remark 2.1, x = (I T)y = 0 whch completes ths part of the proof. 3): As proved above we have λ < 1 for all λ σ(t) wth correspondng egenvector y / N(A). Thus f λ = 1 for some egenvalue λ of T then the correspondng egenvector y must be n the null space of A. Hence Ay = 0. But then Ty = y and thus λ = 1. We menton that we need to prove explctly (5.8) snce we do not have an explct representaton of a nonsngular matrx M MS such that M MS A = I T MS. The exstence of such a matrx,.e., of a splttng nduced by T MS [2] s only obtaned after the theorem s proved. Any splttng nduced by such a matrx M MS s thus P-regular.

11 Renhard Nabben and Danel B. Szyld 11 We also comment on the fact that n some cases one may want to have a symmetrc operator, and n such a case, the natural multplcatve operator s T SMS = (I P 1 )(I P 2 ) (I P p )(I P p )(I P p ) (I P 1 ). (5.9) It follows that Theorem 6.1 apples to ths case as well, and that a posteror, there exsts a nonsngular matrx M SMS such that M SMS A = I T SMS. We can characterze the convergence factor (3.2) of ths Symmetrc Multplcatve Schwarz teratons as γ =γ(t SMS ) = max z N(A) z T z=1 (z, T SMS z). (5.10) 6. Inexact local solvers. In ths secton we study the effect of varyng how exactly (or nexactly) the local problems are solved. The convergence of these very practcal verson of the methods s based on the same deas used to prove that of the standard Schwarz teratons n sectons 4 and 5. The nfluence of dfferent level of nexactness s analyzed usng our comparson theorem 3.6. Very often n practce, nstead of solvng the local problems A y = z exactly, such lnear systems are approxmated by à z where à s an approxmaton of A ; see, e.g., [6], [41], [44]. The expresson à z often represents an approxmaton to the soluton of the system A z = v usng some steps of an (nner) teratve method. By replacng A wth à n (2.4) one obtans the damped addtve Schwarz teratons wth nexact local solvers, and ts teraton matrx s then T AS, = I R T à R A. (6.1) The teraton matrces T AS, and T AS, n (2.4) and (6.1) are nduced by splttngs A = M N and A = M Ñ where Here M = π T M M [ à O O D = = R T A R T à R = R = E M O, (6.2) E M O. (6.3) ] π, and thus M = π T [ à O O D ] π. (6.4) The fact that the matrx (6.3) s nonsngular follows n the same manner as n the proof that (6.2) s nonsngular n Theorem 4.2. In the case consdered n ths paper we assume, as s generally done (see, e.g., [21, Ch ]), that the nexact local solvers correspond to symmetrc postve defnte matrces and satsfy à A. (6.5) For examples of splttngs for whch the nequalty (6.5) holds see, e.g., [33]. A stuaton worth mentonng where (6.5) holds s when A s semdefnte and the nexact local solver s defnte. Ths process s usually called regularzaton; see, e.g., [15], [25].

12 12 Schwarz for semdefnte systems Theorem 6.1. Let A be a symmetrc postve semdefnte matrx such that each prncpal submatrx s postve defnte. Let b R(A) and x 0 / N(A). Let à and Ā be nexact local solvers of A satsfyng à Ā A. Let T AS, s obtaned by replacng à by Ā n (6.1), = 1,...,p. Let the dampng factor 0 < < 2/p. Then the nexact addtve Schwarz teratons defned by (6.1) and T AS, are convergent, and the splttngs nduced by these teraton matrces are P-regular. Wth the stronger hypothess that 0 < < 1/p, we also have that γ(t AS, ) γ( T AS, ) γ( T AS, ), and the splttngs nduced by these teraton matrces are strong P-regular. Proof. Snce à A we have and thus, usng Lemma 4.1 A 1 2 R T à à A. (6.6) 1 R A 2 A 1 2 R T A R A 1 2 I. Smlar nequaltes are obtaned wth Ā. The rest of the convergence proof proceeds n the same manner as that of Theorem 4.2. Consder the matrces (6.2) and (6.3) whch are symmetrc postve defnte usng M as n (2.12) and M as n (6.4). From (6.6), we have that M O. Ths mples M M and N Ñ. By Remark 4.3, we have that N O,.e., that the splttngs are strong P-regular. The same results are obtaned n the case of Ā. The theorem follows from Theorem 3.6. As was the case wth Theorem 4.2, we can replace p n the restrcton on the dampng parameter wth q, the number of colors,.e., we guarantee convergence of addtve Schwarz wth nexact local solvers for < 2/q. Snce Theorem 6.1 apples n partcular to the symmetrc postve defnte case, we have agan double the nterval of admssble dampng factors for the addtve Schwarz teraton wth nexact local solvers, cf. [1]. Remark 6.2. An alternatve proof of the second part of Theorem 6.1 can be obtaned by consderng the two convergence factors, γ(t AS, ) gven by (4.4) for the exact case, and the second gven by γ( T AS, ) = 1 mn  /2 w N(A) (w,â w)=1 0=1 w T  /2 R T à M R  1/2 w (6.7) for the nexact case. Snce σ(â/2 R TA R  1/2 ) = {0} σ(a ) and σ(â/2 R Tà R  1/2 ) = {0} σ(ã ), and snce à A, we have that w T  /2 R T à R  1/2 w w T  /2 R T A R  1/2 w, = 1,..., p, whch mples that γ( T AS, ) γ(t AS, ). For smplcty, n Theorem 6.1, we assumed that the nexact versons use the same dampng parameter. It s evdent from the proofs that f the dampng parameter for the nexact verson s smaller, say <, the same conclusons hold. The mplcaton of Theorem 6.1 s that by replacng the local solvers A wth the approxmate counterparts Ã, the addtve Schwarz teraton s expected to take more teratons. In practce, a solve wth à should be suffcently less expensve so that the overall method s cheaper.

13 Renhard Nabben and Danel B. Szyld 13 Next we consder the multplcatve Schwarz method wth nexact local solvers on the subdomans. Here we assume that the approxmatons à satsfy Ths assumpton mples that à + ÃT A 0. (6.8) A = à (à A ) are P-regular splttngs. Usng (6.4), the nexact multplcatve Schwarz teraton matrx s gven by T = (I E p M p A)(I E p M p A) (I E 1 M 1 A). (6.9) Lemma 6.3. Let A be a symmetrc postve semdefnte matrx. Let x, y R n such that y = (I E M A)x where M s defned n (6.4) wth à satsfyng (6.8). Then the followng dentty holds: (y x) T E ( M T + M A)E (y x) 0. (6.10) Proof. The proof proceeds as that of Lemma 5.1. We have that (5.4) holds, but nstead of (5.5) we have Ãy 1 = (à A )x 1 A 12 x 2. We then obtan y T Ay x T Ax = x T 2 A 21 (y 1 x 1 ) + (y T 1 x T 1 )A 12 x 2 + y T 1 A y 1 x T 1 A x 1 = (x T 1 (à A ) T y T 1 ÃT )(y 1 x 1 ) + (y T 1 x T 1 )((à A )x 1 Ãy 1 ) + y T 1 A y 1 x T 1 A x 1 = ( x T 1 A (y T 1 x T 1 )ÃT )(y 1 x 1 ) + (y T 1 x T 1 )( A x 1 Ã(y 1 x 1 )) + y T 1 A y 1 x T 1 A x 1 = (y T 1 x T 1 )(à + ÃT A )(y 1 x 1 ) = (y x) T E ( M T + M A)E (y x) 0, where the last nequalty follows from (6.8) and the form of the matrces M n (6.4). Theorem 6.4. Let A be a symmetrc postve semdefnte matrx such that each prncpal submatrx s postve defnte. Let b R(A) and x 0 / N(A). Then the multplcatve Schwarz teraton wth teraton matrx (6.9) wth M defned n (6.4) and wth nexact local solvers satsfyng (6.8) converges to the soluton of Ax = b. Proof. We need to prove that the teraton matrx T s convergent,.e., we need to prove condtons 1), 2) and 3) of Defnton 3.1. The proof s smlar to the proof of Theorem 5.2. The only dfference appears n provng condton 1). Here we use Lemma 6.3 and obtan z T T T A Tz z T Az < 0. for all z / N(A), and the rest of the proof follows. A symmetrc verson of Multplcatve Schwarz wth nexact local solvers can also be constructed n a way smlar to (5.9), and ts convergence factor can be characterzed n a way smlar to (5.10). We menton that a comparson analogous to that of the second part of Theorem 6.1 s not vald for multplcatve Schwarz, not even n the defnte case. A counterexample can be found n [40].

14 14 Schwarz for semdefnte systems 7. Varyng the amount of overlap. We study here how varyng the amount of overlap between sub-blocks (subdomans) nfluences the convergence rate of addtve Schwarz. Let us consder two sets of sub-blocks (subdomans) of the matrx A, as defned by the sets (2.11), such that one has more overlap than the other,.e., let wth p Ŝ = Ŝ S, = 1,...,p, (7.1) p S = S. Of course, each set Ŝ defnes an ˆn n matrx ˆR, where ˆn s the cardnalty of Ŝ, and the correspondng n n matrx Ê = ˆR T ˆR, as n (2.10). The relaton (7.1) mples that I Ê E O. (7.2) Smlarly, f ˆπ s such that ˆR = [I O] ˆπ, wth I the dentty n Rˆn, we denote by  the correspondng prncpal submatrx of A,.e.,  = ˆR A ˆR T = [I O] ˆπ A ˆπ T [I O] T, and, as n (2.12) defne ˆM = ˆπ T [  O O ˆD ] ˆπ, (7.3) where ˆD = dag(â ) O, and  s the (n ˆn ) (n ˆn ) complementary prncpal submatrx of A as n (2.9). As n (2.13), we have here also the fundamental dentty Ê ˆM = ˆR T  ˆR, = 1,...,n. We want to compare ˆM wth M, although  and A are of dfferent sze. Wthout loss of generalty, we can assume that the permutatons π and ˆπ concde on the set S, and that the ndexes n S are the frst n elements n Ŝ. In fact, we can assume that ˆπ = π. Thus, A s a prncpal submatrx of Â, and ˆM has the same dagonal as M. We wll apply to these the followng result for symmetrc postve defnte matrces whch can be found, e.g., n [21]. Lemma 7.1. Let A be a symmetrc postve defnte matrx and the form of the matrces M n (6.4). Let A be a symmetrc postve defnte matrx, and A = R AR T, R a restrcton operator, so that A s a prncpal submatrx of A. Then R TA R A. We consder the case of damped addtve Schwarz wth teraton matrx (2.4), and the teraton matrx correspondng to the larger overlap s ˆT AS, = I ˆR T  ˆR A. (7.4) Theorem 7.2. Let A be a symmetrc postve semdefnte matrx such that each prncpal submatrx s postve defnte. Let b R(A) and x 0 / N(A). Consder two sets of sub-blocks of A defned by (7.1), and the two correspondng addtve Schwarz

15 Renhard Nabben and Danel B. Szyld 15 teratons (2.4) and (7.4). Let the dampng factor 1/p, whch mples n partcular that the addtve Schwarz methods are convergent. Then, γ( ˆT ) γ(t ). Proof. As mentoned above assume that the all prncpal submatrces of A of order less than n are nonsngular. Let Q = E M = R TA R and ˆQ = Ê ˆM = ˆR TÂ ˆR. Snce A s a prncpal submatrx of Â, by Lemma 7.1 we have that ˆQ Q. Therefore, ˆM = ˆQ Q = M O. As shown n Remark 4.3, these splttngs are strong P-regular, and the theorem follows from Theorem 3.6. We note that an alternatve proof smlar to that n Remark 6.2 can be appled ˆR = ˆQ Q = R T A here, usng the relaton ˆR TÂ R just proved. Theorem 7.2 ndcates that the more overlap there s, the faster the convergence of the algebrac addtve Schwarz method. As a specal case, we have that overlap s better than no overlap. Ths s consstent wth the analyss for grd-based methods; see, e.g., [4], [41]. Of course, the faster convergence rate brngs assocated an ncreased cost of the local solvers, snce now they have matrces of larger dmenson and more nonzeros. In the cted references a small amount of overlap s recommended, and the ncrease n cost s usually offset by faster convergence. We should menton that wth an ncrease of overlap, the number of colors of the graph may decrease, so that the dampng factor may need to be revsed. In all cases, the maxmum restrcton s < 1/p. A comparson analogous to that of Theorem 7.2 s not vald for multplcatve Schwarz, not even n the defnte case. A counterexample can be found n [40]. 8. Varyng the number of blocks. We address here the followng queston. If we partton a block nto smaller blocks, how s the convergence of the Schwarz method affected? We show that for the addtve Schwarz method the more sub-blocks (subdomans) the slower the convergence. In a lmtng case, f we have a sngle varable n each block and there s no overlap, ths s the classc Jacob method, and our results ndcate that ths has asymptotcally slower convergence than any sets of blocks for addtve Schwarz. As n the stuatons descrbed n sectons 6 and 7, the slower convergence may be partally compensated by less expensve local solvers, snce they are of smaller dmenson. Formally, consder each block of varables S parttoned nto k sub-blocks,.e., we have S j S, j = 1,...,k, (8.1) k S j = S, and S j S k = f j k. Each set S j has assocated matrces R j and j=1 E j = R T j R j. Snce we have a partton, E j E, j = 1,..., k, and k j=1 E j = E, = 1,...,p. (8.2)

16 16 Schwarz for semdefnte systems We defne the matrces A j = R j AR T j, and M j correspondng to the set S j n the manner already famlar to the reader (see, e.g., (7.3)), so that E j M j = R T j A j R j, j = 1,...,k, = 1,...,p. Gven a fxed dampng parameter, the teraton matrx of the refned partton s then k T = I E j M j A, (8.3) j=1 cf. (2.4), and an nduced strong P-splttng (assumng the proper restrcton on ) A = M N s gven by M = k E j M j. j=1 Theorem 8.1. Let A be a symmetrc postve semdefnte matrx such that each prncpal submatrx s postve defnte. Let b R(A) and x 0 / N(A). Consder two sets of sub-blocks of A defned by (2.11) and (8.1), respectvely, and the two correspondng addtve Schwarz teratons defned by (2.4) and (8.3). Let k = max k, and let the dampng factors be 1/p, and = /k 1/(kp). Ths mples that n partcular the addtve Schwarz methods are convergent. Then, γ(t ) γ( T ). Proof. As n the proof of Theorem 7.2 we have, usng Lemma 7.1, that k Therefore, Q j k Q, and j=1 M M Q j = E j M j = k Q j k j=1 M Q = E M. Q = km, whch s equvalent to = (1/k) M. The theorem now follows usng Theorem 3.6 and the fact that these are strong P-regular splttngs, as shown n Remark 4.3. As n the prevous sectons a comparson analogous to that of Theorem 8.1 s not vald for multplcatve Schwarz, not even n the defnte case. Agan, a counterexample can be found n [40]. 9. Two-level schemes. We consder now two-level schemes,.e., those n whch an addtonal step s taken, correspondng to a coarse grd correcton. In the nonsngular case, ths addtonal step makes Schwarz methods optmal n the sense that the condton number of the precondtoned matrx M A s ndependent of the mesh sze; see, e.g., [38], [41], [44]. In our settng, for the coarse grd correcton consder an addtonal subspace V 0 of V, and the correspondng projecton P 0 = R0 T A 0 R 0A = R0 T(R 0AR0 T) R 0 A. There are several cases we consder here: addtve Schwarz wth coarse grd correcton, wth teraton matrx gven by T ASc, = T AS, R T 0 A 0 R 0A = I =0 R T A R A = I P ; (9.1) =0

17 Renhard Nabben and Danel B. Szyld 17 multplcatve Schwarz wth coarse grd correcton, wth teraton matrx gven by T MSc = T MS (I P 0 ) = 0 (I P ), or n the symmetrzed case by T SMSc = (I P 0 )T SMS (I P 0 ); multplcatve Schwarz addtvely corrected, known as two-level hybrd I Schwarz method, wth teraton matrx gven by =p H I, = I P 0 (I T MS ) = I (G 0 + M MS )A, where G 0 = R T 0 A 0 R 0; and the two-level hybrd II Schwarz method, whch s addtve Schwarz multplcatve corrected, wth teraton matrx gven by H II, = T AS, (I P 0 ). We begn our analyss wth the addtve Schwarz teraton wth coarse grd correcton. By comparng the teraton matrces n (9.1) and (2.4), one can see that Theorem 4.2 s vald n ths case as well, wth the excepton that the dampng factor needs to be less than 2/(p+1). Therefore we have that the matrx T ASc, s a convergent matrx, and that the nduced splttng defned by M ASc, = p =0 RT A R s P-regular. We can also show that coarse grd correcton does not ncrease (and may decrease) the convergence factor of the teratons. Theorem 9.1. Let A be a symmetrc postve semdefnte matrx such that each prncpal submatrx s postve defnte. Then γ(t ASc, ) γ(t AS, ). Proof. We use the fact that G 0 = R0 TA 0 R 0 0 to conclude that M ASc, = (MAS + G 0) M AS. The theorem now follows by the applcaton of Theorem 3.6. A characterzaton smlar to (4.4) apples to ths two-level method, wth one more term n the sum. Thus, an alternatve proof of ths theorem usng ths characterzaton can be done n a manner smlar to that n Remark 6.2. Next, we consder the multplcatve Schwarz teratons wth coarse grd correcton. It s not hard to see that Theorem 5.2 apples to ths case as well, so that T MSc and T SMSc are convergent. We conclude by mentonng that the coarse grd correctons can be appled to the methods wth nexact solvers descrbed n secton 6 as well, and snce the analyss s very smlar, we do not repeat t. Acknowledgement. We thank Mchele Benz and the referees for ther comments on an earler verson of ths paper, whch helped mprove our presentaton. REFERENCES [1] Mchele Benz, Andreas Frommer, Renhard Nabben, and Danel B. Szyld. Algebrac theory of multplcatve Schwarz methods. Numersche Mathematk, 89: , [2] Mchele Benz and Danel B. Szyld. Exstence and unqueness of splttngs for statonary teratve methods wth applcatons to alternatng methods. Numersche Mathematk, 76: , [3] Abraham Berman and Robert J. Plemmons. Nonnegatve Matrces n the Mathematcal Scences. Academc Press, New York, Updated edton, Classcs n Appled Mathematcs, vol. 9, SIAM, Phladelpha, 1994.

18 18 Schwarz for semdefnte systems [4] Petter E. Bjørstad and Olof B. Wdlund. To overlap or not to overlap: A note on a doman decomposton method for ellptc problems. SIAM Journal on Scentfc and Statstcal Computng, 10: , [5] Pavel Bochev and Rchard B. Lehoucq. On the fnte element soluton of the pure Neumann problem. SIAM Revew, 47:50 66, [6] James H. Bramble, Joseph E. Pascak, and Apostol T. Vasslev. Analyss of non-overlappng doman decomposton algorthms wth nexact solves. Mathematcs of Computaton, 67:1 19, [7] Rafael Bru, Francsco Pedroche, and Danel B. Szyld. Addtve Schwarz teratons for Markov chans. SIAM Journal on Matrx Analyss and Applcatons, 27: , [8] Tony F. Chan and Tarek P. Mathew. Doman decomposton methods. Acta Numerca, pages , [9] Joan-Josep Clment and Carmen Perea. Some comparson theorems for weak nonnegatve splttngs of bounded operators. Lnear Algebra and ts Applcatons, :77 106, [10] George Csordas and Rchard S. Varga. Comparsons of regular splttngs of matrces. Numersche Mathematk, 44:23 35, [11] Maksymlan Dryja. An addtve Schwarz algorthm for two- and three-dmensonal fnte element problems. In Tony F. Chan, Roland Glownsk, Gérard A. Meurant, Jacques Pèraux, and Olof B. Wdlund, edtors, Second Internatonal Symposum on Doman Decomposton Methods for Partal Dfferental Equatons, pages , Phladelpha, SIAM. [12] Maksymlan Dryja and Olof B. Wdlund. An addtve varant of the Schwarz alternatng method for the case of many subregons. Techncal Report 339, also Ultracomputer Note 131, Department of Computer Scence, Courant Insttute, New York Unversty, [13] Maksymlan Dryja and Olof B. Wdlund. Some doman decomposton algorthms for ellptc problems. In Iteratve Methods for Large Lnear Systems, pages , San Dego, Calforna, Academc Press. Proceedng of the Conference on Iteratve Methods for Large Lnear Systems held n Austn, Texas, October 19-21, 1988, to celebrate the sxty-ffth brthday of Davd M. Young, Jr. [14] Maksymlan Dryja and Olof B. Wdlund. Towards a unfed theory of doman decomposton algorthms for ellptc problems. In Tony F. Chan, Roland Glownsk, Jacques Péraux, and Olof B. Wdlund, edtors, Thrd Internatonal Symposum on Doman Decomposton Methods for Partal Dfferental Equatons, held n Houston, Texas, March 20-22, 1989, pages 3 21, Phladelpha, SIAM. [15] Maksymlan Dryja and Olof B. Wdlund. Schwarz methods of Neumann-Neumann type for three-dmensonal ellptc fnte element problems. Communcatons n Pure and Appled Mathematcs, 48: , [16] Ludwg Elsner. Comparsons of weak regular splttngs and multsplttng methods. Numersche Mathematk, 56: , [17] Roland W. Freund and Marls Hochbruck. On the use of two QMR algorthms for solvng sngular systems and applcatons n Markov Chan modellng. Numercal Lnear Algebra wth Applcatons, 1: , [18] Andreas Frommer and Danel B. Szyld. Weghted max norms, splttngs, and overlappng addtve Schwarz teratons. Numersche Mathematk, 83: , [19] Andreas Frommer and Danel B. Szyld. An algebrac convergence theory for restrcted addtve Schwarz methods usng weghted max norms. SIAM Journal on Numercal Analyss, 39: , [20] Mchael Grebel and Peter Oswald. On the abstract theory of addtve and multplcatve Schwarz algorthms. Numersche Mathematk, 70: , [21] Wolfgang Hackbusch. Iteratve Soluton of Large Sparse Systems of Equatons. Sprnger, New York, Berln, Hedelberg, [22] Roger A. Horn and Charles R. Johnson. Matrx Analyss. Cambrdge Unversty Press, Cambrdge, [23] Wllam J. Kammerer and M. Zuhar Nashed. On the convergence of the Conjugate Gradent method for sngular lnear operator equatons. SIAM Journal on Numercal Analyss, 9: , [24] Herbert B. Keller. On the soluton of sngular and semdefnte lnear systems by teraton. SIAM Journal on Numercal Analyss, 2: , [25] Axel Klawonn and Olof B. Wdlund. A doman decomposton method wth Lagrange multplers and nexact solvers for lnear elastcty. SIAM Journal on Scentfc Computng, 22: , [26] Young-Ju Lee, Jnbao Wu, Jnchao Xu, and Ludml Zkatanov. A sharp convergence estmate

19 Renhard Nabben and Danel B. Szyld 19 of the method of subspace correctons for sngular systems. Techncal Report AM259, Department of Mathematcs, Pennsylvana State Unversty, September [27] Young-Ju Lee, Jnbao Wu, Jnchao Xu, and Ludml Zkatanov. On the convergence of teratve methods for semdefnte lnear systems. SIAM Journal on Matrx Analyss and Applcatons, 2006, to appear. [28] Young-Ju Lee, Jnchao Xu, and Ludml Zkatanov. Successve subspace correcton method for sngular system of equatons. In Ismael Herrera, Davd E. Keyes, Olof B. Wdlund, and Robert Yates, edtors, Fourteenth Internatonal Conference on Doman Decomposton Methods, pages , Mexco, UNAM. Avalable also at [29] Ivo Marek and Danel B. Szyld. Comparson theorems for weak splttngs of bounded operators. Numersche Mathematk, 58: , [30] Ivo Marek and Danel B. Szyld. Comparson theorems for the convergence factor of teratve methods for sngular matrces. Lnear Algebra and ts Applcatons, 316:67 87, [31] Ivo Marek and Danel B. Szyld. Comparson of convergence of general statonary teratve methods for sngular matrces. SIAM Journal on Matrx Analyss and Applcatons 24:68 77, [32] Ivo Marek and Danel B. Szyld. Algebrac Schwarz methods for the numercal soluton of Markov chans. Lnear Albegra and ts Applcatons, 386:67 81, [33] Renhard Nabben. A note on comparson theorems of splttngs and multsplttngs of Hermtan postve defnte matrces. Lnear Algebra and ts Applcatons, 233:67 80, [34] Renhard Nabben. Comparsons between addtve and multplcatve Schwarz teratons n doman decomposton methods. Numersche Mathematk, 95: , [35] Renhard Nabben and Danel B. Szyld. Convergence theory of restrcted multplcatve Schwarz methods. SIAM Journal on Numercal Analyss, 40: , [36] James M. Ortega. Numercal Analyss, A Second Course. Academc Press, New York, Reprnted by SIAM, Phladelpha, Classcs n Appled Mathematcs, vol. 3. [37] J. M. Ortega and W. C. Rhenboldt. Iteratve Soluton of Nonlnear Equatons n Several Varables. Academc Press, San Dego, Reprnted by SIAM, Phladelpha, Classcs n Appled Mathematcs, vol. 30. [38] Alfo Quarteron and Alberto Vall. Doman Decomposton Methods for Partal Dfferental Equatons. Oxford Scence Publcatons, Clarendon Press, Oxford, [39] Lothar Rechel and Qang Ye. Breakdown-free GMRES for sngular systems. SIAM Journal on Matrx Analyss and Applcatons, 26: , [40] Marco Schntker. Ene algebrasche Konvergenztheore der Schwarz-Verfahen für symmetrsch postv defnte Matrzen, Examensarbet, Unverstät Belefeld, [41] Barry F. Smth, Petter E. Bjørstad, and Wllam D. Gropp. Doman Decomposton: Parallel Multlevel Methods for Ellptc Partal Dfferental Equatons. Cambrdge Unversty Press, Cambrdge, New York, Melbourne, [42] Wllam J. Stewart. Introducton to the Numercal Soluton of Markov Chans. Prnceton Unversty Press, Prnceton, New Jersey, [43] Danel B. Szyld. Equvalence of convergence condtons for teratve methods for sngular equatons. Numercal Lnear Algebra wth Applcatons, 1: , [44] Andrea Tosell and Olof Wdlund. Doman Decomposton Methods - Algorthms and Theory, volume 34 of Seres n Computatonal Mathematcs. Sprnger, Berln, Hedelberg, New York, [45] Rchard S. Varga. Matrx Iteratve Analyss. Prentce-Hall, Englewood Clffs, New Jersey, Second Edton, revsed and expanded, Sprnger, Berln, Hedelberg, New York, [46] Zbgnew I. Woźnck. Nonnegatve splttng theory. Japan Journal of Industral and Appled Mathematcs, 11: , [47] Jnchao Xu. Iteratve methods by space decomposton and subspace correcton. SIAM Revew, 34: , 1992.