Bulletin of the. Iranian Mathematical Society

Transcription

1 ISSN: X (Print) ISSN: (Online) Bulletin of the Iranian Matheatical Society Vol. 41 (2015), No. 4, pp Title: Convergence analysis of the global FOM and GMRES ethods for solving atrix equations AXB = C with SPD coefficients Author(s): M. Mohseni Moghada, A. Rivaz, A. Tajaddini and F. Saberi Movahed Published by Iranian Matheatical Society

2 Bull. Iranian Math. Soc. Vol. 41 (2015), No. 4, pp Online ISSN: CONVERGENCE ANALYSIS OF THE GLOBAL FOM AND GMRES METHODS FOR SOLVING MATRIX EQUATIONS AXB = C WITH SPD COEFFICIENTS M. MOHSENI MOGHADAM, A. RIVAZ, A. TAJADDINI AND F. SABERI MOVAHED (Counicated by Davod Khojasteh Salkuyeh) Abstract. In this paper, we study convergence behavior of the global FOM (Gl-FOM) and global GMRES (Gl-GMRES) ethods for solving the atrix equation AXB = C where A and B are syetric positive definite (SPD). We present soe new theoretical results of these ethods such as coputable exact expressions and upper bounds for the nor of the error and residual. In particular, the obtained upper bounds for the Gl-FOM ethod help us to predict the behavior of the Frobenius nor of the Gl-FOM residual. We also explore the worst-case convergence behavior of these ethods. Finally, soe nuerical experients are given to show the perforance of the theoretical results. Keywords: Convergence analysis; Global FOM; Global GMRES; Global Lanczos algorith; Worst-case behavior. MSC(2010): Priary: 65F10; Secondary: 15A Introduction In this paper, we consider the atrix equation (1.1) AXB = C, where A R n n, B R s s, C R n s and the atrices A and B are syetric positive definite (SPD). Different ethods are devoted to find the special solution structures of the atrix equation AXB = C such as diagonal, triangular, reflexive, syetric, centro-syetric, skew-syetric or least squares solution X; see [7, 8, 13, 18] and the references therein. During the last years, several projection ethods have been proposed to solve the atrix equations. The ain idea developed in these ethods is to Article electronically published on August 16, Received: 18 May 2014, Accepted: 27 June Corresponding author. 981 c 2015 Iranian Matheatical Society

3 Convergence analysis 982 construct suitable bases of the Krylov subspace and project the large proble into a saller one; see [9 11, 20, 21] for ore details. For exaples, Beik and Salkuyeh [2] have proposed two global Krylov subspace ethods for solving the following general coupled linear atrix equations (1.2) p A ij X j B ij = C i, i = 1,..., p. j=1 Furtherore, using the Schur copleent forula, Beik [1] has studied soe convergence results of the Gl-GMRES for solving the generalized Sylvester equations and the atrix equations (1.2). In the case where B is the identity atrix I s s, Jbilou et al. [14] have presented the properties of the Gl-FOM and Gl-GMRES ethods for solving a atrix equation AX = C. Then, using the product, Bouyouli et al. [4] have studied the convergence properties of the Gl-FOM and Gl-GMRES ethods. When A is a SPD atrix and B = I s s with s = 1, the Conjugate Gradient (CG) and the MINRES ethods are two well-known Krylov subspace approaches for solving the linear syste Ax = b. An overview of the convergence analysis of these ethods is given in [17]. Also, an interesting work was recently done by Bouyouli et al. [5]. Based on the relationship between the CG and Lanczos ethods, they have derived soe convergence results for the error and the residual of the CG ethod. In this paper, we discuss convergence analysis of the Gl-FOM and Gl- GMRES ethods for solving the atrix equation (1.1). Our ain interest is to understand the behavior of the error and residual of these ethods. These ethods use a global Lanczos-based algorith onto a atrix Krylov subspace. In addition, since these ethods are the global orthogonal residual and global inial residual ethods, we call the the G-OR-L and G-MR-L ethods, respectively. This paper is organized as follows. In section 2, we give soe preliinary tools. We also recall the atrix Krylov subspaces, then we give a brief description of the global Arnoldi algorith and its properties. In section 3, we present the G-OR-L and G-MR-L ethods and their atheatical properties for solving the atrix equation (1.1). We also prove soe convergence results of these ethods. In particular, we discuss the iportance of the spectral inforation of A and B on the convergence behavior of the Frobenius nor of the G-OR- L residual. In section 4, we investigate the worst-case convergence behavior of these ethods. For the special case AX = C where A is a diagonalizable atrix, we also prove the worst-case convergence behavior of the Gl-GMRES ethod. Furtherore, in coparison with the proof given in [3], our proof is shorter. Nuerical experients are presented in the last section.

4 983 Mohseni Moghada, Rivaz, Tajaddini and Saberi Movahed 2. Preliinaries We use the following notations. The notation R n s denotes the real vector space of n s atrices. e j denotes the jth colun of an identity atrix of a suitable diension. Let X, Y R n s. The vector vec(x) denotes the vector of R ns defined by vec(x) = [x T 1,..., x T s ] T where x i is the ith colun of X. The Frobenius inner product is defined by X, Y F = trace(y T X) where trace( ) denotes the trace of the square atrix Y T X. The associated nor is the Frobenius nor denoted by F. The notation X F Y eans that X, Y F = 0. The Kronecker product of atrices A R k l and B R n is defined by A B = [a ij B]. For this product, we have the following properties. For ore details we refer to [4]. (1) (A B)(C D) = (AC BD), provided that AC and BD exist. (2) (A B) 1 = A 1 B 1, provided that A 1 and B 1 exist. (3) (A B) T = A T B T. (4) vec(abc) = (C T A)vec(B), provided that ABC exists. (5) If A R k l, B R l and C R k, then trace(abc) = vec(a T ) T (I k B)vec(C). We give the following proposition whose proof is straightforward. Proposition 2.1. Assue that A R n n and B R s s are SPD. The ap, (A,B) : R n s R n s R defined as X, Y (A,B) = trace(y T AXB) is an inner product denoted by the (A, B)-inner product. Moreover, the associated nor of the (A, B)-inner product is X (A,B) = vec(x) B A, X R n s. Let E = [E 1, E 2,..., E p ] R n ps and F = [F 1, F 2,..., F l ] R n ls, where E i and F j are n s atrices. The atrices E T F and E T (A,B) F are defined by (E T F ) ij = E i, F j F and (E T (A,B) F ) ij = E i, F j (A,B), respectively. One can see that the product satisfies the following properties; see [4] for ore details. Proposition 2.2. Let E, F, M R n ps, L R p p, K R s s, y R s and α R. Then we have (1) E T (F (L I s )(I p K)) = (E T (F (I p K)))L. (2) E T (F (I s L)(y I p )) = (E T (F (I s L)))y. (3) E T (AE(I p B)) = E T (A,B) E = ẼT (B A)Ẽ, where Ẽ = [vec(e 1), vec(e 2 ),..., vec(e p )] The generalized atrix Krylov subspace. Let A R n n, B R s s be arbitrary atrices. The generalized atrix Krylov subspace associated with

5 Convergence analysis 984 the triplet (A, V, B) where V R n s is defined by: (2.1) GK i (A, V, B) = span{v, AV B,..., A i 1 V B i 1 }. The following definition of [16, p. 411] is a basic tool to describe the structure of GK i (A, V, B). Definition 2.3. Let p(x, y) = k i,j=0 c ijx i y j be a polynoial in two variables, x and y, with real coefficients c ij. p(c :D) is to be defined a atrix of the for p(c :D) = k i,j=0 c ij(c i D j ) where C R and D R n n. Reark 2.4. Suppose that K i (G, v) = span{v, Gv,..., G i 1 v} is the classical atrix Krylov subspace where v = vec(v ) and G = B T A. The ap T : GK i (A, V, B) K i (G, v) given by Z vec(z) is an isoorphis. So, GK i (A, V, B) is the set of all atrices Z such that vec(z) = p k (B T : A)v in which p k (x, y) = k j=0 c jx j y j and k i 1. To construct an orthonoral basis of GK k (A, V, B), we apply the global Arnoldi algorith which can be suarized in Algorith 1. 1: Set β = V F and V 1 = V/β. 2: for j = 1,..., k do 3: W j = AV j B 4: for i = 1,..., j do 5: h ij = W j, V i F 6: W j = W j h ij V i 7: end for 8: h j+1,j = W j F. If h j+1,j = 0 then stop. 9: V j+1 = W j/h j+1,j. 10: end for Let V k = [V 1,..., V k ]. It is easy to see that V T k V k = I k and V T k V k+1 = 0 k 1. In addition, we have AV k (I k B) = [AV 1 B,..., AV k B]. Fro Algorith 1, it follows that AV j B = j+1 i=1 h ijv i, for j = 1,..., k. Hence, we get 2 k AV k (I k B) = [ h i1 V i,..., h i V i ] + h k+1,k [0,..., V k+1 ]. So, it is clear that i=1 (2.2) AV k (I k B) = V k (H k I s ) + h k+1,k V k+1 (e T k I s ), i=1 where H k = [h ij ] is an upper Hessenberg atrix whose nonzero entries h ij are defined by the generalized global Arnoldi algorith.

6 985 Mohseni Moghada, Rivaz, Tajaddini and Saberi Movahed Using the relation (2.2), the part (3) of Proposition 2.2 iplies that H k = Ṽ T k (B A)Ṽk, where Ṽk = [vec(v 1 ),..., vec(v k )]. In the particular case where A, B are SPD, we state an iportant property of the atrix H k, which is easy to prove. Proposition 2.5. If A, B are SPD, then the atrix H k is SPD and tridiagonal. Moreover, all entries of the three diagonals of H k are positive. As a consequence of the above proposition, when A and B are SPD, then the atrices V 1, V 2,..., V k constructed by Algorith 1 satisfy the three-ter recurrence h j+1,j V j+1 = AV j B h jj V j h j 1,j V j 1. Therefore, the generalized global Arnoldi algorith is transfored to the following algorith. In fact, this algorith is a version of the global Lanczos algorith. We call this algorith the generalized global Lanczos (GGL) algorith. Algorith 1 The generalized global Lanczos (GGL) algorith 1: Set β = V F, V 1 = V/β, h 0,1 0 and V : for j = 1,..., k do 3: W j = AV jb h j 1,jV j 1 4: h jj = W j, V j F 5: W j = W j h jjv j 6: h j+1,j = W j F. If h j+1,j = 0 then stop. 7: V j+1 = W j /h j+1,j. 8: end for 3. Convergence analysis In this section, we present soe convergence results for the Gl-FOM and Gl- GMRES ethods for solving the atrix equation (1.1). Both of these ethods use the GGL algorith to construct an orthonoral basis of a generalized atrix Krylov subspace. In addition, since these ethods are the global orthogonal residual and global inial residual ethods, we call the the G-OR-L and G-MR-L ethods, respectively The G-OR-L ethod. Assue that X 0 R n s is an initial guess and R 0 = C AX 0 B is its corresponding residual. The G-OR-L ethod consists in generating approxiate solutions of the for X or = X 0 + Z with Z GK (A, R 0, B) such that (3.1) R or := C AX or B F GK (A, R 0, B), where = 1, 2,.... The atrix R or is called the th residual associated with X or.

7 Convergence analysis 986 Let {V 1,..., V } be the basis of GK (A, R 0, B) constructed by the GGL algorith. The approxiation X or can be written as (3.2) X or = X 0 + V (α or I s ), where V = [V 1,..., V ]. It is also easy to see that α or solves the linear syste (3.3) H α or = R 0 F e 1. Moreover, as H is SPD, α or is the unique solution of (3.3). Now, we show how to copute the Frobenius nor of the residual without coputing explicitly the residual. Proposition 3.1. Under the sae assuptions as in Proposition 2.5, the residual atrix R or satisfies the following relation (3.4) R or F = R 0 F i=1 h i+1,i det(h ). So, R or F = 0 if and only if h +1, = 0. Proof. We have R or = R 0 AV (I B)(α or I s ). So, using the relations (2.2) and (3.3), we obtain R or = R 0 V (H α or I s ) h +1, (e T α or )V +1 = h +1, (e T α or )V +1. Therefore, R or F = h +1, α or() where α or() is the last coponent of α or. On the other hand, fro (3.3) and the Craer rule, it follows that α or() = ( 1) +1 R 0 F 1 i=1 h i+1,i det(h ) Using Proposition 2.5, we have h i+1,i > 0, i = 1,..., 1. Hence R or F = 0 if and only if h +1, = 0. We study further the G-OR-L ethod by considering the properties of the error associated with X or, i.e., X X or where X is the solution of the atrix equation (1.1). Since the G-OR-L ethod is an orthogonal projection ethod onto the generalized atrix Krylov subspace GK i (A, R 0, B), we have the iniization property of the error of the G-OR-L ethod. It is not hard to prove the following theore. Theore 3.2. Assue that X is the solution of (1.1) and X 0 R n s. Then is the th approxiation of the G-OR-L ethod if and only if X or (A,B) = in X X (A,B). X X 0+GK (A,R 0,B).

8 987 Mohseni Moghada, Rivaz, Tajaddini and Saberi Movahed By considering the iniization property of the (A, B) nor of the error of the G-OR-L ethod, we will try to establish ore properties of the error. First, we give two expressions for X X or (A,B). In the following, we assue that D 0 = X X 0 and D = X X or Theore 3.3. The error X X or relation. associated with X or satisfies the following (16-i) (A,B) = R or F V T +1 (A 1,B 1 ) V +1. Also, suppose that K +1 = [R 0,..., A R 0 B ]. If {R 0,..., A R 0 B } is a linearly independent set, then (16-ii) 2 (A,B) = 1 e T 1 (KT +1 (A 1,B 1 ) K +1 ) 1 e 1. Proof. (i) Since D = A 1 R or B 1, we have 2 (A,B) = vec(d ) T (B A)vec(D ) = vec(r or ) T (B 1 A 1 )vec(r or ) = R or 2 F vec(v +1 ) T (B 1 A 1 )vec(v +1 ) = R or 2 F (V T +1 (A 1,B 1 ) V +1 ). (ii) We have X or = X 0 + K (α or I s ) such that R or B)(α or I s ), where K = [R 0,..., A 1 R 0 B 1 ]. So, 2 (A,B) = trace ((X X or ) T A(X X or )B) = R 0 AK (I = D 0 K (α or I s ), R or F = D 0, R or F = D 0, R 0 AK (I B)(α or I s ) F. Fro [12, Theore ], the atrix K T (A,B) K is nonsingular. Now, the orthogonality relation (3.1) yields α or = (K T (A,B) K ) 1 (K T R 0 ). In addition, we have R 0 = AD 0 B. Therefore, 2 (A,B) = DT 0 (A,B) D 0 (D0 T (A,B) K )α or = D T 0 (A,B) D 0 (D0 T (A,B) K )(K T (A,B) K ) 1 (K T (A,B) D 0 ) ( [ ] D0 = T (A,B) D 0 D0 T / (A,B) K K T (A,B) D 0 K T K (A,B) K T (A,B) K ), where M/F is the Schur copleent of F in M. As K +1 = [AD 0 B, AK (I B)] and A 1 K +1 (I +1 B 1 ) = [D 0, K ],

9 Convergence analysis 988 we get ( A 1 K +1 (I +1 B 1 ) ) [ T D T K+1 = 0 (A,B) D 0 D0 T (A,B) K K T (A,B) D 0 K T (A,B) K On the other hand ( A 1 K +1 (I +1 B 1 ) ) T K+1 = K T +1 (A 1,B 1 ) K +1, so we obtain 2 (A,B) = ( K T +1 (A 1,B 1 ) K +1 / K T (A,B) K ) = det(kt +1 (A 1,B 1 ) K +1 ) det(k T (A,B) K ) 1 = e T 1 (KT +1. (A 1,B 1 ) K +1 ) 1 e 1 Reark 3.4. (1) Proposition 3.1 together with the relation (16-i) follows that h +1, = 0 if and only if (A,B) = 0. (2) Fro [12, Theore ], the atrix K+1 T (A 1,B 1 ) K +1 is SPD, so e T 1 (K+1 T (A 1,B 1 ) K +1 ) 1 e 1 > 0. Now, we introduce soe upper bounds for (A,B). First, we state the following relations: (B A) = λ in (B A) = 1 λ in (A)λ in (B) = A 1 2 B 1 2, B A 2 = λ ax (B A) = λ ax (A)λ ax (B) = A 2 B 2. Therefore, κ(b A) = κ(a)κ(b), where κ(z) = Z 2 Z 1 2. In the following, the expression UB is used to denote Upper Bound. Theore 3.5. The error X X or (UB.1) (A,B) R or F ]. associated with X or satisfies the relations κ(a)κ(b). A 2 B 2 (UB.2) (UB.3) (A,B) 2 (A,B) Ror 2 R or F κ(a)κ(b) + 1. V+1 T (A,B) V +1 κ(a)κ(b) F ( κ(a)κ(b) + 1 A 2 B 2 ). (UB.4) where γ = (A,B) γ R or F, κ(a)κ(b) R 0 F A 2 B 2 + α or 2.

10 989 Mohseni Moghada, Rivaz, Tajaddini and Saberi Movahed Proof. (1): Let G = B A and v = vec(v +1 ). If we apply the Courant-Fisher theore [12, Theore ], then v T G 1 v 1 λ in (A)λ in (B) = κ(a)κ(b). A 2 B 2 The relation (16-i) together with the above relation follows the inequality (UB.1). (2): Since v T G 1 v = V T +1 (A 1,B 1 ) V +1 and v T Gv = V T +1 (A,B) V +1, the Kantorovich inequality [12, Theore ] iplies that (3.5) V T +1 (A 1,B 1 ) V (V T +1 (A,B) V +1 ) (κ(a)κ(b) + 1) 2. κ(a)κ(b) Multiplying both sides of the relation (3.5) by R or 2 F follows the inequality (UB.2). On the other hand, using the Courant-Fisher theore, we obtain which follows the inequality (UB.3). (3): We have (3.6) and (3.7) 1 v T Gv 1 λ in (G) = κ(a)κ(b), A 2 B 2 2 (A,B) = Gvec(X X or ), vec(x X or ), C AX or B 2 F = A(X X or )B 2 F = vec(a(x X or )B) 2 2 = G 2 vec(x X or ), vec(x X or ). Let y = vec(x X or ) and z = y/ y 2. Using a result given in [15], we get Gz, z 2 G 2 z, z, or equivalently, Gy, y 2 y 2 2 G 2 y, y. Since vec(x X or ) 2 = F, it follows fro (3.6) and (3.7) that (A,B) F C AX or B F. Fro the relation X X or = D 0 V (α or I s ), we have F D 0 F + V (α or I s ) F. Therefore, as V (α or I s ) F = α or 2 and D 0 F = vec(d 0 ) 2 = (B A) 1 vec(r 0 ) 2 A 1 2 B 1 2 vec(r 0 ) 2, the proof copletes. It can be shown that the (A, B) nor of the error at each step of the G-OR- L ethod satisfies X X or (A,B) X X 0 (A,B). However, the Frobenius nor of the residual at each step of the G-OR-L ethod ay oscillate. In the next result, we show an upper bound for R or F.

11 Convergence analysis 990 Theore 3.6. Assue that R or is the G-OR-L residual obtained at step. Then (3.8) F in{α 1 R 0 F, α 2 R 0 F }, where α 1 = A κ(a)κ(b) and α 2 = 2 B 2 R or (V T 1 (A,B)V 1 )κ(a)κ(b) ( κ(a)κ(b)+1 2 Proof. The Courant-Fisher theore together with the relation (16-i) iplies that R or 2 F 2 (A,B) A 2 B. 2 On the other hand, siilar to Theore 3.5, we obtain κ(a)κ(b) X X 0 (A,B) R 0 F, A 2 B 2 X X 0 (A,B) Therefore, using the above relations, we have R 0 F κ(a)κ(b) V1 T (A,B) V 1 κ(a)κ(b) X X 0 2 (A,B) 2 (A,B) κ(a)κ(b) R F R or 2 F, A 2 B 2 A 2 B 2 and, X X 0 2 (A,B) 2 (A,B) (κ(a)κ(b) + 1)2 R 0 2 F 4(V1 T (A,B) V 1 )κ(a)κ(b) Ror 2 F. A 2 B 2 Now, since X X 0 2 (A,B) 2 (A,B) 0, the results follow. Reark 3.7. Since R or F in{α 1 R 0 F, α 2 R 0 F }, we expect that the Frobenius nor of the residuals will not oscillate too uch when α 1 and α 2 are sall. On the other hand, it is easy to see that α 2 (κ(a)κ(b) + 1)/2, so we have R or F in{ κ(a)κ(b) R 0 F, ( κ(a)κ(b) ) R 0 F }. Hence, we conclude that when κ(a) = 1 and κ(b) = 1, then R or F R 0 F. We also consider the especial case where κ(a) and κ(b) are close to 1. We will coe back to these points in the nuerical illustrations section. ) The G-MR-L ethod. Let X 0 R n s be an initial guess and R 0 = C AX 0 B be its corresponding residual. The G-MR-L ethod consists in generating approxiate solutions of the for X r = X 0 + Z with Z GK (A, R 0, B) such that (3.9) R r := C AX r B F AGK (A, R 0, B)B,

12 991 Mohseni Moghada, Rivaz, Tajaddini and Saberi Movahed where = 1, 2,.... The atrix R r X r. is called the th residual associated with Proposition 3.8. Assue that {V 1,..., V } is the basis of GK (A, R 0, B) constructed by the GGL algorith. The approxiation X r can be written as (3.10) X r where α r = X 0 + V (α r I s ), is the unique solution of the linear syste (3.11) (H 2 + h 2 +1,e e T )α r = R 0 F H e 1. Proof. We have X r B)(α r I s ) where α r = X 0 + V (α r relation (3.9) yields (W T W )α r (2.2) and Proposition 2.2, we obtain I s ) such that R r = R 0 AV (I R. Let W = AV (I B). The orthogonality = W T R 0. Since R 0 = R 0 F V 1, using W T W = H 2 + h 2 +1,e e T, and W T R 0 = R 0 F H e 1. As H 2 + h 2 +1,e e T is SPD, the solution of (3.11) exists and is unique. Proposition 3.9. The residual atrix R r satisfies the following relation ( (3.12) R r = h +1, α r() h +1, V (H 1 e I s ) V +1 ), where α r() is the last coponent of α r. In addition, we get h 2 +1, H 1 e (3.13) R r F = h +1, α r() So, R r F = 0 if and only if h +1, = 0. Proof. We have R r = R 0 AV (I B)(α r I s ). Using the relations (2.2) and (3.11), it follows that R r = R 0 V (H α r I s ) h +1, (e T α r )V +1 = R 0 R 0 F V (e 1 I s ) + h 2 +1,(e T α r )V (H 1 e I s ) h +1, (e T α r )V +1 ( = h +1, (e T α r ) h +1, V (H 1 e I s ) V +1 ). Therefore, R r F = (R r ) T R r = h +1, α r() Fro (3.11) and the Craer rule, we obtain α r() = ( 1) +1 R 0 F 1 i=1 h i+1,i h 2 +1, H 1 det(h + h 2 +1, H 1 e e T ). e Using Proposition 2.5, we have h i+1,i > 0, i = 1,..., 1. Hence R r F = 0 if and only if h +1, = 0.

13 Convergence analysis 992 The next result shows that the Frobenius nor of the residual has the iniization property. Theore Let X 0 R n s be an initial guess and R 0 = C AX 0 B be its corresponding residual. Then X r is the th approxiation of the G-MR-L ethod if and only if C AX r B F = in C AXB F. X X 0 +GK (A,R 0,B) Proof. The proof is siilar to that of Theore 3.2 and is oitted. 4. The worst-case behavior of the G-OR-L and G-MR-L ethods The worst-case convergence behavior of any well known Krylov subspace ethods for noral atrices is described by the in-ax approxiation proble on the discrete set of the atrix eigenvalues [17], in p P ax p (λ). λ σ(a) We investigate the worst-case convergence behavior of the G-OR-L and G- MR-L ethods The worst-case behavior of the G-OR-L ethod. First, we prove the following lea. Lea 4.1. Let X X 0 + GK (A, R 0, B) where X 0 R n s. If X is the solution of (1.1), then vec(x X) = p (B : A)vec(D 0 ) where p (x, y) = i=0 c ix i y i with p (0, 0) = 1. Proof. We have X = X i=1 d ia i 1 R 0 B i 1 where d i R, i = 1,..., 1. As R 0 = AD 0 B, we get vec(x X) = i=0 c i(b A) i vec(d 0 ) such that c 0 = 1 and c i = d i, i = 1,...,. Now, if p (x, y) = i=0 c ix i y i, then the result follows. Let A = V T DV and B = U T ΛU with V T V = I n and U T U = I s, where D and Λ are the diagonal atrices whose eleents are the eigenvalues µ 1,..., µ n of A and the eigenvalues λ 1,..., λ s of B, respectively. If V = [v 1,..., v n ] and U = [u 1,..., u s ], then vec(d 0 ) can be written as vec(d 0 ) = s n i=1 j=1 a ij(u i v j ) where a ij R. Fro [16, Theore 1, p. 411], we obtain (4.1) p (B : A)vec(D 0 ) = s i=1 j=1 n a ij p (λ i, µ j )(u i v j ).

14 993 Mohseni Moghada, Rivaz, Tajaddini and Saberi Movahed Theore 4.2. Let X be the solution of the atrix equation (1.1). If X or the th approxiation of the G-OR-L ethod, then (4.2) (A,B) X X 0 (A,B) in p P ax λ σ(b) p (0,0)=1 µ σ(a) p (λ, µ), where P denotes the set of polynoials in two variables x, y of the for p (x, y) = i=0 c ix i y i with value one at the origin. Proof. Using Theore 3.2 and Lea 4.1, we obtain (A,B) = in p (B : A)vec(D 0 ) B A. p P p (0,0)=1 Let, be the standard inner product in R ns. Fro (4.1), it is straightforward to verify that p (B : A)vec(D 0 ) 2 B A = p (B : A)vec(D 0 ), (B A)p (B : A)vec(D 0 ) = p (B : A)vec(D 0 ), p (B : A)(B A)vec(D 0 ) s n = a 2 ijp 2 (λ i, µ j )λ i µ j. i=1 j=1 As vec(d 0 ) 2 (B A) = s i=1 n j=1 a2 ij λ iµ j, we get p (B : A)vec(D 0 ) B A ax p (λ i, µ j ) vec(d 0 ) B A i,j = ax λ σ(b) µ σ(a) p (λ, µ) vec(d 0 ) B A. is Finally, since vec(d 0 ) B A = X X 0 (A,B), the proof copletes The worst-case behavior of the G-MR-L ethod. In this section, we study the worst-case behavior of the G-MR-L ethod. The ain result of this section is derived with the assuptions that the atrices A and B are SPD. Theore 4.3. If X r (4.3) is the th approxiation of the G-MR-L ethod, then C AX r B F C AX 0 B F in p P ax λ σ(b) p (0,0)=1 µ σ(a) p (λ, µ), where P denotes the set of polynoials in two variables x, y of the for p (x, y) = i=0 c ix i y i with value one at the origin.

15 Convergence analysis 994 Proof. Let X X 0 + GK (A, R 0, B) and R = C AXB. Then vec(r) = vec(r 0 ) c i (B A) i vec(r 0 ) i=1 = (U V ) T ( I c i (Λ D) i) (U V )vec(r 0 ) i=1 = (U V ) T p (Λ : D)(U V )vec(r 0 ), which p (x, y) = 1 + i=1 c ix i y i where c i R, i = 1,...,. Hence, we have C AXB F = vec(r) 2 (U V ) T 2 p (Λ : D) 2 (U V ) 2 vec(r 0 ) 2. Now, using [16, Theore 1, p. 411], it follows that C AXB F ax λ σ(b) µ σ(a) Together with Theore 3.10, we conclude p (λ, µ) R 0 F. C AX r B F = in C AXB F X X 0 +GK (A,R 0,B) in p P ax λ σ(b) p (0,0)=1 µ σ(a) p (λ, µ) R 0 F. Reark 4.4. In the process of proving Theore 4.3, we also consider the following cases: (1): A and B are diagonalizable. When A and B are diagonalizable, i.e., A = V 1 DV and B = U 1 ΛU with D = diag(µ 1,..., µ n ) and Λ = diag(λ 1,..., λ s ), then we obtain the following convergence bound (4.4) C AX r B F C AX 0 B F κ(v )κ(u) in p P ax λ σ(b) p (0,0)=1 µ σ(a) p (λ, µ). Moreover, if we assue that B = I s, then we have σ(b) = {1} and κ(u) = 1. In this case, we deduce that Theore 4.3 coincides with the result of the Gl- GMRES given in [3, Theore 5]. (2): A and B are noral atrices. If A and B are noral atrices, then Theore 4.3 holds. In addition, if B = I s, then Theore 4.3 reduces to Theore 7 in [3]. Reark 4.5. Interesting related work was recently done by Bellalij et al. [3] for solving a atrix equation AX = C. They presented the worst-case convergence behavior of the Gl-GMRES ethod for diagonalizable atrices. Although all presented results in [3] are correct, as it was discussed in Reark 4.4, when B =

16 995 Mohseni Moghada, Rivaz, Tajaddini and Saberi Movahed I s, Theore 4.3 leads to Theores 5 and 7 in [3]. Furtherore, a coparison between our proof process with that of [3] shows that Theore 4.3 presents an easier proof for the worst-case convergence behavior of the Gl-GMRES ethod. 5. Nuerical illustrations The G-OR-L and G-MR-L ethods are suarized in the following algorith. It is iportant to notice that when the iterate becoes large, the coputational requireents increase. To reedy this proble, we use the algorith in a restarted ode, i.e., X 0 = X or (X 0 = X r ) where X or (X r ) is the last coputed approxiate solution obtained with the G-OR-L (G-MR-L) ethod. Algorith 2 (The G-OR-L() and G-MR-L() ethods) 1: Choose X 0. 2: Copute R 0 = C AX 0 B and set V 1 = R 0 / R 0 F. Use the generalized global Lanczos algorith to copute the basis {V 1, V 2,..., V } of GK (A, R 0, B) and the atrix H. 3: G-OR-L: copute the approxiate solution X or = X 0 + V (α or I s) where α or solves the equation (3.3). 4: If the approxiation X or is suitable then stop, else set X 0 = X or and goto 2. 5: G-MR-L: copute the approxiate solution X r = X 0 + V (α r solves the equation (3.11). 6: If the approxiation X r is suitable then stop, else set X 0 = X r and goto 2. I s ) where α r In this section, we present soe nuerical exaples to illustrate the quality of the presented theoretical results. To do so, we use the atrices A 1 (n) = tridiag( 1, 10, 1) and B 1 (s) = tridiag( 1, 10, 1), A 2 (n) = and B 2(s) = where n and s are the order of atrices A and B, respectively , Also, we use soe atrices fro the Matrix-Market website at MatrixMarket. The entries of the atrix C in all the exaples are rando values uniforly distributed on [0, 1] and X 0 = 0. All nuerical tests run in Matlab. Exaple 5.1. In this exaple, we seek two goals. First, we exaine the influence of the condition nubers of the atrices A and B on the speed of convergence of the G-OR-L() and G-MR-L() ethods. Then, we study the behavior of the Frobenius nor of the residual of the G-OR-L() and G- MR-L() ethods. Also, as stopping criteria, we use C AX or B F 10 6 and C AX r B F 10 6 for the G-OR-L() and G-MR-L() ethods, respectively.

17 Convergence analysis 996 Tables 1 and 2 report the final Frobenius nor of the residuals, the nuber of iterations and the CPU tie required to convergence for each ethod. We note that the nuber of iterations refers to the nuber of restarts. As shown in Tables 1 and 2, we see that the nuber of iterations increases when A or B have very large condition nubers. However, if the condition nubers of A and B are sall, then the nuber of iterations decreases. Table 1. Effectiveness of the G-OR-L and G-MR-L ethods: = 3. (A, B) G-OR-L(3) G-MR-L(3) n = 2000, s = 100 Residuals e e-008 (A 1, B 1) κ(a) = 1.5 Nuber of iterations 6 6 κ(b) = CPU tie (sec) n = 1000, s = 500 Residuals e e-007 (A 2, B 2) κ(a) = 3 Nuber of iterations κ(b) = 3 CPU tie (sec) Table 2. Effectiveness of the G-OR-L and G-MR-L ethods: = 20. (A, B) G-OR-L(20) G-MR-L(20) n = 900, s = 10 Residuals e e-007 (GR3030, B 1) κ(a) = 3.8e + 02 Nuber of iterations κ(b) = CPU tie (sec) n = 100, s = 10 Residuals e e-007 (NOS4, B 1) κ(a) = 2.7e + 03 Nuber of iterations κ(b) = CPU tie (sec) n = 10, s = 468 Residuals e e-007 (A 1, NOS5) κ(a) = Nuber of iterations κ(b) = 2.9e + 04 CPU tie (sec) According to the aforeentioned stopping criteria, we investigate further this exaple by considering the convergence behavior of the Frobenius nor of the residual at each iteration of the G-OR-L() and G-MR-L() ethods, respectively. The G-OR-L ethod. In Figures 1 5, the left plots show the Frobenius nor of the G-OR-L residuals and the right plots show the values of α 1 and α 2. As shown in Figures 1 3, we see that α 1 and α 2 are sall. Moreover, as it is expected fro Reark 3.7, we observe that the Frobenius nor of the residuals associated with the atrices (A 1 (2000), B 1 (100)), (A 2 (1000), B 2 (500)) and (GR3030, B 1 (10)) decreases. Figures associated with the atrices (NOS4, B 1 (10)) and (B 1 (10), NOS5), especially (B 1 (10), NOS5), illustrate that α 1 and α 2 are large. Therefore, we expect that the rate of changes of the Frobenius nor of the residuals

18 997 Mohseni Moghada, Rivaz, Tajaddini and Saberi Movahed associated with these atrices increases. As we see in Figures 4 and 5, the Frobenius nor of the residuals associated with the atrices (NOS4, B 1 (10)) and (B 1 (10), NOS5), especially (B 1 (10), NOS5), oscillates too uch. Figure 1. Illustration of R or F associated with (A 1 (2000), B 1 (100)). Figure 2. Illustration of R or F associated with (A 2 (1000), B 2 (500)). Figure 3. Illustration of R or F associated with (GR3030, B 1 (10)). Figure 4. Illustration of R or F associated with (NOS4, B 1 (10)). The G-MR-L ethod. It can be shown that the Frobenius nor of the residual at each step of the G-MR-L ethod satisfies R r F R 0 F.

19 Convergence analysis 998 Figure 5. Illustration of R or F associated with (B 1 (10), NOS5). Therefore, since we use the G-MR-L ethod in a restarted ode, we expect a onotonically decreasing behavior of the Frobenius nor of the residual. The plots of Figure 6 show that the Frobenius nor of the residual is onotonically decreasing. Figure 6. Illustration of R r F associated with the atrices (A 1 (2000), B 1 (100)), (A 2 (1000), B 2 (500)), (GR3030, B 1 (10)), (NOS4, B 1 (10)) and (B 1 (10), NOS5). We suarize what we have stated so far. As we have seen, the Frobenius nor of the residual of the G-OR-L() ay oscillate. In contrast, the Frobenius nor of the residual of the G-MR-L() shows a onotonically decreasing behavior. Therefore, this contrast can be used as one of the reasons for the observed difference in the speed of convergence of the G-OR-L() and G-MR-L() ethods. Exaple 5.2. In this exaple, we study the behavior of the (A, B) nor of the error of the G-OR-L() ethod. The test atrices are the sae as the previous exaple. Also, the iterations are stopped when (A,B) In Table 3, we report the final values of the (A, B) nor of the errors and the final upper bounds derived in Theore 3.5. The obtained results indicate that these upper bounds are noticeable. However, the upper bound (UB.2) is better than other upper bounds. Finally, Figure 7 shows the values of the (A, B) nor of the errors and the upper bound (UB.2) together with the nuber of iterations.

20 999 Mohseni Moghada, Rivaz, Tajaddini and Saberi Movahed Table 3. The final values of (A,B) and the quality of the upper bounds (UB.1) (UB.4). (A, B) (A,B) (UB.1) (UB.2) (UB.3) (UB.4) (A 1(2000), B 1(100)) e e e e e-007 (A 2(1000), B 2(500)) e e e e e-006 (GR3030, B 1(10)) e e e e e-006 (NOS4, B 1(10)) e e e e e-005 (B 1(10), NOS5) e e e e e-006 Figure 7. Illustration of (A,B) and the values of the upper bound (UB.2). 6. Conclusion and further works We have considered the convergence behavior of the Gl-FOM and Gl-GMRES ethods for solving the atrix equation AXB = C where A and B are SPD. More precisely, soe new theoretical results of these ethods, such as coputable exact expressions, upper bounds for the nor of the error and residual and the worst-case convergence behavior of these ethods, have been established. In particular, our upper bounds for the nor of the Gl-FOM error depend on the condition nubers of the atrices A and B and the inforation generated by the Gl-FOM ethod. In the nuerical test section, we have explored the convergence behavior of these ethods. The nuerical results show the efficiency of the theoretical results. The relationship between the nor of the Gl-GMRES residual and the spectral inforation of A and B and the inforation generated by the Gl-GMRES ethod can be the subject of further investigations. Furtherore, the theoretical results, presented in this paper, can be used for the Gl-FOM and Gl-GMRES ethods in order to solve the generalized Sylvester equations with SPD coefficients.

21 Convergence analysis 1000 References [1] F. P. A. Beik, Theoretical results on the global GMRES ethod for solving generalized Sylvester atrix equations, Bull. Iranian Math. Soc. 40 (2014), no. 5, [2] F. P. A. Beik and D. K. Salkuyeh, On the global Krylov subspace ethods for solving general coupled atrix equation, Coput. Math. Appl. 62 (2011), no. 12, [3] M. Bellalij, K. Jbilou and H. Sadok, New convergence results on the global GMRES ethod for diagonalizable atrices, J. Coput. Appl. Math. 219 (2008), no. 2, [4] R. Bouyouli, K. Jbilou, R. Sadaka and H. Sadok, Convergence properties of soe block Krylov subspace ethods for ultiple linear systes, J. Coput. Appl. Math. 196 (2006), no. 2, [5] R. Bouyouli, G. Meurant, L. Soch and H. Sadok, New results on the convergence of the conjugate gradient ethod, Nuer. Linear Algebra Appl. 16 (2009), no. 3, [6] J. B. Conway, A Course in Functional Analysis, Springer-Verlag, New York, [7] D. Cvetković-Iliíc, The reflexive solutions of the atrix equations AXB = C. Coput. Math. Appl. 51 (2006), no. 6-7, [8] H. Dai, On the syetric solutions of linear atrix equations, Linear Algebra Appl. 131 (1990) 1 7. [9] M. Heyouni, The global Hessenberg and CMRH ethods for linear systes with ultiple right-hand sides, Nuer. Algoriths 26 (2001), no. 4, [10] M. Heyouni, Extended Arnoldi ethods for large low-rank Sylvester atrix equations, Appl. Nuer. Math. 60 (2010), no. 11, [11] M. Heyouni and A. Essai, Matrix Krylov subspace ethods for linear systes with ultiple right-hand sides, Nuer. Algoriths 40 (2005), no. 2, [12] R. A. Horn and C. R. Johnson, Matrix Analysis, Cabridge University Press, Cabridge, [13] G. X. Huang, F. Yin and K. Guo, An iterative ethod for the skew-syetric solution and the optial approxiate solution of the atrix equation AXB = C, J. Coput. Appl. Math. 212 (2008), no. 2, [14] K. Jbilou, A. Messaoudi and H. Sadok, Global FOM and GMRES algoriths for atrix equations, Appl. Nuer. Math. 31 (1999), no. 1, [15] F. Kittaneh, Notes on soe inequalities for Hilbert space operators, Publ. Res. Inst. Math. Sci. 24 (1988), no. 2, [16] P. Lancaster and M. Tisenetsky, The Theory of Matrices with Applications, Acadeic Press, New York, [17] J. Liesen and P. Tichý, Convergence analysis of Krylov subspace ethods, GAMM Mitt. Ges. Angew. Math. Mech. 27 (2004), no. 2, [18] Y. A. Peng, X. Hu and L. Zhang, An iteration ethod for the syetric solutions and the optial approxiation solution of the atrix equation AXB = C, Appl. Math. Coput. 160 (2005), no. 3, [19] Y. Saad, Iterative Methods for Sparse Linear Systes, PWS Press, New York, [20] K. Zhang and C. Gu, Flexible global generalized Hessenberg ethods for linear systes with ultiple right-hand sides, J. Coput. Appl. Math. 263 (2014) [21] K. Zhang and C. Gu, A global transpose-free ethod with quasi-inial residual strategy for the Sylvester equations, Filoat 27 (2013), no. 8, (Mahood Mohseni Moghada) Departent of Matheatics and Coputer Science, Shahid Bahonar University of Keran, P.O.Box , Keran, Iran E-ail address: ohseni@uk.ac.ir

22 1001 Mohseni Moghada, Rivaz, Tajaddini and Saberi Movahed (Azi Rivaz) Departent of Matheatics and Coputer Science, Shahid Bahonar University of Keran, P.O.Box Keran, Iran E-ail address: (Azita Tajaddini) Departent of Matheatics and Coputer Science, Shahid Bahonar University of Keran, P.O.Box , Keran, Iran E-ail address: (Farid Saberi Movahed) Departent of Matheatics and Coputer Science, Shahid Bahonar University of Keran, P.O.Box , Keran, Iran E-ail address: