PROJECTED GMRES AND ITS VARIANTS

Transcription

1 PROJECTED GMRES AND ITS VARIANTS Reinaldo Astudillo Brígida Molina Centro de Cálculo Científico y Tecnológico (CCCT), Facultad de Ciencias, Universidad Central de Venezuela (UCV), Ciudad Universitaria, Av. Los Estadios, Los Chaguaramos, Caracas-Venezuela. Abstract: In this work, we propose a new Krylov iterative method to solve systems of linear equations. This method is a variant of the well-known GMRES and is based on modifications over the constraints imposed on the residual vector, i.e., this vector is projected in another subspace and impose the constraints over this projection, because of this, we called the method: Projected GMRES (PRGMRES). Additionally, we develope two versions of PRGMRES: the PRGMRES with Biorthogonalization (BPRGMRES) and the Inexact PRGMRES (IPRGMRES). Experimental results are presented to show the good performances of the new methods, compared to FOM(m) and GMRES(m). Keywords: restarted GMRES, Krylov Subspace methods, Petrov-Galerkin conditions, unsymmetric linear systems. 1. INTRODUCTION In a variety of engineering and scientific applications we need to solve different systems of differential equations and these are solved numerically discretizing by mean of finite differences or finite element methods. The process of discretization, in general, leads to a linear system of the form: Ax = b (1) where A R n n is a sparse unsymmetric and nonsingular matrix, b R n the right hand side, and x R n is the unknown vector.

2 Due to computational and memory costs, compute x by factorization methods like: LU, QR, etc. can be very expensive for large values of n, besides these methods can be numerically unstable (see [1] and [2]). In this paper we will present a brief overview of general projection methods including the Krylov methods. At the same time, in section 3 we will discuss the modifications to restriction over general projection methods and how to produce the projected Krylov methods. We also present in section 4, preliminary numerical experiments and, finally, in section 5 we expose concluding remarks. 2. PROJECTION METHODS Definition: A projection method to solve (1) onto the subspace K (search subspace) and orthogonal to L (subspace of constraints) is a process which finds an approximate solution x by imposing the conditions that x belong to K and that the new residual vector is to be orthogonal to L (Petrov- Galerkin conditions). This can be written as : x K (2) b A x L. Let V =[v 1,...,v m ],an m matrix whose column-vectors form a basis of K and, similarly, W = [w 1,...,w m ] a n m matrix whose column-vectors form a basis of L, thus a prototype of projection method algorithm can be described as (see [3] and [4]): Algorithm 1 Prototype Projection Method 1: while no convergence do 2: choose V =[v 1,...,v m ] and W =[w 1,...,w m ] for K and L. 3: r b + A x 4: y (W T AV ) 1 W T r 5: x x 0 +Vy 6: end while 2.1 Krylov Methods The Krylov methods are projection methods where the search subspace K is the Krylov subspace: K m K m (A,r 0 )=Spanr 0,Ar 0,,A 2 r 0,...,,A m 1 r 0 } GMRES [5] is a Krylov method that computes an approximate solution x m at step k by doing an oblique projection onto the Krylov subspace K m (A,r 0 ) of size m, it means: x k K m (3) b Ax k AK m. Through the Arnoldi process [6], can be obtained a matrix V m =[v 1,...,v m ] whose columns are an orthonormal basis of K m (A,r 0 ) and an upper Hessenberg matrix H m.

3 These matrices satisfy the well-known relations given by: Then, the equation (3) can be written as: AV m = V m H m + h m+1,m v m+1 e T m = V m+1 H m, Vm T AV m = H m. x m = x 0 +V m y m where y m = arg min y R βe(m+1) m 1 H m y 2 (4) with β = r 0 2. FOM [7] on the other hand, is an orthogonal projection method where the equation (4) has been replaced by H m y m = βe (m) 1 while the rest of the algorithm remains unchanged. 3. A NEW PROTOTYPE PROJECTION METHOD If a subspace Z of dimension j contains the residual vector b A x, the expression (2) remain unchanged if we write: x k K T j Z (b Ax (5) k) L where T j Z is a orthogonal projector over Z. We propose to solve: x k K T j Z (b Ax (6) k) L. The main difference between the expressions (5) and (6) is T j Z which is a projector over a subspace Z where the residual vector b A x is not necessarily contained into this subspace. This is a generalization of (5). 3.1 Projected GMRES (PRGMRES) From (6), we choose K = K m (A,r k ), L = AK m (A,r k ) (such as GMRES) and Z = K m+1 (A,r k 1 ) (the previous Krylov subspace), which can be written as: x k = x 0 +V m y m (AV m ) T Z m+1 Zm+1 T r k = (7) 0 where V m is a matrix whose column-vectors represent the orthogonal Krylov basis, Z m+1 the previous Krylov basis and x 0 is an initial guess to the solution. Finally, using some algebraic manipulations, we rewrite (7) as: x k = x 0 +V m y m y m = min y R m Zm+1 T (r (8) 0 V m+1 H m y) 2 and together with the conditions (8), the PRGMRES algorithm is now described as:

4 Algorithm 2 PRGMRES(m) with Arnoldi modificated 1: Choose Z m+1 a orthogonal basis 2: r 0 b Ax 0, β r 0 2 y v 1 r 0 /β 3: for j 1,...,m do 4: w j Av j 5: for j 1,..., j do 6: h ij < w j,v i > 7: w j w j h ij v i 8: end for 9: h j+1, j w j 2 10: if h j+1, j = 0 then 11: Stop 12: end if 13: v j+1 w j /h j+1, j 14: end for 15: y m min y Zm+1 T (r 0 V m+1 H m y) 2 16: x m x 0 +V m y m 17: Z m+1 V m+1 18: x 0 x m go to PRGMRES(m) with biorthogonalization (BPRGMRES) In the previous algorithm 2, we have to solve: y m = min y R m Z T m+1 (r 0 V m+1 H m y) 2. An option is to make a process of biorthogonalization between the basis Z m+1 and V m+1 such as: < vi,z j > 0 if i = j < v i,z j >= 0 if i j then the expressions (8) can be written like: where D m+1 is a diagonal matrix. y m = min y Z T m+1 (βv 1 V m+1 H m y) 2 = min y βz T m+1 v 1 Z T m+1 V m+1 H m y 2 = min y βd 11 e 1 D m+1 H m y PRGMRES(m) Inexact (IPRGMRES) Another idea about the conditions (8) is to suppose that Z m+1 and V m+1 are biorthogonal bases, this is: Z T m+1 V m+1 = D m+1 Z T m+1r 0 = βd 11 e 1 where D m+1 is a diagonal matrix and d ii =< z i,v i > for i = 1,...,(m + 1).

5 4. NUMERICAL RESULTS We compare the classical restarted versions of FOM and GMRES with the projected versions presented in this work, on the following nonsymmetric matrices from the Harwell-Boeing library [8]: Matrix BCSSTK14 (n = 1806, cond(a)= ) Matrix FS (n = 680, cond(a)= ) Matrix Orsirr 1 (n = 1030, cond(a)= ) Matrix Sherman 1 (n = 1000, cond(a)= ). In all cases, we run experiments without preconditioning techniques. All experiments were run on Pentium IV 1.8 GHz, 256 Mb RAM, using MATLAB 7.0, and we stopped the process when: b Ax 2 b 2 < (9) Results are showed on tables 1 to 4 for each matrix studied. The parameter m is the positive integer used to restart the algorithms. For all cases, the initial guess was choosen as x 0 =(0,...,0) t. For each test matrix the rigth hand side vector is selected such that the solution vector is x =(1,1,...,1) t. A maximum of 3000 iterations were allowed for all methods. The symbol means that the associated method failed to satisfy (9) at the maximum number of iterations. Table 1. Number of iterations for the matrix BCSSTK14 m FOM GMRES PRGMRES BPRGMRES IPRGMRES 3 **** **** **** **** **** **** **** **** **** **** **** **** **** **** **** In general, we observe a very competitive behavior in the numerical tests between the proposed algorithms and classical restarted FOM and GMRES. However, we would like to point out some observations and conclusions about these numerical tests:

6 Table 2. Number of iterations for the matrix FS m FOM GMRES PRGMRES BPRGMRES IPRGMRES 9 **** **** **** 2636 **** 10 **** **** **** 2508 **** 15 **** **** **** **** Table 3. Number of iterations for the matrix Orsirr 1 m FOM GMRES PRGMRES BPRGMRES IPRGMRES 4 **** **** **** **** **** **** **** **** **** **** **** **** **** **** Table 4. Number of iterations for the matrix Sherman 1 m FOM GMRES PRGMRES BPRGMRES IPRGMRES 2 **** **** **** **** **** **** **** **** ****

7 The methods PRGMRES, BPRGMRES, IPRGMRES have a nonmonotone behavior (see figure 1). For small values of m, the proposed algorithms outperform the classic restarted versions of FOM and GMRES. This is certainly an interesting feature of the projected versions for very large problems when storage is crucial. For large values of m the new methods converges in the same number of iterations of GM- RES. In our experiments the classic versions failed to converge more frequently than the projected versions. We have no theoretical explanation for the interesting results reported in this section Matriz orsirr 1 = GMRES PRGMRES BPRGMRES IPRGMRES Figure 1. Evolution of the residual norm between the different methods with m = 15. Matrix Orsirr 1 5. FINAL REMARKS In this work we have proposed a new family of methods (projected methods) to solve systems of linear equations based on the equation (6), and developed three new Krylov methods that, in the numerical tests have a competitive behavior with FOM and GMRES. In the near future, we would like to study the performance of the new methods from a theoretical point of view, and to extend the projected approach to biorthogonal type methods like QMR and TFQMR. Acknowledgements This work was partially supported by FONACIT under the Programme of Human Resources Formation and by CDCH-UCV project

8 REFERENCES [1]. Biswa Nath Datta. Numerical Linear Algebra and Applications. Brooks/Cole Publishing Company, Pacific Grove, California, [2]. G. H. Golub and C. F. Van Loan, Matrix Computations, Third Edition, The Johns Hopkins University Press, Baltimore (1996). [3]. C. Brezinski. Multiparameter descent methods. Linear Algebra and Appl., 296: , [4]. Y. Saad. Iterative Methods for Sparse Linear Systems, 2nd edition. SIAM, Philadelphia, PA, [5]. Y. Saad and M. H. Schultz. GMRES: A generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM J. Sci. Stat. Comput., 7: , [6]. W.E. Arnoldi. The principle of minimized iteration in the solution of the matrix eigenvalue problem. Quart. Appl. Math., 9:17 19, [7]. Y. Saad. Krylov subspaces methods for solving large unsymmetric linear systems. Mathematics of Computation, 37: , [8]. I. Duff, R. Grimes and J. Lewis. User s guide for the Harwell-Boeing sparse matrix collection (release i). Technical Report TR-PA-92-86, CERFACS, Toulouse, France, 1992.