RESTARTED FULL ORTHOGONALIZATION METHOD FOR SHIFTED LINEAR SYSTEMS

BIT Nuerical Matheatics 43: 459 466, 2003. 2003 Kluwer Acadeic Publishers. Printed in The Netherlands 459 RESTARTED FULL ORTHOGONALIZATION METHOD FOR SHIFTED LINEAR SYSTEMS V. SIMONCINI Dipartiento di Mateatica, Università di Bologna, and Istituto di Mateatica Applicata e Tecnologie Inforatiche del CNR, via Ferrata 1, 27100 Pavia, Italy. eail: val@iati.cnr.it. Abstract. RestartedGMRESisknowntobeinefficientforsolvingshiftedsysteswhenthe shifts are handled siultaneously. Variants have been proposed to enhance its perforance. We show that another restarted ethod, restarted Full Orthogonalization Method (FOM, can effectively be eployed. The total nuber of iterations of restarted FOM applied to all shifted systes siultaneously is the sae as that obtained by applying restarted FOM to the shifted systewith slowest convergence rate, while the coputational cost grows only sub-linearly with the nuber of shifts. Nuerical experients are reported. AMS subject classification: 65F10, 65F15, 15A06, 15A18. Key words: Shifted algebraic linear systes. Krylov subspace ethods. Iterative non syetric solver. 1 Introduction. Given a real large n n nonsyetric atrix A, we are interested in the siultaneous solution of the shifted nonsingular linear syste (1.1 (A σix = b, for several (say a few hundreds; see, e.g., [2] tabulated values of the paraeter σ. (I stands for the identity atrix. This type of proble arises in any applications, such as control theory, structural dynaics, tie dependent PDEs and quantu chroodynaics; see [1, 2, 5, 6] and references therein. One such application is discussed in the experients section. Krylov subspace techniques are particularly appealing since they rely on a shift invariance property, which allows to obtain approxiation iterates for all paraeter values by only constructing one approxiation subspace. Indeed, the Krylov subspace K (A, v =span{v,av,...,a 1 v} satisfies K (A, v =K (A σi, v. In the linear syste setting, the generating vector v is the residual associated with a starting approxiate solution. A widely known and appreciated schee, the Generalized Miniu residual ethod (GMRES, has been shown to be effective on (1.1, see [1]. However, in several circustances, the approxiation Received August 2002. Revised Noveber 2002. Counicated by Olavi Nevanlinna.

460 V. SIMONCINI space required to satisfactorily approxiate all shifted systes appears to be too large, and the ethod needs to be restarted, taking the current syste residual as new generating vector. Coputational efficiency can be aintained after restart if the new Krylov subspace is the sae for all shifted systes. This happens when the generating vectors, in our case the current residuals, are collinear. Since in general the coputed GMRES shifted residuals are not collinear, after the first restart, restarted GMRES can only be applied to each shifted syste separately. Several alternative strategies can be considered within the Krylov subspace setting with restarting: 1 Run GMRES on a seed syste, e.g. the syste with zero shift, and then generate the approxiate solutions to the shifted systes iposing collinearity with the coputed seed residual. This idea was investigated in [5]. Note that only the seed syste residual is iniized, whereas no iniization property is satisfied by the shifted systes residuals. 2 Take as seed syste one of the shifted systes, on which restarted GMRES is applied, and generate approxiate solutions to the reaining shifted systes iposing collinearity. Possibly a new seed is selected at each restart, choosing as seed the syste with larger residual nor. This is a siple variant of the previous schee, which was used in [2] to cure isconvergence of the original approach in [5]. Note that in general, at each restarting phase a different syste residual is iniized. 3 Use a restarted Krylov subspace ethod other than restarted GMRES, that naturally generates collinear residuals. To the best of our knowledge, the third option above has not been considered in the past. The ai of this contribution is to draw attention to the fact that another restarted ethod, the Full Orthogonalization ethod (FOM, can be naturally applied to shifted systes since all residuals are naturally collinear. It will becoe apparent that the convergence history is the sae as that obtained by applying restarted FOM to each shifted syste independently, while coputational cost and eory requireents 1 are substantially reduced, since a single approxiation space is constructed for all shifted systes. Alternative approaches that do not require restarting are based on CG and Lanczos recurrences. These include the shifted TFQMR ethod, proposed by R. Freund [4], and the Shifted two-sided Lanczos ethod [9]. Meory requireents however ay liit their applicability to general shifted probles, since additional long vectors need to be stored for each shifted syste siultaneously handled. We refer to [2, 9] for nuerical experiences on applications on which variants of these solvers are particularly effective. We refer to [6] for a coparison of unrestarted FOM and GMRES in the syetric shifted case. All experients were run using Matlab 6 on one processor of a Sun Enterprise 4500. Exact arithetic is assued throughout the paper. 1 Order n eory requireents are the sae as for standard restarted FOM if this is applied to each shifted syste sequentially.

RESTARTED FOM FOR SHIFTED LINEAR SYSTEMS 461 2 The algorith. Given a nonsyetric linear syste Ax = b and a starting guess x 0,consider the Krylov subspace K (A, r 0, where r 0 = b Ax 0. Letting V be an orthogonal basis of K (A, r 0 andh be the projection and restriction of A onto K (A, r 0, then the FOM approxiation x = x 0 +z to x is deterined in x 0 +K (A, r 0 so that the associated residual r = b Ax is orthogonal to the approxiation space K (A, r 0. More precisely, x = x 0 + V y,wherey solves the reduced syste H y = β 0 e 1,withβ 0 = r 0 [7]. Here and below, e i denotes the ith colun of the identity atrix, whose diension is clear fro the context; is the vector 2 nor. If the obtained approxiate solution is not sufficiently accurate, then the FOM ethod is restarted, by using r = b Ax as new starting residual. The generation of the Krylov subspace can be carried out by eans of the Arnoldi algorith, which constructs V,H so that the first basis vector v 1 is v 1 = r 0 /β 0,and (2.1 AV = V H + h +1, v +1 e T. e T indicates the real transpose of e. Consider now the shifted syste (1.1. Shifting transfors (2.1 into (2.2 (A σiv = V (H σi +h +1, v +1 e T, where I is the identity atrix of size. Thanks to (2.2, the only difference in FOM is that y is coputed by solving the reduced shifted syste (H σi y = β 0 e 1. Therefore, the expensive step of constructing the orthogonal basis V is perfored only once for all values of σ of interest, σ {σ 1,...,σ s }, whereas s reduced systes of size need be solved. This is the case if the right hand sides are collinear. In the following, we shall assue that x 0 =0so that all shifted systes have the sae right-hand side. Restarting can also be eployed in the shifted case. The key fact is that the FOM residual r is a ultiple of the basis vector v +1, that is r = h +1, v +1 (y ; see e.g. [7]. In the last expression, (y is the th coponent of the vector y. The next proposition shows that collinearity still holds in the shifted case when FOM is applied. Proposition 2.1. For each i =1,...,s,letx (i = V y (i be a FOM approxiate solution to (A σ i Ix = b in K (A σ i I,b, withv satisfying (2.2 with σ = σ i. Then there exists β (i Proof. Fori =1,...,s,wehave r (i Setting β (i = b (A σ i Ix (i R such that r (i = r 0 (A σ i IV y (i = b (A σ i Ix (i = V β 0 e 1 V H y (i + σ iv y (i h +1,v +1 (y (i = h +1, v +1 (y (i. = h +1, (y (i, i =1,...,s,wehaver (i = β (i v +1. = β (i v +1.

462 V. SIMONCINI The Shifted FOM ethod can be restarted by using any of the shifted syste residuals r (i, i =1,...,s as new generating vector. Let thus ˆv 1 be the first vector of the new basis ˆV after restart 2.Notethat ˆv 1 = r (i /β (i = ±v +1, i =1,...,s. Hence the new proble reads: For each i =1,...,s, find ˆx (i = x (i + ˆV ŷ (i with Range( ˆV =K (A, ˆv 1 whereŷ (i solves the reduced syste (Ĥ σ i I ŷ = β (i e 1. For each shifted syste, the new residual can be coputed as ˆr (i = r (i (A σ i I ˆV ŷ (i = ĥ+1,ˆv +1 (ŷ (i, showing that all new residuals are collinear to the +1st basis vector, hence the process can be repeated. In particular, we have shown that collinearity holds also when the original right-hand sides are different but collinear. The final algorith can be written as follows. Algorith. Restarted Shifted FOM: Given A, b, x 0 =0,,{σ 1,...,σ s }, I = {1,...,s}: 1. Set r 0 = b, β (i = r 0,x (i = x 0,i=1,...,s.Setv 1 = r 0 /β (i. 2. Generate V,H associated with K (A, v 1 3. For each i I y (i =(H σ i I 1 e 1 β (i Update x (i x (i + V y (i 4. Eliinate converged systes. Update I. IfI = exit. 5. Set β (i 6. Set v 1 v +1.Goto2 = h +1, (y (i for each i I The convergence history of the shifted restarted ethod on each syste is the sae as the convergence of the usual restarted ethod applied individually to each shifted syste. This can be clearly observed by noticing that both the sequence of Krylov subspace systes and approxiation iterates are the sae as those coputed by standard restarted FOM. In particular, this iplies that Shifted Restarted FOM only generates one sequence of bases {V } for all shifted systes siultaneously solved and that no degradation of perforance is caused by the inforation sharing. 3 Nuerical experients. In this section we report the results of soe nuerical experients we have carried out. We copare the Froer and Glässner correction of restarted GMRES( for shifted systes (hereafter g s (, its variant proposed in [2] (hereafter g sv ( and restarted FOM(. Restarted Arnoldi-type ethods 2 To avoid indexing overwheling we shall adopt the hat sybol ˆ for the coputed quantities after restart.

RESTARTED FOM FOR SHIFTED LINEAR SYSTEMS 463 are known to suffer when A is indefinite. The presence of the shift ay exacerbate the situation aking the shifted syste particularly hard to solve when the spectru of the coefficient atrix A σi surrounds the origin. Nuerical experients with restarted FOM are less coon in the literature than with restarted GMRES; see e.g. [8, 7]. Perforance evaluation of g s ( and g sv ( can be found in [5] and in [2], respectively. Far fro being exhaustive, our nuerical experients ai to describe typical convergence behavior of the discussed ethods in situations where the shift ay deteriorate convergence. Our convergence stopping criterion is based on the residual nor, r (i. Depending on the proble at hand, different strategies ay be ore relevant; see the discussion below on the structural dynaics proble. 10 5 10 10 g s (10,σ 1 10 5 nor of relative residual 10 0 10 5 g s (10,σ 2 g s (10,σ 1 nor of relative residual 10 0 10 5 g s (10,σ 2 g sv (10,σ 1 g sv (10,σ 2 fo(10,σ 2 fo(10,σ 1 g sv (10,σ 1 g sv (10,σ 2 10 10 0 10 20 30 40 50 60 70 80 nuber of restarts 10 10 fo(10,σ 1 fo(10,σ 2 0 20 40 60 80 100 nuber of restarts Figure 3.1: Left: Exaple with bidiagonal atrix and σ 1 = 1, σ 2 = 1. Right: Exaple with operator L(u andσ 1 =0.5, σ 2 = 0.5. The first exaple considers a 100 100 upper bidiagonal atrix A with diagonal the vector d =[0.01, 0.02, 0.03, 0.04, 10, 11,, 105] and super-diagonal the vector of all ones; see e.g. [8]. The right hand side is the vector of all ones, noralized to have unit nor, and we consider two values for the shift paraeter, σ = 1, 1. The results are reported in the left plot of Figure 3.1 for =10and stopping tolerance ε =10 8. While g s (10 stagnates on both shifted systes, its variant is able to converge, although the convergence is slowed down, copared with running restarted GMRES on each shifted syste separately. Indeed, restarted GMRES with = 10 would converge in 16 restarts on A + I and in 22 restarts on A I. Therefore, the restarting strategy that chooses as seed the slower converging shifted syste in practice affects the convergence of all systes. Note also that a different ordering of the shifts at start tie (when all residuals equal the right hand side b would yield a quite different convergence history of g sv on the two systes. We next consider the 100 100 atrix corresponding to the discretization of the operator L(u = u +10u x on the unit square with Dirichlet boundary

464 V. SIMONCINI conditions. We setb and stopping tolerance as before, and σ = 0.5, 0.5. Results are suarized in the right plot of Figure 3.1. Standard restarted GMRES applied to each shifted syste separately with = 10 would converge in 4 restarts on A +0.5 I and in 172 restarts on A 0.5 I. The difference in perforance is ore pronounced on this exaple. Note that g sv converges very rapidly on σ 2 so that only σ 1 is processed in later restarts. The atrices in our next exaple ste fro a structural dynaics engineering proble, whose algebraic forulation was discussed in detail in [9]. Direct frequency analysis leads to the solution of the following algebraic linear syste (3.1 (σ 2 A + σb + Cx = b, for several values of the frequency-related paraeter σ. Here A, B and C are coplex syetric. Linearization yields the syste [( ( ] [ ] [ ] B C A 0 y b C T + σ 0 0 C T = (T σsz = f. x 0 Note that only a portion of the approxiation to z is eployed to approxiate x in (3.1. We refer to [9] for a detailed analysis of the connections between the two algebraic approxiation probles and for stopping criteria that take the original proble (3.1 into account. 10 0 10 1 nor of relative residual 10 2 g sv (10, σ 2 10 3 fo(10,σ 1 fo(10,σ 2 g sv (10,σ 1 10 4 0 10 20 30 40 50 60 70 80 90 100 nuber of restarts Figure 3.2: Exaple fro structural dynaics. σ 1 =(5 2π 1,σ 2 =(8 2π 1. Whenever S is nonsingular, we can write the proble above as in (1.1, naely (TS 1 σiẑ = f, Sz =ẑ. We consider test case C in [9] and for the sake of siplicity, we only report results for two paraeters σ 1 =(5 2π 1,σ 2 = (8 2π 1. The linearized proble has diension 3947 and the right-hand side

RESTARTED FOM FOR SHIFTED LINEAR SYSTEMS 465 is f = e 1160 + e 2298. Convergence curves with shifted restarted FOM and with g sv and = 10 are displayed in Figure 3.2. The stopping tolerance was set to ε =10 3, a loose value which is typical in this kind of applications [9]. FOM is faster than g sv with both shifts. Again, different shifting affects the whole coputation in g sv : restarted GMRES with = 10 would converge in 9 restarts for σ 1 andin29restartsforσ 2. This suggests that in g sv other strategies should be devised to select the seed syste, to prevent convergence delay. 4 Conclusions and further rearks. We have shown that a known Arnoldi-based ethod, the restarted Full Orthogonalization Method, can efficiently solve shifted algebraic linear systes, at the cost that grows only odestly as the nuber of shifts increases. Liited nuerical experients see to show its copetitiveness with respect to other restarted ethods. Note also that the various ethods can be cobined by switching to any of the strategies at restart tie. Our experience sees to show that known probles associated with restarting (see, e.g., [8], are exacerbated in the shifting setting in GMRES-based ethods; other strategies to select the collinearity restarting vector should perhaps be considered. The natural ipleentation of FOM in the shifted context is particularly attractive when both A and b are real, while the shift is coplex. More precisely, let σ C and consider (A σix = b A R n n,b R n. The subspace K (A, b aswellash and V are real, whereas only the approxiate solution in the projected space, y =(H σi 1 e 1 β,hascoplex entries; see [3] for siilar considerations. At restart tie, all (coplex residuals are collinear to the + 1st basis vector, which is real. Therefore, the new approxiation basis after restart can still be iposed to be real, while the collinearity coefficients β (i are coplex. As a consequence, at each restart, the expensive step of constructing the orthogonal basis is done in real arithetic, whereas ost of the reaining coputation, of order, is done in coplex arithetic. Acknowledgent. We thank Daniel Szyld for carefully reading an earlier draft of this anuscript. REFERENCES 1. B. N. Datta and Y. Saad, Arnoldi ethods for large Sylvester-like observer atrix equations,and an associated algorith for partial spectru assignent, Linear Algebra Appl., 154 156 (1991, pp. 225 244. 2. A. Feriani, F. Perotti, and V. Sioncini, Iterative syste solvers for the frequency analysis of linear echanical systes, in Coputer Methods in Applied Mechanics and Engineering, 190 (13 14 (2000, pp. 1719 1739.

466 V. SIMONCINI 3. R. F. Freund, On conjugate gradient type ethods and polynoial preconditioners for a class of coplex non-heritian atrices, Nuer. Math., 57 (1990, pp. 285 312. 4. R. F. Freund, Solution of shifted linear systes by quasi-inial residual iterations, in Nuerical Linear Algebra, L. Reichel, A. Ruttan, and R. S. Varga, eds., W. de Gruyter. Berlin, 1993, pp. 101 121. 5. A. Froer and U. Glässner, Restarted GMRES for shifted linear systes, SIAM J. Sci. Coput., 19(1 (1998, pp. 15 26. 6. K. Meerbergen, The solution of paraetrized syetric linear systes, SIAMJ. on Matrix Analysis and Applications, 24(4 (2003, pp. 1038 1059. 7. Y. Saad, Iterative Methods for Sparse Linear Systes, The PWS Publishing Copany, Boston, 1996. 8. V. Sioncini, On the convergence of restarted Krylov subspace ethods, SIAM J. Matrix Anal. Appl., 22 (2000, pp. 430 452. 9. V. Sioncini and F. Perotti, On the nuerical solution of (λ 2 A + λb + Cx = b and application to structural dynaics, SIAM J. Scientific Coput., 23 (2002, pp. 1876 1898.