Nahid Emad. Abstract. the Explicitly Restarted Block Arnoldi method. Some restarting strategies for MERAM are given.

Transcription

1 A Generalzaton of Explctly Restarted Block Arnold Method Nahd Emad Abstract The Multple Explctly Restarted Arnold s a technque based upon a multple use of Explctly Restarted Arnold method to accelerate ts convergence. Ths paper presents the Multple Explctly Restarted Arnold Method (MERAM) to compute a few egen-elements of large non-hermtan matrces and shows that t generalzes the Explctly Restarted Block Arnold method. Some restartng strateges for MERAM are gven. Key words: large egenproblem, Arnold method, block Arnold method, explct restartng, parallel programmng. AMS subect classcatons: 65F5, 65F50 and 65Y05. Introducton Hybrd methods combne several numercal methods whch cooperate to ecently compute a soluton to a gven problem. Examples of methods that could be used to develop an hybrd method are Specc methods, convergence acceleraton technques, and precondtonng methods. Hybrd methods can solve lnear systems, such as the one ntroduced by Breznsk & Redvo-Zagla [2], or solve egenproblems, such as the one developed by Saad [7] or the Implctly Restarted Arnold method proposed by Sorensen [0, 6]. In ths paper, we present a new approach to compute a few egen-elements of a large sparse non-hermtan matrx based on a hybrd method usng the Explctly Restarted Arnold method (ERAM). Ths technque nvolves multple uses of ERAM. It makes use of several derently parameterzed ERAM for the benet of the same applcaton. The experments on a network of parallel machnes [5, 3,4]have gven a very good acceleraton of convergence wth respect to the Explctly Restarted Arnold method. An mportant property of ths technque s that t generalzes the Explctly Restarted Block Arnold Method. Furthermore, the concept of ths technque may be appled to some other restarted proecton methods. To present the Multple Explctly Restarted Arnold algorthm n the context of Krylov proecton methods, a bref presentaton of these methods s gven n the next secton. An overvew of the Arnold method and ts ERAM varant s provded n the thrd secton. The actual Multple Explctly Restarted Arnold method s presented n the fourth secton. The secton also ncludes the restartng strategy and stoppng crteron selected n MERAM. Secton 5 s devoted to a presentaton of MERAM as a generalzaton of the Explctly Restarted Block Arnold method. Fnally, possble applcatons of the concept are dscussed along wth the concluson. 2 Krylov Subspace Methods Let A be a complex non-hermtan matrx of dmenson n n and K m be a subspace of C n.theorthogonal proecton method on the subspace K m ams to approxmate an egen-par ( u )ofaby a par of Rtz value, (m) 2C,and Rtz vector, u (m) 2K m, satsfyng the Galerkn condton: Laboratore PRSM, Unverste Versalles St-Quentn, 45, av. des Etats-Uns, Versalles, France (emad@prsm.uvsq.fr) 6th IMACS World Congress ( c 2000 IMACS)

2 A Generalzaton of ERBAM 2 () (A ; (m) I)u (m)?k m In a Krylov subspace method, K m s the Krylov subspace dened by (2) K m U = Span(U AU A m; U) where U s spanned by asetfx x g of ntal guess. The Block Krylov subspace methods are characterzed by the choce >. Some very popular technques for ndng a few egen-elements of a large sparse matrx A s the Pont Krylov subspace methods, ( =) wth v = x. They approxmate s egen-elements of A by those of a matrx of order m obtaned by an orthogonal proecton onto a m-dmensonal subspace K m v wth s m n. Let w w m be an orthogonal bass of K m v and W m be the matrx whose columns are w w m. The condton () s equvalent to search (m) (3) 2Cand y (m) 2C m n: (H m ; (m) I)y (m) =0 where the m m matrx H m s dened by H m = Wm H AW m wth Wm H the transpose conugate of W m and u (m) = W m y (m). Thus, some egenvalues of A can be approxmate by the ones of the matrx H m. They can be found by buldng an orthogonal bass of K m v and by solvng the problem (3). There are derent ways of buldng such bass. The most often used one s the Arnold's orthogonalzaton process. 3 Arnold Method Let w be the normalzed ntal guess v=kvk 2. The well-known Arnold process generates an orthogonal bass w w m of Krylov subspace K m v by usng the Gram-Schmdt orthogonalzaton process: Arnold Reducton : AR(A k m v). For = k + 2 m do: h =(Aw w P ), for = 2 w = Aw ; h = w. h + = k w k 2. w + =w =h +. The h and w computed bytheabove process consttute respectvely the non-zero elements of an upper Hessenberg matrx H m and the columns of a matrx W m. The matrces H m, W m and A satsfy the equaton: (4) AW m = W m H m + f m e H m where f m = h m+ m w m+ and e m s the mth vector of the canoncal bass of C m. If stopped before completon, ths process ntroduced by E. Arnold n 95 [] could gve good approxmatons to the egenvalues stuated at the extremtes of the spectrum. The s desred Rtz values (wth largest/smallest real part or largest/smallest magntude) (m) =( (m) (m) s ) and ther assocate Rtz vectors U (m) =(u (m) u (m) s ) can be computed as follows : ones. Basc Arnold Algorthm : BAA(A s m v r s (m) U (m) ).. Compute an AR(A m v)step. 2. Compute the egen-elements of H m and select the s desred ones. 3. Compute the s assocate Rtz vectors u (m) = W m y (m). 4. Compute r =( s ) t wth = k(a ; (m) I)u (m) k 2. If the accuracy of the computed Rtz elements s not satsfactory the proecton can be restarted onto a new K m v. We suppose that the egenvalues and correspondng egenvectors of Hm are re-ndexed so that the rst s Rtz pars are the desred

3 A Generalzaton of ERBAM 3 3. Explctly Restarted Arnold Method Ths verson of Arnold method conssts of restartng a BAA process wth a new K m v dened wth the same subspace sze and an updated startng vector. Ths method s called Explctly Restarted Arnold Method (ERAM). Startng wth an ntal vector v, t computes BAA. The startng vector s updated and a BAA process s restarted untl the accuracy of the approxmated soluton s satsfactory (usng approprate methods on the computed Rtz vectors). Ths update s desgned to force the vector n the desred nvarant subspace. Ths goal can be reached by some polynomal restartng strateges proposed n [7] and dscussed n the secton 4.. Explctly Restarted Arnold Algorthm.. Start. Choose a parameter m and an ntal vector v. 2. Iterate. Compute a BAA(A s m v r s (m) U (m) )step. 3. Restart. If g(r) >tolthen use (m) and U (m) to update the startng vector v and go to 2. where tol s a tolerance value and g s a functon that wll be dened n the next secton. 4 Multple Explctly Restarted Arnold Method Ths method ams to restart the Arnold method wth a new K m v bulted wth an updated startng vector and a new subspace sze. Ths means that, n ths verson of Arnold method nether the parameter m nor the ntal vector v are xed. The new parameter m can be gven by m = m + q wth q a postve natural number. It s clear that, under the hypothess that v does not belong to any desred nvarant subspace, m has to be as large as possble. An mportant well-known shortcomng of ths choce s ts alarmngly large storage space requrement and computaton cost for large m. Toovercome ths storage dependent shortcomng, a constrant on the subspace sze m s mposed. More precsely, m has to belong to the dscrete nterval I m =[m nf m sup ]. The bounds m nf and m sup may be chosen n functon of the avalable computaton and storage resources and have to fulll m nf m sup n. We suppose that u (k) s \better" than u (`) f (k) (`). Let m m` wth m 2 I m ( `). An algorthm of ths method to compute s (s m ) desred Rtz elements of A s the followng (see[5] for more detals on ths method): Multple Explctly Restarted Arnold Algorthm.. Start. Choose ` startng vectors fv () v (`) g and a set of subspace szes fm m`g. 2. Iterate. For = ` do: (a) Compute a BAA(A s m v () r () (m) U (m) ) step. (b) If g(r () ) tol then stop. 3. Restart. Update the vectors fv () v (`) g and go to 2. where r () s the vector of the resdual norms at the th teraton. The restartng strategy to update v () can be derent from the one to update v () (for 6= ). In other words, the vector v () of step 3 can be computed by: (5) v () = f () (U (mk l ) ) where U (mk ) =(u (mk ) u (mks s ) )and u (mk ) s "the best" th Rtz vector computed at the teraton k wth k 2 [ ]. The denton of the functon f () can be based onto the technques proposed by Y.Saadn[7] and wll be dscussed n secton 4.. Ths functon has to be chosen n order to force the vector v () nto the desred nvarant subspace. Clearly, ths algorthm s equvalent to a partcular use of several Explctly Restarted Arnold methods. It allows to update the restart vector v () of an ERAM by takng account thenterestng egen-nformaton obtaned by the other ones. Another advantage of ths algorthm s ts structure n vewpont of large coarse parallelsm. In fact, each teraton of step 2 can be executed on an ndependent process. Let us suppose that we have ` processes and that the

4 A Generalzaton of ERBAM 4 th step of the nner loop of the algorthm s assgned to the th process. We suppose also that after a BAA step, each process sends ts output of nterest to all the other processes. Then, the computaton of the restartng vectors v () on the th process can be done by a combnaton of \the best" computed Rtz elements avalable on t. 4. Restartng Strateges - Convergence The restartng strategy s a crtcal part of smple and multple Explctly Restarted Arnold algorthms. Saad [8] proposed to restart the teraton of ERAM wth a vector precondtonng so that t s forced to be n desred nvarant subspace. It concerns a polynomal precondtonng appled to the startng vector of ERAM. Ths precondtonng ams at computng the restartng vector so that ts components are nonzero n the desred nvarant subspace and zero n the unwanted nvarant subspace: (6) v(k) =p(a)v where v(k) skth restart vector of ERAM and p s a polynomal n the space of polynomals of degree <m. One approprate possblty todenep s a Chebyshev polynomal determned from some knowledge on the dstrbuton of the egenvalues of A. Ths restartng strategy s very ecent to accelerate the convergence of ERAM and s dscussed n detals n [8, 7]. Another possblty to dene the polynomal p s to compute the restartng vector wth a lnear combnaton of s desred Rtz vectors: (7) where u (mk) u (mk) 2K m mples u (mk) v(k) = sx = u (mk) denotes th Rtz vector computed at the teraton k. Ths restartng P strategy s polynomal because s = (A)v for some polynomal and then v(k) = = (A)v. There are several ways to choose the scalar values n (7). One choce can be equal to the th resdual norm. Some other choces can be =, = or = s ; + for s (see [9] for more detals). However, f the desred egenvalues are not well-separated, the restartng strategy (7) appled to ERAM wll cause many teratons back and forth toward the wanted nvarant subspace. Ths means that SRN s a non-monotonc functon. SRN s the sum of the resdual norms correspondng to the desred egenvalues as a functon of the number of restart. To dene a restartng strategy for MERAM, the functons f () n (5) have to be dened. The denton of these functons consttutes a very mportant part of ths algorthm. However, a smple choce for f () can be the use of the same technques than the ones proposed for ERAM. the analogous strategy of the restartng strategy (7) for MERAM wth the hypothess that ( (mp) u (mp) ) s better than ( (mq ) (8) v () = f () (U (mk` )) = u (mq) sx = )f (p) u (mk ) (q) and call t the hybrd restart strategy: We can vew (` ; ) ERAM process of the Multple Explctly Restarted Arnold method as the convergence accelerators of the last one. The experments n [5, 3, 4] llustrate the facts that ths ERAM process converges more rapdly than an Explctly Restarted Arnold algorthm wth the same subspace sze and startng vector. In other words, these experments show that the restartng vector computed wth the hybrd restart s more n the desred nvarant subspace than the one computed wth (7). 4.2 Stoppng Crteron The resdual norms dened n the former secton verfy the well known equaton [7]: (9) = k(a ; (m) I)u (m) k 2 = h m+ m e H m y(m) where e m s the mth vector of the canoncal bass of C m. Ths allows to compute the resdual norms nexpensvely because the th resdual norm s equal to the last component oftheegenvectors y (m) multpled by h m+ m. Let r be the resdual norms vector ( s ) t. Once ths vector s computed, to stop the Restarted Arnold algorthms we have to dene a stoppng crteron. Ths can be done by denng the functon g of these algorthms. Some typcal examples can be gven by (0) g(r) =krk

5 A Generalzaton of ERBAM 5 or by () g(r) = sx = where are some scalar values. The same functon g, appled to r () =( () () s ) t, can be used to dene the restart crteron n the MERAM. 5 MERAM versus Explctly Restarted Block Arnold Method 5. Explctly Restarted Block Arnold Method (ERBAM) As we have seen n secton 2, the Block Krylov subspace methods allow tond( (m) u (m) ) soluton of () wth K m U dened by (2), U = spanfx x g and >. We suppose the vectors x x are lnearly ndependent. The Explctly Restarted Block Arnold Method permts the extracton of the egenvalues whose multplcty s less or equal than the block sze. Let W be the orthonormal matrx whose columns are x x. The block Arnold process generates a set of W W m matrces. The m columns of these matrces buld an orthogonal bass of Krylov subspace K m U : Block Arnold Reducton : BAR(A k m W ). For = k 2 m do: H = W H AW,for = 2 W = AW ; P = W H. Compute QR decomposton of W : W = Q R = W + H +, The H matrces computed by the above process consttute the blocks of an upper block Hessenberg matrx H m. The egenvalues of ths matrx approach the correspondng ones of A. Let W m be the (n m )-sze matrx whose columns are W W m. Then, the analogous of the relaton (4) s (2) AW m = W m H m + F m E H m where F m = W m+ H m+ m and E m s the matrx of the last s columns of I ns. The block verson of Basc Arnold algorthm of secton 3 s unchanged but, n the rst step, we compute BAR(A m W ) nstead of AR(A m v). We call ths algorthm BBAA(A s m W r (m) U (m) ). 5.2 MERAM as a Generalzaton of ERBAM Both Explctly Restarted Block Arnold and Multple Explctly Restarted Arnold algorthms begn wth a set of startng vectors, but they are not equvalent. To pont out the derences between them, suppose ` = s the number of startng vectors and m s the sze of the proecton subspaces for both methods. Ths means that we have m = = m = m n MERAM. Then, the proecton subspace wth Explctly Restarted Block Arnold algorthm s K m U (wth U = spanfx x g) whle MERAM proect the problem to solve onto subspaces K m x,, K m x. Now, under the hypothess that the degree of the mnmal polynomal of U s greater than m, these Krylov subspaces fulll the followng relaton: (3) K m U = K m x K m x We have then dm(k m U )=m ) dm(k m x )=m for all 2 [ ] and dm(u) = We suppose dm(k m U )=m. Buldng up an orthogonal bass (x x xm xm )ofk m U amounts to buldng up an orthogonal bass X =(x xm ) for each subspace K m x,. The nverse s true only f X s

6 A Generalzaton of ERBAM 6 orthogonal to X for all 2 [ ] and 6=. Ths means that the block Arnold reducton whch s a Gram Schmdt orthogonalzaton of Krylov vectors of K m U s a partcular case of the Multple Restarted Arnold proectons whch are themselves Gram Schmdt orthogonalzaton of Krylov vectors of subspaces K m x ( ). In the rst method s desred egenvalues are approxmated by the ones of a m-sze block Hessenberg matrx, whle n the second, they are approxmated by s \best" egenvalues of Hessenberg matrces of order m. We notce also that n the Explctly Restarted Block Arnold method the relaton m s= has to be true whle n MERAM m have to fulll the more restrctve relaton m s. 6 Concluson We have presented the Multple Explctly Restarted Arnold Method and some of ts characterstcs. We have seen that MERAM allows to restart the teratons of each of ts ERAM process wth a vector precondtonng so that t s forced to be n the desred drecton. Ths precondtonng can be seen as a partcular applcaton of an ERAM process to the startng vector of the other ones. We have seen that MERAM can be consdered as a generalzaton of ERBAM. To gve a relable comparson between these methods we are plannng now to mplement them n the same programmng envronnement and compare ther numercal results. The concept of the MERAM s smple and can be extended to some other Explctly Restarted proecton methods lke GMRES or Hermtan/non-Hermtan Lanczos methods. Furthermore, MERAM accelerates the convergence of ERAM[5, 3, 4]. It can cooperate wth convergence acceleraton methods smlar to the teratve Chebyshev polynomals. The MERAM conssts n several ERAM nteractng wth some protocol. A great advantage of MERAM s ts large coarse parallelsm. Each ERAM core - BAA(A s m v r (m) U (m) ) - can be executed ndependently and asynchronously from the others. Consequently, ` ndependent large coarse tasks can be executed concurrently. An optmal parallel verson of MERAM s gven n [5]. References [] W. E. Arnold, The Prncple of Mnmzed Iteraton n the Soluton of Matrx Egenvalue Problems, Quart. Appl. Math., 9:pp. 7-29, 95 [2] C. Breznsk and M. R. Zagla, A hybrd procedure for solvng lnear systems, Numersche Mathematk, 67, pp. -9, 994. [3] G. Edlal, S. Petton and N. Emad, Interleaved Parallel Hybrd Arnold Method for a Parallel Machne and a Network of Workstatons. Conference on Informaton, Systems, Analyss and Synthess (ISAS'96), Jullet 996, Orlando, Florda, USA. [4] G. Edlal, Contrbuton a la parallelsaton des methodes teratves hybrdes pour matrce creuse sur archtectures heterogenes, PhD of the Unversty of Perre et Mare Cure (Pars VI) n french, 994. [5] N. Emad, S. Petton et G. Edlal, Iteratve Hybrd Arnold Method. Rapport PRSM 98/039, 998. [6] R. Lehoucq, D.C. Sorensen and P.A Vu, ARPACK: Fortran subroutnes for solvng large scale egenvalue problems, Release 2., avalable form netlbornl.gov n the scalapack drectory, 994. [7] Y. Saad, Numercal Methods for Large Egenvalue Problems, Manchester Unversty Press, 993. [8] Y. Saad, Chebyshev acceleraton technques for solvng nonsymmetrc egenvalue problems, Math. Com., 42, , 994. [9] Y. Saad,Varatons on Arnold's Method for Computng Egenelements of Large Unsymmetrc Matrces, Lnear Algebra Applcatons, 34, , 980. [0] D.C. Sorensen, Implctly restarted Arnold/Lanczos Methods for Large Scale Egenvalue Calculatons, In D. E. Keyes, A. Sameh, and V. Venkatakrshnan, eds. Parallel Numercal Algorthms, pages 9-66, Dordrecht, 997, Kluwer.