The FEAST Algorithm for Sparse Symmetric Eigenvalue Problems

Transcription

1 The FEAST Algorthm for Sparse Symmetrc Egenvalue Problems Susanne Bradley Department of Computer Scence, Unversty of Brtsh Columba Abstract The FEAST algorthm s a method for computng extreme or nteror egenvalues of a matrx or matrx pencl. Each step of ths algorthm nvolves repeatedly solvng lnear systems (whch are complex and generally ll-condtoned) along a contour n the complex plane. Most of the prevous research done on the FEAST algorthm has used drect solvers for these systems. In ths report, we consder nstead usng teratve solvers for FEAST. Focusng on the standard symmetrc egenvalue problem, we explore the use of MINRES, as well as GMRES wth and wthout precondtonng. Through numercal experments, we conclude that FEAST can be employed effectvely wth a precondtoned teratve solver to fnd egenpars of large, sparse matrces. 1 Introducton Consder the generalzed egenvalue problem, whch s to fnd a scalar λ and an n-dmensonal vector x such that Ax = λbx for A, B C n n. If B s nonsngular, ths s equvalent to computng an egenpar of the matrx M := B 1 A. The FEAST method [14] fnds all such egenpars that have λ n some regon Γ n the complex plane. The computatonal backbone of FEAST s the soluton of several lnear systems along a contour n the complex plane. If A and B are large and sparse, the current mplementaton of FEAST solves these systems usng the Intel MKL-Pardso solver [15]. Despte the avalablty of ths opton, much of the prevous work n the FEAST lterature has focused on matrces of sze 500 to 4,000 [10, 11, 22, 21]. Some exceptons are: the orgnal FEAST paper [14], whch employed FEAST on a matrx of sze 12,450; the work of and Güttel et al. [9], whch had a maxmum matrx sze of 176,622; and Aktulga et al. [1], whch explored FEAST for standard egenvalue problems of dmensons as hgh as 327,000. To our knowledge, the only work that has explored the use of teratve solvers n FEAST s that of Krämer et al. [11], whch performed emprcal testng of dfferent varants of FEAST. In t, GMRES was employed wth dfferent tolerances on a matrx of sze 1,059. The purpose of ths project s to mplement FEAST n Matlab wth teratve lnear solvers and explore what mpact ths has on the performance of the method. We focus prmarly on the standard egenvalue problem (.e. B s the dentty) wth real symmetrc A. Emal: smbrad@cs.ubc.ca 1

2 1.1 Overvew of the FEAST method We now descrbe the dervaton of the FEAST algorthm n the case where A s Hermtan and B s Hermtan postve defnte (for a dervaton n the non-hermtan case, see [21]). The man computatonal work of FEAST les n applyng a ratonal functon to M that approxmates the ndcator functon π(µ) = 1 2πı Γ 1 dγ, (1) γ µ where Γ s a contour n the complex plane that encloses the desred egenvalues. Cauchy s ntegral theorem tells us that π(µ) = 1 for µ Γ, and π(µ) = 0 otherwse. Defne g(µ; z) := 1 z µ for a scalar nput µ and parameter z. By conventon, we defne the functon g(m; z) for a matrx M as g(m; z) = (zi M) 1. If we assume that M s dagonalzable as s the case wth Hermtan A and B we can wrte M = XΛX 1, meanng that g(m; z) : = (zi M) 1 = (zxx 1 XΛX 1 ) 1 = X(zI Λ) 1 X 1 = Xg(Λ; z)x 1, (2) where g(λ; z) s the dagonal matrx wth each dagonal entry λ of Λ replaced by g(λ; z). Our focus s on Hermtan matrces, whch have real egenvalues. Therefore, we assume that our nterval of nterest s gven by I := [λ, λ + ] R. For some complex contour Γ enclosng I, we estmate the ntegral π(µ) by numercal quadrature. We assume we have m quadrature nodes n the upper half of the complex plane, and m nodes n the lower half. We therefore wsh to construct a functon of the form ρ m (µ) := 2m =1 w z µ, wth ρ m (µ) π(µ). For matrx nput, the functon s defned as ρ m (M) : = = 2m =1 2m =1 w (z I M) 1 w (z B A) 1 B. (3) From Equaton (2), we see that ρ m (M) wll have the same egenvectors x as M, but ts egenvalues wll be gven by ρ m (λ ), where λ are the orgnal egenvalues of M. Rather than computng ρ m (M) drectly, we postmultply by a set of vectors Q = [q 1, q 2,..., q p ] to compute ρ(m)q. Ths entals solvng 2m lnear systems, each wth multple rght-hand sdes Bq j. If ρ m s a good approxmaton to the ndcator functon π (Equaton (1)), ρ(m)q approxmates the applcaton of the spectral projector to Q. Specfcally, we wll have ρ m (M)Q X Γ X H Γ BQ, where X Γ are the (B-orthogonal) egenvectors correspondng to the egenvalues of M n Γ. In FEAST, we use the vectors resultng from the approxmate spectral projecton n a Raylegh-Rtz procedure, and the resultng Rtz vectors form the rght-hand sde vectors Q for the 2

3 next teraton. Pseudocode for the FEAST method (as presented n [22]) s gven n Algorthm 1. Constructng the flter functon ρ m and selectng a search space dmenson p are dscussed n Sectons 3 and 4, respectvely. Algorthm 1: FEAST Input: Hermtan A, h.p.d. B, nterval I = [λ, λ + ], search space dmenson p Output: All egenpars (λ, x) of M = B 1 A such that λ I 1 Construct ρ m (µ) := 2m =1 w z µ 2 Pck p random n-vectors Q (0) = [q 1, q 2,..., q p ]; 3 for k = 1, 2,... untl convergence do 4 Y (k) ρ(m) Q (k 1) (see Equaton (3)); // Raylegh-Rtz procedure: to approxmate the ndcator functon for I; 5 Obtan reduced p p system: Â (k) Y H (k) AY (k), ˆB(k) Y H (k) BY (k); 6 Solve Â(k) ˆX (k) = ˆB (k) ˆX(k) ˆΛ(k) for ˆX (k), ˆΛ (k) ; 7 Set Q (k) Y (k) ˆX(k) ; 8 end 2 Survey of other egenvalue solvers In ths secton we wll go over some other establshed methods for egenvalue problems. 2.1 Standard egenvalue problem To compute the entre spectrum of a (generally farly small) real matrx, a common method s the QR algorthm [7], whch computes the Schur decomposton of a gven matrx. Namely, we factor A = Q H TQ, where Q s orthonormal (or untary, f A s complex), and T s upper trangular. Here the egenvalues of A are gven by the dagonal entres of T, and the columns of Q are called the Schur vectors of A. Sometmes, computng a full Schur factorzaton s too expensve, or we aren t nterested n the entre spectrum of a matrx. Several methods compute only a subset of the spectrum. The smplest of these s the power method [17, chapter 4]: gven a nonzero ntal vector v 0, the sequence A k v 0 wll eventually converge to the egenvector correspondng to the largest modulus egenvalue λ 1 of A, provded that λ 1 s sem-smple. We can generalze ths to subspace teratons: nstead of a sngle vector, we consder the sequence X (k) := A k X (0), where X (0) s a system of vectors [x 0, x 1,..., x m ]. Under a few assumptons, the columns of X (k) converge n drecton to the m domnant Schur vectors of A (see [17, chapter 5]). For ths to occur, we must force the columns of X (k) to be lnearly ndependent otherwse, all columns wll converge to the sngle domnant egenvector. Ths s done by computng a QR decomposton X (k) = QR at each step and settng X (k) := Q, whch ensures that the vectors wll be orthogonal. Raylegh-Rtz methods Power methods and subspace teratons are rarely used alone: they more often form a buldng block for more sophstcated algorthms. A large class of methods uses the Raylegh- Rtz procedure [17, chapter 4]: gven an orthonormal bass V n s that approxmates some porton of the spectrum of A, we compute the egenpars of a smaller (s s) matrx H := V H AV. If (λ, x) s an egenpar 3

4 of H then (λ, Vx) s called a Rtz par of A. If the range of V approxmates the range of the egenvectors of nterest suffcently well, the Rtz pars wll be a good approxmaton of the egenpars of the orgnal matrx. The best-known methods of ths type are the Arnold and Lanczos methods, n whch V s s an orthonormal bass for the Krylov subspace K s := span{v 0, Av 0, A 2 v 0,..., A s 1 v 0 } for some nonzero ntal vector v 0. Ths results n an upper Hessenberg matrx H s (or trdagonal, n the case of the Lanczos method). In general, the Rtz pars wll approxmate the extremal egenpars of A farly quckly, but nteror egenpars wll converge more slowly. Some methods for example, those of Davdson [3] and Jacob-Davdson [20] apply precondtonng to the search vectors obtaned n an Arnold procedure, whch can speed up convergence for some matrces. Flterng methods Computng nteror egenvalues s more challengng. The most conceptually straghtforward technque s the shft-and-nvert teraton, n whch we perform subspace or Krylov teratons usng (A σi) 1 nstead of A. Ths matrx has the same egenvectors as A, but ts egenvalues are λ new = 1/(λ σ), = 1,..., n. Hence, the largest-magntude egenvalues λ new of the nverted matrx wll be those correspondng to λ near σ. We then have faster convergence of egenpars near σ, but at the cost of multple lnear system solves wth a matrx that wll generally be ndefnte, and may be ll-condtoned f A has egenvalues very close to σ. For matrces that are too large to nvert, t s common to use flterng technques that elmnate portons of the search space n the drecton of unwanted egenvectors. For example, one can replace the standard power teratons wth polynomal teratons of the form z q = p q (A)z 0, wth p q beng a degree-q polynomal constructed based on some knowledge of the spectrum of A, accordng to what porton of the spectrum s of nterest. Examples of ths class of method nclude the explctly and mplctly restarted Arnold and Lanczos methods [8], and the polynomal fltered Lanczos method developed n [6]. An alternatve to polynomal flterng s to construct flters based on ratonal functons of A [9]. FEAST s an example of ths class of method, as ts subspace teratons use a ratonal functon ρ that approxmates the ndcator functon over the nterval of nterest [22]. Wthn the same group s the Contour Integral Raylegh-Rtz (CIRR) method [19]. Lke FEAST, ths method uses ratonal functons derved from Cauchy s ntegral theorem to fnd egenpars n a certan nterval; but CIRR nstead uses these functons to approxmate moment matrces, whch are then used n a Raylegh-Rtz procedure. 2.2 Generalzed egenvalue problem Of the Raylegh-Rtz technques lsted above for standard egenvalue problems, only FEAST and CIRR are drectly applcable to the generalzed egenvalue problem. In FEAST n the generalzed case, we use numercal ntegraton to estmate the ntegral π(m) := 1 2πı Γ (zb A) 1 Bq j dz, for dfferent rght-hand sdes q j. The CIRR method s smlar, but nstead approxmates several ntegrals of the form s k := 1 (z γ) k (zb A) 1 Bq 2πı j dz, k = 0, 1,..., m 1, Γ 4

5 for some γ nsde Γ. Note that n FEAST we only approxmate the ntegral s 0. For detals and explanaton of CIRR, we refer to [18, 19]. 3 Numercal quadrature In ths secton we focus on computng quadrature nodes and weghts whch we wll denote by z and w, respectvely so that 2m w ρ m (µ) = (4) z µ =1 approxmates the ndcator functon (as defned by the contour ntegral π(µ) n Equaton (1)) over the nterval [ 1, 1]. We are assumng wthout loss of generalty that the pencl (A, B) has been transformed lnearly such that the wanted egenvalues are n [ 1, 1]. We derve node-weght pars based on Gaussan and trapezodal quadrature n sectons 3.1 and 3.2, respectvely, and dscuss n secton 4 how the quadrature functon affects the convergence of FEAST. Integral parameterzaton For a gven scalng parameter S, we defne a famly of ellpses Γ S ntersectng the real lne at 1 and 1 by: Γ S = {γ : γ = γ(θ) = Seıθ + S 1 e ıθ } S + S 1, θ [0, 2π). (5) For S =, ths gves the unt crcle, whle for fnte S t gves a vertcally flattened ellpse. By varable substtuton, we can then evaluate the ntegral π(µ) by π(µ) = 1 2π γ (θ) 2π dθ =: f µ (θ)dθ, (6) 2πı 0 γ(θ) µ 0 where f µ (θ) = 1 (Se ıθ S 1 e ıθ )/(S + S 1 ) 2π (Se ıθ + S 1 e ıθ )/(S + S 1 ) µ. 3.1 Gaussan quadrature As n [14], we use m Gauss quadrature nodes θ (G) on the nterval [0, π] and another m nodes θ (G) on [π, 2π], all wth correspondng quadrature weghts ω (G). The quadrature approxmaton for π(µ) s then gven by π(µ) 2m =1 ω (G) f µ (θ (G) ) =: ρ (G) m (µ). We then defne the mapped Gauss nodes and weghts as z (G) (G) = Seıθ + S 1 e ıθ(g) S + S 1, w (G) = ω 2π Se ıθ(g) S 1 e ıθ(g) S + S 1, 5

6 and then wrte ρ (G) m (µ) = 2m =1 w (G) z (G) µ. (7) By constructon, the nodes and weghts both appear n complex conjugate pars. Specfcally, for = 1,..., m, z = z m+ and w = w m+. Note that, for real µ, Therefore, whch means we can rewrte Equaton (7) as ( ) w w z µ =. z µ ( ) w m+ z m+ µ = w, = 1,..., m, z µ ρ (G) m (µ) = 2 Re ( m =1 ) w (G) z (G). (8) µ Analogously, for real symmetrc matrces A, B, and a real matrx Q, we can wrte ρ (G) m (M)Q := 2m =1 w (G) (z (G) B A) 1 BQ = 2 Re ( m =1 w (G) (z (G) B A) 1 BQ Hence, n the strctly real case, we can perform the quadrature step of FEAST usng only m of the 2m quadrature nodes. Ths means the requred number of lnear system solves s now mp nstead of 2mp, where p s the number of columns of Q. 3.2 Trapezodal quadrature The constructon of the trapezodal approxmant ρ (T m ) s smlar to the Gaussan case, but we begn wth the trapezod rule to estmate 2π 0 f µ (θ)dθ. We use equspaced quadrature nodes θ (T ) = π( 1/2)/m and equal weghts ω (T ) = π/m for = 1,..., 2m. Defnng the mapped nodes z (T ) and weghts w (T ) from θ (T ) and ω (T ) n the same way as n the Gaussan case, we can now wrte ρ (T ) m (µ) = In ths case, note that for = 1,..., m, z (T ) = z (T ) 2m reasonng as we dd n the Gaussan case to wrte ρ (T ) m (µ) = 2 Re 2m =1 w (T ) z (T ) µ. (9) ) and w(t = w (T ) 2m. Therefore, we can use the same ( m =1 ) w (T ) z (T ) µ ). (10) 6

7 Fgure 1: Gaussan quadrature wth (half) number of quadrature ponts m = 3, 5, 8, and shape parameters S =, 2, Top fgures show quadrature nodes for m = 3, and bottom fgures show the value of ρ(µ) for postve µ only (as ρ s symmetrc about µ = 0). Smaller values of S result n more rapd decay outsde the nterval, at the expense of added oscllatons nsde the nterval. for all real µ. The analogous result also holds for the matrx case, assumng real symmetrc A, B, and real Q. 4 Convergence of the FEAST method We now descrbe how the choce of quadrature functon ρ affects convergence of the desred egenpars. Ths wll also shed lght on how to select a search space dmenson p for a partcular FEAST nstance. We gnore the presence of any errors n the computaton of ρ(m)q (k). For convergence analyss of FEAST wth lnear solver errors, we refer the reader to [22]. Assume that our search space has dmenson p, and that the egenvalues λ j, j = 1,..., n of M are 7

8 Fgure 2: ρ (T ) m (µ) for the nterval ( 1, 1) wth S =, 2, 1.25, and wth (half) number of nodes equal to 3, 5, 8. Plots show the results of the ntegraton for real µ. The functon s even, so we only show postve µ values. Smaller values of S result n more rapd decay outsde the nterval, at the expense of added oscllatons nsde the nterval. ordered such that ρ(λ 1 ) ρ(λ 2 )... ρ(λ n ). Let x j denote the egenvector correspondng to λ j. Let Q (k) := span(q (k) ), and let q denote the number of egenvalues of M n the nterval of nterest [λ, λ + ]. The followng result was proven n [22]: Theorem 1 (Tang and Polzz, 2014). At teraton k of FEAST wth quadrature rule ρ, for each j = 1, 2,..., q, there s a vector s j Q (k) such that x j s j B α ρ(λ p+1 )/ρ(λ j ) k, where α s a constant. In partcular, (I P (k) )x j B α ρ(λ p+1 )/ρ(λ j ) k, where P (k) s the B-orthogonal projector onto the space Q (k). Note that, for the standard egenvalue problem, the B-norm reduces to the 2-norm. In ths case, Theorem 1 tells us that the dstance between an egenvector x j and the span of the Rtz vectors s attenuated by a factor of ρ(λ p+1 )/ρ(λ j ) after each FEAST teraton. Selectng a search space sze From Theorem 1, we see that FEAST wll converge more quckly the smaller the value of ρ(λ p+1 ). For a gven quadrature functon ρ, ths value wll be monotoncally nondecreasng as the dmenson p of our search space s ncreased. But, because ncreasng p requres solvng addtonal lnear systems at each teraton, there s a trade-off between the FEAST convergence rate and the amount of work per teraton. In the orgnal FEAST paper [14] and n [22], t was suggested that one should generally use p = 1.5q, where q s the number of egenpars n the search nterval. (Ths, of course, requres a pror estmate of q, but technques to address ths problem have been developed n [12].) The reasonng behnd ths recommendaton was that, when ρ(µ) s an 8-pont Gaussan quadrature functon over a crcular contour for the nterval [ 1, 1], ρ(1.5) Therefore, f the spectrum of M s approxmately evenly dstrbuted, we would expect to see (I P (k) )x j B 0 at a rate of 10 3k. 8

9 However, Güttel et al. [9] later explored the use of dfferent quadrature functons wth ellptcal contours, nstead of just crcular ones. From Fgures 1 and 2, we see that for both quadrature types, functons based on ellptcal contours decay more quckly outsde the nterval of nterest, at the expense of some oscllaton and early decay wthn the nterval. We would expect and ths was verfed expermentally n [9] that by usng ellptcal contours, we can acheve a desrable convergence rate wth a smaller search space than we could wth crcular contours. 5 FEAST wth teratve lnear solvers The man work of FEAST s n the soluton of the lnear systems to approxmate the spectral projector. For standard problems wth real A, these equatons are of the form (z I A)y,j = q j, = 1,..., m, j = 1,..., p. (11) If A s large and sparse, we would lke to be able to solve these equatons wth teratve solvers. However, the matrces wll generally have egenvalues near the contour ponts, makng the systems ll-condtoned. Addtonally, the shfts z are complex, meanng that the systems wll be non-hermtan even f A s symmetrc. 5.1 Iteratve solver canddates For each of the p rght-hand sde vectors q j, we must solve m shfted systems (wth complex shfts). If we use a Krylov solver, we can solve these more effcently as Krylov subspaces are nvarant under shfts namely, f V s s a bass for an s-dmensonal Krylov subspace and H s := V H s ( A)V s, then V H s (z I A)V s = z I + H s, where H s s upper Hessenberg. Symmetrc case When A s symmetrc, H s s trdagonal, and therefore so s z I + H s. Thus, for a gven rght-hand sde q j, each shfted system reduces to a trdagonal matrx, all wth the same Krylov bass vectors V s. Ths means that even though our matrces are non-hermtan, we can stll formulate a three-term recurrence relaton between the bass vectors v j [16]. Hence, we can solve these systems wth the MINRES method [23]. Non-symmetrc case When A s non-symmetrc [21], we can stll use the same Krylov subspace for the shfted systems, but MINRES s no longer applcable. We can alternatvely use the GMRES method. The GMRES-DR-Sh method of [4] s one approach for takng advantage of shfted systems wth multple rght-hand sdes, although n ts orgnal mplementaton t can t be used wth precondtonng. 5.2 Iteratve solvers wth precondtonng Ideally, we would lke to use a precondtoner P to accelerate the convergence of the lnear solvers we ve just descrbed. 9

10 MINRES : In general, we can precondton MINRES as long as the precondtoner P s symmetrc postve defnte. However, matters are complcated for our problem because the systems are shfted by a complex value z. Assume we have factored P = P 1/2 P 1/2 (wth P 1/2 symmetrc), and consder the splt precondtoned system: P 1/2 (z I A)P 1/2 (P 1/2 y,j ) = P 1/2 q j. If (z I A) was real symmetrc, t would follow that the teraton matrx P 1/2 (z I A)P 1/2 would be real symmetrc as well. However, n our case the shft z s complex, whch would mean that the precondtoned matrx, whle stll beng symmetrc, wll be non-hermtan. Wth the complex elements of the system not necessarly restrcted to a shft along the dagonal, standard MINRES may not be applcable, as the Krylov bass vectors could no longer be expressed as a three-term recurrence. There are approaches that could be used to overcome ths: for example, we could use precondtonng f we used the MINRES varant for complex symmetrc systems developed by Cho [2], but an nvestgaton of ths s beyond the scope of ths report. GMRES : When we don t precondton GMRES, we can easly take advantage of the shfted systems. Snce the Krylov subspaces are nvarant under shfts, we can construct a sngle Krylov bass for each rght-hand sde. Matters are more complcated f we wsh to use precondtonng. When we precondton the shfted systems, they wll no longer be shfted by the dentty; thus, ther Krylov spaces are no longer equvalent, and reusng nformaton effcently between the dfferent systems s not as straghtforward. There are precondtoned methods that can mprove effcency by takng advantage of related lnear systems and multple rght-hand sdes see, for example, the work of Parks et al. [13] but an exploraton of these methods s left as a subject for future work. 6 Numercal experments All experments focus on the standard egenvalue problem wth real symmetrc A. We use MINRES-FEAST to denote FEAST wth MINRES lnear solvers, and GMRES-FEAST to denote the use of GMRES. All code s mplemented n Matlab. We use the Parallel Computng toolbox wth 4 workers n the parallel pool to solve the p dfferent rght-hand sdes n Equaton (11). Experments were run on a commodty PC wth a 3.60 GHz Intel Core processor and 16 GB of memory. We check for convergence of FEAST by measurng the resdual norms of the computed Rtz pars (λ, x) that have λ [λ, λ + ]. Specfcally, we say the method has converged f Ax λx 2 < tol for all such egenpars (wth x 2 = 1), where tol s a user-specfed resdual tolerance. The followng experments are all performed wth tol = Test matrces We begn wth a bref descrpton of the three real symmetrc matrces used n the followng experments. We use the Trefethen 2000 matrx 1, the Andrews matrx 2 and the c-bg matrx from the Schenk IBMNA

11 group 3, all of whch were obtaned from the Unversty of Florda Sparse Matrx Collecton [5]. Table 1 summarzes some propertes of the matrces, and sparsty patterns are shown n Fgure 3. Matrx Trefethen 2000 Andrews c-bg n 2,000 60, ,241 Nonzeros 41, ,154 2,340,859 Full egen-range [1.12, 1.74e+04] [0.0, 36.49] [ , ] nnz(l+u), wth AMD 1,699, ,019, ,569,235 Table 1: Summary of the dfferent test matrces used n our numercal experments. 6.2 GMRES-FEAST vs. MINRES-FEAST In ths experment we explore the performance of FEAST wth MINRES, GMRES, and GMRES precondtoned wth ILUTP usng a drop tolerance of We use no restartng wth ether GMRES solver. Ths experment s performed on the symmetrc postve defnte 2,000 2,000 Trefethen 2000 matrx. For our experment, we fnd the q = 20 egenpars wth λ [31.2, 113.5]. We use a search space dmenson p = 26 wth the same random ntal subspace Q (0) for all methods. The tolerance of all lnear solvers s We use 8-pont Gaussan quadrature wth shape parameter S = 2.0. Accordng to Theorem 1, n the absence of lnear solver errors ths should gve us a convergence factor of for the slowest-convergng egenpar (note that we can t generally compute ths value n practce, as t requres knowledge of the egenvalues of the matrx). Results In all cases, FEAST took three teratons to converge. Table 2 summarzes the number of lnear solver teratons and computatonal tme requred for each teraton. MINRES GMRES GMRES-ILUTP Iteraton 1 (FEAST) Iteraton 2 (FEAST) Iteraton 3 (FEAST) Total Solver teratons 166, ,213 36, ,835 Tme (s) Solver teratons 146, ,305 27, ,777 Tme (s) Solver teratons 1,487 1,513 1,509 4,509 Tme (s) Table 2: MINRES-FEAST vs. GMRES-FEAST, for fndng 20 egenpars of the Trefethen 2000 matrx. MINRES and GMRES refer to unprecondtoned solvers, wth Krylov subspaces recycled between shfted systems, and GMRES-ILUTP refers to GMRES precondtoned wth ILUTP(τ = 0.01), wth all systems solved ndependently. Solver teratons denotes the total number of lnear solver teratons used for all m p = 208 lnear system solves at each FEAST teraton. Total tme for GMRES-ILUTP ncludes factorzaton tme

12 (a) Trefethen 2000 (b) Andrews (c) c-bg Fgure 3: Sparsty patterns of the matrces Trefethen 2000; (upper left quarter of) Andrews; and c-bg. At the frst teraton when all solvers have the same random rght-hand sde vectors MINRES requres an average of 801 teratons per lnear system solve. For GMRES wthout precondtonng, the average s 706 teratons per lnear system solve; however, due to orthogonalzaton costs, these teratons are much more expensve, and the computatonal tme ncreases substantally. We get much better results when we precondton GMRES, nstead of takng advantage of the shfts as we do n the unprecondtoned verson. Computng the ILUTP factorzatons takes a total of 0.05 seconds for all eght systems, and the resultng factorzatons L + U have between 2,500 and 2,800 nonzero entres. By precondtonng, we go from an average of 706 teratons per lnear system to an average of 7 teratons. 12

13 6.3 Precondtoned GMRES-FEAST for large matrces From the prevous experment, we saw that we ganed more computatonal savngs by precondtonng the lnear systems than by explotng symmetry (as wth MINRES) or takng advantage of equvalent Krylov subspaces across shfted systems (as wth MINRES and unprecondtoned GMRES). We now explore the scalablty of precondtoned GMRES-FEAST by testng t on two large matrces: the Andrews matrx (sze 60,000) and the c-bg matrx (sze 345,241). In both cases, we fnd 100 egenpars (extreme for Andrews and nteror for c-bg) usng a random ntal subspace of sze p = 130. We use 8-pont trapezodal quadrature wth S = All shfted matrces are precondtoned wth ILUTP and all solver tolerances are set to In both cases, convergence of all desred egenpars s acheved n three teratons. A summary of the results s gven n Table 3. Iteraton counts for both the FEAST algorthm and the lnear solvers suggest that our approach s effectve for large matrces. Matrx Andrews (60,000 60,000) c-bg (345, ,241) Expermental settng Interval [λ, λ + ] [0.0, 1.3] [45.2, 50.0] ILUTP drop tolerance 5.0 1e e 3 Results Average nnz(l+u) (over all shfts) 2,767,412 5,697,775 Total ILUTP factorzaton tme (s) 1, Average GMRES teratons per LS solve Max GMRES teratons for any LS solve FEAST teratons to convergence 3 3 Average tme per FEAST teraton (s) 8, ,578.3 Table 3: Results of FEAST for fndng 100 egenpars of the matrces Andrews (sze 60,000) and c-bg (sze 345,241). 7 Conclusons In ths report we explored the use of teratve solvers wthn the FEAST method to compute egenpars of sparse symmetrc matrces. We conclude that usng FEAST wth precondtoned GMRES s a promsng approach, and we demonstrated the effectveness of the method by computng egenpars of matrces of dmensons up to the hundreds of thousands. There are several possble ways we mght employ teratve solvers even more effectvely wthn FEAST. We observed n our experments that MINRES teratons are cheap, but that wthout precondtonng, the teraton counts tend to be rather hgh. If we apply a complex symmetrc verson of MINRES, as was developed n [2], we could then use MINRES solvers wth precondtonng. Wthn FEAST, we solve a few (often eght) dfferent systems, for several rght-hand sdes. Our precondtoned approach doesn t attempt to explot any relatonshp between the dfferent systems, and none of the methods explored here have taken advantage of the fact that each system s beng solved 13

14 repeatedly for multple rght-hand sdes. Krylov subspace recyclng methods partcularly those that can be used n conjuncton wth precondtonng, such as n [13] represent a promsng area of future research. In performng the numercal experments, we observed that we requre tght lnear solver tolerances (generally somewhere between 10 9 and ) to obtan relable FEAST convergence to Further research s needed to determne whether there may be ways to reduce the need for very accurate lnear system solves. Ths may also mean that, under some crcumstances (such as when we fnd we need a solver tolerance of ), we gan lttle from usng teratve nstead of drect solvers. A queston for future research s to determne more precsely the crcumstances under whch a partcular FEAST nstance would beneft most from the use of teratve solvers. A lmtaton of ths work s that t doesn t take full advantage of one of the key benefts of FEAST, whch s that FEAST s nherently parallelzable: we can parallelze the solutons of the separate lnear systems as we ve done here, or we can partton the nterval of nterest and run FEAST on each segment n parallel (see [1, 9]), or both. We ve made lmted use of parallelzaton here, but could acheve better results wth more computng resources and more sophstcated parallelzaton schemes. Another possble drecton for future work n ths area would be the use of parallel teratve solvers. References [1] H. M. Aktulga, L. Ln, C. Hane, E. G. Ng, and C. Yang. Parallel egenvalue calculaton based on multple shft-nvert Lanczos and contour ntegral based spectral projecton method. Parallel Comput., 40(7): , July [2] S. T. Cho. Mnmal resdual methods for complex symmetrc, skew symmetrc, and skew Hermtan systems. CoRR, abs/ , [3] M. Crouzex, B. Phlppe, and M. Sadkane. The Davdson method. SIAM J. Sc. Comput., 15(1):62 76, Jan [4] D. Darnell, R. B. Morgan, and W. Wlcox. Deflated GMRES for systems wth multple shfts and multple rght-hand sdes. Lnear Algebra and ts Applcatons, 429(10): , [5] T. A. Davs and Y. Hu. The Unversty of Florda sparse matrx collecton. ACM Trans. Math. Softw., 38(1):1:1 1:25, Dec [6] H. Fang and Y. Saad. A fltered Lanczos procedure for extreme and nteror egenvalue problems. SIAM Journal on Scentfc Computng, 34(4):A2220 A2246, [7] J. G. F. Francs. The QR transformaton a untary analogue to the LR transformaton, Part 1. The Computer Journal, 4(3): , [8] G. H. Golub and C. F. Van Loan. Matrx Computatons (3rd Ed.). Johns Hopkns Unversty Press, Baltmore, MD, USA, [9] S. Güttel, E. Polzz, P. T. P. Tang, and G. Vaud. Zolotarev quadrature rules and load balancng for the FEAST egensolver. SIAM Journal on Scentfc Computng, 37(4):A2100 A2122,

15 [10] J. Kestyn, E. Polzz, and P. T. P. Tang. FEAST egensolver for non-hermtan problems. CoRR, abs/ , [11] L. Krämer, E. D. Napol, M. Galgon, B. Lang, and P. Bentnes. Dssectng the FEAST algorthm for generalzed egenproblems. CoRR, abs/ , [12] E. D. Napol, E. Polzz, and Y. Saad. Effcent estmaton of egenvalue counts n an nterval. CoRR, abs/ , [13] M. L. Parks, E. de Sturler, G. Mackey, D. D. Johnson, and S. Mat. Recyclng Krylov subspaces for sequences of lnear systems. SIAM Journal on Scentfc Computng, 28(5): , [14] E. Polzz. A densty matrx-based algorthm for solvng egenvalue problems. CoRR, abs/ , [15] E. Polzz. FEAST egenvalue solver v3.0: User gude. CoRR, abs/ , [16] Y. Saad. Iteratve methods for sparse lnear systems. SIAM, [17] Y. Saad. Numercal methods for large egenvalue problems. SIAM, [18] T. Sakura and H. Sugura. A projecton method for generalzed egenvalue problems usng numercal ntegraton. Journal of Computatonal and Appled Mathematcs, 159(1): , th Japan- Chna Jont Semnar on Numercal Mathematcs; In Search for the Fronter of Computatonal and Appled Mathematcs toward the 21st Century. [19] T. Sakura and H. Tadano. CIRR: a Raylegh-Rtz type method wth contour ntegral for generalzed egenvalue problems. Hokkado Math. J., 36(4): , [20] G. L. G. Slejpen and H. A. Van der Vorst. A Jacob-Davdson teraton method for lnear egenvalue problems. SIAM Journal on Matrx Analyss and Applcatons, 17(2): , [21] P. T. P. Tang, J. Kestyn, and E. Polzz. A new hghly parallel non-hermtan egensolver. In Proceedngs of the Hgh Performance Computng Symposum, HPC 14, pages 1:1 1:9, San Dego, CA, USA, Socety for Computer Smulaton Internatonal. [22] P. T. P. Tang and E. Polzz. FEAST as a subspace teraton egensolver accelerated by approxmate spectral projecton. SIAM Journal on Matrx Analyss and Applcatons, 35(2): , Copyrght , Socety for Industral and Appled Mathematcs; Last updated [23] H. A. Van der Vorst. Iteratve Krylov methods for large lnear systems. Cambrdge monographs on appled and computatonal mathematcs. Cambrdge Unversty Press, Cambrdge, UK, New York,