Preconditioned Locally Harmonic Residual Method for Computing Interior Eigenpairs of Certain Classes of Hermitian Matrices

Transcription

1 MITSUBISHI ELECTRIC RESEARCH LABORATORIES Preconditioned Locay Harmonic Residua Method for Computing Interior Eigenpairs of Certain Casses of Hermitian Matrices Vecharynski, E.; Knyazev, A. TR November 2014 Abstract We propose a Preconditioned Locay Harmonic Residua (PLHR) method for computing severa interior eigenpairs of a generaized Hermitian eigenvaue probem, without traditiona spectra transformations, matrix factorizations, or inversions. PLHR is based on a short-term recurrence, easiy extended to a bock form, computing eigenpairs simutaneousy. PLHR can take advantage of Hermitian positive definite preconditioning, e.g., based on an approximate inverse of an absoute vaue of a shifted matrix, introduced in [SISC, 35 (2013), pp. A696- A718]. Our numerica experiments demonstrate that PLHR is efficient and robust for certain casses of arge-scae interior eigenvaue probems, invoving Lapacian and Hamitonian operators, especiay if memory requirements are tight. arxiv This work may not be copied or reproduced in whoe or in part for any commercia purpose. Permission to copy in whoe or in part without payment of fee is granted for nonprofit educationa and research purposes provided that a such whoe or partia copies incude the foowing: a notice that such copying is by permission of Mitsubishi Eectric Research Laboratories, Inc.; an acknowedgment of the authors and individua contributions to the work; and a appicabe portions of the copyright notice. Copying, reproduction, or repubishing for any other purpose sha require a icense with payment of fee to Mitsubishi Eectric Research Laboratories, Inc. A rights reserved. Copyright c Mitsubishi Eectric Research Laboratories, Inc., Broadway, Cambridge, Massachusetts 02139

2

3 PRECONDITIONED LOCALLY HARMONIC RESIDUAL METHOD FOR COMPUTING INTERIOR EIGENPAIRS OF CERTAIN CLASSES OF HERMITIAN MATRICES EUGENE VECHARYNSKI AND ANDREW KNYAZEV Abstract. We propose a Preconditioned Locay Harmonic Residua (PLHR) method for computing severa interior eigenpairs of a generaized Hermitian eigenvaue probem, without traditiona spectra transformations, matrix factorizations, or inversions. PLHR is based on a short-term recurrence, easiy extended to a bock form, computing eigenpairs simutaneousy. PLHR can take advantage of Hermitian positive definite preconditioning, e.g., based on an approximate inverse of an absoute vaue of a shifted matrix, introduced in [SISC, 35 (2013), pp. A696 A718]. Our numerica experiments demonstrate that PLHR is efficient and robust for certain casses of arge-scae interior eigenvaue probems, invoving Lapacian and Hamitonian operators, especiay if memory requirements are tight. Key words. Eigenvaue, eigenvector, Hermitian, absoute vaue preconditioning, inear systems 1. Introduction. We are interested in computing a subset of eigenpairs of the generaized Hermitian eigenvaue probem (1.1) Av = λbv, A = A C n n, B = B > 0 C n n, that correspond to eigenvaues cosest to a given shift σ, which is rea and points to the eigenvaues in the interior of the spectrum. We refer to (1.1) as an interior eigenprobem, and ca the targeted eigenpairs interior eigenpairs. Interior eigenpairs are of fundamenta interest in some physica modes, e.g., for first-principes eectronic structure anaysis of materias using a semi-empirica potentia or a charge patching method [37, 38, 39]. There, the requested eigenvaues correspond to energy eves around a materia-dependent energy shift σ, and the eigenvectors represent the associated wave functions. We assume that the size n of the matrices is so arge that eigenvaue sovers based on expicit matrix transformations are impractica. Other conventiona ways of soving the interior eigenvaue probems are based on spectra transformations that aow reducing the interior eigenprobem to an easier task of computing extreme (smaest or argest) eigenvaues of a transformed penci. Such transformations incude the socaed shift-and-invert (SI) and foded spectrum (FS) approaches; see, e.g., [2]. Whie both techniques are extensivey used, they face serious issues as n increases. In particuar, SI reies on repeated soution of arge shifted inear systems, which generay becomes extremey inefficient or infeasibe for arge n. Simiar difficuties are experienced in methods reying on contour integration, e.g., [29]. The FS approach for standard, i.e., with identity B = I, eigenprobems is inversefree. However, in exact arithmetic, it is equivaent to soving an eigenprobem for the matrix (A σi) 2, which squares the condition number, resuting in severey tightened custering of the targeted eigenvaues. For the generaized, i.e., with B I, eigenprobems, FS requires inverting the matrix B, which can be probematic. Additionay, Preiminary version posted at The resuts presented in this work are partiay based on the PhD thesis of the first coauthor [34]. Computationa Research Division, Lawrence Berkeey Nationa Laboratory, Berkeey, CA (eugene.vecharynski[at]gmai.com) Mitsubishi Eectric Research Laboratories; 201 Broadway Cambridge, MA (knyazev[at]mer.com) 1

4 2 EUGENE VECHARYNSKI AND ANDREW KNYAZEV avaiabe preconditioning, normay based on approximating an inverse of A σb, may not be appicabe for FS, which shoud be preconditioned by an approximate inverse of the squared operator. For certain casses of eigenprobems, mutigrid (MG) agorithms have proved to be very efficient; see, e.g., [3, 10, 19, 20] and references therein. However, the appicabiity of such eigensovers faces the same kind of imitations as exist for the MG inear sovers in that either geometric (grid) information must be avaiabe at soution time or the probem s structure shoud be appropriate for the agebraic MG approaches. Another popuar technique for interior eigenprobems is based on combining poynomia transformations, caed fiters, with the standard Lanczos method, as in fitered Lanczos procedure [6]. The approach cannot easiy be extended to generaized eigenprobems and does not take advantage of preconditioners that may be avaiabe. Residua minimization scheme, direct inversion in the iterative subspace (RMM- DIIS), see [40], provides an aternative, but is known to be unreiabe if the initia approximations are not cose enough to the targeted eigenpairs. RMM-DIIS has to be preceded by, e.g., generaized Davidson [22] or Jacobi Davidson [30] methods, such as, e.g., in Vienna ab initio simuations (VASP) package [18]. Combining harmonic Rayeigh Ritz (RR) procedure [2] with the Davidson s famiy of methods has recenty aimed at suppementing RMM-DIIS in VASP; see [12]. Success of the Davidson type methods is often determined by two interdependent agorithmic components: the maximum aowed size of the search subspace and the quaity of the preconditioner, typicay given as a form of an approximate sove of the system with a shifted matrix A σb. The maximum size of the search subspace has to be increased if the preconditioner quaity deteriorates. In practica computations, preconditioners that ack the desired quaity are common, especiay if the shift σ targets the eigenvaues that are deep in the interior of the spectrum. We deveop new preconditioned iterative schemes for interior eigenprobems that exhibit a ower sensitivity to the preconditioner quaity and at the same time ead to fixed and reativey modest memory requirements. A connection between iterative methods for singuar homogeneous Hermitian indefinite systems and eigenprobems in Section 2 motivates our deveopment. Our base method, caed Preconditioned Locay Harmonic Residua (PLHR) and presented in Section 3, computes a singe eigenpair. In Section 4, we generaize PLHR to bock iterations, where targeted eigenpairs, as determined by σ, are computed simutaneousy, simiar to LOBPCG [15]. PLHR requires that the preconditioner is Hermitian positive definite (HPD). In Section 5, we suggest that the preconditioner T shoud approximate the inverted matrix absoute vaue A σb. Preconditioners of this kind, caed the absoute vaue (AV) preconditioners, have been recenty introduced in [35]. In particuar, [35] describes an MG scheme for approximating the inverse of the AV of the shifted Lapacian. Thus, the approach can be directy appied as a PLHR preconditioner for computing interior eigenpairs of the Lapacian matrix, which is demonstrated in our numerica experiments in Section 6. Furthermore, the suitabe HPD preconditioners are readiy avaiabe in eectronic structure cacuations, where we combine PLHR with the existing state-of-the-art preconditioner [32], aso in Section 6. Whie, in this work, we focus ony on severa mode probems with aready avaiabe HPD preconditioners, the proposed PLHR approach can be appicabe to broader casses of eigenprobems that admit appropriate HPD preconditioning. Investigation of such probems, as we as the deveopment of the corresponding preconditioners, is a matter of future research, and is outside the scope of the current paper.

5 COMPUTING INTERIOR EIGENPAIRS OF HERMITIAN MATRICES 3 An important novety of the proposed approach is a modification of the harmonic RR procedure, obtained by a proper utiization of the HPD preconditioner T in the Petrov-Gaerkin condition for extracting the approximate eigenvectors, incuding its rea arithmetic impementation. The new extraction technique, that we ca the T - harmonic RR procedure in Section 3, is critica. Our tests demonstrate remarkabe robustness if the T -harmonic procedure is used. The gains are especiay evident if the preconditioner quaity deteriorates, whereas the memory requirements remain fixed. 2. Preconditioned nu space computations. We start with an easier probem of finding a nu space component of a Hermitian matrix. Let λ q be a targeted eigenvaue of (1.1) cosest to the shift σ. We assume that λ q is known. Then eigenprobem (1.1) turns into the singuar homogeneous inear system (2.1) (A λ q B)v = 0. It is cear that (2.1) has a non-trivia nu space soution determining an eigenvector v q associated with λ q. Thus, in the ideaized setting, where λ q is avaiabe, an eigensover for (1.1) can be given by an appropriate soution scheme for inear system (2.1). In this sense, inear sovers for (2.1) can be viewed as prototypica, or ideaized, methods for computing the eigenpair (λ q, v q ). The described connection between inear and eigenvaue sovers has been emphasized in iterature; see, e.g., [13, 15]. For exampe, in [15], a specia case is considered where λ q is the smaest eigenvaue of (1.1) and, hence, the system (2.1) is Hermitian positive semidefinite. A proper choice of the inear sover a three-term recurrent form of the preconditioned conjugate gradient (PCG) method has ed to derivation of the popuar LOBPCG agorithm for finding extreme eigenpairs; see [15, 16]. We foow a simiar approach. Assuming that λ q is known, we seect efficient preconditioned inear sovers capabe of computing a non-trivia soution of (2.1). Viewing these inear sovers as ideaized eigensovers, we then extend them to the practica case where λ q is unknown. Since the targeted eigenvaue λ q can be ocated anywhere in the interior of the spectrum of the penci A λb, the Hermitian coefficient matrix of system (2.1) is generay indefinite in contrast to [15], which makes PCG inappicabe. We ook for suitabe preconditioned short-term recurrent Kryov subspace type methods that can be appied to singuar Hermitian indefinite systems of the form (2.1) and guarantee inear convergence. An optima technique in this cass of methods is the preconditioned minima residua (PMINRES) agorithm [25]. Whie most commony appied to non-singuar systems, the method is aso guaranteed to work for a cass of singuar consistent Hermitian systems, such as (2.1). The preconditioner for PMINRES shoud normay be HPD, and the convergence of the method is governed by the spectrum distribution of the preconditioned matrix. However, the standard impementation of the agorithm is deepy rooted in the Lanczos procedure, where the avaiabiity of the matrix A λ q B is crucia at every stage of the process. Since this assumption is not reaistic in the context of eigenvaue computations, where ony an approximation of λ q is at hand, PMINRES fais to provide a proper insight into the structure of oca subspaces that determine a new approximate soution. Simiar arguments appy to methods that are mathematicay equivaent to PMINRES, such as, e.g., orthodir(3) [9, 27, 42]. Returning back for a moment to the specia case where λ q is the smaest eigenvaue of (1.1), a preconditioned steepest descent (PSD) inear sover can be viewed

6 4 EUGENE VECHARYNSKI AND ANDREW KNYAZEV as a predecessor of the PCG inear sover, on the one hand, and as restarted preconditioned Lanczos inear sover, on the other hand. We use this viewpoint to describe an anaog of a PSD-ike inear sover for the indefinite case. To that end, we restart PMINRES in such a way that inear convergence is preserved (possiby with a ower rate), whereas the number of steps between the restarts is kept to a minimum possibe. It is shown in [34] that in order to maintain PMINRES convergence, the number of steps between the restarts shoud be no smaer than two. Thus, the simpest convergent residua minimizing iteration for (2.1) is (2.2) v (i+1) = v (i) + α (i) T r (i) + β (i) T (A λ q B) T r (i), i = 0, 1,... ; where r (i) = (λ q B A) v (i) and the iteration parameters α (i), β (i) are chosen to minimize the T -norm of the residua r (i+1), i.e., are such that (2.3) r (i+1) T = min u K (i) r (i) (A λ q B)u T, where (2.4) K (i) = span { T r (i), T (A λ q B)T r (i)}. Here and throughout, for a given HPD matrix M, the corresponding vector M-norm is defined as M = (, M ) 1/2. Theorem 2.1 ([34]). Given an HPD preconditioner T, the iteration (2.2) (2.4) converges to a nontrivia soution of (2.1), provided that the initia guess x (0) has a non-zero projection onto the nu space of A λ q B. Moreover, if λ q is a simpe eigenvaue 1 of A λb and µ 1 µ 2... µ q 1 < µ q = 0 < µ q+1 <... µ n are the eigenvaues of the preconditioned matrix T (A λ q B), then the residua norm reduction is given by (2.5) where (2.6) κ = ( µn µ q+1 ( µ1 µ q 1 r (i+1) T r (i) T κ 1 κ + 1 < 1, ) ( 1 + µ ) n µ q+1, if µ 1 µ q 1 µ n µ q+1 µ q 1 ) ( 1 + µ ) 1 µ q 1, if µ 1 µ q 1 > µ n µ q+1. µ q+1 The proof of Theorem 2.1 reies on the anaysis of iterations of the form (2.2), where the parameters α (i) and β (i) are fixed. In this case, (2.2) can be viewed as a stationary Richardson type scheme for a poynomiay preconditioned system, and a standard convergence anaysis (see, e.g., [1, Theorem 5.6]) appies to determine the vaues of the parameters that yied an optima convergence, which is given by (2.5) (2.6). Since the convergence of the stationary iteration cannot be better then that of (2.2) with the residua minimizing parameters (2.3), bound (2.5) (2.6) immediatey appies to method (2.2) (2.4). For more detais, we refer the reader to [34]. 1 This assumption is made ony to simpify the statement of the theorem. The theorem can be simiary formuated for the case of mutipe eigenvaue.

7 COMPUTING INTERIOR EIGENPAIRS OF HERMITIAN MATRICES 5 A natura way to enhance (2.2) (2.4) is by introducing an additiona term that hods information from the previous step. By anaogy with a three-term recurrent form of PCG, consider the scheme (2.7) v (i+1) = v (i) +α (i) T r (i) +β (i) T (A λ q B) T r (i) +γ (i) (v (i) v (i 1) ), i = 0, 1,..., where v ( 1) = 0. Here, the scaar parameters α (i), β (i), and γ (i) are chosen according to oca minimaity condition (2.3) with { (2.8) K (i) = span T r (i), T (A λ q B)T r (i), v (i) v (i 1)}. Since K (i) in (2.8) contains the subspace in (2.4), the empoyed minimaity condition (2.3) impies that convergence of (2.7) is not worse than that of (2.2). Hence, the resuts of Theorem 2.1 aso appy to (2.7) with (2.3) and (2.8). In this case, however, bound (2.5) (2.6) is ikey to be an overestimate, and in practice the presence of the additiona vector in the recurrence eads to a faster convergence. In fact, the scheme (2.7) has been observed to exhibit behavior simiar to PMINRES up to the occurrence of superinear convergence [34], which is a consequence of the goba optimaity of the atter. Additionay, iteration (2.7) reveas the structure of oca subspaces that are used to determine the improved approximate soution, which we expoit in the next section for constructing the tria subspaces in the context of the eigenvaue cacuations. For these reasons, we choose (2.7) with (2.3) and (2.8) to be a base inear sover for the nu space probem (2.1) and, in what foows, use it as a starting point for deriving preconditioned interior eigensovers. 3. The Preconditioned Locay Harmonic Residua method. We now present an approach for computing an eigenpair (λ q, v q ) of (1.1) that corresponds to the eigenvaue cosest to a given shift σ. The method is motivated by the preconditioned nu space finders discussed in the previous section. Our idea is to extend the base inear sover (2.7) to the case of eigenvaue computations by introducing a series of approximations into recurrence (2.7) and optimaity condition (2.3). The nu space finding scheme (2.7) suggests that, if λ q is known, the improved eigenvector approximation v (i+1) beongs to the subspace { (3.1) span v (i), T (A λ q B)v (i), T (A λ q B)T (A λ q B)v (i), v (i 1)}. Ceary, in practice, the exact vaue of λ q is unavaiabe and, hence, the computation of (3.1) cannot be performed. In order to obtain a computabe subspace in the context of eigenvaue probem (1.1), we approximate λ q by the Rayeigh Quotient (RQ) λ (i) λ(v (i) ) = (v (i), Av (i) )/(v (i), Bv (i) ). As a resut, at each iteration i, the foowing tria subspace is introduced: { (3.2) Z (i) = span v (i), w (i), s (i), v (i 1)}, where w (i) = T (Av (i) λ (i) v (i) ) is the preconditioned residua for probem (1.1), and s (i) = T (Aw (i) λ (i) w (i) ).

8 6 EUGENE VECHARYNSKI AND ANDREW KNYAZEV Next, we address the question of extracting an approximate eigenpair from the subspace Z (i). Let us reca that minimaity principe (2.3), utiized by the base nu space finder, impies the orthogonaity reation (3.3) r (i+1) (A λ q B)v (i+1) T (A λ q B)K (i), where T denotes the orthogonaity in the T -based inner product and K (i) is defined in (2.8). Foowing the anaogy with the inear sover, in the context of the eigenvaue probem, we introduce a simiar condition: (3.4) (A θb)v (i+1) T (A σb)z (i), v (i+1) B = 1. Here, we find a B-unit vector v (i+1) and a scaar θ C that ensure the orthogonaity of the vector (A θb)v (i+1) to the subspace (A σb)z (i). Condition (3.4) has been obtained from (3.3) as a resut of two modifications. First, the known eigenvaue λ q in the residua r (i+1) of inear system (2.1) has been repaced by an unknown θ, giving rise to the residua ike vector (A θb)v (i+1) for the eigenvaue probem. Note that this vector shoud be distinguished from the standard eigenresidua (A λ (i+1) B)v (i+1), where λ (i+1) is the RQ. Since the orthogonaity of (A θb)v (i+1) in (3.4) is invariant with respect to the norm of v (i+1), for definiteness, we request that the vector has a unit B-norm. The second modification has been the repacement of the subspace (A λ q B)K (i) in the right-hand side of (3.3) by (A σb)z (i), where Z (i) is the tria subspace defined in (3.2). We expect that (A σb)z (i) captures we the idea subspace (A λ q B)K (i). Note that in the case where λ q is known, K (i) Z (i). Hence, if σ is cose to the targeted eigenvaue, both subspaces are cose to each other. We next discuss a procedure for computing the pair (θ, v (i+1) ) in (3.4) The T -harmonic Rayeigh Ritz procedure. Motivated by (3.4), et us consider a genera probem, where we are interested in finding a number of pairs (θ, v) that approximate eigenpairs of (1.1) corresponding to the eigenvaues cosest a given shift σ, such that each v beongs to a given m-dimensiona subspace Z, and (3.5) (A θb)v T (A σb)z, v B = 1. One can immediatey recognize that (3.5) represents the Petrov-Gaerkin condition [2, 28] formuated with respect to the T -based inner product. In the standard case with T = I, this condition eads to the we known harmonic Rayeigh-Ritz (RR) procedure, where approximate eigenpairs are deivered by the harmonic Ritz pairs [22, 23, 24]. The corresponding vectors Av θbv are sometimes caed the harmonic residuas; see, e.g., [36]. Let a basis of Z be given by coumns of an n-by-m matrix Z, so that any eement of the subspace can be expressed in the form v = Zy, where y is a vector of coefficients. Then the orthogonaity constraint (3.5) transates into Z (A σb)t (A θb)zy = 0, Zy B = 1. This is equivaent to the eigenvaue probem (3.6) Z (A σb)t (A σb)zy = ξz (A σb)t BZy, Zy B = 1, where ξ θ σ.

9 COMPUTING INTERIOR EIGENPAIRS OF HERMITIAN MATRICES 7 It is cear that the smaest in the absoute vaue eigenvaues ξ correspond to the vaues of θ cosest to the shift σ. Thus, if (ξ, y) are the eigenpairs of (3.6) associated with the eigenvaues of the smaest absoute vaue, then the candidate approximations to the desired eigenpairs of the origina probem (1.1) can be chosen as (θ, Zy). Definition 3.1. Let (ξ, y) be an eigenpair of (3.6). We ca (θ, Zy) a T - harmonic Ritz pair, where θ = ξ + σ and v = Zy are a T -harmonic Ritz vaue and vector, respectivey. The corresponding vector Av θbv is then referred to as the T -harmonic residua with respect to the subspace Z (or, simpy, the T -harmonic residua). In order to see why T -harmonic Ritz pairs can be expected to deiver reasonabe approximations to the interior eigenpairs, et us introduce the substitution Q = T 1 2 (A σb)z and rewrite (3.6) as (3.7) Q (T 1 2 B)(A σb) 1 B(T 1 2 B) 1 Qy = τq Qy, τ = 1 ξ 1 θ σ. We note that (3.7) is the projected probem in the RR procedure for the matrix H = (T 1 2 B)(A σb) 1 B(T 1 2 B) 1 with respect to the subspace spanned by the coumns of Q. The soution of this probem yieds the Ritz pairs (τ, Qy) that tend to approximate we the exterior eigenpairs of H; see, e.g., [26, 28, 31]. Since the matrix H is simiar through T 1 2 B to the shift-and-invert operator (A σb) 1 B, each of its eigenpairs is of the form (1/(λ σ), T 1 2 Bv), where (λ, v) is an eigenpair of (1.1). Thus, the desired interior eigenvaues of (1.1) near the target σ correspond to the exterior (argest in the absoute vaue) eigenvaues of H, which can be we approximated by the Ritz vaues of (3.7). The associated Ritz vectors Qy then represent approximations to the eigenvectors T 1 2 Bv of H, i.e., T 1 2 (A σb)zy Qy T 1 2 Bv. This impies that Zy αv for some scaar α The PLHR agorithm. We now summarize the deveopments of the preceding sections and introduce an iterative scheme that at each step constructs a owdimensiona tria subspace (3.2) and uses the T -harmonic RR procedure to extract an approximate eigenvector from this subspace. The extraction is based on soving a 4-by-4 (3-by-3, at the initia step) eigenvaue probem (3.6), which defines an appropriate T -harmonic Ritz pair. The T -harmonic Ritz vector is then used as a new eigenvector approximation. We ca this scheme the Preconditioned Locay Harmonic Residua (PLHR) agorithm; see Agorithm 1 beow. The initia guess v (0) in Agorithm 1 can be chosen as a random vector. In practica appications, however, certain information about the soution is avaiabe, and v (0) can give a reasonabe approximation to the eigenvector of interest. Utiizing such initia guesses typicay eads to a substantia decrease in the iteration count and is therefore advisabe. Note that each PLHR iteration constructs a residua of the eigenvaue probem that can be used to determine the convergence in step 2 of Agorithm 1. In some appications, appropriate stopping criteria rey on tracking the difference between eigenvaue approximations, evauated in step 8, at the subsequent iterations. An important property of Agorithm 1 is that, in contrast to the origina probem (1.1), the T -harmonic probem (3.6) is no onger Hermitian. As such, (3.6) can have compex eigenpairs, i.e., the iteration parameters y = (α, β, γ, δ) T can be compex. Thus, the PLHR agorithm shoud be impemented using compex arithmetic, even if A and B are rea. This feature is undesirabe, e.g., because of the need to

10 8 EUGENE VECHARYNSKI AND ANDREW KNYAZEV Agorithm 1: Preconditioned Locay Harmonic Residua (PLHR) agorithm Input: The matrices A = A and B = B > 0, a preconditioner T = T > 0, the shift σ, and the initia guess v (0) for the eigenvector; Output: An eigenvaue λ cosest to σ and the associated eigenvector v, v B = 1; 1: v v (0) ; v v/ v B; λ (v, Av); p [ ]; 2: whie convergence not reached do 3: Compute the preconditioned residua w T (Av λbv); 4: Compute s T (Aw λbw); 5: Find the eigenpair (ξ, y) of (3.6), where Z = [v, w, s, p], such that ξ is the smaest; y = (α, β, γ, δ) T (at the initia step, y = (α, β, γ) T and δ = 0); 6: p βw + γs + δp; 7: v αv + p; 8: v v/ v B; λ (v, Av); 9: end whie 10: Return (λ, v). doube the required storage to accommodate compex numbers. A rea arithmetic version of the method for the rea case is presented in Section 4.1. In Agorithm 1, we foow the common practice of discarding the harmonic Ritz vaues and repacing them with the RQs associated with the newy computed Ritz vectors; see, e.g., [22, 23] for the standard harmonic case. In particuar, we skip the θ( ξ + σ) vaues (determined in step 5 of Agorithm 1) and repace them with λ(v) = (v Av)/(v Bv), where v = Zy is the T -harmonic Ritz vector. Note that, at each iteration i, Agorithm 1 (step 6) constructs a conjugate direction that can be expressed, in the superscripted notation, as p (i+1) = v (i+1) α (i) v (i) β (i) w (i) + γ (i) s (i) + δ (i) p (i), and the method extracts an approximate eigenvector from the subspace spanned by v (i), w (i), s (i), and p (i). In exact arithmetic, this subspace is the same as Z (i) defined in (3.2). The above formua is known to yied an improved numerica stabiity in cacuating the tria subspaces in the LOBPCG agorithm [15]. We expect that a simiar property wi hod for PLHR, and therefore foow the same stye for computing the conjugate directions in Agorithm 1. If it is possibe to store additiona vectors that contain resuts of the matrix-vector mutipications with A, B, and T (A σb), then Agorithm 1 can be impemented with two matrix-vector mutipications invoving A and B, and four appications of the preconditioner T. Two of the preconditioning operations resut from the construction of the tria subspace in steps 3 and 4, whereas the other two are the consequence of the T -harmonic extraction Possibe variations. The choice of the tria subspaces and of the extraction procedure, motivated by the preconditioned nu space finding framework in Section 2, is not unique. For exampe, one can define the s-vector as s (i) = T (Aw (i) θ (i) Bw (i) ), where instead of the RQ λ (i) the current T -harmonic vaue θ (i) is used. Aternativey, it is possibe to set s (i) = T (Aw (i) σbw (i) ), i.e., use the fixed σ instead of λ (i). However, our numerica experience suggests that these options do not ead to any better resuts compared to the origina choice of s (i) in (3.2). The minima residua condition (2.3) can aso motivate an extraction procedure that is different from the T -harmonic RR described in Section 3.1. In particuar, et

11 COMPUTING INTERIOR EIGENPAIRS OF HERMITIAN MATRICES 9 us assume that λ R is some approximation to the targeted eigenvaue λ q and v (i) is the current approximate eigenvector. In this case, one can extend (2.3) to minimize the T -norm of the residua-ike vector r = Av λbv, such that (3.8) v (i+1) = argmin Av λbv T. v Z (i), v B =1 This minimization principe represents the refinement procedure [11], performed with respect to the T -norm. Assuming that Z is a basis of the current tria subspace Z (i), one can show that (3.8) yieds the new approximate eigenvector v (i+1) = Zy min, where y min minimizes the biinear form θ 2 (y) = (y, Z (A λb)t (A λb)zy) (y, Z BZy) over a vectors y of dimension 4. This is equivaent to soving the eigenvaue probem Z (A λb)t (A λb)zy = θ 2 Z BZy, where the desired y min is the eigenvector associated with the smaest eigenvaue. The described approach can be of interest, in particuar, if v (i) is aready reasonaby cose to the targeted eigenvector. In this case, λ in (3.8) can be set to the current RQ λ (i). After the minimizer v (i+1) is constructed and the corresponding RQ and the new tria subspace Z (i+1) are computed, the procedure is repeated once again. Such an approach, caed the Preconditioned Locay Minima Residua (PLMR) method, which combines the tria subspaces (3.2) with the eigenvector extraction in (3.8), has been described in [34]. 4. The bock PLHR agorithm. In this Section, we extend PLHR to the bock case, where severa eigenpairs cosest to σ are computed simutaneousy. We start by introducing the bock notation. Let Λ = diag{λ 1, λ 2,..., λ k } denote a k-by-k diagona matrix of the targeted eigenvaues, ordered according to their distances from the shift σ, so that λ σ λ j σ for < j, where j = 1, 2,..., k. Let V be an n-by-k matrix of the associated eigenvectors. We assume that V (i) = [v (i) 1, v(i) 2,..., v(i) k ], Λ(i) = diag{λ (i) 1, λ(i) 2,..., λ(i) k } are the matrices of the approximate eigenvectors and eigenvaues at iteration i, respectivey. The diagona entries of Λ (i) are the RQs λ (i) j λ(v (i) j ) = (v (i) j, Av (i) j )/(v (i) j, Bv (i) j ), evauated at the corresponding vectors v (i) j, such that λ (i) σ λ (i) j σ for < j. Given the approximate eigenpairs in V (i) and Λ (i), et us define the bock W (i) [w (i) 1, w(i) 2,..., w(i) k ] = T (AV (i) BV (i) Λ (i) ) of the preconditioned residuas w (i) j = T (Av (i) j λ (i) j Bv(i) j ), and the bock S (i) [s (i) 1, s(i) 2,..., s(i) k ] = T (AW (i) BW (i) Λ (i) ), of s-vectors s (i) j = T (Aw (i) j λ (i) j spanned by the coumns of V (i), W (i), S (i), and V (i 1) (V ( 1) = 0), i.e., (4.1) Z (i) = span { v (i) 1,..., v(i) k Bw(i) j ). We can then introduce a tria subspace Z(i), w(i) 1... w(i) k, s(i) 1,..., s(i) k, v(i 1) 1,... v (i 1) k }.

12 10 EUGENE VECHARYNSKI AND ANDREW KNYAZEV Ceary, (4.1) is a bock generaization of the tria subspace (3.2) constructed at each singe-vector PLHR iteration. Let Z be a matrix whose coumns represent a basis of (4.1). Anaogousy to Agorithm 1, we require that new eigenvector approximations V (i+1) are the T -harmonic Ritz vectors extracted from (4.1). Thus, the vectors V (i+1) are defined by soving the 4k-by-4k (or, at the initia step, 3k-by-3k) eigenvaue probem (3.6), and correspond to the k T -harmonic Ritz vaues that are cosest to the shift σ. The entire approach, referred to as the bock PLHR (BPLHR), is summarized in Agorithm 2. Agorithm 2: The bock PLHR (BPLHR) Agorithm Input: Output: The matrices A and B, a preconditioner T = T > 0, the shift σ, and the initia guess V (0) for k eigenvectors; Diagona matrix Λ of eigenvaues cosest to the target σ and the matrix V of the associated eigenvectors; 1: V V (0) ; P [ ]; 2: Normaize coumns of V to have a unit B-norm; Λ diag (V AV ); 3: whie convergence not reached do 4: Compute the preconditioned residuas W T (AV BV Λ); 5: Compute S T (AW BW Λ); 6: Set Z [V, W, S, P ]. B-orthonormaize the coumns of Z. Let Ẑ = [ ˆV, Ŵ, Ŝ, ˆP ] be the matrix of the resuting B-orthonorma coumns. 7: Find eigenpairs of the projected probem (3.6) with Z Ẑ. Define Y [YV T, YW T, YS T, YP T ] T to be the matrix of k eigenvectors of (3.6) corresponding to the smaest in the absoute vaue eigenvaues. 8: Compute P Ŵ YW + ŜYS + ˆP Y P ; 9: Compute new approximate eigenvectors V ˆV Y V + P ; 10: Normaize coumns of V to have a unit B-norm; Λ diag (V AV ); 11: end whie 12: Perform the standard RR procedure for (1.1) with respect to V. Update V to contain the Ritz vectors and Λ the corresponding Ritz vaues. 13: Return (Λ, V ). Note that the set of approximate eigenvectors constructed by Agorithm 2 is generay not B-orthogona. This is a consequence of the T -harmonic projection that is based on soving the non-hermitian reduced probem (3.6). The eigenvectors Y of this probem are non-orthogona. Therefore the corresponding T -harmonic Ritz vectors V = ZY are aso non-orthogona. As iterations proceed and the approximate eigenpairs get coser to the soution, the (near) B-orthogonaity of the coumns of V starts to show up naturay, due to the intrinsic property of the symmetric eigenvaue probem. However, since in many appications the required accuracy of the soution may be ow, in order to ensure that the returned eigenvector approximations are B-orthonorma, the agorithm performs the post-processing (step 12 of Agorithm 2), where the fina bock V is rotated to the B-orthogona set of the Ritz vectors. For the purpose of numerica stabiity, at each iteration, Agorithm 2 B-orthogonaizes the set of vectors Z = [V, W, S, P ] that span the tria subspace and performs the T -harmonic RR procedure with respect to the B-orthonorma basis Ẑ = [ ˆV, Ŵ, Ŝ, ˆP ]. This is done by first B-orthonormaizing the coumns of V, so that the coumn space of ˆV and V is the same. Then the bock W is B-orthogonaized against ˆV and the resuting set of vectors is B-orthonormaized to obtain the bock Ŵ.

13 COMPUTING INTERIOR EIGENPAIRS OF HERMITIAN MATRICES 11 The remaining bocks Ŝ and ˆP are obtained in the same manner, by orthogonaizing against the currenty avaiabe B-orthogona bocks. The BPLHR agorithm requires storage for 16k vectors of Z, AZ, BZ, and T (A σb)z. If the storage is avaiabe, it can be impemented using 2 matrix-bock mutipications invoving A and B, and 4 appications of the preconditioner T to a bock per iteration. Note that in the case of standard eigenvaue probem (B = I), the memory requirement reduces to storing 12k vectors, and the two matrix-bock mutipications with B are no onger needed. Additiona cost reductions can resut from ocking the converged eigenpairs. In our BPLHR impementation, this is done by the standard soft ocking procedure, simiar to [16] The BPLHR agorithm in rea arithmetic. Each BPLHR iteration reies on the soution of the projected probem (3.6), which is generay non-hermitian. As a consequence, (3.6) can have compex eigenpairs and, therefore, compex arithmetic shoud be assumed for Agorithm 2. However, in the case where A and B are rea symmetric, the use of the compex arithmetic is unnatura, since the soution of (1.1) is rea. Moreover, the presence of compex-vaued iterations transates into the undesirabe requirement to doube the memory aocation for each vector, or bock of vectors, utiized in the computation. The goa of the present subsection is to discuss ways to overcome this issue and introduce a rea arithmetic version of the BPLHR agorithm. Let A and B be rea, where A is symmetric and B symmetric positive definite (SPD). Let T be an SPD preconditioner, and assume that at the current iteration of Agorithm 2 the tria subspace is given by a rea matrix Z. In this case the eft- and right-hand side matrices in the T -harmonic probem (3.6) are rea, impying that the eigenvaues ξ are either rea or appear in the compex conjugate pairs. Specificay, if (ξ, y) is a compex eigenpair of (3.6) then ( ξ, ȳ) is aso an eigenpair of (3.6). Ceary, if y is a rea eigenvector of (3.6) then the corresponding T -harmonic Ritz vector v = Zy is aso rea. If y is compex, then v = Zy is compex, and the conjugate eigenvector ȳ gives v = Zȳ, i.e., the presence of compex soutions in (3.6) yieds compex conjugate T -harmonic Ritz vectors v and v. Compex soutions of (3.6) and, subsequenty, the compex conjugate T -harmonic Ritz vectors, are not rare in practice. Their presence indicates that the extraction procedure is attempting to approximate eigenpairs associated with a (rea) eigenvaue of mutipicity greater than one. In particuar, if v = v R +iv I C n and v = v R iv I C n is a pair of the compex conjugate T -harmonic Ritz vectors, where v R R n and v I R n are the rea and imaginary parts of v, then the eigenvaue approximations given by the corresponding RQs coincide, i.e., (4.2) λ(v) = λ( v) = v R Av R + vi Av I vr Bv R + vi Bv. I Thus, v and v indeed approximate eigenvectors corresponding to the same eigenvaue. Let Y denote the matrix of eigenvectors of (3.6) associated with k smaest magnitude eigenvaues. For simpicity, we assume that if y is a compex eigenvector in Y, then the eigenvector ȳ is aso incuded into Y, i.e., Y contains compex conjugate coumns y and ȳ (the case where Y contains ony the coumn y wi be discussed beow). Then, for a given Y, et us define a rea matrix Y = [Y 0 Y R Y I ], where the subbock Y 0 consists of the rea coumns of Y, whereas Y R and Y I contain the rea and imaginary parts of the compex coumns of Y. More precisey, corresponding to each compex conjugate pair of coumns y = y R + iy I and ȳ = y R iy I in Y, we define two

14 12 EUGENE VECHARYNSKI AND ANDREW KNYAZEV rea coumns y R and y I, where the former is paced to Y R and the atter to Y I. It is assumed that a coumns y R and y I appear in Y R and Y I in the same order. That is, if y R is the jth coumn in the subbock Y R, then y I is the jth coumn of Y I. Ceary, Y R and Y I contain the same number of coumns, which is equa to the number of compex eigenvectors in Y divided by two. Thus, the sizes of Y and Y are identica. Let V = ZY be the bock of approximate eigenvectors extracted by Agorithm 2 from the tria subspace defined by Z at the current BPLHR step, and et [V, W, S, P ] be the basis of the tria subspace computed from V at the next iteration. It is cear that each bock in this basis wi have compex coumns, provided that Y contains compex eigenvectors of (3.6). Additionay, according to the assumption on Y, for each compex coumn v of the bock V, there exists a conjugate coumn v in V. Beow we show that it is possibe to repace [V, W, S, P ] by a rea basis [V, W, S, P ] that spans exacty the same tria subspace. This idea wi ead to a rea arithmetic version of the BPLHR agorithm. Our approach is based on the observation that span{v R, v I } = span{v, v} over the fied of compex numbers. Therefore, if we define (4.3) V [V 0 V R V I ] = [ZY 0 ZY R ZY I ], then its coumn space wi be exacty the same as that of V. Here the subbock V 0 of V contains the rea coumns of the matrix V, whereas V R and V I correspond to the rea and imaginary parts of V s compex coumns, respectivey. Thus, one can repace the bock V of T -harmonic Ritz vectors by the corresponding rea-vaued bock V without changing the coumn span. Note that both V and V have k coumns. However, a coumns of V are rea. Therefore, pacing them into the new tria subspace does not ead to any increase in storage, in contrast to using V, whose compex coumns woud require extra memory. Simiary, since the presence of the compex conjugate pair of coumns v and v in V yieds the conjugate coumns w and w in the bock W, the atter can be repaced by the rea bock (4.4) W [W 0 W R W I ] = [T (AV 0 BV 0 Λ 0 ) T (AV R BV R Λ 1 ) T (AV I BV I Λ 1 )], where Λ 0 is the diagona matrix of the RQs λ(v) = v Av/v Bv evauated at the coumns v of V 0 ; and Λ 1 is diagona matrix that, for each coumn v R of V R and the corresponding v I of V I, contains the RQ defined in (4.2). Note that the matrix W can be expressed as W = T (AV BV Λ ), where Λ = diag(λ 0, Λ 1, Λ 1 ). Finay, using the same argument, the bock S constructed by Agorithm 2 can be repaced by the rea bock (4.5) S [S 0 S R S I ] = [T (AW 0 BW 0 Λ 0 ) T (AW R BW R Λ 1 ) T (AW I BW I Λ 1 )], i.e., S = T (AW BW Λ ). Thus, instead of using the possiby compex basis [V, W, S, P ], constructed by Agorithm 2 for the next BPLHR iteration, one can set up the rea basis [V, W, S, P ] that defines the same tria subspace. The bock P is constructed exacty in the same way as P in Agorithm 2, with the difference that the coefficients given by Y are repaced by those in Y. As a resut, P represents a combination of the new V and of the subbock of Z that corresponds to the current eigenvector approximations. Ceary, this definition ensures that P is rea, and hence the same tria subspace is spanned by [V, W, S, P ] and [V, W, S, P ]. The above discussion is based on the assumptions that the matrix Y of eigenvectors of (3.6) contains the conjugate ȳ for each compex coumn y. In practice,

15 COMPUTING INTERIOR EIGENPAIRS OF HERMITIAN MATRICES 13 however, this may not necessariy be the case. Since k is a fixed parameter, one can encounter the situation where a the compex coumns y have the corresponding conjugate coumns ȳ in Y, but there exists a coumn ỹ = ỹ R + iỹ I whose conjugate is not among the k coumns of Y. This happens if ỹ corresponds to the kth eigenvaue of (3.6), whereas the conjugation of ỹ corresponds to the (k + 1)st eigenvaue, where the eigenvaues are numbered in the ascending order of their magnitudes. In this case, we ignore the imaginary part of ỹ and construct the matrix Y to be of the form Y = [Y 0 Y R ỹ R Y I ], where Y R and Y I contain the rea and imaginary parts of those vectors y that appear in Y in conjugate pairs. Given Y, we proceed in the same fashion as has been described above. We introduce the bock V [V 0 V R ṽ R V I ] of approximate eigenvectors and the diagona matrix Λ = diag(λ 0, Λ 1, λ, Λ 1 ) of the corresponding eigenvaue approximations, where ṽ R = Zỹ R and λ is the RQ evauated at ṽ R, i.e., λ λ(ṽ R ) = (ṽ R, Aṽ R )/(ṽ R, Bṽ R ). The bocks W and S then take the form W = T (AV BV Λ ) = [W 0 W R w R W I ] and S = T (AW BW Λ ) = [S 0 S R s R S I ], where w R = Aṽ R λbv R and s R = A w R λb w R. Ceary, if the imaginary part of ỹ is ignored, i.e., the vector ṽ I = Zỹ I is not in V, then the coumns of [V, W, S, P ] span a sighty smaer subspace than those of [V, W, S, P ] in Agorithm 2. This may ead to a sight convergence deterioration of the rea arithmetic scheme compared to the origina BPLHR version. Therefore, prior to the run, we suggest increasing the bock size (at east) by 1 and running the agorithm for the extended bock size. This removes the possibe effects of cutting in between the conjugate pair when eigenvectors of (3.6) are seected into Y. The detais of the rea arithmetic version of BPLHR agorithm are summarized in Agorithm 3. Note that if at each step of the agorithm probem (3.6) gives ony rea eigenvectors Y, then Agorithm 3 becomes equivaent to the origina version of BPLHR in Agorithm Preconditioning. In order to motivate preconditioning strategy, et us again consider the ideaized tria subspaces (3.1). Assuming that the targeted eigenvaue λ q is known, we address the question of defining an optima preconditioner T which ensures that the corresponding eigenvector v q is exacty in the tria subspace (3.1). A possibe choice of such a preconditioner is given by T = (A λ q B) [14]. In this case, v q v (i) T (A λ q B)v (i) = (I K)v (i), where K = (A λ q B) (A λ q B). Ceary, v q is in (3.1). The fact that v q is an eigenvector foows from the observation that K is an orthogona projector onto the range of A λ q B. Hence, I K projects v (i) onto the nu space of A λ q B, which gives the desired eigenvector. Note that v q is nonzero provided that v (i) has a nontrivia component in the direction of the targeted eigenvector. In genera, neither the ideaized subspaces (3.1) nor the optima preconditioner T = (A λ q B) are avaiabe at the PLHR iterations. However, the above anaysis suggests that practica preconditioners can be defined as approximations of (A λ q B). For exampe, one can aim at constructing T, such that T (A σb) 1. This gives rise to the inexact shift-and-invert type preconditioning, which represents a traditiona approach for preconditioning eigenvaue probems; see, e.g., [8, 15]. Since preconditioners for PLHR shoud be HPD, the definition T (A σb) 1 may not aways be suitabe. Whie one can expect to construct the HPD inexact shiftand-invert preconditioners for computing eigenpairs associated with the eigenvaues

16 14 EUGENE VECHARYNSKI AND ANDREW KNYAZEV Agorithm 3: The rea arithmetic version of the BPLHR agorithm Input: A symmetric matrix A, an SPD matrix B, an SPD preconditioner T, the shift σ, and the rea initia guess V (0) for k eigenvectors; Output: Diagona matrix Λ of eigenvaues cosest to the shift σ and the matrix V of the associated eigenvectors; 1: V V (0) ; P [ ]; 2: Normaize coumns of V to have a unit B-norm; Λ diag (V AV ); 3: whie convergence not reached do 4: Compute the preconditioned residuas W T (AV BV Λ); 5: Compute S T (AW BW Λ); 6: Set Z [V, W, S, P ]. B-orthonormaize the coumns of Z. Let Ẑ = [ ˆV, Ŵ, Ŝ, ˆP ] be the matrix of the resuting B-orthonorma coumns. 7: Find eigenpairs of the projected probem (3.6) with Z Ẑ. Sort the eigenvaues ξ in the ascending order of their absoute vaues, ensuring that in the sorted set every compex eigenvaue ξ is immediatey foowed by its conjugate ξ. 8: Seect k eigenvectors of (3.6) associated with the smaest magnitude eigenvaues into the matrix Y. 9: Move rea coumns of Y into Y 0. Define Y R and Y I to be the matrices containing the rea and imaginary parts of the coumns of Y that appear in compex pairs. 10: If Y contains a compex coumn ỹ, such that the conjugate of ỹ is not in Y, then set ỹ R to be the rea part of ỹ. Otherwise, ỹ R [ ]. Define Y = [Y 0 Y R ỹ R Y I] [YV T, YW T, YS T, YP T ] T. 11: Compute P Ŵ YW + ŜYS + ˆP Y P ; 12: Compute new approximate eigenvectors V ˆV Y V + P. Define the coumn partitioning V = [V 0 V R ṽ R V I]( ẐY ) according to Y (ṽ R [ ] if ỹ R = [ ]). 13: For each coumn v of V 0 compute the RQ λ(v) v Av/v Bv and pace it to the diagona of Λ 0. For each coumn v R of V R and the corresponding v I of V I, compute the diagona entry of Λ 1 by (4.2). Set λ ṽraṽ R/ṽRBṽ R ( λ [ ] if ṽ R = [ ]). 14: Normaize coumns of V to have a unit B-norm; Λ diag(λ 0, Λ 1, λ, Λ 1). 15: end whie 16: Perform the standard RR procedure for (1.1) with respect to V. Update V to contain the Ritz vectors and Λ the corresponding Ritz vaues. 17: Return (Λ, V ). not far away from the ends of the spectrum, the approach wi resut in the indefinite preconditioning as the eigenvaues deeper in the interior of the spectrum are sought. In order to define a preconditioner that is HPD and, at the same time, that preserves the desirabe effects of shift-and-invert, et us consider the operator T = A λ q B, where A λ q B is defined as a matrix function [7] of A λ q B. Simiar to T = (A λ q B), this choice of preconditioner is aso optima with respect to the ideaized tria subspaces (3.1). Indeed, if T = A λ q B, then v q v (i) T (A λ q B)T (A λ q B)v (i) = (I K)v (i), where K = ( A λ q B (A λ q B) ) 2 = ( (A λq B) (A λ q B) ) 2 = (A λq B) (A λ q B). Thus, v q beongs to (3.1) and represents the wanted eigenvector, since I K projects v (i) onto the nu space of A λ q B. As discussed above, the computation of ideaized tria subspaces and an optima preconditioner may be infeasibe in practice. Instead, however, one can construct T that approximates the action of the (pseudo-) inverted absoute vaue operator. In particuar, we can define T A σb 1. In the next section, we demonstrate that

17 COMPUTING INTERIOR EIGENPAIRS OF HERMITIAN MATRICES 15 such preconditioners exist for certain casses of probems and indeed ead to a rapid and robust convergence if used to construct the PLHR tria subspaces (3.2) (or (4.1), in the bock case) and to perform the T -harmonic projection introduced in Section Numerica experiments. We organize our numerica experiments into three sets. The first set concerns a mode generaized eigenvaue probem of a sma size. This exampe is mainy of theoretica nature and is intended to demonstrate severa features of the convergence behavior of the proposed method. In the second test set, we consider a arger mode probem, where a practica AV preconditioner is used. Specificay, we address a probem of computing a subset of interior eigenpairs of a discrete Lapacian. As an SPD preconditioner we use the mutigrid (MG) scheme proposed in [35] in the context of soving symmetric indefinite Hemhotz type inear systems. We show that exacty the same AV preconditioner can be empoyed for the interior eigenvaue computations in the BPLHR agorithm. The third series of experiments aims at a particuar appication. We consider severa Hamitonian matrices that arise in eectronic structure cacuations, and appy BPLHR to revea eigenvectors (wave functions) that correspond to the eigenvaues (energy eves) around a given reference energy. The preconditioners avaiabe for this type of probems are diagona and SPD [32]. In our experiments, we compare BPLHR with the a bock version of the Generaized Davidson (BGD) method that is based on the harmonic projection [12, 22]. This choice has been motivated by severa factors. First, we want to restrict our comparisons to the cass of bock methods which rey on the same type of computationa kernes, such as mutipication of a bock of vectors by a matrix or a preconditioner, dense matrix-matrix mutipication, etc. Secondy, the BGD agorithm represents a state-of-the-art approach for interior eigenvaue computations in a number of critica appications; see, e.g., [12]. In particuar, we demonstrate that BPLHR can give a more robust soution option compared to BGD in cases where memory is imited and the avaiabe preconditioner is of a moderate quaity A sma mode probem: the finite eement (FE) Lapacian. Let us consider the eigenvaue probem (6.1) u(x, y) = λu(x, y), (x, y) Ω = (0, 1) (0, 1), u Γ = 0, where = 2 / x / y 2 is the Lapace operator and Γ denotes the boundary of the unit square Ω. Assume that we are interested in approximating a number of eigenpairs (λ, u(x, y)) of (6.1) that are cosest to a given shift σ. Discretizing (6.1) with standard biinear finite eements resuts in the agebraic generaized eigenvaue probem Av = λbv, where A and B are the SPD stiffness and mass matrices, respectivey. In this exampe, we assume a reativey sma number of finite eements, 50 aong each side of the unit square, which resuts in the tota of 2, 401 degrees of freedom, i.e., the size of the matrix probem is n = 2, 401. Due to the sma probem size, in this exampe, we can mode high-quaity AV preconditioners by perturbations of the operator A σb 1. In particuar, we define T = A σb 1 +E, where E is a random SPD matrix, such that E ɛ (A σb) 1 and ɛ is a parameter that sets up the preconditioning quaity; denotes the spectra norm. As ɛ increases, so does the norm of the perturbation E, which means that T becomes more distant from A σb 1 and hence the quaity of the preconditioner deteriorates. The matrix E is fixed during the iterations, but is updated for every new run. We present typica resuts.