A Tuned Preconditioner for Inexact Inverse Iteration Applied to Hermitian Eigenvalue Problems

Transcription

1 IMA Journal of Numercal Analyss (2005) Page of 29 do: 0.093/manum/ A Tuned Precondtoner for Inexact Inverse Iteraton Appled to Hermtan Egenvalue Problems MELINA A. FREITAG AND ALASTAIR SPENCE Department of Mathematcal Scences, Unversty of Bath, Claverton Down, BA2 7AY, Unted Kngdom. [Receved on ; revsed on ] In ths paper we consder the computaton of an egenvalue and correspondng egenvector of a large sparse Hermtan postve defnte matrx usng nexact nverse teraton wth a fxed shft. For such problems the large sparse lnear systems arsng at each teraton are often solved approxmately by means of symmetrcally precondtoned MINRES. We consder precondtoners based on the ncomplete Cholesky factorsaton and derve a new tuned Cholesky precondtoner whch shows consderable mprovement over the standard precondtoner. Ths mprovement s analysed usng the convergence theory for MIN- RES. We also compare the spectral propertes of the tuned precondtoned matrx wth those of the standard precondtoned matrx. In partcular, we provde both a perturbaton result and an nterlacng result, and these results show that the spectral propertes of the tuned precondtoner are smlar to those of the standard precondtoner. For Raylegh quotent shfts, comparson s also made wth a technque ntroduced by Smoncn and Eldén [Inexact Raylegh quotent-type methods for egenvalue computatons, BIT, 42 (2002), pp ] whch nvolves changng the rght hand sde of the nverse teraton step. Several numercal examples are gven to llustrate the theory descrbed n the paper. Keywords: Inexact nverse teraton, precondtoned teratve methods, Hermtan matrces.. Introducton We consder the problem of computng an egenvalue and the correspondng egenvector of a Hermtan postve defnte matrx A C n n, that s Ax = λ x, λ R, x C n \ {0}, (.) usng nexact nverse teraton wth a fxed shft. We assume that the matrx A s very large and sparse and so to explot the structure teratve technques, n partcular, precondtoned MINRES, may be used to solve the lnear shfted systems (A σi)y = x (.2) arsng n nverse teraton, where the shft σ s chosen to be close to any egenvalue. Hence, the lnear system s solved to a prescrbed tolerance only. It s well known that usng exact solves nverse teraton wth a fxed shft acheves lnear convergence (see for example Demmel (997) or Parlett (998)). Also, a famous result due to Ostrowsk (957) (see also Parlett (998)) shows that Raylegh quotent teraton yelds cubc convergence for a close enough startng guess. Theory for nexact nverse teraton wth a fxed shft can be found n Golub & Ye (2000), La et al. (997) and Smt & Paardekooper (999) where t was proved that for an approprately chosen decreasng tolerance nexact nverse teraton wth a fxed shft acheves lnear convergence. The convergence theory for nexact Raylegh quotent teraton s gven n Smt & Paardekooper (999) and Berns-Müller et al. (2006). In these papers t was proved that nexact Raylegh quotent teraton wth fxed tolerance acheves quadratc convergence whereas wth an IMA Journal of Numercal Analyss Insttute of Mathematcs and ts Applcatons 2005; all rghts reserved.

2 2 of 29 M. A. FREITAG AND A. SPENCE approprately chosen decreasng tolerance t recovers the cubc convergence attaned when exact solves are used. Inexact nverse teraton s an example of an nner-outer teratve method. The outer teraton s the basc teratve nverse teraton algorthm, requrng the solve of (.2) wth a fxed or varable shft σ at each step, wth the nner teraton beng the nexact teratve soluton of (.2). Inner-outer methods have been used n other settngs, see, for example, Golub & Ye (999). In order to reduce the number of nner teratons needed to solve (.2), precondtonng becomes necessary. Snce A s Hermtan postve defnte, t s natural to use an ncomplete Cholesky factorsaton of A to construct a symmetrcally precondtoned form of (.2). Specfcally, f LL s an ncomplete Cholesky factorsaton of A then one apples the teratve solver (for example, MINRES) to L (A σi)l ỹ = L x, y = L ỹ, (.3) rather than to (.2). For a fxed shft, t s known that (see for example Berns-Müller et al. (2006), La et al. (997)) the number of nner teratons used by a Krylov solver appled to (.3) ncreases steadly as the outer teraton proceeds. We shall show that wth a smple rank-one change to the precondtoner, whch we call tunng the precondtoner, ths steady ncrease n the number of nner teratons can be stopped, and ndeed consderable mprovements n the total nner teraton count can be acheved. In Fretag & Spence (2007) the concept of tunng the precondtoner to mprove the outer convergence of a varant of nverse teraton was ntroduced, but no analyss of the tuned precondtoner was gven. Based on Fretag & Spence (2007) and an earler verson of ths paper, Fretag & Spence (2005), Robbé et al. (2006) analysed a subspace verson of tunng for the standard egenproblem and ntroduced the concept of an deal precondtoner. In ths paper we extend that analyss to obtan a detaled descrpton of the performance of MINRES, and n partcular show that the tuned precondtoner should not exhbt growth n the number of nner teratons as the outer teraton proceeds. Then we provde a careful spectral analyss that explans the dfferences between the teraton matrces for the tuned and standard cases. Ths nvolves the formulaton of a nonstandard egenvalue perturbaton problem, whch s analysed by a modfcaton of the Bauer-Fke theorem (see Golub & Loan (996)) and a novel nterlacng property (n the sprt of (Wlknson, 965, p. 94 ff) and Golub & Loan (996)). These results show that the spectral propertes of the tuned precondtoner are smlar to those of the standard precondtoner. For the case of Raylegh quotent shfts, the dea from Smoncn & Eldén (2002) s to modfy the rght hand sde of the precondtoned system (.3) so that the new rght hand sde s close to an approxmate null-vector of the teraton matrx (see Secton 5). Ths new strategy reduces the number of nner teratons for each solve of (.2), but destroys the cubc outer convergence for Raylegh quotent teraton, achevng only quadratc outer convergence. (Note that ths strategy requres that the shfts tend to the desred egenvalue and so s not an opton when the shft s fxed.) We compare the use of the tuned precondtoner wth the approach of Smoncn & Eldén (2002) and fnd that the tuned precondtoner s also superor n terms of overall teraton count. In Secton 2 of ths paper we dscuss the theory of nexact nverse teraton wth a fxed shft, the convergence theory for MINRES, and then go over the use of the standard ncomplete Cholesky precondtoner. We also dscuss the applcaton of these results to the soluton of the shfted systems n nexact nverse teraton. In Secton 3 we make a comparson wth the deal precondtoner of Robbé et al. (2006) and prove the man theorem (Theorem 3.2) about the performance of MINRES appled to the tuned precondtoned shfted system. Numercal results are presented to show the superorty of the tuned precondtoner over the standard precondtoner. In Secton 4 we provde a detaled analyss of the spectra of both the tuned and untuned teraton matrces and dscuss the consequences for MINRES

3 A tuned precondtoner for nexact nverse teraton 3 of 29 as teratve solver for the nner teratons. In Secton 5 the tuned precondtoner s appled to nexact Raylegh quotent teraton. Agan, the tuned precondtoner s superor to the the standard precondtoner. Numercal results are also presented comparng the performance of the tuned precondtoner wth the approach of Smoncn & Eldén (2002). Secton 6 summarses the man results of the paper. We denote the egenpars of A by (λ,x ), =,...,n, and use = Inexact nverse teraton wth a fxed shft In ths secton we revse the theory for nexact nverse teraton wth a fxed shft for the calculaton of a smple egenvalue of the standard Hermtan egenvalue problem (.), and then go on to dscuss the use of MINRES as the teratve solver and precondtonng. A fxed shft method s unlkely to be of nterest on ts own, but t mght well be used to provde a good startng guess for the egenvector to feed nto the Raylegh quotent teraton. Also, results for fxed shfts are of nterest when usng subspace based methods, lke the Lanczos method. The followng algorthm s a verson of nexact nverse teraton wth a fxed shft to fnd any wellseparated smple egenvalue of a Hermtan matrx. ALGORITHM 2. (INEXACT INVERSE ITERATION WITH A FIXED SHIFT) Gven σ and x (0) wth x (0) =. For = 0,,2,... () Choose τ (), (2) Solve (A σi)y () = x () nexactly, that s, (A σi)y () x () τ (), (3) Compute approxmate egenvector x (+) = y() y (), (4) Compute approxmate egenvalue λ (+) = x (+) Ax (+), (5) Evaluate egenvalue resdual r (+) = (A λ (+) I)x (+), (6) Test for convergence. The followng theorem states the convergence theory for nexact nverse teraton wth a fxed shft. It follows drectly from Theorem 2.2 n Berns-Müller et al. (2006), where a detaled proof s gven (see also Lemma 2.2 n Golub & Ye (2000)). THEOREM 2. (CONVERGENCE OF INEXACT INVERSE ITERATION WITH A FIXED SHIFT) Let (.) be the standard egenvalue problem for a Hermtan matrx A and consder the applcaton of Algorthm 2. to fnd a smple egenpar (λ,x ) of A. Assume σ s closer to λ than to any other egenvalue of A, and that x (0) s close enough to the desred x. Then, f a decreasng tolerance s chosen for the nexact solves n the nverse teraton Algorthm 2., say τ () = C r () n step (), then lnear convergence s acheved for small enough τ (0) and C. Proof. Followng Parlett (998), f we wrte x () = cosθ () x + snθ () x (), x() x, (2.)

4 4 of 29 M. A. FREITAG AND A. SPENCE wth x = x () = and θ() = (x (),x ), then the egenvalue resdual defned by wth λ () = x () Ax () satsfes (see (Parlett, 998, Theorem.7.)) r () = (A λ () I)x (), (2.2) snθ () λ 2 λ () r () sn θ () λ n λ. (2.3) Thus the choce of τ () = C r () asks that the solve tolerance n step (2) of Algorthm 2. decreases wth the error angle θ (). From (Berns-Müller et al., 2006, Lemma 2.) we have tanθ (+) λ σ snθ () +τ () λ 2 σ cosθ () τ (), whch, wth the choce of τ () yelds lnear convergence for small enough C. Note that for the specal case of a tolerance τ () = 0 we obtan the well known lnear convergence acheved by exact nverse teraton. 2. Convergence theory of MINRES In order to understand the performance of the nner teraton part of the nexact nverse teraton algorthm we revse some convergence theory of MINRES. Frst, we quote a theorem about the convergence of MINRES when appled to Bz = b (2.4) for the case of nterest here. Ths s a specal case of Theorem 3. of Berns-Müller et al. (2006), but smlar results are well known n the lterature (see, for example Greenbaum (997) and Hackbusch (994)). THEOREM 2.2 Suppose that the symmetrc matrx B has egenvalues µ,...,µ n wth correspondng egenvectors w,...,w n. Let µ be well separated from {µ } n =2. Furthermore, let κ = max =2,...,n µ mn =2,...,n µ be the reduced condton number of B, assume max =,...,n µ µ = µ µ n and defne P to be the orthogonal proecton along w onto span{w 2,...,w n }. If z k s the result of applyng MINRES to (2.4) wth startng value z 0 = 0 then b Bz k 2 max =2,...,n f all the elements of {µ } n =2 have the same sgn and µ µ b Bz k 2 max κ =2,...,n µ κ + otherwse. In addton, f the number of teratons satsfes κ k + 2 ( k µ µ κ P µ κ + ) b, (2.5) k 2 ( log2 µ µ n + log P b µ τ P b ), (2.6)

5 or respectvely, then b Bz k τ. A tuned precondtoner for nexact nverse teraton 5 of 29 ( k 2+κ log2 µ ) µ n + log P b, µ τ REMARK 2. Note that the bounds n ths Theorem are worst case bounds and may ndeed be worse than the trval bound k () n. In general these bounds are often used to gve qualtatve rather than quanttatve nformaton, snce n practce convergence of MINRES can be much faster. Also, for smplcty we shall consder only the case of a smple extreme egenvalue, snce the convergence theory for MINRES s easest. Therefore, n ths paper we concentrate on the frst case, where {µ } n =2 have the same sgn; all results generalse to the second case. We now apply ths theorem to the soluton of (A σi)y () = x (), wth B = A σi, b = x (), µ = λ σ, w = x and P x () = snθ () x (), (2.7) usng (2.). Thus f k () denotes the number of nner teratons used by MINRES to solve (A σi)y () = x () nexactly as n step (2) of Algorthm 2., then ( ) κ k () + log2 λ λ n 2 λ σ + log snθ() τ (). (2.8) Wth λ σ fxed and τ () = C r (), we see, usng (2.3), that the rght hand sde of (2.8) s bounded ndependent of. Hence we nfer that the number of nner teratons used by MINRES wll not ncrease as the outer teraton proceeds. Ths nce property s not mantaned when precondtonng s appled as we dscuss next. 2.2 Precondtoned nexact nverse teraton wth a fxed shft In ths subsecton we consder the applcaton of a precondtoner n the soluton of the lnear system n step (2) of Algorthm 2.. Let A n the standard egenvalue problem (.) be Hermtan postve defnte and consder the ncomplete Cholesky factorsaton LL, that s, A = LL + E, (2.9) where E s the symmetrc error matrx assocated wth the ncomplete decomposton of A. Then, nstead of solvng (A σi)y () = x () n step (2) of Algorthm 2. nexactly, we solve the Hermtan system L (A σi)l ỹ () = L x (), y () = L ỹ (), (2.0) to a tolerance τ () L so that x () (A σi)y () τ (). Ths does not change the lnear outer rate of convergence of the nexact nverse teraton algorthm. However, the rght hand sde L x () s no longer close to the egenvector correspondng to the egenvalue of L (A σi)l closest to zero and ths changes the nner teraton behavour as the outer teraton proceeds as we now explan. Apply Theorem 2.2 wth B = L (A σi)l, b = L x (), τ = τ() L and wth κ L denotng the correspondng

6 6 of 29 M. A. FREITAG AND A. SPENCE reduced condton number of L (A σi)l, to obtan the followng bound on k () : κl ( k () L + 2 log2 µ µ n µ ) + log P L x () L τ () (2.) The key pont to note s that there s no reason for P L x () to behave lke sn θ () as s the case when there s no precondtonng. So, usng P L x () L, (2.) provdes κl k () L + 2 ( log2 µ µ n L L + log ) µ τ (), (2.2) and the rght hand sde ncreases wth for a decreasng τ (). Ths ndcates that there wll be growth n the number of nner teratons used by MINRES to solve (2.0). Ths s ndeed observed n practce as s seen n Fgure (sold lne wth crcles). In order to recover the reassurng property of a constant number of nner teratons for precondtoned MINRES, a dfferent approach has to be chosen. Smoncn & Eldén (2002) alter the rght hand sde n (2.0), but for outer convergence ths strategy requres that the shft tends to the desred egenvalue as s the case for Raylegh quotent teraton. In ths paper we try the alternatve approach of changng the precondtoner to recover the nce property of a constant number of nner teratons at each outer step. Ths dea s explaned n the next secton. REMARK 2.2 In ths paper we shall assume that a good precondtoner for A s also a good precondtoner for A σ I. Ths s the approach taken n Smoncn & Eldén (2002) and t s lkely to be the case f A arses from a dscretsed partal dfferental equaton where a talor-made precondtoner for A may be avalable. 3. The tuned precondtoner In ths secton we ntroduce a new precondtoner to be appled to (A σi)y () = x (), so that the lnear outer convergence s retaned, but whch provdes the advantage of cheap nner solves. Ths approach s motvated by the tuned precondtoner that was ntroduced n Fretag & Spence (2007) for the nonsymmetrc generalsed egenproblem but needs a more careful treatment to retan the Hermtan structure. Addtonally, n ths secton and n Secton 4 we are able to provde theoretcal results for the tuned precondtoner that are not avalable n the nonsymmetrc case dscussed n Fretag & Spence (2007). 3. An deal precondtoner In ths subsecton we dscuss a rather hypothetcal case. Assume we know the sought egenvector x and that nstead of solvng (2.0) n step (2) of Algorthm 2. we solve the precondtoned Hermtan system L (A σi)l ỹ = L x, y = L ỹ, (3.) where L s chosen such that the rght hand sde of (3.) s an egenvector of L (A σi)l correspondng to the egenvalue closest to zero. We shall see below that ths s acheved f we ask that the precondtoner LL should satsfy LL x = Ax, (3.2)

7 A tuned precondtoner for nexact nverse teraton 7 of 29 and so x s an egenvector of both A and LL. Hence, n addton to LL beng close to A as s usual n precondtonng we requre that LL acts exactly lke A n the drecton of x. From (3.2) t s easy to see that L AL L x = L x, that s, L x s an egenvector of L AL correspondng to the egenvalue. Also L x = λ L x, and so L x s an egenvector of L (A σi)l correspondng to the egenvalue (λ σ)/λ, whch ustfes the asserton made after (3.). We now have the followng lemma, that tells us about the exstence and constructon of the deal precondtoner P = LL and ts (theoretcal) mpact on the soluton of (3.). LEMMA 3. Let P = LL be the postve defnte precondtoner gven by (2.9) and assume t has the egendecomposton VPV = D = dag(η,...,η n ), where 0 < η... η n. Introduce u := (A P)x = Ex. For x u 0 defne Then () Px = Ax. (2) If where (Vu ) s the frst entry of Vu, then P s postve defnte. Now assume (3.4) holds and that s the Cholesky factorsaton of P. Then (3) L x = λ L x. ( ) λ σ (4),L x s an egenpar of L (A σi)l. λ P = P+ u u x u. (3.3) x u < (Vu ) 2 η or x u > 0 (3.4) P = LL (3.5) (5) Wth startng guess set to zero, MINRES solves (3.) n exactly one step. Proof. () Px = Px + u = Ax. (2) (a) Obvous. (b) Standard rank-one perturbaton theory (Golub & Loan, 996, Theorem 8.5.3) shows that the egenvalues of P = P + u u x u = V (D + Vu u V x u )V are gven by the zeros of h(λ) = + ( n ) (Vu ) 2 x u. Wth x = η λ u < 0, the smallest zero of h(λ) s less than η. Now the root of + ( (Vu ) 2 ) x u provdes a lower bound for the smallest egenvalue of P, whch, wth η λ the requrement λ > 0, gves the result. (3) Follows from LL x = Ax = λ x.

8 8 of 29 M. A. FREITAG AND A. SPENCE (4) L (A σi)l (L x ) = λ L (A σi)l L x = λ σ λ (L x ). (5) Follows from standard theory for Krylov subspace methods. REMARK 3. (a) Gven P, P s the deal precondtoner for MINRES, snce t converges n one step. A smlar deal precondtoner s employed n Robbé et al. (2006) to analyse subspace teraton for the nonsymmetrc egenvalue problem. (b) Lemma 3. shows that there s a range (Vu ) 2 x u 0 where the deal precondtoner s not η postve defnte. The lower bound of ths range wll be large, f η s small, that s, f P s a poor precondtoner. Also f Px s close to Ax, say, for example, f E n (2.9) were small, then x u would be close to zero. However, n ths case, there would be no need for tunng. For the practcal tuned precondtoner dscussed n the next subsecton the condtons correspondng to (3.4) are nvestgated n the examples n Secton 3.3, and ndeed, are shown to hold n all cases consdered. Of course, n practce the precondtoner (3.3) cannot be used snce x s not avalable. However, ts form suggests a practcal tuned precondtoner. 3.2 The practcal tuned precondtoner At the th step n Algorthm 2. defne u () = (A P)x (). (3.6) Assumng that x () u () 0 the practcal tuned precondtoner s obtaned by replacng P n (3.3) by P gven by P = P+ u() u (), (3.7) x () u () where the unknown x s replaced by ts approxmaton x (). Clearly P tends to P as x () x. Assume also x () u () < (Vu() ) 2 η or x () u () > 0, (3.8) where V and η are defned as n Lemma 3.. Then P s postve defnte. If we can prove smlar results to Lemma 3., we can expect to obtan a sgnfcant beneft n the teratve soluton of L (A σi)l ỹ () = L x (), y () = L ỹ (), (3.9) where P = L L s the Cholesky decomposton of P. Frst note that the tuned precondtoner P satsfes the tunng condton P x () = Ax (), (3.0) and we wll use ths condton several tmes. We now state a Lemma about P.

9 A tuned precondtoner for nexact nverse teraton 9 of 29 LEMMA 3.2 Let P be gven by (3.3) and P be gven by (3.7). Further let u be gven as n Lemma 3. and u () as n (3.6). Assume (3.8) holds and let Then R () = x() x () x () u () x x x u. R () C tanθ (), (3.) where θ () s gven n (2.) and C s ndependent of for large enough. Furthermore P P =, wth C 2 tanθ (), (3.2) where C 2 s ndependent of for large enough. If P exsts then, for P exsts and Ths term can be bounded ndependent of for large enough. C 2 tanθ () < P, (3.3) P P C 2 tanθ () P. (3.4) Proof. Wrte x () as (2.), then a straghtforward but lengthy calculaton gves (3.). Clearly, we have P P = (A P)R () (A P) and (3.2) s readly obtaned. Further, we can bound P = (P+ ) (I+P ) P, and (3.3) gves (3.4). Next, we have a Lemma that provdes bounds on L and L. LEMMA 3.3 Let P = LL and P = L L be the Cholesky factorsatons of P and P and assume (3.2) and (3.3) hold. Then L C 3 and C 4 L C 5, (3.5) where C 3, C 4 and C 5 are ndependent of for large enough. Proof. Frst note that where P = P+ = L(I+D () )L, D () = L L. (3.6) For large enough, I+D () s symmetrc postve defnte, and D () C 6 tanθ () C 7. Drmač et al. (994) show that the Cholesky factorsatons I+D () = (I+F () )(I+F () ) exst wth F () C 8 D () where C 8 depends on the matrx dmenson but s ndependent of. Hence, we may wrte the Cholesky factor of P as L = L(I+F () ) wth for some constant C 3 ndependent of. F () ) L and so L L ( I+ F () ) L (+C 8 tanθ () ) C 3, L For the upper bound on L observe that L = (I + F () L C 8 D () L C 5

10 0 of 29 M. A. FREITAG AND A. SPENCE snce D () C 7 for large enough. For the lower bound use (I+F () )L Reorderng gves L (I+F () ) L (+ F () ) L. L L + F () ) L +C 7 C 8 C 4 = L and hence for large enough from whch the stated result holds. The followng proposton shows that the egenvalues of L (A σi)l and L (A σi)l are close to each other, where L L s the Cholesky factorsaton of P. PROPOSITION 3. Let L (A σi)l have egenvalues and egenvectors ẑ (), and L (A σi)l have egenvalues ˆξ wth egenvectors ẑ. Assume σ s not an egenvalue of A. Then, for each, ˆξ 0 and () ˆξ ˆξ ˆξ wth D () gven by (3.6) and C 6 ndependent of. Proof. Wth L = L(I+F () ) we have that may be wrtten as L L () (A σi)l z = Ths egenvalue problem s a perturbaton of ˆξ () D () C 6 tanθ (), (A σi)l ẑ () = () ˆξ (I+D () ) z (), () ˆξ ẑ () z () L (A σi)l z = ˆξ z, = (I+F () ) ẑ (). and an analyss smlar to the proof of Theorem 4. provdes the stated results. The followng Theorem shows that f (3.0) holds then the rght hand sde L x () n (3.9) s an approxmaton to the egenvector of the teraton matrx ( L (A σi)l ) correspondng to the egenvalue nearest zero. The dea s to show frst that λ () σ λ (),L x() s an approxmate egenpar of L (A σi)l drecton of L (cf. (4) n Lemma (3.)) and then use the fact that L x() s approxmately n the x (). Ths follows snce (2.2) and (3.0) gve L L x() = λ () x () + r () and hence L x () λ () L x () = L r (), wth r () C 9 tanθ () for some constant C 9 usng (2.3). ˆξ () the egen- THEOREM 3. Let L (A σi)l () have egenvalues ˆξ and egenvectors ŵ (), wth value nearest zero. Let P denote the orthogonal proecton onto span{ŵ () 2,...,ŵ() n }. Assume (3.0) holds, let r () be defned by (2.2) and assume λ () 0. Then, for small enough r () we have L x () c () 3 ŵ() C 0 r () (3.7)

11 and for some C 0 ndependent of for large enough. Proof. Straghtforward manpulaton shows that and that λ () σ (L A tuned precondtoner for nexact nverse teraton of 29 (A σi)l P L x () C 0 r () (3.8) )L x () = λ () σ λ () L x () + σ L λ () r (), λ () s the Raylegh quotent of L (A σi)l wth respect to the vector L x(). Then standard perturbaton theory for smple egenvalues of symmetrc matrces (see (Parlett, 998, Chapter ) or (Stewart & Sun, 990, page 250)) shows that L λ () σ λ () (A σi)l wth correspondng egenvector ŵ () near L x(). We obtan sn (ŵ (),L x() ) σ δ () L λ () r () where δ () = mn σ L r (), δ λ has a smple egenvalue =2,...,n ˆξ () λ () σ λ () ˆξ () near where n the last bound we have used λ () > λ due to the propertes of the Raylegh quotent, and the results of Proposton 3. and λ () λ = c sn(θ () ) 2 for some constant c whch yeld δ () () = mn ˆξ =2,...,n λ () σ mn =2,...,n ˆξ λ σ C 6 ˆξ tanθ () > C λ () where C s a constant ndependent of for large enough. After normalsng the vectors we obtan λ c () ŵ () L x() L x() σ L r (), δ λ where c () := cos (ŵ (),L x() ). Multplyng by L x() λ () and usng L x() λ () c () ŵ () L where c () s chosen approprately. Hence c () 3 ŵ() L r () λ () x () L x () L x() δ = L σ λ () L r (), λ x () + L r () λ () we get ( ) L x() σ δ λ () + λ λ () L r (). (3.9) Wth λ () > λ and (3.5) we obtan (3.7), snce all the terms n the brackets of (3.9) can be bounded ndependent of for large enough. Fnally, we have P L x () = P (L x () c () 3 ŵ() ) C r ()

12 2 of 29 M. A. FREITAG AND A. SPENCE snce P ŵ () = 0 and P =. For our purposes, the mportant result n Lemma 3. s (3.8), whch wth (2.3) mples that P L x () = O( snθ () ). Ths s smlar to the correspondng result n the unprecondtoned case gven by (2.7) and s mportant when analysng the lower bound for the number of teratons needed by MINRES. We have the followng consequence of Theorem 2.2 (compare wth (2.8) for the unprecondtoned case). THEOREM 3.2 Assume the condtons of Lemma 3. and that (3.8) and (3.0) hold. Consder the applcaton of MINRES to the nexact soluton of L (A σi)l ỹ () = L x (). (3.20) Assume L (A σi)l satsfes the condtons on B n Theorem 2.2. Further assume that we seek () () the smallest egenvalue, such that ˆξ < 0 and { ˆξ } n =2 > 0. Denote the reduced condton number by κl. Then the number of nner teratons needed by MINRES to solve (3.20) to a tolerance τ () L, where τ () = C r (), satsfes κ ( k () L + log2 2 ˆξ () () ˆξ n () ˆξ + log P L ) x () L and the rght hand sde of (3.2) can be bounded ndependent of for large enough. τ () (3.2) Proof. The bound on the teraton number (3.2) follows from (2.6) appled to (3.20), wth τ replaced by τ () L and P b replaced by P L x (). The bound (3.8) and the frst bound n (3.5) show that log P L x () τ () L log C r () L τ () log C C 3 C s ndependent of for large enough. The frst term n the brackets n (3.8) can be bounded usng Proposton 3.: () () ˆξ ˆξ n () ˆξ ˆξ ˆξn +C 5 ( ˆξ + ˆξn ) tanθ() ˆξ. C 5 ˆξ tanθ () Snce tanθ () s decreasng, the frst term n (3.2) can be bounded ndependent of for large enough. Furthermore we have κ L = max () =2,...,n ˆξ () mn =2,...,n ˆξ max =2,...,n ˆξ +C 5 tanθ() mn =2,...,n ˆξ C 5 tanθ (), (3.22) whch can also be bounded ndependent of for large enough. Theorem 3.2 ndcates that f we can fnd a postve defnte precondtoner that satsfes (3.0) then we expect no growth n the nner teraton count for MINRES usng the tuned precondtoner as the outer teraton proceeds. Numercal results confrmng ths effect are gven n Fgures and 4. We shall return n Secton 4 to the assumpton about the egenvalues of L (A σi)l satsfyng the condtons on B n Theorem 2.2. In the rest of ths secton we llustrate the performance of the tuned precondtoner by two numercal examples. Note that by applyng the second case n Theorem 2.2 a modfcaton of Theorem 3.2 also holds for nteror egenvalues though we do not gve examples of ths case here.

13 A tuned precondtoner for nexact nverse teraton 3 of 29 It s mportant to note that replacng P by P nvolves mnmal extra computatonal work. Indeed for the mplementaton of P rather than P at each () (that s, at each outer teraton) only a sngle extra back substtuton wth P s needed for the tuned precondtoner P. Ths s proved usng the Sherman- Morrson formula (see (Demmel, 997, p. 95)) for the nverse of a matrx wth a rank-one change. 3.3 Numercal examples We now present two numercal examples to llustrate the theory n ths secton. EXAMPLE 3. (PROBLEM FROM MARKET (2004)) Here we consder the matrx nos5.mtx from the Market (2004). It s a real symmetrc postve defnte matrx of sze wth 572 nonzero entres. Its frst two egenvalues are gven by st 2nd egenvalue We consder a fxed shft strategy and seek the smallest egenvalue. We compare the costs of the followng two dfferent methods: (a) Standard ncomplete Cholesky precondtoner: Algorthm 2. wth step (2) mplemented by solvng (2.0), where LL s the ncomplete Cholesky factorsaton of A. (b) Tuned ncomplete Cholesky precondtoner: Algorthm 2. wth step (2) mplemented by solvng (3.9), where P s gven by (3.7). For the nexact solves we use precondtoned MINRES wth τ () = mn{τ,τ r () }, τ = 0.. (3.23) and for the ncomplete Cholesky decomposton we use a drop tolerance of 0.. Then the number of nonzero entres n L s 032 and E 7.7e+04 (wth A 5.8e+05). We use a startng guess x (0) of all ones and a fxed shft of σ = 58. The computatons stop once the egenvalue resdual satsfes r () < 0 8. Fgure shows the number of nner teratons used by methods (a) and (b), where we see the steady ncrease n nner teratons needed by the standard precondtoner n method (a), but essentally constant number of nner teratons needed to solve (3.9) usng the tuned precondtoner as n method (b). Ths supports the result of Theorem 3.2. Fgure 2 plots the resdual norms aganst the total number of teratons, whch agan shows the superorty of the tuned precondtoner n terms of the total number of teratons. In Fgure 3 we plot the rght hand sdes of the lower bounds (2.2) and (3.2) respectvely, whch agan agrees wth the theory, though, as noted n Remark 2., these bounds should not be used quanttatvely. Indeed, the bound for method (a) exceeds the trval bound k () n for large enough. However, the bound for method (b) only overestmates the actual number of nner teratons by a factor of roughly.5. Next, n Table, we present the values n condton (3.8), whch ensures that P s postve defnte. We see tht (3.8) holds at each outer teraton. Also, we see that κl quckly becomes ndependent of, as stated after (3.22). Both methods have a relatvely hgh number of outer teratons, snce we have only lnear convergence wth convergence rate λ σ As we would expect there s almost no dfference λ 2 σ

14 4 of 29 M. A. FREITAG AND A. SPENCE between methods (a) and (b) as regards the number of outer teratons and the overall outer convergence rate. These results are not presented here snce we are prmarly nterested n the nner teratons used by precondtoned MINRES. The tuned precondtoner s clearly much better than the standard precondtoner n terms of the total number of teratons standard precondtonng,2232 nner teratons tuned precondtonng,940 nner teratons 0 0 standard precondtonng tuned precondtonng nner teratons resdual norms \ r^{()}\ outer teratons sum of nner teratons FIG.. Number of nner teratons aganst outer teratons for methods (a) and (b) FIG. 2. Resdual norms aganst total sum of teratons for methods (a) and (b) Bounds on number of nner teratons for method (a) Bounds on number of nner teratons for method (b) 350 nner teratons outer teratons FIG. 3. The bound (2.2) for method (a) and (3.2) for method (b) To summarse, the number of nner teratons per outer teraton grows steadly for the standard ncomplete Cholesky precondtoner as expected, whlst t stays roughly constant for the tuned precondtoner. Also, the tuned precondtoner requres about half the total number of nner teratons than the standard precondtoner. Ths suggests that tuned precondtoner has a clear advantage over the standard ncomplete Cholesky precondtoner. Smlar behavour s observed n our second example. EXAMPLE 3.2 (ELLIPTIC OPERATOR PROBLEM FROM LINDSTRÖM & ELDÉN (2002),SIMONCINI & ELDÉN (2002)) The matrx A(t) s a symmetry-preservng central fnte dfference approxmaton of

15 A tuned precondtoner for nexact nverse teraton 5 of 29 Iteraton x () u () (Vu () ) 2 /η κl Table. Values of x () u () and (Vu () ) 2 /η as well as reduced condton number κ L the self-adont ellptc operator A (t)u = ((+ tx)u x ) x +((+ ty)u y ) y on an equdstant grd on the unt square wth Drchlet boundary condtons and 50 nodes n each dmenson. Ths leads to a matrx A(t) of the sze 2500 wth 2300 nonzero entres. The two smallest egenvalues of A() are as follows. st 2nd egenvalue We are nterested n approxmatng the smallest egenpar of A(4). Our startng approxmaton x (0) s gven by the vector of all ones. We wll compare the costs of methods (a) and (b) from Example 3. For the nexact solves we use precondtoned MINRES wth (3.23) and for the ncomplete Cholesky decomposton we use a drop tolerance of 0.. We use a fxed shft of σ = Agan, the computatons stop once the egenvalue resdual satsfes r () < 0 8. Fgure 4 shows the number of nner teratons used by methods (a) and (b). Fgure 5 plots the resdual norms aganst the total number of teratons. Furthermore, Table 2 shows that condtons (3.8) are satsfed for each. Methods (a) and (b) requre the same number of outer teratons and the same outer convergence rate. In terms of the total number of teratons the tuned precondtoner s clearly much better than the standard precondtoner. We also observe that the theoretcal bounds n Fgure 6 overestmate the actual number of nner teratons by a factor of around 2 or less. The number of nner teratons per outer teraton grows steadly for the standard ncomplete Cholesky precondtoner as expected, whlst t stays roughly constant for the tuned precondtoner (see Fgure 4).

16 6 of 29 M. A. FREITAG AND A. SPENCE standard precondtonng,457 nner teratons tuned precondtonng,267 nner teratons 0 3 standard precondtonng tuned precondtonng 55 nner teratons resdual norms \ r^{()}\ outer teratons sum of nner teratons FIG. 4. Number of nner teratons aganst outer teratons for methods (a) and (b) FIG. 5. Resdual norms aganst total sum of teratons for methods (a) and (b) Bounds on number of nner teratons for method (a) Bounds on number of nner teratons for method (b) 0 00 nner teratons outer teratons FIG. 6. Bound on number of nner teratons aganst outer teratons for methods (a) and (b) Also, the tuned precondtoner requres about half the total number of nner teratons than the standard precondtoner. Indeed ths superorty s seen n other numercal experments not reproduced here, and overall, t appears that the tuned precondtoner has a clear advantage over the standard ncomplete Cholesky precondtoner. 4. Spectral analyss for the tuned precondtoner In Secton 3 we proved varous propertes of the tuned precondtoner P = L L gven by (3.7) by comparson wth the deal (but unknown) precondtoner gven by (3.3). In ths secton we shall present a drect comparson of the spectral propertes of L (A σi)l and L (A σi)l snce the analyss does not nvolve. However the analyss s dentcal to a comparson of the spectral propertes of L (A σi)l and L (A σi)l (whch we do not repeat) and the numercal results presented are for the practcal precondtoner P = L L. We shall show that there s a close relatonshp between the respectve spectra, and so f LL s a

17 A tuned precondtoner for nexact nverse teraton 7 of 29 Iteraton x () u () (Vu () ) 2 /η κl Table 2. Values of x () u () and (Vu () ) 2 /η as well as reduced condton number κ L good precondtoner for A σi then LL wll also be a good precondtoner. Specfcally, we make the comparson usng both a perturbaton analyss and an nterlacng analyss. Frst recall that for =,...,n, A has egenpars (λ,x ), L (A σi)l has egenpars (µ,w ) and L (A σi)l has egenpars (ξ,ŵ ). If we consder the problem of fndng the smallest egenvalue of A, say λ, usng a shft σ between λ and λ 2 (the next smallest egenvalue of A) then A σi has one negatve egenvalue, λ σ, and n postve egenvalues, {λ σ} n =2. Sylvester s Inerta Theorem readly shows that both L (A σi)l and L (A σi)l have one negatve egenvalue and n postve egenvalues. Thus, n ths case, the assumpton n Theorem 3.2 that L (A σi)l satsfes the condtons on B n Theorem 2.2 s satsfed. We emphasse that our theory s applcable to an nteror egenvalue, n whch case Theorem 2.2 can be altered to apply to a B wth a more general spectrum. Frst, we note that f L (A σi)l ŵ = ξ ŵ, (4.) then L (A σi)l ŵ = ξ (I+γvv )ŵ, (4.2) where ŵ = L L ŵ, v = L u and γ = x. Hence, we fnd that (4.) s equvalent to the generalsed u egenvalue problem (4.2) and compare the egenvalues of L (A σi)l w = µ w (4.3) wth those of (4.2). Also, Sylvester s nerta theorem shows that f (3.4) holds, then +γvv s postve defnte and +γv v > 0. (4.4) In Secton 4. we wll present a perturbaton result comparng the egenvalues ξ of (4.) to the egenvalues µ of (4.3), whch s a modfcaton of the theorem by Bauer and Fke (see, for example (Golub & Loan, 996, Theorem 7.2.2)). In Secton 4.2 we obtan a nonstandard nterlacng result to compare the spectra of the standard and tuned precondtoned systems.

18 8 of 29 M. A. FREITAG AND A. SPENCE 4. Perturbaton Theory The followng theorem yelds a perturbaton result for the egenvalues µ and ξ of (4.3) and (4.). THEOREM 4. (PERTURBATION PROPERTY) Assume σ s not an egenvalue A. Defne S = L (A σi)l and consder the two egenvalue problems and Sw = µw (4.5) Sw = ξ(i+γvv )w. (4.6) Then µ and ξ are nonzero. Also, let ξ be a soluton of (4.6). Then mn µ ξ µ Λ(S) ξ γv v. (4.7) Proof. If σ s not an egenvalue of A then A σi s nonsngular, and Sylvester s Inerta Theorem shows that µ and ξ n (4.5) and (4.6) respectvely cannot be zero. Wrte equaton (4.6) as (S ξ I)w = ξ γvv w. Now, let µ ξ (for µ = ξ the result (4.7) follows mmedately). Then S ξ I s nonsngular and w = ξ(s ξ I) γvv w. Takng norms we obtan w ξ (S ξ I) γ vv w and hence ξ mn µ ξ v, µ Λ(S) γ v yeldng (4.7) after rearrangement. COROLLARY 4. Interchangng the roles of (4.5) and (4.6) we have mn ξ µ ξ Λ(S,(I+γvv )) µ γv v, (4.8) where Λ(S,(I+γvv )) s the spectrum of the generalsed egenproblem (4.6). In Secton 4.3 we use ths perturbaton result to estmate the change n the condton number of the system matrx of (3.9) compared to the condton number of the system matrx of (2.0), whch s mportant for the performance of the teratve solver. 4.2 Interlacng property The followng two Lemmata lead to an nterlacng result (Theorem 4.2) between the egenvalues µ of (4.5) and ξ of (4.6), whch leads to an nterlacng result between the egenvalues of the matrces n (4.3) and (4.). Here we use deas from Wlknson (see Wlknson (965)) and Golub and van Loan (Golub & Loan, 996, Lemma 8.5.2, Theorem 8.5.3).

19 LEMMA 4. Consder the egenvalue problems and A tuned precondtoner for nexact nverse teraton 9 of 29 L (A σi)l w = µw (4.9) L (A σi)l ŵ = ξ ŵ, (4.0) where L s the Cholesky factor of P gven by (3.3). Then we can rewrte the second equaton as or Dt = ξ(i+γzz )t (4.) (D ξ γzz )t = ξ t, (4.2) where L L Qt = ŵ, z = Q v wth v = L u as n (4.2) and S = QDQ s the symmetrc real Schur decomposton of S = L (A σi)l,.e. D s a dagonal matrx D = dag(µ,...,µ n ) contanng the egenvalues of L (A σi)l. Proof. We already know from (4.3) and (4.2) that wth S = L (A σi)l we can rewrte equatons (4.9) and (4.0) as Sw = µw (4.3) and Sw = ξ(i+γvv )w, (4.4) Then, by usng the real symmetrc Schur decomposton of S = QDQ, where D = dag(µ,...,µ n ) and settng Q w = t and Q v = z we obtan (4.), that s Dt = ξ(i+γzz )t. In Golub & Loan (996) the effcent computaton of the egenvalues and egenvectors of a dagonal plus rank- matrx was descrbed establshng also an nterlacng property between the egenvalues of the dagonal matrx and the perturbed matrx (see also Wlknson (965)). Here, problem (4.) s a generalsed egenvalue problem rather than a standard egenproblem wth rank- change but we shall prove that for ths problem an nterlacng property also holds. The proofs of the followng Lemma and Theorem follow the lnes of the proofs of Lemma and Theorem n Golub & Loan (996). LEMMA 4.2 Suppose D = dag(µ,...,µ n ) R n n has the property that µ <... < µ n. Assume that γ 0 and that z has no zero components. If then z t 0 and D ξ I s nonsngular. (D ξ γzz )t = ξ t, t 0 Proof. If ξ were an egenvalue of D then ξ = µ for some and hence wth e beng the th canoncal vector we have 0 = e [(D ξ)t ξ γ(z t)z] = ξ γ(z t)z. Snce γ, z and ξ are nonzero (f ξ were zero then D would be sngular and σ would be an egenvalue of A) we must have z t = 0 and so Dt = ξ t. However D has dstnct egenvalues µ and therefore t span{e }. But then 0 = z t = z, yeldng a contradcton. Thus ξ s not an egenvalue of D and hence D ξ I s nonsngular and z t 0. We use ths result to prove the followng theorem.

20 20 of 29 M. A. FREITAG AND A. SPENCE THEOREM 4.2 (INTERLACING PROPERTY) Consder the two egenvalue problems and L (A σi)l w = µw (4.5) L (A σi)l ŵ = ξ ŵ, (4.6) and assume condton (3.4) holds. Suppose D = dag(µ,...,µ n ) R n n and that the dagonal entres satsfy µ <... < µ n. Let γ = x. Furthermore let z and t be defned as n Lemma 4.. Assume that u γ 0 and that z has no zero components. Let Dt = ξ (I+γzz )t, (4.7) where ξ are the egenvalues, wth ξ... ξ n and t are the correspondng egenvectors. Also, let µ <... < µ p < 0 < µ p+ <... < µ n, where p s the number of negatve egenvalues of L (A σi)l. Then (a) The ξ are the n zeros of f(ξ) = ξ γz (D ξ I) z. (b) The egenvector t s a multple of (D ξ I) z. (c) If γ > 0, then and whle, f γ < 0 then and µ < ξ < µ 2 < ξ 2 <... < µ p < ξ p < 0 0 < ξ p+ < µ p+ < ξ p+2 < µ p+2 <... < ξ n < µ n, ξ < µ < ξ 2 < µ 2 <... < ξ p < µ p < 0 0 < µ p+ < ξ p+ < µ p+2 < ξ p+2 <... < µ n < ξ n. Proof. From Lemma 4. we know that we can reduce problems (4.5) and (4.6) to (4.7). If (D ξ γzz )t = ξ t, then (D ξ I)t ξ γ(z t)z = 0. (4.8) From Lemma 4.2 we know that (D ξ I) s nonsngular. Thus t span((d ξ I) z) thereby establshng (b). Applyng z (D ξ I) to both sdes of equaton (4.8) we get z t( ξ γz (D ξ I) z) = 0. By Lemma (4.2), z t 0 and so ths shows that f ξ s an egenvalue of the generalzed problem (4.7) then f(ξ) = 0, establshng (a). To show the nterlacng property (c) we need to look more carefully at the equaton f(ξ) = ξ γ n = z 2 µ ξ.

21 A tuned precondtoner for nexact nverse teraton 2 of 29 In order to fnd the roots of f(ξ) = 0 the followng equalty has to be satsfed z 2 n ξ = γ = µ ξ. (4.9) Hence, the roots of f(ξ) = 0 can be found by determnng the ntersecton ponts of f (ξ) = ξ Note that the dervatve of f 2 (ξ) s gven by and f 2 (ξ) = γ n = z 2 µ ξ. z 2 f 2 n (ξ) = γ = (µ ξ) 2 and thus the dervatve s ether strctly postve or strctly negatve, dependng on the sgn of γ. Also, note that for ξ ± we get f 2 (ξ) 0. Furthermore, snce γ 0 and the µ are dstnct, f(ξ) has n zeroes. Dependng on the sgn of γ we have the followng stuatons: If γ > 0, then f 2 (ξ) > 0, that s f 2(ξ) s monotonely ncreasng between ts poles at ξ = µ, where µ are the egenvalues of (4.5). The hyperbola f (ξ) s monotonely decreasng for all ξ n (,0) and (0, ). The plot n Fgure 7 llustrates the stuaton for n = 4 and p = 2. We see that due to the f 2 (ξ) 0 f (ξ) µ µ 2 µ 3 µ FIG. 7. Intersecton ponts of f (ξ) and f 2 (ξ) for γ > 0 monotoncty propertes of f (ξ) and f 2 (ξ) there s exactly one ntersecton pont of f (µ) and f 2 (ξ)

22 µ µ 2 µ 3 µ 4 22 of 29 M. A. FREITAG AND A. SPENCE between each of the poles at ξ = µ except n the nterval contanng zero. In ths case there s an ntersecton pont between µ p and zero and a second ntersecton pont between zero and µ p+. Next, we show that there are no ntersecton ponts ξ > µ n and ξ < µ, that s that the ntersecton ponts are shfted towards the orgn wth respect to the poles. For ξ ± we get f 2 (ξ) 0 and snce f 2 (ξ) s monotonely ncreasng f 2 (ξ) approaches zero from below (for ξ ) or from above (for ξ ). The decreasng hyperbola f (ξ) does exactly the opposte and therefore the two curves cannot ntersect for ξ > µ n and ξ < µ. On the other hand, f γ < 0, then f 2 (ξ) < 0 and therefore f 2(ξ) s monotonely decreasng between ts poles at ξ = µ and the hyperbola f (ξ) s monotonely decreasng for all ξ n (,0) and (0, ). The plot n Fgure 8 llustrates the stuaton for n = 4 and p = 2. Agan we observe that due to the f 2 (ξ) 0 f (ξ) FIG. 8. Intersecton ponts of f (ξ) and f 2 (ξ) for γ < 0 monotoncty propertes of f (ξ) and f 2 (ξ) there s exactly one ntersecton pont of f (ξ) and f 2 (ξ) between each of the poles at ξ = µ wth the excepton that there s no ntersecton between the poles µ p < 0 and µ p+ > 0. Next, we show that there are two further ntersecton ponts, one for ξ > µ n and one for ξ < µ, and hence the ntersecton ponts are shfted away from the orgn wth respect to the poles. Consder ξ. Both functons f (ξ) and f 2 (ξ) are monotonely decreasng and approachng zero. In order to show that they ntersect we need to show that f (ξ)> f 2 (ξ) for ξ, snce, obvously close to the pole ξ = µ n + δ, δ 0, f (ξ) < f 2 (ξ). Hence, for f (ξ) > f 2 (ξ) for ξ, we have to show z 2 n ξ > γ = µ ξ for ξ

23 whch s equvalent to Takng the lmt we obtan A tuned precondtoner for nexact nverse teraton 23 of 29 > γξ and usng Q v = z ths s equvalent to n = > γ z 2 µ ξ n = for ξ. z 2 = γz z +γv v > 0 whch holds from (4.4). In order to show that f (ξ) < f 2 (ξ) for ξ a smlar analyss apples. Thus we have shown that the egenvalues are shfted away from the orgn for γ < 0. Hence, we see that for γ > 0 the egenvalues ξ are moved towards the orgn, nterlacng the egenvalues µ, whereas for γ < 0 the egenvalues ξ are moved away from the orgn nterlacng the egenvalues µ. Theorem 4.2 s proved n the specal case of no multple egenvalues µ and no zero components of z. Just as n (Golub & Loan, 996, Theorem 8.5.4) these restrctons are easly removed. THEOREM 4.3 Consder the two egenvalue problems and L (A σi)l w = µw (4.20) L (A σi)l ŵ = ξ ŵ, (4.2) and assume condton (3.4) holds. Suppose D = dag(µ,...,µ n ) R n n and let γ = x u. Furthermore let z and t be defned as n Lemma 4.. Assume that γ 0 and let Dt = ξ (I+γzz )t, (4.22) where ξ are the egenvalues, wth ξ... ξ n and t are the correspondng egenvectors. Also, let µ... µ p < 0 < µ p+... µ n, where p s the number of negatve egenvalues of L (A σi)l. Then the same nterlacng result as n Theorem 4.2 (c) holds, except that the strct nequaltes change to equaltes for z = 0 and n case of multple egenvalues µ of L (A σi)l. Proof. We only need to show the result for z = 0 and n case of multple µ. For other cases the result follows from Theorem 4.2. If z = 0 then from (4.) we obtan De = ξ(i+γzz )e = ξ e, where e s the th canoncal vector. Hence ξ = µ wth correspondng egenvector e whch s even better than nterlacng. Furthermore, f µ = µ + we can transform the problem to a problem wth a zero component of z. Let U = G(, +,θ) be a (orthogonal) Gvens rotaton n the (, + ) plane wth the property that z + = 0, that s Uz = [z,..., z,0,z +2,...,z n ] = z.

24 24 of 29 M. A. FREITAG AND A. SPENCE It s not hard to show that U DU = D. Hence U (D ξ γzz )U = D ξ γ z z and usng the prevous observaton for z + = 0 we get µ + = µ s an egenvalue ξ of the generalzed problem (4.7) wth correspondng egenvector Ue +. REMARK 4. Combnng the results of Theorem 4. and 4.2 we obtan one sded bounds for the largest and smallest egenvalues of L (A σi)l n terms of the largest and smallest egenvalues of L (A σi)l. Thus we also obtan bounds on the condton number of L (A σi)l n terms of the condton number of L (A σi)l. Furthermore we can conclude that any egenvalue clusterng propertes of L (A σi)l are preserved n L (A σi)l. Thus we are able to obtan qualtatve and quanttatve nformaton about the qualty of L as a precondtoner compared wth L. We note that all the results n ths subsecton hold dentcally for the practcal tuned precondtoner P = L L, provded (3.8) holds. Numercal results are gven for ths case below. 4.3 Consequences for the Tuned Precondtoner Here we merely compare the varous terms whch appear n (2.) and (3.2), whch gve bounds for the nner teratons n MINRES usng the standard and tuned precondtoners respectvely. As before, we assume that n our nvestgaton all the egenvalues µ 2,...,µ n of L (A σi)l are postve and µ, the extremal egenvalue s negatve. Thus (usng Slvester s Inerta Theorem) the egenvalues ξ 2,...,ξ n of L (A σi)l are postve and ξ s negatve. We have to compare the reduced condton numbers κl = µ n and κl µ 2 = ξ n ξ 2 as well as the terms µ µ n µ and ξ ξ n. ξ If γ > 0 then, from Theorem 4.2, the egenvalues ξ are shfted towards the orgn wth respect to the egenvalues µ. Hence ξ n µ n holds and from (4.7) we get Combnng both bounds yelds µ 2 + γv v ξ 2. κ L = ξ n ξ 2 µ n µ 2 (+ γv v ) = κ L(+ γv v ), (4.23) whch s an upper bound on the change to the reduced condton number due to tunng. Usng a smlar consderaton we obtan for γ > 0. µ µ n µ (+ γv v ) ξ ξ n ξ (4.24)

25 A tuned precondtoner for nexact nverse teraton 25 of 29 For γ < 0 a smlar dscusson also yelds (4.23) and 4.4 Numercal example ξ ξ n ξ (+ γv v ) µ µ n. (4.25) µ We consder a numercal example to support our theory n ths secton and compare the reduced condton numbers. EXAMPLE 4. We consder the matrx nos5.mtx from the Matrx Market lbrary Market (2004). Ths s a matrx of sze 468. We use a shft σ = 55 whch s close to the smallest egenvalue of (A σi) and whch leads to exactly one negatve egenvalue of (A σi). Agan, we choose x to be a random perturbaton from the egenvector belongng to the smallest egenvalue. Note that n ths case γ > 0 and condton (3.4) s ensured. Table 3. Results for Example 4.. The table gves values for u x = γ, + γv v, κl and κ L and for dfferent drop tolerances DROP TOLERANCE u x + γv v κ L κ L Table 3 shows the results for Example 4.. Wth regard to the soluton of the precondtoned shfted lnear systems usng MINRES, we observe that the change n the condton numbers and thus the change n the convergence rate s moderate. In fact, t only changes n the thrd or fourth sgnfcant dgt. We also observe that the perturbaton of the reduced condton number (4.23) s not sharp. The actual perturbaton of the reduced condton number s rather small, wth for bothdrop tolerances κ L κ L κ L (+ γv v ). 5. Numercal examples for nexact Raylegh quotent teraton In ths secton we present some numercal results to show the use of tunng when Raylegh quotent shfts are used. Also, we compare the performance of the tuned precondtoner wth the technque ntroduced by Smoncn & Eldén (2002). We do not provde any convergence theory: results n ths area can be found n Smt & Paardekooper (999), Smoncn & Eldén (2002) and Berns-Müller et al. (2006). Consder step (2) of Algorthm 2. where the shft σ s chosen as ρ () = x () Ax (). (5.) For the soluton by MINRES of the precondtoned system n the nexact Raylegh quotent method we have for the standard precondtoner where A = LL + E as n (2.9), and L (A ρ () I)L ỹ () = L x (), y () = L ỹ (), (5.2) L (A ρ () I)L ỹ () = L x (), y () = L ỹ (), (5.3)