A HIERARCHICAL LOW-RANK SCHUR COMPLEMENT PRECONDITIONER FOR INDEFINITE LINEAR SYSTEMS

Transcription

1 A HERARCHCAL LOW-RANK SCHUR COMPLEMENT PRECONDTONER FOR NDEFNTE LNEAR SYSTEMS GEOFFREY DLLON, YUANZHE X, AND YOUSEF SAAD Abstract. Nonsymmetric and highy indefinite inear systems can be quite difficut to sove via iterative methods. This paper combines ideas from the Mutieve Schur Low-Rank preconditioner deveoped by Y. Xi, R. Li, and Y. Saad [SAM J. Matrix Ana., 37 (216, pp ] with cassic bock preconditioning strategies in order to hande this case. The method to be described generates a tree structure T that represents a hierarchica decomposition of the origina matrix. This decomposition gives rise to a bock structured matrix at each eve of T. An approximate inverse based on the bock LU factorization of the system is computed at each eve via a ow-rank property inherent in the difference between the inverses of the Schur compement and another bock of the reordered matrix. The ow-rank correction matrix is computed by severa steps of the Arnodi process. Numerica resuts iustrate the robustness of the proposed preconditioner with respect to indefiniteness for a few discretized Partia Differentia Equations (PDEs and pubicy avaiabe test probems. Key words. bock preconditioner, Schur compements, mutieve, ow-rank approximation, Kryov subspace methods, domain decomposition, Nested Dissection ordering. AMS subject cassifications. 65F8, 65F1, 65F5, 65N55, 65Y5 1. ntroduction. n this paper we focus on the soution of arge nonsymmetric sparse inear systems Ax = b (1.1 via Kryov subspace methods where A C n n and b C n. When soving (1.1 it is often necessary to combine one of these Kryov methods with some form of preconditioning. For exampe, a right-preconditioning method woud sove the system AM u = b, M u = x, in pace of (1.1. Other variants incude eft and 2-sided preconditioners. deay, M is an approximation to A such that it is significanty easier to sove the inear systems with it than with the origina A. A commony used preconditioner is the ncompete LU (LU factorization of A, where A LU = M. LU preconditioners can be very effective for certain types of inear systems. However, if the origina matrix A is poory conditioned or highy indefinite then LU methods can fai due to very sma pivots or unstabe factors [1, 36]. Here, note that by indefinite matrix we refer to matrices that have eigenvaues on both sides of the imaginary axis. LU methods are aso known to have poor performance on high-performance computers, e.g., those with GPUs [29] or nte Xeon Phi processors. Agebraic mutigrid (AMG is another popuar technique for soving probems arising from discretized PDEs. Mutigrid methods are provaby optima for a wide range of SPD matrices and aso perform we in parae. However, without speciaization, mutigrid wi fai on even midy indefinite probems. Considerabe efforts must be made to make mutigrid work on these types of inear systems and even then the method must be taiored to the specific PDE being soved (e.g. mutigrid for Maxwe s equations. Sparse approximate inverses emerged in the 199s as aternatives to LU factorizations [7, 11, 19]. These methods were mosty abandoned due to their high cost both in terms of arithmetic and memory usage. A subsequent cass of preconditioners were based on rank-structured matrices [8]. Two such types of matrices are H 2 -matrices [2, 21] and hierarchicay semiseparabe (HSS matrices [42, 43, 44]. Both of these forms are the resut of a partition of the origina matrix where some of the off-diagona bocks are approximated by ow rank matrices. These ideas have been used to deveop both sparse direct sovers and preconditioners [45]. Recenty, a new cass of approximate inverse preconditioners was deveoped. The first of these was the Mutieve Low-Rank (MLR preconditioner descibed in [28]. Within the domain decomposition framework came the Schur compement ow-rank (SLR preconditioner [3], the Fast contour integra (FC preconditioner [41] foowed by the Mutieve Schur compement Low-Rank (MSLR preconditioner [39]. The MSLR preconditioner uses a mutieve Hierarchica interface decomposition (HD ordering [22] aong This work was supported by NSF under grant DMS and by the Minnesota Supercomputing nstitute Address: Department of Computer Science & Engineering, University of Minnesota, Twin Cities. {gdion,yxi,saad}@umn.edu 1

2 with an efficient Schur compement approximation. This approach is shown to be much ess sensitive to indefiniteness than the cassica LU and domain decomposition based methods. However, MSLR is designed for symmetric probems. n this paper, we present a preconditioner that incorporates a modified hierarchica ow rank approximation of the inverse Schur compement from the MSLR preconditioner into a bock preconditioner based on the bock LU factorization of A. The resuting method wi be caed a Generaized Mutieve Schur compement Low-Rank (GMSLR preconditioner. Two characteristics of the proposed methods are worth highighting. First GMSLR is designed to be appicabe to a wide range of probems. The preconditioner is nonsymmetric and it changes at each iteration, since it incorporates inner soves. As a resut we use fexibe GMRES [34] as the acceerator. The method aso performs we with symmetric matrices. As observed in [5, Section 1.1.2], the oss of symmetry incurred by appication of a nonsymmetric preconditioner is not a major concern provided that good approximations to certain bocks of A are avaiabe. The numerica experiments wi confirm this observation. Second, a property that is inherited from MSLR is that the GMSLR preconditioner computes a recursive, mutieve approximation to the inverse of the Schur compement. GMSLR is a bock preconditioner with inner sub-soves required at every outer iteration. These inner soves can themseves be preconditioned in order to reduce computationa costs. One of these required inner soves is with the Schur compement, i.e., we must sove Sy = g. For most probems, this inverse Schur compement approximation turns out to be an effective preconditioner for these inner soves. We have deveoped a code consisting of a set of C/C++ routines that impement the GMSLR preconditioner. This code was used in the numerica resuts section to sove SPD, symmetric indefinite, nonsymmetric, and compex non-hermitian inear systems. The code reorders the matrix, buids the preconditioner, and then soves the inear system. nte MKL routines are used aong with thread-eve paraeism via OpenMP. A truy parae code using MP is in progress. This paper is organized as foows. n Section 2 we briefy review the HD ordering. Section 3 has a brief overview of bock preconditioning that motivates the need for the ow-rank property of the inverse of the Schur compement. The detais of the Schur compement approximation are given in Section 4. n Section 5 we present the preconditioner construction process. A two eve anaysis of the preconditioned eigenvaues is presented in Section 6. Then, in Section 7, we present some numerica resuts from test probems and probems from the SuiteSparse matrix coection [15]. Concuding remarks and some ideas for future work are found in Section HD ordering. Reordering the origina system matrix A is essentia for the performance of direct as we as iterative methods [6, 27, 33, 37]. GMSLR uses one such reordering technique known as the Hierarchica nterface Decomposition (HD [22]. This ordering is appicabe to a wide cass of sparse matrices, not just those that originate from PDEs. An HD ordering can be obtained in a number of ways. A particuar method for obtaining such an ordering is the we known nested dissection method [18]. Nested dissection recursivey partitions the adjacency graph of A into bipartite subgraphs. The vertices whose remova eads to two disjoint subgraphs are caed vertex separators. Each eve of bisection produces a new separator and new subgraphs. This eve information can be represented by an HD tree T. The matrix itsef is reordered according to eve, starting with eve and ending with eve L. Since we assume that A is arge, sparse, and nonsymmetric, then an HD ordering has the mutieve, recursive structure ( B F A = and C E C = A +1 for = : L 1. (2.1 n this notation, A denotes the origina matrix A after HD ordering whereas A L is the submatrix associated with the L th eve connector. The B bock itsef has a bock diagona structure due to the bock independent set ordering [37], making soves with B ideay suited for parae computation. Figure 2.1 shows an exampe of the HD ordering for a 3D convection-diffusion operator discretized with the standard 7-point finite difference stenci. 3. Bock Preconditioning. Domain decomposition reordering gives rise to inear systems of the form ( B F A =, (3.1 E C 2

3 Fig. 2.1: A 4-eve HD ordered 3D convection-diffusion matrix with zero Dirichet boundary conditions. The origina matrix is discretized on a reguar grid with the standard 7-point stenci. The red ines separate the different eves. see [2, 9]. Simiar bock structured matrices aso arise from the discretization of systems of partia differentia equations. n these couped systems, the individua bocks usuay correspond to differentia/integra operators, however in this context they represent different sets of unknowns (interior, interface, couping that resut from domain decomposition. There is a arge body of work on preconditioning these systems mosty from the point of view of sadde point systems, see [4, 5, 25, 31, 32]. For exampes of preconditioning other couped systems of PDEs, see [12, 23, 24]. At the initia eve GMSLR uses a bock trianguar preconditioner of the form ( B F P = S (3.2 where B is an approximation to the (1, 1 bock of A and S is an approximation to the Schur compement S = C E B F. n the idea case where B = B and S = S, it is we known that the matrix A P idea has a quadratic minima poynomia, which means that GMRES wi converge in two iterations [25, 32]. Therefore the tota cost of the procedure based on the idea form of (3.2 is 2 inear soves with B and two inear soves with S pus additiona sparse matrix-vector products. This is made cear by ooking at the factored form of P idea : ( ( P idea = B F B = S ( ( F S. (3.3 This choice corresponds to using ony the upper trianguar part of the bock LU factorization of A as a preconditioner. f both parts of this factorization are used, i.e., if our preconditioner is of the form ( ( ( ( P B = F S E B, (3.4 3

4 then in the idea case we have an exact inverse of A and a Kryov method wi converge in a singe iteration at the tota cost of two soves with B and one sove with S. Thus, in a, using (3.4 saves one S sove over (3.3. The scenario just described invoves idea preconditioners (3.3 and (3.4 which are however not practica since they invove the exact computation of S. n practice, B and S are approximated, at the cost of a few extra outer iterations. With these approximations in pace it turns out that there is itte difference in practice between these two options and, based on our experience, we prefer to use (3.2. This issue wi be revisited at the end of Section Simiar to [3], we sove inear systems with the B bocks by using incompete LU (LU factorizations. Approximations to the Schur compement are typicay taiored specificay to the probem being studied (e.g. the pressure convection diffusion [17] and east-squares commutator [16] preconditioners for Navier- Stokes. However, in our framework, the bock form of A is the resut of a reordering of the unknowns and so our Schur compement approximation is inherenty agebraic and not based on the physics of the probem. We base our Schur compement approximation on ideas from [3, 39]. 4. Schur compement approximation. GMSLR is an extension of the MSLR preconditioner of [39] based on approximating the bock LDU factorization of (2.1: ( ( ( B B A = E B F (4.1 S at every eve =,... L 1. We write the Schur compement as S = ( E B F C C ( G C. (4.2 Let the compex Schur decomposition of G be G = E B F C = W R W H (4.3 where W is unitary and R is an upper trianguar matrix whose diagona contains the eigenvaues of G. Substituting (4.3 into (4.2 we get that S = ( W R W H C = W ( R W H C. (4.4 Then, the Sherman-Morrison-Woodbury formua yieds the inverse of S : which reduces to S = C W ( R W H S = C = C [ + W (( R W H ] (4.5 + C [ W ( R ] W H. (4.6 Some observations about the matrix S C wi be stated in the next section. n our agorithm, we do not compute the fu Schur decomposition of G, just the k k eading submatrix of R and the first k Schur vectors. The resuting inverse Schur compement approximation is given in the foowing proposition. Proposition 4.1. Let G = E B F C, =... L 1 and G = W R W H be its Schur decomposition at eve. Let W,k be the matrix of the first k Schur vectors (k < s of W. f we define R,k to be the k k eading principa submatrix of R, then the approximate th eve inverse Schur compement is given by S,k where S,k = C ( + W,k H,k W,k H. (4.7 H,k = [( R,k ]. (4.8 This inverse Schur compement approximation (4.7 wi be used at every eve =,..., L 1. Finay, due to the potentia size of the C bocks, we can ony afford to factor C L (i.e., at the top eve since it is the smaest of a the C bocks. For L 1 we use a sighty modified version of the recursive scheme of [39] for approximating the action of C on a vector. The detais of this approximation wi be shown in Section 5. 4

5 4.1. Low rank property of S C. Consider the inverse Schur compement formua given by (4.6. n this section we caim that for certain probems, the matrix S C is of ow rank. f this is the case, then (4.7 wi be a good approximation to (4.6. The ony assumption we make on the bocks B, C is that they have LU factorizations, i.e., n practice we wi use incompete LU factorizations, so instead B = L B U B, C = L C U C. (4.9 B L B U B, C L C U C. Note that for arge, 3D probems, the number of interface points (i.e., the size of the C bock can be quite arge, making this factorization too costy. This is part of the motivation for the mutieve decomposition. To see that S C is of ow-rank, again define the matrix G by G = E B F C = (C S C. (4.1 Let γ i, i = 1,..., s be the eigenvaues of G (and aso R and define X C (S (4.6 the eigenvaues θ 1, θ 2,, θ s, θ s of X are given expicity by θ i = C. By equation γ i 1 γ i, i = 1,... s (4.11 since ( G = G ( G. As ong as the eigenvaues γ i of G are not custered at 1, the eigenvaues θ i of X wi be we separated. This in turn means that S C can be approximated by a ow rank matrix. This was studied in detai in [39, Section 2] for the symmetric case, where a theoretica bound for the numerica rank was estabished Buiding the ow-rank correction. We use Arnodi s method [1] to buid the ow rank correction matrices in (4.7. This approximation can be efficient if the desired eigenpairs of G are on the periphery of the spectrum. However, as we sha see in the numerica resuts, this is simpy not the case for some of the more indefinite probems. A particuar remedy is to take more steps of Arnodi s method. Taking m steps of Arnodi s method on G yieds the Kryov factorizations: G U m = U m H m + h m+1,m u m+1 e T m U T mg U m = H m where U m is an orthonorma matrix and H m is a Hessenberg matrix whose eigenvaues (aso caed Ritz vaues are good estimates to the extreme eigenvaues of G. We then take the compex Schur factorization of H m Q H H m Q = T. (4.12 We can reorder the k eigenvaues cosest to 1 we wish to defate so that they appear as the first k diagona entries of T [3, 38]. We approximate the ow-rank matrices in (4.7 by R,k T 1:k,1:k and W,k U m Q :,1:k. ( Preconditioner construction process. n this section we show how the ow-rank property discussed in the previous section is used to buid an efficient preconditioner. The ony assumption we make is that each of the B, C bocks are non-singuar eve scheme. We iustrate the steps taken to sove Ax = b with a 3-eve exampe. Step : Appy a 3-eve HD ordering to the origina matrix A and right hand side b. Ca the resuting reordered matrix and right hand side A, b respectivey. Step 1: At this eve (ony we use the bock trianguar matrix U = ( ( B F B = S 5 ( ( F S

6 as a right preconditioner for A i.e., we sove A U u = b. Here we approximatey factor B by LU and approximate the Schur compement by S S = C ( + W H W T where H and W are taken from (4.8 and (4.13 respectivey. To sove with C, we refer to (2.1 and move from eve to eve 1. Step 2: At eve 1, we have ( C = A B 1 = 1 F ( 1 B 1 S 1 ( E 1 B1 where S 1 is approximated by C 1 pus a ow-rank correction: S1 S 1 = C1 ( + W 1H 1 W1 T. Next we move up a eve again to define an approximate inverse for C 1, referring again to (2.1. Step 3: At eve 2 we have: ( C1 = A B 2 = 2 F ( 2 B 2 S 2 ( E 2 B2. Our earier anaysis suggests that we approximate S 2 by C 2 pus a ow-rank correction term, i.e., S2 S 2 = C2 ( + W 2H 2 W2 T. At this eve, we decide that C 2 is sufficienty sma and compute its LU factorization: C 2 L C2 U C2. n order to appy the preconditioner U, the actua agorithm starts at eve 2 and proceeds up to eve. For this particuar exampe, that means we start forward-backward soving with the LU factorization of C 2 since C2 is needed in order to appy S2. Now that the action of S 2 is avaiabe we can then approximate A 2 and the pattern continues unti we hit eve, i.e., L C2 U C2 C2 S 2 A 2 S 1 A 1 S U. Once C (or its action on a vector is avaiabe, the ow-rank correction matrices W, H can be computed Genera Case. When computing the eigenvaues and eigenvectors of the matrix G, we need to be abe to compute matrix vector products with the matrix E B F C at each eve. We aready have the factors of B, so any matrix-vector product with B can be computed with one forward and one backward substitution. The same does not hod true for C, since we ony compute its factorization at eve L 1. However, we aready have an approximate factorization of A +1 and since C = A +1 we can use this approximation to appy C to a vector. The construction of the preconditioner is summarized in Agorithm 1. The detais of the recursivey defined product of C with a vector b are given in Agorithm 6

7 2. Agorithm 1 Generaized Mutieve Schur Low-Rank (Construction phase 1: procedure GMSLR 2: Appy an L-eve reordering to A (A = reordered matrix. 3: for eve from L 1 to do 4: if = L 1 then 5: Compute LU factorization of C L, C L L CL U CL 6: end if 7: Compute LU factorization of B, B L B U B. 8: Perform k steps of the Arnodi process Ca Agorithm 2 to appy C [V, K ] = Arnodi(E U B L B F C, k 9: Compute the compex Schur decomposition K = W T W T. 1: Compute W,k = V W and set R,k = T 1:k,1:k. 11: Compute H = ( R,k = R,k ( R k. 12: end for 13: end procedure Agorithm 2 Approximation of y = C b for 1 and y = U b 1: procedure RecursiveSove(, b 2: if = L 1 then 3: return y = U C L L C L b 4: ese 5: Spit b = (b T 1, b T 2 T conformingy with the bocking of C 6: Compute z 1 = U B +1 L B +1 b 1 7: Compute z 2 = b 2 E +1 z 1 8: if 1 < L 1 then 9: Compute w 2 = W +1,k+1 H +1 W+1,k T +1 z 2 1: Compute y 2 = RecursiveSove( + 1, z 2 + w 2 11: Compute y 1 = z 1 U B +1 L B +1 F +1 y 2 12: ese 13: Sove the system S y 2 = z 2 with S as a right preconditioner 14: Compute y 1 = U B L B (b 1 F y 2 15: end if 16: return y = (y1 T, y2 T T 17: end if 18: end procedure This construction process shares many of the same efficiencies of the MSLR preconditioner. Namey, the HD ordering gives rise to B matrices that are bock-diagona in structure, and so a of these bocks can be factored in parae. Secondy, the trianguar soves associated with B can aso be done in parae for each bock. n addition, whie Agorithm 2 generay provides an accurate approximation to C, we must point out that due to the presence of the inner sove at eve = (Line 13 of Agorithm 2, GMSLR is (potentiay more expensive per iteration than MSLR. This expense can be essened somewhat by the fact that the inner sove can ony require 1-2 digits of accuracy without radicay affecting the convergence rate of the outer sove. 6. Eigenvaue Anaysis. This section studies the spectra of inear systems preconditioned by GM- SLR. We ony consider a 2 eve decomposition since the recursive nature of both agorithms makes the anaysis difficut. n what foows, et B denote an approximation to B and S the GMSLR approximation to the Schur compement S = C E B F respectivey. GMSLR starts with a 2 2 bock partition of 7

8 (a Eigenvaues of shifted Lapacian preconditioned by GMSLR with a rank 2 correction. (b Eigenvaues of shifted Lapacian preconditioned by GMSLR with a rank 2 correction. Fig. 6.1: Spectra of (6.3. These figures were obtained by using a 2 eve reordering (to compement the above anaysis and the exact LU decomposition of the B bocks. the origina matrix A, i.e., ( B F A = E C (6.1 where B is n B n B and C is s s. As was aready seen, the GMSLR preconditioner is based on the bock-lu factorization of (6.1, so at eve we have ( ( ( B F A = B F = E C E B = L S U, and the preconditioner Ũ is A simpe cacuation shows that ( ( Ũ = B F A Ũ = ( ( S B B ( B B F E B S S S.. (6.2 f we assume that B = B, then (6.2 simpifies to ( A Ũ = E B, (6.3 S S which has eigenvaues λ(a Ũ = {1, λ(s S }. Convergence wi be rapid if the eigenvaues of S S are aso cose to 1. To iustrate the infuence the rank has on convergence, we show the spectra of (6.3 for a sma test probem in Figure 6.1. Here A is the discretized shifted Lapacian operator u cu = f with c =.5 and homogeneous Dirichet boundary conditions. For reference, this 8 8 matrix has 35 negative eigenvaues. Since this matrix is sma, we use the LU factorization of B in order for the preconditioned system to resembe (6.3 as much as possibe. Figure 6.1(a shows that with ony a rank 2 correction, the spectrum of A Ũ is not particuary we custered. Figure 6.1(b shows that with a rank 2 correction, the spectrum of A Ũ is tighty custered around 1. For this particuar probem, GMSLR with a rank 2 ow rank correction converges in 6 outer iterations whie the rank 2 corrected version converges in 23. 8

9 7. Numerica experiments. Our goa is the efficient soution of arge 2D and 3D probems. The experiments were a run on a singe node of the Mesabi Linux custer at the Minnesota Supercomputing nstitute. This node has 64 GB memory and consists of two sockets each having a tweve core 2.5 GHz nte Haswe processor. This preconditioner was written in C++ and compied by nte s C++ compier using O3 optimization. Simpe thread-eve paraeism was achieved with OpenMP with a maximum of 24 threads. The B bocks are factored by the LUT routine from TSOL. The nte Math Kerne Library (MKL was used for many BLAS and LAPACK routines. We use fexibe GMRES [34] with a fixed restart size of 4 as the outer sover, denoted by GMRES(4. The inner sove in step 14 of Agorithm 2 is aso done with FGMRES. Uness otherwise noted, we foow the methodoogy of [28, 35, 39] where the right hand side vector b is given by Ae = b where e is the vector of a ones. The HD ordering was obtained by the function PartGraphRecursive from the METS [26] package. The diagona bocks of each B, C were reordered using the approximate minimum degree (AMD ordering [13, 14] in order to reduce fi-in generated by their LU factorizations. n our experiments the reported preconditioner construction time comes from the factorization of the B bocks and the computation of the ow-rank correction matrices. The reordering time is regarded as preprocessing and is therefore not reported. Simiary, the iteration time is the combined time spent on the inner and outer soves. The parameters we are most interested in varying are: the number of eves in the HD and the maximum rank used in the ow-rank correction, i.e. the number of steps of Arnodi s method. We use the foowing notation in the resuts that foow: fi = nnz(prec nnz(a ; p-t: wa cock time to buid the preconditioner (in seconds; its: number of outer iterations of preconditioned GMRES(4 required for r k 2 < 1 6. We use F to indicate that GMRES(4 did not converge after 5 iterations; i-t: wa cock time for the iteration phase of the sover. This time is not reported when GMRES(4 does not converge, as indicated by ; rk: max rank used in buiding the ow-rank corrections Probem 1. We begin our tests with the symmetric indefinite probem: u cu = f in Ω, u = on Ω, (7.1 where Ω = (, 1 3. The discretization is via finite differences with the standard 7-point stenci in 3D. This test probem is usefu for testing robustness with respect to definiteness. For reference, GMRES preconditioned by standard AMG fais to converge when appied to (7.1 with even a sma positive shift on a reguar mesh Varying the number of eves. First, we study the effect of adding more eves to the preconditioner. We sove (7.1 with c > in order to make the probem indefinite. n the cases where c >, we shift the diescretized Lapacian operator by s, where s = h 2 c for mesh size h. For this first exampe, we set s =.5. The associated coefficient matrix has 163 negative eigenvaues. The maximum rank was fixed at 5. As Figure 7.1 shows, the LU fi-factor curve is monotonicay decreasing whie the ow-rank correction fi-factor increases monotonicay. The optima number of eves occurs when these two quantities are roughy equa. For this particuar exampe, we pick nev opt = 6 as it strikes the right baance of fi, iteration count, and tota computationa time as shown in Tabe 7.1. Finay, reca that we coud have used the inexact version of (3.4 instead of (3.2. For SPD probems there is not a significant difference in the resuts obtained by either preconditioner. However, as shown in Tabe 7.2, for an indefinite probem such as (7.1 with s =.5, (3.2 performs better. The ikey expanation for this behavior is that (3.2 invoves fewer soves with the B matrices which are highy indefinite and therefore admit poor LU factorizations Varying the maximum rank in the ow-rank corrections. Next, we keep the number of eves fixed, but increase the maximum rank. We again sove (7.1 with s =.5 discretized on a 32 3 reguar grid. The LU fi factor is constant because we are keeping the number of eves fixed at 6. The fi factor from the ow rank corrections increases at an amost constant rate. ncreasing the maximum rank has the unfortunate effect of increasing the fi-factor and the preconditioner construction time. As we see in Tabe 9

10 ev LU fi LRC fi fi p-t i-t its Tabe 7.1: The fi factor and iteration counts for soving (7.1 with s =.5 on a 32 3 grid with the FGMRES-GMSLR method. Here, the maximum rank for the LRC matrices was fixed at 5. Fig. 7.1: ustration of the fi factors from LU and ow-rank corrections versus different eves in Tabe , the effect of increasing the rank (at east for this mode probem is difficut to predict. As a genera rue, it seems as though a arge maximum rank is unavoidabe for highy indefinite probems ncreasingy indefinite probems. The mode probem (7.1 becomes significanty more difficut to sove as s increases. Here, we increase s from to 1 whie tuning the maximum rank and number of eves to compensate for soving this increasingy difficut probem. We report the resuts that give the best baance between iteration count and fi in Tabe 7.4. The fi factor increases dramaticay for two reasons: first, we must increase the rank of the ow rank correction and second, we must keep the number of eves ow, which means the LU factors wi be denser. f the rank is too ow or the number of eves is too high, GMRES(4 simpy wi not converge. We are forced to dramaticay increase the rank for these highy indefinite probems due to how we construct the ow rank correction. Reca that the construction of the ow rank correction is based on finding approximate eigenvaues of the matrix E U B L B F C using Arnodi s method. When B is indefinite, as is the case here, the eigenvaues we seek get pushed deeper inside the spectrum, i.e. they become interior eigenvaues. Since the Arnodi process recovers extreme eigenvaues quicky, we are forced to take more steps of Arnodi in order to approximate these eigenvaues Probem 2. The second probem of interest is nonsymmetric: u α u cu = f in Ω, u = on Ω, (7.2 where Ω = (, 1 3, α R 3. This probem is simpy a shifted convection-diffusion equation, again discretized by the 7-point finite difference stenci. As before we shift the discretized convection-diffusion operator by s where s = h 2 c. 1

11 GMSLR - U ony GMSLR - U nev p-t i-t its p-t i-t its Tabe 7.2: Comparison between GMSLR with ony using U and GMSLR with L and U on (7.1 with s =.5 on a 32 3 grid. The maximum rank was fixed at 5. rank LU fi LRC fi fi p-t i-t its Tabe 7.3: teration counts for soving (7.1 with s =.5 on a 32 3 grid with the FGMRES-GMSLR method. The number of eves was fixed at Varying the number of eves. n this next set of experiments we fix α = [.1,.1,.1] and sove (7.2 in 3D with no shift and then with a shift of s =.25. As before, we start by increasing the number of eves. The resuts of the first probem with a maximum rank of 2 are in Tabe 7.5. These resuts are comparabe to those obtained from the SPD probem (7.1 with s =, i.e., for this probem, the convergence rate is not adversey affected by the oss of symmetry. Next, we sove (7.2 with s =.25. The shift significanty increases the number of eigenvaues with negative rea parts, so we increase the maximum rank to 5. The resuts can be found in Tabe 7.6. t is interesting to note that the fi from the ow rank correction is amost exacty the same as in Tabe 7.1. This is due to the fact that both probems used a maximum rank of 5 to buid the ow-rank corrections Probem 3. The third mode probem is a Hemhotz equation of the form ( ω2 v(x 2 u(x, ω = s(x, ω. (7.3 n this formuation, is the Lapacian operator, ω the anguar frequency, v(x the veocity fied, and s(x, ω is the externa forcing function with corresponding time-harmonic wave fied soution u(x, ω. The computationa domain is the unit cube Ω = (, 1 3 where we again use the seven-point finite difference discretization on a reguar mesh. The Perfecty Matched Layer (PML boundary condition is used on a faces of Ω. The resuting inear systems are compex non-hermitian. f we assume that the mean of v(x is 1 in (7.3, then the wave number is ω/(2π and λ = 2π/ω the waveength. The number of grid points in each dimension is N = qω/(2π where q is the number of points per waveength. As a resut, the discretized system is n = N 3 N 3. We test the performance of the GMSLR preconditioner on 6 cubes with q = 8 and report the resuts in Tabe 7.7. Since q is fixed, an increase in wave number means an increase in N, so the higher frequency probems ead to much arger inear systems. These probems are much more sensitive to the number of eves used. Overa, we use a smaer number of eves (as compared to the rea-vaued test probems, which eads to arger fi factors. n these experiments, we set the inner sove toerance to 1. The number of outer iterations required ony increases from 6 to 13 whie the matrices grow from 2 3 to 8 3. The fi factors do increase as the probem gets arger, but not by too much. The ast probem has a arger 11

12 s nev max rank fi p-t i-t its Tabe 7.4: Resuts of soving symmetric inear systems with increasing shift vaues s on a 32 3 reguar mesh with GMSLR. ev LU fi LRC fi fi p-t i-t its Tabe 7.5: The fi factor and iteration counts for soving (7.2 with no shift and α = [.1,.1,.1] on a 32 3 grid with the FGMRES-GMSLR method. Here, the maximum rank for the LRC matrices was fixed at 2. fi factor due to the fact that the maximum rank used is significanty higher than the other exampes whie the number of eves remains the same Other probems. To further iustrate the robustness of the GMSLR preconditioner, we tested it on severa arge, nonsymmetric matrices from the SuiteSparse Matrix Coection [15]. These matrices come from a wide range of appication areas, not just PDEs. As a benchmark, we aso tested LUT for these nonsymmetric matrices. nformation about the matrices is shown in Tabe 7.8. Tabe 7.9 shows the resuts of these experiments. The LUT parameters were chosen such that the fi of both methods was comparabe. Resuts are shown in Tabe 7.9, where F indicates a faiure to converge in 5 iterations. As can be seen, for these probems, GMSLR is superior to LUT. t is worth adding that LUT is a highy sequentia preconditioner both in its construction and its appication. n contrast, GMSLR is by design a domain decomposition-type preconditioner that offers potentia for exceent paraeism. 8. Concusion. The GMSLR preconditioner combines severa ideas. First is the HD ordering method, which has a recursive mutieve structure. The (1, 1 bock of each eve of this structure is bock diagona, which means that soves with this bock are easiy paraeizabe. Motivated by the bock LU factorization of the reordered matrix, we use a bock trianguar preconditioner at the bottom eve of the HD tree. For the other eves, we use approximate inverse factorizations thanks to the reationships between the different eves. Finay, we approximate the inverse Schur compement of each eve of the HD tree via a ow-rank property. Because it is essentiay an approximate inverse preconditioner, GMSLR is capabe of soving a wide range of highy indefinite probems that woud be difficut for standard methods such as LU. The numerica experiments we showed confirm this. Additiona benefits of GMSLR incude its inherent paraeism and its fast construction. GMSLR is aso promising for use in eigenvaue computations. n severa eigenvaue methods, such as shift and invert and rationa fitering [4], one has to sove highy indefinite systems. The factorization of these systems can be sow and costy for arge 3D probems. We pan on investigating the use of Kryov subspace methods preconditioned by GMSLR to sove such systems. Among our other objectives, we are aso panning to impement and pubicy reease a fuy parae, domain-decomposition based, version of GMSLR. 12

13 ev LU fi LRC fi fi p-t i-t its Tabe 7.6: The fi factor and iteration counts for soving (7.2 with s =.25 and α = [.1,.1,.1] on a 32 3 grid with the FGMRES-GMSLR method. Here, the maximum rank for the LRC matrices was fixed at 5. ω/(2π q n = N 3 nev rk fi p-t i-t its Tabe 7.7: Resuts from soving (7.3 on a sequence of 3D meshes with GMSLR. Here q denotes the number of points per waveength. Acknowedgements. The authors woud ike to thank the Minnesota Supercomputing nstitute for the use of their extensive computing resources. REFERENCES [1] W. E. Arnodi, The principe of minimized iterations in the soution of the matrix eigenvaue probem, Quartery of Appied Mathematics, 9 (1951, pp [2] O. Axesson and B. Poman, Bock preconditioning and domain decomposition methods, Journa of Computationa and Appied Mathematics, 24 (1988, pp [3] Z. Bai and J.W. Demme, On swapping diagona bocks in rea Schur form, Linear Agebra and its Appications, 186 (1993, pp [4] M. Benzi, Preconditioning techniques for arge inear systems: A survey, J. of Computationa Physics, 182 (22, pp [5] M. Benzi, G. H. Goub, and J. Liesen, Numerica soution of sadde point probems, Acta Numerica, 14 (25, pp [6] M. Benzi, D. Szyd, and A. Van Duin, Orderings for incompete factorization preconditioning of nonsymmetric probems, SAM J. Sci. Comput., 2 (1999, pp [7] M. Benzi and M. Tůma, A sparse approximate inverse preconditioner for nonsymmetric inear systems, SAM J. Sci. Comput., 19 (1998, pp [8] D. Cai, E. Chow, Y. Saad, and Y. Xi, SMASH: Structured matrix approximation by separation and hierarchy., Preprint ys-216-1, Dept. Computer Science and Engineering, University of Minnesota, Minneapois, MN, (216. [9] E. Chow and Y. Saad, Approximate inverse techniques for bock-partitioned matrices, SAM J. Sci. Comput., 18 (1997, pp [1], Experimenta study of LU preconditioners for indefinite matrices, J. Comput. App. Math., 86 (1997, pp [11], Approximate inverse preconditioners via sparse-sparse iterations, SAM J. Sci. Comput., 19 (1998, pp [12] E. C. Cyr, J. N. Shadid, R. S. Tuminaro, R.P. Pawowski, and L. Chacón, A new approximate bock factorization preconditioner for two-dimensiona incompressibe (reduced resistive MHD, SAM J. Sci. Comput., 35 (213, pp. B71 B73. [13] P.R. Amestoy T.A. Davis and.s. Duff, An approximate minimum degree ordering agorithm, SAM J. Matrix Ana. App., 17 (1996, pp [14], Agorithm 837: An approximate minimum degree ordering agorithm, ACM Trans. Math. Software, 3 (24, pp

14 Matrix Order nnz SPD Origin CoupCons 416,8 22,322,336 no structura probem AtmosModd 1,27,432 8,814,88 no atmospheric mode AtmosModL 1,489,752 1,319,76 no atmospheric mode Cage14 1,55,785 27,13,349 no directed weighted graph Transport 1,62,111 23,5,731 no CFD probem Tabe 7.8: Set of nonsymmetric test matrices from the SuiteSparse Matrix Coection. Matrix GMSLR LUT fi nev rank p-t i-t its fi p-t i-t its CoupCons AtmosModd AtmosModL Cage Transport F Tabe 7.9: Comparison between GMSLR and LUT preconditioners for soving the above probems. LUT parameters were chosen so that the fi factor was cose to that of GMSLR. Both sets of tests use the same reordered matrix. [15] T.A. Davis and Y. Hu, The University of Forida Sparse Matrix Coection, ACM Trans. Math. Software, 38 (211, p. 1. [16] H. Eman, V. Howe, J. Shadid, R. Shutteworth, and R. Tuminaro, Bock preconditioners based on approximate commutators, SAM J. Sci. Comput., 27 (26, pp [17] H. C. Eman, D. J. Sivester, and A. J. Wathen, Finite Eements and Fast terative Sovers, Oxford University Press, Oxford, 25. [18] A. George, Nested dissection of a reguar finite eement mesh, SAM J. Numer. Ana., 1 (1973, pp [19] M. Grote and T. Hucke, Parae preconditioning with sparse approximate inverses, SAM J. Sci. Comput., 18 (1997, pp [2] W. Hackbusch, A sparse matrix arithmetic based on H-matrices. Part : ntroduction to H-matrices, Computing, 62 (1999, pp [21] W. Hackbusch and B. N. Khoromskij, A sparse H-matrix arithmetic. Part : Appication to muti-dimensiona probems, Computing, 64 (2, pp [22] P. Hénon and Y. Saad, A parae mutistage LU factorization based on a hierarchica graph decomposition, SAM J. Sci. Comput., 28 (26, pp [23] V. E. Howe and R. C. Kirby, Bock preconditioners for finite eement discretization of incompressibe fow with therma convection, Numerica Linear Agebra with Appications, 19 (212, pp [24] V. E. Howe, R. C. Kirby, and G. Dion, Bock preconditioners for couped fuids probems, SAM Journa of Scientific Computing, 35 (213, pp. S368 S385. [25]. C. F. psen, A note on preconditioning nonsymmetric matrices, SAM Journa on Scientific Computing, 23 (21, pp [26] G. Karypis and V. Kumar, A fast and high quaity mutieve scheme for partitioning irreguar graphs, SAM J. Sci. Comput, 2 (1998, pp [27] E.-J. Lee and J. Zhang, Hybrid reordering strategies for LU preconditioning of indefinite sparse matrices, Journa of Appied Mathematics and Computing, 22 (26, pp [28] R. Li and Y. Saad, Divide and conquer ow-rank preconditioners for symmetric matrices, SAM J. Sci. Comput., 35 (213, pp. A269 A295. [29], GPU-acceerated preconditioned iterative inear sovers, J. Supercomput., 63 (213, pp [3] R. Li, Y. Xi, and Y. Saad, Schur compement-based domain decomposition preconditioners with ow-rank corrections, Numerica Linear Agebra with Appications, 23 (216, pp [31] K.-A. Marda and R. Winther, Preconditioning discretizations of systems of partia differentia equations, Numerica Linear Agebra with Appications, 18 (21, pp [32] M. F. Murphy, G. H. Goub, and A. J. Wathen, A note on preconditioning for indefinite inear systems, SAM Journa on Scientific Computing, 21 (2, pp [33] D. Osei-Kuffour, R. Li, and Y. Saad, Matrix reordering using mutieve graph coarsening for LU preconditioning, SAM J. Sci. Comput., 37 (215, pp. A [34] Y. Saad, A fexibe inner-outer preconditioned GMRES agorithm, SAM J. Sci. Comput., 14 (1993, pp [35], LUM: A muti-eimination LU preconditioner for genera sparse matrices, SAM J. Sci. Comput., 17 (1996, 14

15 pp [36], terative Methods for Sparse Linear Systems, SAM, Phiadephia, 2nd ed. ed., 23. [37] Y. Saad and B. Suchome, ARMS: An agebraic recursive mutieve sover for genera sparse inear systems, Numer. Linear Agebra App., 9 (22, pp [38] G.W. Stewart, Agorithm 56: Hqr3 and exchng: Fortran subroutines for cacuating and ordering the eigenvaues of a rea upper Hessenberg matrix, ACM Transactions on Mathematica Software, 2 (1976, pp [39] Y. Xi, R. Li, and Y. Saad, An Agebraic Mutieve Preconditioner with Low-Rank Corrections for Sparse Symmetric Matrices, SAM J. Matrix Ana. App., 37 (216, pp [4] Y. Xi and Y. Saad, Computing partia spectra with east-squares rationa fiters, SAM Journa on Scientific Computing, 38 (216, pp. A32 A345. [41], A rationa function preconditioner for indefinite sparse inear systems, SAM Journa on Scientific Computing, 39 (217, pp. A1145 A1167. [42] Y. Xi and J. Xia, On the stabiity of some hierarchica rank structured matrix agorithms, SAM Journa on Matrix Ana. App., 37 (216, pp [43] Y. Xi, J. Xia, S. Cauey, and V. Baakrishnan, Superfast and stabe structured sovers for Toepitz east squares via randomized samping, SAM J. Matrix Ana. App., 35 (214, pp [44] J. Xia, S. Chandrasekaran, M. Gu, and X.S. Li, Fast agorithms for hierarchicay semiseparabe matrices, Numer. Linear Agebra App., 17 (21, pp [45] J. Xia, Y. Xi, S. Cauey, and V. Baakrishnan, Fast sparse seected inversion, SAM Journa on Matrix Ana. App., 36 (215, pp