A Block Compression Algorithm for Computing Preconditioners

Transcription

1 A Block Compression Algorithm for Computing Preconditioners J Cerdán, J Marín, and J Mas Abstract To implement efficiently algorithms for the solution of large systems of linear equations in modern computer arquitectures, it is convenient to unravel the block structure of the coefficient matrix that is present in many applications of the physics and the engineering This is specially important when a preconditioned iterative method is used to compute an approximate solution Identifying such a block structure is a graph compression problem and several techniques have been studied in the literature In this work we consider the cosine algorithm introduced by Y Saad This algorithm groups two rows of the matrix if the corresponding angle between them in the adjacency matrix is small enough The modification that we propose considers also the magnitude of the nonzero entries of the rows with the aim of computing a better block partition 1 Introduction Very often Numerical Linear Algebra applications give raise to systems of linear equations Ax = b, (1) J Cerdán Instituto de Matemática Multidisciplinar, Universitat Politècnica de València, jcerdan@immupves J Marín Instituto de Matemática Multidisciplinar, Universitat Politècnica de València, jmarinma@immupves J Mas Instituto de Matemática Multidisciplinar, Universitat Politècnica de València, jmasm@immupves 1

2 2 J Cerdán, J Marín, and J Mas where A R n n is a large and nonsingular sparse matrix These problems are usually solved by iterative Krylov methods since they require less computational time and storage than counterpart direct methods based on Gaussian elimination It is well known than the convergence of iterative Krylov methods is improved if a good preconditioner is used, see [10] The aim of the preconditioning technique is to improve the condition number or the eigenvalue distribution of the coefficient matrix On the other hand the cost of computing the preconditioner and its application inside the iterative method should be negligible and performed as efficiently as possible To this end several block versions of the most popular preconditioners have been proposed for a variety of structured problems For example, matrices arising from the discretization of partial differential equations have often a natural block structure [5] This structure consists of small and dense submatrices that can be treated as individual entries of the matrix With appropiate sparse storage formats and using basic linear algebra subroutines (BLAS) the computational performance can be improved [8] Examples of this block preconditioning approach can be found in the literature [2, 3, 4, 6, 7] In every case an improvement in the efficiency compared to its point version is reported Finding the block structure of a matrix is a graph compression problem, and has been studied by different authors [1, 9] In [11] the author propose the cosine algorithm that finds an approximate block structure since it allows some zero entries in the dense blocks The goal of this paper is to modify the algorithm such that besides the nonzero pattern, the magnitude of the entries is considered The paper is organized as follows The cosine algorithm is revised in Section 2 Section 3 is devoted to the motivation and description of the proposed modification Numerical results of experiments with different matrices are shown in Section 4 Finaly, Section 5 summarizes the work 2 The cosine algorithm In this section we describe the cosine algorithm proposed by Y Saad in [11] Let us start with an example that ilustrates the matrix graph compression problem Consider the next symmetric nonzero pattern: , where each represents a nonzero entry This matrix has a block structure of sizes 3, 2, 1 and 1, respectively The adjacency graph G = (V,E) of a matrix consists of the node set V that corresponds to the rows or columns (unknowns) of the matrix, and

3 A Block Compression Algorithm for Computing Preconditioners 3 the set E such that there is an edge from node i to node j if a i j 0 The nodes can be grouped into four subgroups: Y 1 = {1,2,3}, Y 2 = {4,5}, Y 3 = {6} and Y 4 = {7} The corresponding restricted graph to each one of these subgroups is complete and it can be represented by one entry on the adjacency matrix of the associated quotient graph where each in the position (i, j) corresponds to a dense block of size Y i Y j, where X is the cardinality of the set X To detect the partition of the nodes and therefore the block structure of a matrix different algorithms have been proposed Algorithms based on hash functions assign a different value to each nonzero row pattern For instance in [1] it is used the hash function hash(u) = (u,w) E These values allow to group two nodes with the same hash value, though further refinements may be necessary Hash-based algorithms are not useful to detect almost complete subgraphs as it is illustrated by the next example It has been obtained from the previous one by introducing some zeros on the dense subblocks, Using a hash function we would not be able to find the same block structure because all the row patterns and their corresponding hash values are different That is, no block structure will be found To detect the same partition of the nodes the algorithm should allow some few zero entries in the dense subblocks One technique that can be used to compute such a kind of approximate block structures is the cosine algorithm It is based on the idea of computing the angle between the rows, or columns, of the adjacency matrix For example, the adjacency matrix C of the previous matrix is , w

4 4 J Cerdán, J Marín, and J Mas where each nonzero entry has been replaced by 1 We note that the (i, j) entry of the matrix CC T is the inner product of the rows i and j Thus, the cosine of the angle between two given rows of the adjacency matrix can be easily obtained by computing the upper triangular part of this matrix Large cosine values correspond to rows with similar nonzero patterns and therefore, an approximate block structure can be detected Algorithm 1 shows the cosine algorithm The vector Group stores the index of the group where each row belongs to If Group(i) = 1 it means that row i is either not grouped or it is the leader row of the i th group Lines 5 to 10 implement the computation of the inner product between rows which is stored in vector Count The cosine evaluation is done in lines 11 to 14 The parameter τ is the minimum value for the cosine between two rows such that they can be grouped together Tipically 1 τ 07 even for some matrices smaller values may be needed to find blocks of moderate size If τ = 1 only rows with exactly the same nonzero pattern are grouped performing in that case as a hash-based algorithm The biggest block sizes are obtained for small τ values but probably at the cost of introducing a large amount of nonzero entries on the block partition Finally, nz C (i) indicates the number of nonzero elements of the i th row of the matrix C and corresponds to its norm Further details can be found in [11] Algorithm 1 Cosine Algorithm 1 Input: Adjacency matrix C and tolerance τ; Output: block partition Compute the pattern C T 2 3 Set Group(i)= 1 and Count(j)= 0 for i, j = 1,,n 4 For i = 1,,n if Group(i)= 1 Do: 5 For { j c i j 0} Do: Let row the j th row of C T 6 For k = nz C T ( j) to 1 Do: 7 8 Let col = row(k) 9 If (col i) break If (Group(col)== 1) Count(col) ++ For {col Count(col) 0} Do: If (Count(col) 2 > τ nz C (i) nz C (col)) Then: Group(col)== i ; Update size of Group(i) Count(col)= Modified Cosine Algorithm In this section we modify the cosine algorithm to take into account the magnitude of the nonzero entries of a matrix In some scenarios, for instance when there is a big difference between the magnitud of the entries, considering only the nonzero pattern for grouping rows can lead to an approximate block partition that may not be a good representation of the significant part structure of the matrix With significant we

5 A Block Compression Algorithm for Computing Preconditioners 5 mean the block structure induced by the largest entries of the matrix For instance, by applying the cosine algorithm to the next two matrices one may find as result the same block structure since both share the same adjacency matrix (which is in fact the matrix on the right) ε 0 ε ε 0 ε 0 ε ε , But, for relative small values of ε, it is reasonably to think that the rows 3 and 4 have closer patterns than thouse corresponding to rows 2 and 3 In that case it could be preferred to identify the slightly different block structure ε 0 ε ε 0 ε 0 ε ε To allow the cosine algorithm to find this structure some changes must be done Figure 1 illustrates the inner product of row i with all the columns of C T, ie, c i C T It is done rowwise and the computational cost is basically the sum of the number Fig 1 Rowwise scheme for the inner product of row i with the columns of C T X indicates the differences in the application of the modified cosine algorithm with respect to the standard one of nonzero entries of each row involved in this inner product This example shows that taking into account the magnitud of the elements the result can be slightly different with respect to the sum obtained with the standard cosine The differences

6 6 J Cerdán, J Marín, and J Mas come first, from the evaluation of which rows of C T are involved Then, the sum by columns may be also affected for the magnitud of the entries Therefore, the corresponding inner product may be different and so the block partition for the matrix Algorithm 2 incorporates these modifications In line 5 for each entry c i j of the adjacency matrix C the magnitude of a i j is checked Row j of C T is not considered for computing rowwise the product c i C T if a i j < ε Another modification is introduced in line 10 where the magnitude of the entries of a row of A T is checked before adding its contribution to the sum by columns Finally, we point out that to compute correctly the row norms used in line 12 to evaluate the cosine, the values of nz C (i) and nz C (col) must be decremented in the same quantity than the number of discarded entries in the corresponding rows Algorithm 2 Modified Cosine Algorithm 1 Input: matrix A, tolerance τ, tolerance ε; Output: Block partition Compute matrices A T, C and C T 2 3 Set Group(i)= 1 and Count(j)= 0 for i, j = 1,,n 4 For i = 1,,n if Group(i)= 1 Do: 5 For { j a i j > ε} Do: Let row the j th row of C T 6 For k = nz C T ( j) to 1 Do: 7 8 Let col = row(k) 9 If (col i) break If ((Group(col)== 1) && ( a T j,col > ε)) Count(col)++ 10 For {col Count(col) 0} Do: If (Count(col) 2 > τ nz C (i) nz C (col)) Then: Group(col)== i ; Update the size of Group(i) Count(col)=0 As additional notes we mention that, since the magnitude of the entries is needed, not only the adjaceny graph of the matrix A but also its entries are needed as an input Moreover, the matrix A T must be computed internally Using appropiate sparse storage schemes, as CSR (Compress Sparse Row), the adjacency matrices C and C T are directly available without additional computational or memory cost With respect to the value of ε we note that different choices for this parameter in lines 5 and 10 can be made This observation makes a difference with respect to the possibility of sparsifying the matrix before the application of the standard cosine algorithm Thus, the modified cosine is more flexible and opens more possibilities to the block partition computation 4 Numerical Experiments In this section the results of some numerical experiments conducted to determine the performance of the modified algorithm are presented Table 1 shows the matrices

7 A Block Compression Algorithm for Computing Preconditioners 7 Table 1 Used matrices Available at Matrix n nnz Aplication HOR Network flow FS Quemical kinetics ORSIRR Oil reservoir UTM1700B Plasma physics UTM Plasma physics BCSSTK Structural engineering ADD Computer component design MEMPLUS Computer component design SAYLR Harwell-Boeing Collection Table 2 Results of the experiments Matrix Cosine Alg Modified Cosine Alg γ ρ τ Its γ ρ τ ε Its HOR FS ORSIRR UTM1700B UTM BCSSTK ADD MEMPLUS SAYLR used, its size (n), the number of nonzero entries (nnz) and the application field from they arise The experiments have been done using MATLAB The block version of the approximate inverse preconditioner AISM [6] has been used to precondition the BiCGSTAB iterative method Table 2 shows, for different values of τ (minimum cosine value) and ε (tolerance to discard entries in the modified algorithm), the average block size obtained (γ), the preconditioner density (ρ) and the iteration count needed to reduce the initial residual by 10 8 The parameters τ and ε have been chosen to get similar preconditioner densities for both algorithms From the results we observe an improvement on the convergence rate of the iterative method with a reduction in the number of iterations from 10 to 20 percent We also note that, in general, this improvement is obtained with a reduction on the preconditioner density and, therefore, a bigger reduction on the computational cost of the iterative solution process We can conclude that it is possible to find a combination of the parameters τ and ε such that the performance of the iterative method can be significantly improved

8 8 J Cerdán, J Marín, and J Mas 5 Conclusions In this work we introduce a modification of the cosine algorithm to compress the graph of a sparse matrix that takes into account the magnitude of the nonzero entries of the rows From the results of the numerical experiments one can deduce that with a good choice of the parameters ε and τ it is possible to reduce the number of iterations needed by the iterative method to get convergence Additional experiments to evaluate different choices for the parameter ε in different points inside the modified cosine algorithm will be done in the future Acknowledgements This work has been supported by The Spanish DGI grant MTM References 1 C Ashcraft Compressed graphs and the minimum degree algorithm, SIAM J Sci Comput, 16(1995), S T Barnard and M J Grote A block version of the SPAI preconditioner, in Proc 9th SIAM Conf on Parall Process for Sci Comp, March M Benzi, R Kouhia, and M Tůma Stabilized and block approximate inverse preconditioners for problems in solid and structural mechanics, Comput Methods Appl Mech Engineering, 190 (2001), R Bridson and W-P Tang Refining an approximate inverse, J Comput Appl Math, 123 (2000), A Chapman, Y Saadn, and L Wigton High order ILU preconditioners for CFD problems, Int J Numer Meth Fluids, 33 (2000), J Cerdán, T Faraj, N Malla, J Marín and J Mas Block Approximate Inverse Preconditioners for Sparse Nonsymmetric Linear Systems, ETNA, 37 (2010), E Chow and Y Saad Approximate inverse techniques for block-partitioned matrices, SIAM J Sci Comput, 18 (1997), JJ Dongarra, IS Duff, DC Sorensen, HA van der Vorst Numerical linear algebra for High-Performance Computer, SIAM, Philadelphia, PA, J O Neil and DB Szyld A block ordering method for sparse matrices, SIAM J Sci Comput, 11 (1990), Y Saad, Iterative Methods for Sparse Linear Systems, PWS Publishing Company, Boston, , Finding exact and approximate block structures for ilu preconditioning SIAM Journal on Scientific Computing, 24 (2003),