Contents. Preface... xi. Introduction...

Transcription

1 Contents Preface... xi Introduction... xv Chapter 1. Computer Architectures Different types of parallelism Overlap, concurrency and parallelism Temporal and spatial parallelism for arithmetic logic units Parallelism and memory Memory architecture Interleaved multi-bank memory Memory hierarchy Distributed memory Hybrid architecture Graphics-type accelerators Hybrid computers Chapter 2. Parallelization and Programming Models Parallelization Performance criteria Degree of parallelism Load balancing Granularity Scalability... 22

2 vi Parallel Scientific Computing 2.3. Data parallelism Loop tasks Dependencies Examples of dependence Reduction operations Nested loops OpenMP Vectorization: a case study Vector computers and vectorization Dependence Reduction operations Pipeline operations Message-passing Message-passing programming Parallel environment management Point-to-point communications Collective communications Performance analysis Chapter 3. Parallel Algorithm Concepts Parallel algorithms for recurrences The principles of reduction methods Overhead and stability of reduction methods Cyclic reduction Data locality and distribution: product of matrices Row and column algorithms Block algorithms Distributed algorithms Implementation Chapter 4. Basics of Numerical Matrix Analysis Review of basic notions of linear algebra Vector spaces, scalar products and orthogonal projection Linear applications and matrices Properties of matrices Matrices, eigenvalues and eigenvectors Norms of a matrix... 80

3 Contents vii Basis change Conditioning of a matrix Chapter 5. Sparse Matrices Origins of sparse matrices Parallel formation of sparse matrices: shared memory Parallel formation by block of sparse matrices: distributed memory Parallelization by sets of vertices Parallelization by sets of elements Comparison: sets of vertices and elements Chapter 6. Solving Linear Systems Direct methods Iterative methods Chapter 7. LU Methods for Solving Linear Systems Principle of LU decomposition Gauss factorization Gauss Jordan factorization Row pivoting Crout and Cholesky factorizations for symmetric matrices Chapter 8. Parallelization of LU Methods for Dense Matrices Block factorization Implementation of block factorization in a message-passing environment Parallelization of forward and backward substitutions Chapter 9. LU Methods for Sparse Matrices Structure of factorized matrices Symbolic factorization and renumbering Elimination trees Elimination trees and dependencies Nested dissections Forward and backward substitutions

4 viii Parallel Scientific Computing Chapter 10. Basics of Krylov Subspaces Krylov subspaces Construction of the Arnoldi basis Chapter 11. Methods with Complete Orthogonalization for Symmetric Positive Definite Matrices Construction of the Lanczos basis for symmetric matrices The Lanczos method The conjugate gradient method Comparison with the gradient method Principle of preconditioning for symmetric positive definite matrices Chapter 12. Exact Orthogonalization Methods for Arbitrary Matrices The GMRES method The case of symmetric matrices: the MINRES method The ORTHODIR method Principle of preconditioning for non-symmetric matrices Chapter 13. Biorthogonalization Methods for Non-symmetric Matrices Lanczos biorthogonal basis for non-symmetric matrices The non-symmetric Lanczos method The biconjugate gradient method: BiCG The quasi-minimal residual method: QMR The BiCGSTAB Chapter 14. Parallelization of Krylov Methods Parallelization of dense matrix-vector product Parallelization of sparse matrix-vector product based on node sets Parallelization of sparse matrix-vector product based on element sets

5 Contents ix Review of the principles of domain decomposition Matrix-vector product Interface exchanges Asynchronous matrix-vector product with non-blocking communications Comparison: parallelization based on node and element sets Parallelization of the scalar product By weight By distributivity By ownership Summary of the parallelization of Krylov methods Chapter 15. Parallel Preconditioning Methods Diagonal Incomplete factorization methods Principle Parallelization Schur complement method Optimal local preconditioning Principle of the Schur complement method Properties of the Schur complement method Algebraic multigrid Preconditioning using projection Algebraic construction of a coarse grid Algebraic multigrid methods The Schwarz additive method of preconditioning Principle of the overlap Multiplicative versus additive Schwarz methods Additive Schwarz preconditioning Restricted additive Schwarz: parallel implementation Preconditioners based on the physics Gauss Seidel method Linelet method

6 x Parallel Scientific Computing Appendices Appendix Appendix Appendix Bibliography Index