Multilevel low-rank approximation preconditioners Yousef Saad Department of Computer Science and Engineering University of Minnesota
|
|
- Jessica Riley
- 5 years ago
- Views:
Transcription
1 Multilevel low-rank approximation preconditioners Yousef Saad Department of Computer Science and Engineering University of Minnesota SIAM CSE Boston - March 1, 2013
2 First: Joint work with Ruipeng Li Work supported by NSF 2 SIAM-CSE March 1, 2013
3 Introduction Preconditioned Krylov subspace methods offer a good alternative to direct solution methods Especially for 3D problems Compromise between performance and robustness... But there are challenges: Highly indefinite systems [Helmholtz, Maxwell,...] Highly ill-conditioned systems [structures,..] Problems with extremely irregular nonzero pattern Recent: impact of new architectures [many core, GPUs] 3 SIAM-CSE March 1, 2013
4 Introduction (cont.) Main issue in using GPUs for sparse computations: Huge performance degradation due to irregular sparsity Matrices: Matrix -name N NNZ FEM/Cantilever 62,451 4,007,383 Boeing/pwtk 217,918 11,634,424 Performance of Mat-Vecs on NVIDIA Tesla C1060 Single Precision Double Precision Matrix CSR JAD DIA CSR JAD DIA FEM/Cantilever Boeing/pwtk SIAM-CSE March 1, 2013
5 Sparse Forward/Backward Sweeps Next major ingredient of precond. Krylov subs. methods ILU preconditioning operations require L/U solves: x U 1 L 1 x Sequential outer loop. for i=1:n for j=ia(i):ia(i+1) x(i) = x(i) - a(j)*x(ja(j)) end end Parallelism can be achieved with level scheduling: Group unknowns into levels Unknowns x(i) of same level can be computed simultaneously 1 nlev n 5 SIAM-CSE March 1, 2013
6 ILU: Sparse Forward/Backward Sweeps Very poor performance [relative to CPU] Matrix N CPU GPU-Lev Mflops #lev Mflops Boeing/bcsstk36 23, , FEM/Cantilever 62, , COP/CASEYK 696, COP/CASEKU 208, Prec: miserable :-) GPU Sparse Triangular Solve with Level Scheduling Very poor performance when #levs is large A few things can be done to reduce the # levels but perf. will remain poor 6 SIAM-CSE March 1, 2013
7 So... 7 SIAM-CSE March 1, 2013
8 Either GPUs must go... 8 SIAM-CSE March 1, 2013
9 or ILUs must go... 9 SIAM-CSE March 1, 2013
10 Or perhaps: Alternative preconditioners? What would be a good alternative? Wish-list: A preconditioner requiring few irregular computations Something that trades volume of computations for speed If possible something that is robust for indefinite case Good candidate: Multilevel Low-Rank (MLR) approximate inverse preconditioners 10 SIAM-CSE March 1, 2013
11 Related work: Work on HSS matrices [e.g., JIANLIN XIA, SHIVKUMAR CHAN- DRASEKARAN, MING GU, AND XIAOYE S. LI, Fast algorithms for hierarchically semiseparable matrices, Numerical Linear Algebra with Applications, 17 (2010), pp ] Work on H-matrices [Hackbusch,...] Work on balanced incomplete factorizations (R. Bru et al.) Work on sweeping preconditioners by Engquist and Ying. Work on computing the diagonal of a matrix inverse [Jok Tang and YS (2010)..] 11 SIAM-CSE March 1, 2013
12 Low-rank Multilevel Approximations Starting point: symmetric matrix derived from a 5-point discretization of a 2-D Pb on n x n y grid A = A = A 1 D 2 D 2 A 2... D D α A α D α+1 D α+1 A α D ny A ny ( ) A11 A 12 A 21 A 22 ( A11 A22) + ( A 21 A 12 ) 12 SIAM-CSE March 1, 2013
13 Corresponding splitting on FD mesh: SIAM-CSE March 1, 2013
14 A 11 R m m, A 22 R (n m) (n m) In the simplest case second matrix is: ( ) A11 A 12 = A 21 A 22 ( A11 A22) + I I Write 2nd matrix as: I I = + I + I I I I I E T = I I E E T 14 SIAM-CSE March 1, 2013
15 Above splitting can be rewritten as A = (A + EE T ) }{{} B EE T B := ( B1 B2) A = B EE T, R n n, E := ( E1 E 2 ) R n n x, Note: B 1 := A 11 + E 1 E T 1, B 2 := A 22 + E 2 E T SIAM-CSE March 1, 2013
16 Shermann-Morrison formula: {}}{ A 1 = B 1 + B 1 E( I E T B 1 E) 1 E T B 1 X A 1 B 1 + B 1 EX 1 E T B 1 X = I E T B 1 E Note: E R n n x, X R n x n x n x = number of points in separator [O(n 1/2 ) for 2-D mesh, O(n 2/3 ) for 3-D mesh] Use in a recursive framework Similar idea was used for computing the diagonal of the inverse [J. Tang YS 10] 16 SIAM-CSE March 1, 2013
17 Multilevel Low-Rank (MLR) algorithm Method: Use lowrank approx. for B 1 E B 1 E U k V T k, U k R n k, V k R n x k, Replace B 1 E by U k Vk T in X = I (ET B 1 )E: X G k = I V k U T k E, ( Rn x n x ) Leads to... Preconditioner: M 1 = B 1 + U k [V T k G 1 k V k]u T k Use recursivity 17 SIAM-CSE March 1, 2013
18 Note: From A 1 = B 1 [I + EX 1 E T B 1 ] could define: M 1 1 = B 1 [I + EG 1 k V ku T k ]. [rationale: approximation made on one side only ] It turns out M 1 and M are equal! We have: M 1 = B 1 + U k H k U T k, with H k = V T k G 1 k V k. No need to store V k : Only keep U k and H k (k k). We can show : H k = (I U T k EV k) 1... and : H k is symmetric 18 SIAM-CSE March 1, 2013
19 Recursive multilevel framework ( A i = B i + E i Ei T, B Bi1 i. Bi2) Next level, set A i1 B i1 and A i2 B i2 Repeat on A i1, A i2 Last level, factor A i (IC, ILU) Binary tree structure: SIAM-CSE March 1, 2013
20 Generalization: Domain Decomposition framework Domain partitioned into 2 domains with an edge separator Ω 2 Ω 1 Matrix can be permuted to: P AP T = ˆB 1 ˆF 1 ˆF 1 T C 1 X X T ˆF 2 T C 2 ˆB 2 ˆF 2 Interface nodes in each domain are listed last. 20 SIAM-CSE March 1, 2013
21 Each matrix ˆB i is of size n i n i (interior var.) and the matrix C i is of size m i m i (interface var.) P AP T = Let: E α = ( B1 B2) and α used for balancing 0 αi 0 X T α EE T with B i = then we have: { D1 = α 2 I D 2 = 1 α 2 X T X. ( ˆB i ˆF 1 Better results when using diagonals instead of αi ˆF T i C i + D i 21 SIAM-CSE March 1, 2013 )
22 Theory: 2-level analysis for model problem Interested in eigenvalues γ j of A 1 B 1 = B 1 EX 1 E T B 1 when A = Pure Laplacean.. They are: γ j = β j = α j = β j 1 α j, j = 1,, n x with: n y /2 k=1 n y /2 k=1 4 ( sin 2 sin 2 sin 2 n y kπ n y +1 kπ n y +1 + sin2 sin 2 n y kπ n y +1 kπ n y +1 + sin2 ) 2, jπ 2(n x +1) jπ 2(n x +1). 22 SIAM-CSE March 1, 2013
23 Decay of the γ j s when nx = ny = β j sqrt(β j ) 1 α j γ j Note β j are the singular values of B 1 E. In this particular case 3 eigenvectors will capture 92 % of the inverse whereas 5 eigenvectors will capture 97% of the inverse. 23 SIAM-CSE March 1, 2013
24 EXPERIMENTS
25 Experimental setting Hardware: Intel Xeon X5675 processor (12 MB Cache, 3.06 GHz, 6-core) C/C++; Intel Math Kernel Library (MKL,version 10.2) Stopping criteria: r i 10 8 r 0 Maximum number of iterations: SIAM-CSE March 1, 2013
26 2-D/3-D model problems (theory) u cu = ( x 2 + y 2 + c ) e xy in (0, 1) 2, + Dirichlet BC FD discret.: n x = n y = 256 Eigenvalues of B 1 i E i X 1 i i = 0, 1, 3 Rapid decay. Ei T B 1 i Lev 1 Lev 2 Lev SIAM-CSE March 1, 2013
27 3-D elliptic PDE u cu = 6 c ( x 2 + y 2 + z 2) in (0, 1) 3, + Dirichlet BC FD discret.: n x = n y = 32, nz = 64 Eigenvalues of B 1 i E i X 1 i i = 0, 1, 3 Rapid decay. Ei T B 1 i Lev 1 Lev 2 Lev SIAM-CSE March 1, 2013
28 Tests: SPD cases SPD cases, pure Laplacean (c = 0 in previous equations) MLR + PCG compared to IC + PCG 2-D problems: #lev= 5, rank= 2 3-D problems: #lev= 5, 7, 10, rank= 2 28 SIAM-CSE March 1, 2013
29 Grid N ICT-CG MLR-CG fill p-t its i-t fill p-t its i-t K K , 048K K K ,097K Set-up times for MLR preconditioners are higher Bear in mind the ultimate target architecture [SIMD...] 29 SIAM-CSE March 1, 2013
30 Symmetric indefinite cases c > 0 in u cu; i.e., shifted by si. 2D case: s = 0.01, 3D case: s = 0.05 MLR + GMRES(40) compared to ILDLT + GMRES(40) 2-D problems: #lev= 4, rank= 5, 7, 7 3-D problems: #lev= 5, rank= 5, 7, 7 ILDLT failed for most cases Difficulties in MLR: #lev cannot be large, [no convergence] inefficient factorization at the last level (memory, CPU time)
31 Grid ILDLT-GMRES MLR-GMRES fill p-t its i-t fill p-t its i-t F F F F F F F 31 SIAM-CSE March 1, 2013
32 General symmetric matrices - Test matrices MATRIX N NNZ SPD DESCRIPTION Andrews/Andrews 60, ,154 yes computer graphics pb. Williams/cant 62,451 4,007,383 yes FEM cantilever UTEP/Dubcova2 65,025 1,030,225 yes 2-D/3-D PDE pb. Rothberg/cfd1 70,656 1,825,580 yes CFD pb. Schmid/thermal1 82, ,458 yes thermal pb. Rothberg/cfd2 123,440 3,085,406 yes CFD pb. Schmid/thermal2 1,228,045 8,580,313 yes thermal pb. Cote/vibrobox 12, ,700 no vibroacoustic pb. Cunningham/qa8fk 66,127 1,660,579 no 3-D acoustics pb. Koutsovasilis/F2 71,505 5,294,285 no structural pb. 32 SIAM-CSE March 1, 2013
33 Generalization of MLR via DD DD: PartGraphRecursive from METIS balancing with diagonals higher ranks used in two problems (cant and vibrobox) Show SPD cases first then non-spd 33 SIAM-CSE March 1, 2013
34 MATRIX ICT/ILDLT MLR-CG/GMRES fill p-t its i-t k lev fill p-t its i-t Andrews cant F Dubcova cfd thermal cfd F thermal SIAM-CSE March 1, 2013
35 MATRIX ICT/ILDLT MLR-CG/GMRES fill p-t its i-t k lev fill p-t its i-t vibrobox F qa8fk F F SIAM-CSE March 1, 2013
36 Conclusion Promising approach Many more avenues to explore: Nonsymmetric case, Implementation on GPUS, Storage for 3D case SIAM-CSE March 1, 2013
Multilevel Low-Rank Preconditioners Yousef Saad Department of Computer Science and Engineering University of Minnesota. Modelling 2014 June 2,2014
Multilevel Low-Rank Preconditioners Yousef Saad Department of Computer Science and Engineering University of Minnesota Modelling 24 June 2,24 Dedicated to Owe Axelsson at the occasion of his 8th birthday
More informationParallel Multilevel Low-Rank approximation preconditioners Yousef Saad Department of Computer Science and Engineering University of Minnesota
Parallel Multilevel Low-Rank approximation preconditioners Yousef Saad Department of Computer Science and Engineering University of Minnesota ICME, Stanford April 2th, 25 First: Partly joint work with
More informationMultilevel preconditioning techniques with applications Yousef Saad Department of Computer Science and Engineering University of Minnesota
Multilevel preconditioning techniques with applications Yousef Saad Department of Computer Science and Engineering University of Minnesota Syracuse University, Sep. 20, 2013 First: Joint work with Ruipeng
More informationThe new challenges to Krylov subspace methods Yousef Saad Department of Computer Science and Engineering University of Minnesota
The new challenges to Krylov subspace methods Yousef Saad Department of Computer Science and Engineering University of Minnesota SIAM Applied Linear Algebra Valencia, June 18-22, 2012 Introduction Krylov
More information1. Introduction. We consider the problem of solving the linear system. Ax = b, (1.1)
SCHUR COMPLEMENT BASED DOMAIN DECOMPOSITION PRECONDITIONERS WITH LOW-RANK CORRECTIONS RUIPENG LI, YUANZHE XI, AND YOUSEF SAAD Abstract. This paper introduces a robust preconditioner for general sparse
More informationPreconditioning Techniques for Large Linear Systems Part III: General-Purpose Algebraic Preconditioners
Preconditioning Techniques for Large Linear Systems Part III: General-Purpose Algebraic Preconditioners Michele Benzi Department of Mathematics and Computer Science Emory University Atlanta, Georgia, USA
More informationSolving Symmetric Indefinite Systems with Symmetric Positive Definite Preconditioners
Solving Symmetric Indefinite Systems with Symmetric Positive Definite Preconditioners Eugene Vecharynski 1 Andrew Knyazev 2 1 Department of Computer Science and Engineering University of Minnesota 2 Department
More informationIncomplete Cholesky preconditioners that exploit the low-rank property
anapov@ulb.ac.be ; http://homepages.ulb.ac.be/ anapov/ 1 / 35 Incomplete Cholesky preconditioners that exploit the low-rank property (theory and practice) Artem Napov Service de Métrologie Nucléaire, Université
More informationRobust Preconditioned Conjugate Gradient for the GPU and Parallel Implementations
Robust Preconditioned Conjugate Gradient for the GPU and Parallel Implementations Rohit Gupta, Martin van Gijzen, Kees Vuik GPU Technology Conference 2012, San Jose CA. GPU Technology Conference 2012,
More informationFine-grained Parallel Incomplete LU Factorization
Fine-grained Parallel Incomplete LU Factorization Edmond Chow School of Computational Science and Engineering Georgia Institute of Technology Sparse Days Meeting at CERFACS June 5-6, 2014 Contribution
More informationSchur complement-based domain decomposition preconditioners with low-rank corrections
NUMERICAL LINEAR ALGEBRA WITH APPLICATIONS Numer. Linear Algebra Appl. (206) Published online in Wiley Online Library (wileyonlinelibrary.com)..205 Schur complement-based domain decomposition preconditioners
More informationITERATIVE METHODS FOR SPARSE LINEAR SYSTEMS
ITERATIVE METHODS FOR SPARSE LINEAR SYSTEMS YOUSEF SAAD University of Minnesota PWS PUBLISHING COMPANY I(T)P An International Thomson Publishing Company BOSTON ALBANY BONN CINCINNATI DETROIT LONDON MADRID
More information9.1 Preconditioned Krylov Subspace Methods
Chapter 9 PRECONDITIONING 9.1 Preconditioned Krylov Subspace Methods 9.2 Preconditioned Conjugate Gradient 9.3 Preconditioned Generalized Minimal Residual 9.4 Relaxation Method Preconditioners 9.5 Incomplete
More informationA robust multilevel approximate inverse preconditioner for symmetric positive definite matrices
DICEA DEPARTMENT OF CIVIL, ENVIRONMENTAL AND ARCHITECTURAL ENGINEERING PhD SCHOOL CIVIL AND ENVIRONMENTAL ENGINEERING SCIENCES XXX CYCLE A robust multilevel approximate inverse preconditioner for symmetric
More informationPreface to the Second Edition. Preface to the First Edition
n page v Preface to the Second Edition Preface to the First Edition xiii xvii 1 Background in Linear Algebra 1 1.1 Matrices................................. 1 1.2 Square Matrices and Eigenvalues....................
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 23: GMRES and Other Krylov Subspace Methods; Preconditioning
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 23: GMRES and Other Krylov Subspace Methods; Preconditioning Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 18 Outline
More informationDomain Decomposition-based contour integration eigenvalue solvers
Domain Decomposition-based contour integration eigenvalue solvers Vassilis Kalantzis joint work with Yousef Saad Computer Science and Engineering Department University of Minnesota - Twin Cities, USA SIAM
More informationFINE-GRAINED PARALLEL INCOMPLETE LU FACTORIZATION
FINE-GRAINED PARALLEL INCOMPLETE LU FACTORIZATION EDMOND CHOW AND AFTAB PATEL Abstract. This paper presents a new fine-grained parallel algorithm for computing an incomplete LU factorization. All nonzeros
More informationFINE-GRAINED PARALLEL INCOMPLETE LU FACTORIZATION
FINE-GRAINED PARALLEL INCOMPLETE LU FACTORIZATION EDMOND CHOW AND AFTAB PATEL Abstract. This paper presents a new fine-grained parallel algorithm for computing an incomplete LU factorization. All nonzeros
More informationc 2015 Society for Industrial and Applied Mathematics
SIAM J. SCI. COMPUT. Vol. 37, No. 2, pp. C169 C193 c 2015 Society for Industrial and Applied Mathematics FINE-GRAINED PARALLEL INCOMPLETE LU FACTORIZATION EDMOND CHOW AND AFTAB PATEL Abstract. This paper
More informationParallelization of Multilevel Preconditioners Constructed from Inverse-Based ILUs on Shared-Memory Multiprocessors
Parallelization of Multilevel Preconditioners Constructed from Inverse-Based ILUs on Shared-Memory Multiprocessors J.I. Aliaga 1 M. Bollhöfer 2 A.F. Martín 1 E.S. Quintana-Ortí 1 1 Deparment of Computer
More informationScientific Computing with Case Studies SIAM Press, Lecture Notes for Unit VII Sparse Matrix
Scientific Computing with Case Studies SIAM Press, 2009 http://www.cs.umd.edu/users/oleary/sccswebpage Lecture Notes for Unit VII Sparse Matrix Computations Part 1: Direct Methods Dianne P. O Leary c 2008
More informationMultigrid absolute value preconditioning
Multigrid absolute value preconditioning Eugene Vecharynski 1 Andrew Knyazev 2 (speaker) 1 Department of Computer Science and Engineering University of Minnesota 2 Department of Mathematical and Statistical
More informationAMS Mathematics Subject Classification : 65F10,65F50. Key words and phrases: ILUS factorization, preconditioning, Schur complement, 1.
J. Appl. Math. & Computing Vol. 15(2004), No. 1, pp. 299-312 BILUS: A BLOCK VERSION OF ILUS FACTORIZATION DAVOD KHOJASTEH SALKUYEH AND FAEZEH TOUTOUNIAN Abstract. ILUS factorization has many desirable
More informationFINDING PARALLELISM IN GENERAL-PURPOSE LINEAR PROGRAMMING
FINDING PARALLELISM IN GENERAL-PURPOSE LINEAR PROGRAMMING Daniel Thuerck 1,2 (advisors Michael Goesele 1,2 and Marc Pfetsch 1 ) Maxim Naumov 3 1 Graduate School of Computational Engineering, TU Darmstadt
More informationA Numerical Study of Some Parallel Algebraic Preconditioners
A Numerical Study of Some Parallel Algebraic Preconditioners Xing Cai Simula Research Laboratory & University of Oslo PO Box 1080, Blindern, 0316 Oslo, Norway xingca@simulano Masha Sosonkina University
More informationEffective matrix-free preconditioning for the augmented immersed interface method
Effective matrix-free preconditioning for the augmented immersed interface method Jianlin Xia a, Zhilin Li b, Xin Ye a a Department of Mathematics, Purdue University, West Lafayette, IN 47907, USA. E-mail:
More informationThe flexible incomplete LU preconditioner for large nonsymmetric linear systems. Takatoshi Nakamura Takashi Nodera
Research Report KSTS/RR-15/006 The flexible incomplete LU preconditioner for large nonsymmetric linear systems by Takatoshi Nakamura Takashi Nodera Takatoshi Nakamura School of Fundamental Science and
More informationSolving Ax = b, an overview. Program
Numerical Linear Algebra Improving iterative solvers: preconditioning, deflation, numerical software and parallelisation Gerard Sleijpen and Martin van Gijzen November 29, 27 Solving Ax = b, an overview
More informationFine-Grained Parallel Algorithms for Incomplete Factorization Preconditioning
Fine-Grained Parallel Algorithms for Incomplete Factorization Preconditioning Edmond Chow School of Computational Science and Engineering Georgia Institute of Technology, USA SPPEXA Symposium TU München,
More informationSome Geometric and Algebraic Aspects of Domain Decomposition Methods
Some Geometric and Algebraic Aspects of Domain Decomposition Methods D.S.Butyugin 1, Y.L.Gurieva 1, V.P.Ilin 1,2, and D.V.Perevozkin 1 Abstract Some geometric and algebraic aspects of various domain decomposition
More informationIncomplete LU Preconditioning and Error Compensation Strategies for Sparse Matrices
Incomplete LU Preconditioning and Error Compensation Strategies for Sparse Matrices Eun-Joo Lee Department of Computer Science, East Stroudsburg University of Pennsylvania, 327 Science and Technology Center,
More informationFast Iterative Solution of Saddle Point Problems
Michele Benzi Department of Mathematics and Computer Science Emory University Atlanta, GA Acknowledgments NSF (Computational Mathematics) Maxim Olshanskii (Mech-Math, Moscow State U.) Zhen Wang (PhD student,
More informationMARCH 24-27, 2014 SAN JOSE, CA
MARCH 24-27, 2014 SAN JOSE, CA Sparse HPC on modern architectures Important scientific applications rely on sparse linear algebra HPCG a new benchmark proposal to complement Top500 (HPL) To solve A x =
More informationDivide and conquer algorithms for large eigenvalue problems Yousef Saad Department of Computer Science and Engineering University of Minnesota
Divide and conquer algorithms for large eigenvalue problems Yousef Saad Department of Computer Science and Engineering University of Minnesota PMAA 14 Lugano, July 4, 2014 Collaborators: Joint work with:
More informationFAST STRUCTURED EIGENSOLVER FOR DISCRETIZED PARTIAL DIFFERENTIAL OPERATORS ON GENERAL MESHES
Proceedings of the Project Review, Geo-Mathematical Imaging Group Purdue University, West Lafayette IN, Vol. 1 2012 pp. 123-132. FAST STRUCTURED EIGENSOLVER FOR DISCRETIZED PARTIAL DIFFERENTIAL OPERATORS
More informationJ.I. Aliaga 1 M. Bollhöfer 2 A.F. Martín 1 E.S. Quintana-Ortí 1. March, 2009
Parallel Preconditioning of Linear Systems based on ILUPACK for Multithreaded Architectures J.I. Aliaga M. Bollhöfer 2 A.F. Martín E.S. Quintana-Ortí Deparment of Computer Science and Engineering, Univ.
More informationParallel Preconditioning Methods for Ill-conditioned Problems
Parallel Preconditioning Methods for Ill-conditioned Problems Kengo Nakajima Information Technology Center, The University of Tokyo 2014 Conference on Advanced Topics and Auto Tuning in High Performance
More informationSolving Large Nonlinear Sparse Systems
Solving Large Nonlinear Sparse Systems Fred W. Wubs and Jonas Thies Computational Mechanics & Numerical Mathematics University of Groningen, the Netherlands f.w.wubs@rug.nl Centre for Interdisciplinary
More informationMultipole-Based Preconditioners for Sparse Linear Systems.
Multipole-Based Preconditioners for Sparse Linear Systems. Ananth Grama Purdue University. Supported by the National Science Foundation. Overview Summary of Contributions Generalized Stokes Problem Solenoidal
More informationOUTLINE ffl CFD: elliptic pde's! Ax = b ffl Basic iterative methods ffl Krylov subspace methods ffl Preconditioning techniques: Iterative methods ILU
Preconditioning Techniques for Solving Large Sparse Linear Systems Arnold Reusken Institut für Geometrie und Praktische Mathematik RWTH-Aachen OUTLINE ffl CFD: elliptic pde's! Ax = b ffl Basic iterative
More informationJae Heon Yun and Yu Du Han
Bull. Korean Math. Soc. 39 (2002), No. 3, pp. 495 509 MODIFIED INCOMPLETE CHOLESKY FACTORIZATION PRECONDITIONERS FOR A SYMMETRIC POSITIVE DEFINITE MATRIX Jae Heon Yun and Yu Du Han Abstract. We propose
More informationContents. Preface... xi. Introduction...
Contents Preface... xi Introduction... xv Chapter 1. Computer Architectures... 1 1.1. Different types of parallelism... 1 1.1.1. Overlap, concurrency and parallelism... 1 1.1.2. Temporal and spatial parallelism
More informationThe Conjugate Gradient Method
The Conjugate Gradient Method Classical Iterations We have a problem, We assume that the matrix comes from a discretization of a PDE. The best and most popular model problem is, The matrix will be as large
More informationA SPARSE APPROXIMATE INVERSE PRECONDITIONER FOR NONSYMMETRIC LINEAR SYSTEMS
INTERNATIONAL JOURNAL OF NUMERICAL ANALYSIS AND MODELING, SERIES B Volume 5, Number 1-2, Pages 21 30 c 2014 Institute for Scientific Computing and Information A SPARSE APPROXIMATE INVERSE PRECONDITIONER
More informationFast Structured Spectral Methods
Spectral methods HSS structures Fast algorithms Conclusion Fast Structured Spectral Methods Yingwei Wang Department of Mathematics, Purdue University Joint work with Prof Jie Shen and Prof Jianlin Xia
More informationNumerical Methods I Non-Square and Sparse Linear Systems
Numerical Methods I Non-Square and Sparse Linear Systems Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 MATH-GA 2011.003 / CSCI-GA 2945.003, Fall 2014 September 25th, 2014 A. Donev (Courant
More informationAlgorithm for Sparse Approximate Inverse Preconditioners in the Conjugate Gradient Method
Algorithm for Sparse Approximate Inverse Preconditioners in the Conjugate Gradient Method Ilya B. Labutin A.A. Trofimuk Institute of Petroleum Geology and Geophysics SB RAS, 3, acad. Koptyug Ave., Novosibirsk
More informationConjugate gradient method. Descent method. Conjugate search direction. Conjugate Gradient Algorithm (294)
Conjugate gradient method Descent method Hestenes, Stiefel 1952 For A N N SPD In exact arithmetic, solves in N steps In real arithmetic No guaranteed stopping Often converges in many fewer than N steps
More informationIterative Methods for Sparse Linear Systems
Iterative Methods for Sparse Linear Systems Luca Bergamaschi e-mail: berga@dmsa.unipd.it - http://www.dmsa.unipd.it/ berga Department of Mathematical Methods and Models for Scientific Applications University
More informationA parameter tuning technique of a weighted Jacobi-type preconditioner and its application to supernova simulations
A parameter tuning technique of a weighted Jacobi-type preconditioner and its application to supernova simulations Akira IMAKURA Center for Computational Sciences, University of Tsukuba Joint work with
More informationAccelerating linear algebra computations with hybrid GPU-multicore systems.
Accelerating linear algebra computations with hybrid GPU-multicore systems. Marc Baboulin INRIA/Université Paris-Sud joint work with Jack Dongarra (University of Tennessee and Oak Ridge National Laboratory)
More informationA sparse multifrontal solver using hierarchically semi-separable frontal matrices
A sparse multifrontal solver using hierarchically semi-separable frontal matrices Pieter Ghysels Lawrence Berkeley National Laboratory Joint work with: Xiaoye S. Li (LBNL), Artem Napov (ULB), François-Henry
More informationAn advanced ILU preconditioner for the incompressible Navier-Stokes equations
An advanced ILU preconditioner for the incompressible Navier-Stokes equations M. ur Rehman C. Vuik A. Segal Delft Institute of Applied Mathematics, TU delft The Netherlands Computational Methods with Applications,
More informationIterative Methods and Multigrid
Iterative Methods and Multigrid Part 3: Preconditioning 2 Eric de Sturler Preconditioning The general idea behind preconditioning is that convergence of some method for the linear system Ax = b can be
More informationDepartment of Computer Science and Engineering
High performance numerical linear algebra: trends and new challenges. Yousef Saad Department of Computer Science and Engineering University of Minnesota HPC days in Lyon April 8, 216 First: A personal
More informationNotes on PCG for Sparse Linear Systems
Notes on PCG for Sparse Linear Systems Luca Bergamaschi Department of Civil Environmental and Architectural Engineering University of Padova e-mail luca.bergamaschi@unipd.it webpage www.dmsa.unipd.it/
More informationA Domain Decomposition Based Jacobi-Davidson Algorithm for Quantum Dot Simulation
A Domain Decomposition Based Jacobi-Davidson Algorithm for Quantum Dot Simulation Tao Zhao 1, Feng-Nan Hwang 2 and Xiao-Chuan Cai 3 Abstract In this paper, we develop an overlapping domain decomposition
More informationEFFECTIVE AND ROBUST PRECONDITIONING OF GENERAL SPD MATRICES VIA STRUCTURED INCOMPLETE FACTORIZATION
EFFECVE AND ROBUS PRECONDONNG OF GENERAL SPD MARCES VA SRUCURED NCOMPLEE FACORZAON JANLN XA AND ZXNG XN Abstract For general symmetric positive definite SPD matrices, we present a framework for designing
More informationMultilevel preconditioning techniques with applications Yousef Saad Department of Computer Science and Engineering University of Minnesota
Multilevel preconditioning techniques with applications Yousef Saad Department of Computer Science and Engineering University of Minnesota Purdue Univ., Math & CS colloquium Purdue, Apr. 6, 2011 Introduction:
More informationSPARSE SOLVERS POISSON EQUATION. Margreet Nool. November 9, 2015 FOR THE. CWI, Multiscale Dynamics
SPARSE SOLVERS FOR THE POISSON EQUATION Margreet Nool CWI, Multiscale Dynamics November 9, 2015 OUTLINE OF THIS TALK 1 FISHPACK, LAPACK, PARDISO 2 SYSTEM OVERVIEW OF CARTESIUS 3 POISSON EQUATION 4 SOLVERS
More informationParallel Algorithms for Solution of Large Sparse Linear Systems with Applications
Parallel Algorithms for Solution of Large Sparse Linear Systems with Applications Murat Manguoğlu Department of Computer Engineering Middle East Technical University, Ankara, Turkey Prace workshop: HPC
More informationPartial Left-Looking Structured Multifrontal Factorization & Algorithms for Compressed Sensing. Cinna Julie Wu
Partial Left-Looking Structured Multifrontal Factorization & Algorithms for Compressed Sensing by Cinna Julie Wu A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor
More informationOn the Preconditioning of the Block Tridiagonal Linear System of Equations
On the Preconditioning of the Block Tridiagonal Linear System of Equations Davod Khojasteh Salkuyeh Department of Mathematics, University of Mohaghegh Ardabili, PO Box 179, Ardabil, Iran E-mail: khojaste@umaacir
More informationPreconditioning Techniques Analysis for CG Method
Preconditioning Techniques Analysis for CG Method Huaguang Song Department of Computer Science University of California, Davis hso@ucdavis.edu Abstract Matrix computation issue for solve linear system
More informationTopics. The CG Algorithm Algorithmic Options CG s Two Main Convergence Theorems
Topics The CG Algorithm Algorithmic Options CG s Two Main Convergence Theorems What about non-spd systems? Methods requiring small history Methods requiring large history Summary of solvers 1 / 52 Conjugate
More informationLecture 8: Fast Linear Solvers (Part 7)
Lecture 8: Fast Linear Solvers (Part 7) 1 Modified Gram-Schmidt Process with Reorthogonalization Test Reorthogonalization If Av k 2 + δ v k+1 2 = Av k 2 to working precision. δ = 10 3 2 Householder Arnoldi
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra for Computational and Data Sciences)
AMS526: Numerical Analysis I (Numerical Linear Algebra for Computational and Data Sciences) Lecture 19: Computing the SVD; Sparse Linear Systems Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical
More informationPASC17 - Lugano June 27, 2017
Polynomial and rational filtering for eigenvalue problems and the EVSL project Yousef Saad Department of Computer Science and Engineering University of Minnesota PASC17 - Lugano June 27, 217 Large eigenvalue
More informationSplitting Iteration Methods for Positive Definite Linear Systems
Splitting Iteration Methods for Positive Definite Linear Systems Zhong-Zhi Bai a State Key Lab. of Sci./Engrg. Computing Inst. of Comput. Math. & Sci./Engrg. Computing Academy of Mathematics and System
More informationMultilevel Preconditioning of Graph-Laplacians: Polynomial Approximation of the Pivot Blocks Inverses
Multilevel Preconditioning of Graph-Laplacians: Polynomial Approximation of the Pivot Blocks Inverses P. Boyanova 1, I. Georgiev 34, S. Margenov, L. Zikatanov 5 1 Uppsala University, Box 337, 751 05 Uppsala,
More informationFast algorithms for hierarchically semiseparable matrices
NUMERICAL LINEAR ALGEBRA WITH APPLICATIONS Numer. Linear Algebra Appl. 2010; 17:953 976 Published online 22 December 2009 in Wiley Online Library (wileyonlinelibrary.com)..691 Fast algorithms for hierarchically
More informationMultilevel preconditioning techniques with applications Yousef Saad Department of Computer Science and Engineering University of Minnesota
Multilevel preconditioning techniques with applications Yousef Saad Department of Computer Science and Engineering University of Minnesota Maillages et EDP, Nancy, June 9, 2010 Introduction: Linear System
More informationThe EVSL package for symmetric eigenvalue problems Yousef Saad Department of Computer Science and Engineering University of Minnesota
The EVSL package for symmetric eigenvalue problems Yousef Saad Department of Computer Science and Engineering University of Minnesota 15th Copper Mountain Conference Mar. 28, 218 First: Joint work with
More informationEfficient Solvers for the Navier Stokes Equations in Rotation Form
Efficient Solvers for the Navier Stokes Equations in Rotation Form Computer Research Institute Seminar Purdue University March 4, 2005 Michele Benzi Emory University Atlanta, GA Thanks to: NSF (MPS/Computational
More informationLecture 11: CMSC 878R/AMSC698R. Iterative Methods An introduction. Outline. Inverse, LU decomposition, Cholesky, SVD, etc.
Lecture 11: CMSC 878R/AMSC698R Iterative Methods An introduction Outline Direct Solution of Linear Systems Inverse, LU decomposition, Cholesky, SVD, etc. Iterative methods for linear systems Why? Matrix
More informationChapter 7 Iterative Techniques in Matrix Algebra
Chapter 7 Iterative Techniques in Matrix Algebra Per-Olof Persson persson@berkeley.edu Department of Mathematics University of California, Berkeley Math 128B Numerical Analysis Vector Norms Definition
More informationEfficient domain decomposition methods for the time-harmonic Maxwell equations
Efficient domain decomposition methods for the time-harmonic Maxwell equations Marcella Bonazzoli 1, Victorita Dolean 2, Ivan G. Graham 3, Euan A. Spence 3, Pierre-Henri Tournier 4 1 Inria Saclay (Defi
More informationLinear Solvers. Andrew Hazel
Linear Solvers Andrew Hazel Introduction Thus far we have talked about the formulation and discretisation of physical problems...... and stopped when we got to a discrete linear system of equations. Introduction
More informationLecture 18 Classical Iterative Methods
Lecture 18 Classical Iterative Methods MIT 18.335J / 6.337J Introduction to Numerical Methods Per-Olof Persson November 14, 2006 1 Iterative Methods for Linear Systems Direct methods for solving Ax = b,
More informationEnhancing Scalability of Sparse Direct Methods
Journal of Physics: Conference Series 78 (007) 0 doi:0.088/7-6596/78//0 Enhancing Scalability of Sparse Direct Methods X.S. Li, J. Demmel, L. Grigori, M. Gu, J. Xia 5, S. Jardin 6, C. Sovinec 7, L.-Q.
More informationAn Adaptive Hierarchical Matrix on Point Iterative Poisson Solver
Malaysian Journal of Mathematical Sciences 10(3): 369 382 (2016) MALAYSIAN JOURNAL OF MATHEMATICAL SCIENCES Journal homepage: http://einspem.upm.edu.my/journal An Adaptive Hierarchical Matrix on Point
More informationA short course on: Preconditioned Krylov subspace methods. Yousef Saad University of Minnesota Dept. of Computer Science and Engineering
A short course on: Preconditioned Krylov subspace methods Yousef Saad University of Minnesota Dept. of Computer Science and Engineering Universite du Littoral, Jan 19-30, 2005 Outline Part 1 Introd., discretization
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra)
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 24: Preconditioning and Multigrid Solver Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 5 Preconditioning Motivation:
More informationSparse Linear Systems. Iterative Methods for Sparse Linear Systems. Motivation for Studying Sparse Linear Systems. Partial Differential Equations
Sparse Linear Systems Iterative Methods for Sparse Linear Systems Matrix Computations and Applications, Lecture C11 Fredrik Bengzon, Robert Söderlund We consider the problem of solving the linear system
More informationParallel Iterative Methods for Sparse Linear Systems. H. Martin Bücker Lehrstuhl für Hochleistungsrechnen
Parallel Iterative Methods for Sparse Linear Systems Lehrstuhl für Hochleistungsrechnen www.sc.rwth-aachen.de RWTH Aachen Large and Sparse Small and Dense Outline Problem with Direct Methods Iterative
More informationLeveraging Task-Parallelism in Energy-Efficient ILU Preconditioners
Leveraging Task-Parallelism in Energy-Efficient ILU Preconditioners José I. Aliaga Leveraging task-parallelism in energy-efficient ILU preconditioners Universidad Jaime I (Castellón, Spain) José I. Aliaga
More informationMathematics and Computer Science
Technical Report TR-2007-002 Block preconditioning for saddle point systems with indefinite (1,1) block by Michele Benzi, Jia Liu Mathematics and Computer Science EMORY UNIVERSITY International Journal
More informationA robust inner-outer HSS preconditioner
NUMERICAL LINEAR ALGEBRA WIH APPLICAIONS Numer. Linear Algebra Appl. 2011; 00:1 0 [Version: 2002/09/18 v1.02] A robust inner-outer HSS preconditioner Jianlin Xia 1 1 Department of Mathematics, Purdue University,
More informationERLANGEN REGIONAL COMPUTING CENTER
ERLANGEN REGIONAL COMPUTING CENTER Making Sense of Performance Numbers Georg Hager Erlangen Regional Computing Center (RRZE) Friedrich-Alexander-Universität Erlangen-Nürnberg OpenMPCon 2018 Barcelona,
More informationInexact inverse iteration with preconditioning
Department of Mathematical Sciences Computational Methods with Applications Harrachov, Czech Republic 24th August 2007 (joint work with M. Robbé and M. Sadkane (Brest)) 1 Introduction 2 Preconditioned
More informationA Robust Preconditioned Iterative Method for the Navier-Stokes Equations with High Reynolds Numbers
Applied and Computational Mathematics 2017; 6(4): 202-207 http://www.sciencepublishinggroup.com/j/acm doi: 10.11648/j.acm.20170604.18 ISSN: 2328-5605 (Print); ISSN: 2328-5613 (Online) A Robust Preconditioned
More informationILAS 2017 July 25, 2017
Polynomial and rational filtering for eigenvalue problems and the EVSL project Yousef Saad Department of Computer Science and Engineering University of Minnesota ILAS 217 July 25, 217 Large eigenvalue
More informationFEM and sparse linear system solving
FEM & sparse linear system solving, Lecture 9, Nov 19, 2017 1/36 Lecture 9, Nov 17, 2017: Krylov space methods http://people.inf.ethz.ch/arbenz/fem17 Peter Arbenz Computer Science Department, ETH Zürich
More informationPreconditioning techniques to accelerate the convergence of the iterative solution methods
Note Preconditioning techniques to accelerate the convergence of the iterative solution methods Many issues related to iterative solution of linear systems of equations are contradictory: numerical efficiency
More informationTR A Comparison of the Performance of SaP::GPU and Intel s Math Kernel Library (MKL) for Solving Dense Banded Linear Systems
TR-0-07 A Comparison of the Performance of ::GPU and Intel s Math Kernel Library (MKL) for Solving Dense Banded Linear Systems Ang Li, Omkar Deshmukh, Radu Serban, Dan Negrut May, 0 Abstract ::GPU is a
More informationDepartment of Computer Science, University of Illinois at Urbana-Champaign
Department of Computer Science, University of Illinois at Urbana-Champaign Probing for Schur Complements and Preconditioning Generalized Saddle-Point Problems Eric de Sturler, sturler@cs.uiuc.edu, http://www-faculty.cs.uiuc.edu/~sturler
More informationOn the design of parallel linear solvers for large scale problems
On the design of parallel linear solvers for large scale problems ICIAM - August 2015 - Mini-Symposium on Recent advances in matrix computations for extreme-scale computers M. Faverge, X. Lacoste, G. Pichon,
More informationAN INTRODUCTION TO DOMAIN DECOMPOSITION METHODS. Gérard MEURANT CEA
Marrakech Jan 2003 AN INTRODUCTION TO DOMAIN DECOMPOSITION METHODS Gérard MEURANT CEA Introduction Domain decomposition is a divide and conquer technique Natural framework to introduce parallelism in the
More informationArchitecture Solutions for DDM Numerical Environment
Architecture Solutions for DDM Numerical Environment Yana Gurieva 1 and Valery Ilin 1,2 1 Institute of Computational Mathematics and Mathematical Geophysics SB RAS, Novosibirsk, Russia 2 Novosibirsk State
More information