Challenges for Matrix Preconditioning Methods

Transcription

1 Challenges for Matrix Preconditioning Methods Matthias Bollhoefer 1 1 Dept. of Mathematics TU Berlin Preconditioning 2005, Atlanta, May 19, 2005 supported by the DFG research center MATHEON in Berlin

2 Outline 1 Challenging application problems Circuit and device simulation Anderson Model a case study 2 Advances in Preconditioning Techniques Matchings Symmetric Matchings Symbolic reorderings techniques Inverse-based Techniques Aggressive Dropping 3 Application Problems Circuit and Device Simulation Anderson Model a case study ILUPACK

3 Outline 1 Challenging application problems Circuit and device simulation Anderson Model a case study 2 Advances in Preconditioning Techniques Matchings Symmetric Matchings Symbolic reorderings techniques Inverse-based Techniques Aggressive Dropping 3 Application Problems Circuit and Device Simulation Anderson Model a case study ILUPACK

4 Circuit and device simulation Circuits Modified nodal approach transient analysis Devices harmonic balance drift diffusion equations

5 Outline 1 Challenging application problems Circuit and device simulation Anderson Model a case study 2 Advances in Preconditioning Techniques Matchings Symmetric Matchings Symbolic reorderings techniques Inverse-based Techniques Aggressive Dropping 3 Application Problems Circuit and Device Simulation Anderson Model a case study ILUPACK

6 The Anderson model of localization A Challenge for modern eigenvalue algorithms Model describes electronic transport properties in disordered systems Wave function probabilities ω = 14.5 ω = 16.5 ω = 18.5

7 The Anderson model of localization A Challenge for modern eigenvalue algorithms Uniform grid in a three dimensional cube 1 1 ε ijk x ijk P p i =1, q j =1, r j =1 xpqr = λx ijk ε ijk [ ω/2, ω/2], ω radomly in [1, 30]. Eigenvalue problem Ax = λx in 3D x quantum mechanic wave function ω 16.5 critical range ω 16.5 fluctuations, but bounded ω 16.5 wave functions are localized

8 Challenges Physically sensible results require large scale simulation, n = m 3 (e.g. m = 100, 200,... ) physically interesting: eigenvectors around λ = 0 at ω c = 16.5 requires eigenvalue solver which compute some eigenvectors around λ = 0 at ω c are fast (some hours up to a few days) are memory efficient (required memory scales n)

9 Challenges Physically sensible results require large scale simulation, n = m 3 (e.g. m = 100, 200,... ) physically interesting: eigenvectors around λ = 0 at ω c = 16.5 requires eigenvalue solver which compute some eigenvectors around λ = 0 at ω c are fast (some hours up to a few days) are memory efficient (required memory scales n) Numerical Approach Preconditioned Eigenvalue Solver

10 Outline 1 Challenging application problems Circuit and device simulation Anderson Model a case study 2 Advances in Preconditioning Techniques Matchings Symmetric Matchings Symbolic reorderings techniques Inverse-based Techniques Aggressive Dropping 3 Application Problems Circuit and Device Simulation Anderson Model a case study ILUPACK

11 What are Matchings Example 0 original A 1 C A

12 What are Matchings Example 0 original A 1 C A Rows Columns Associated graph

13 What are Matchings Example 0 original A 1 C A Rows Columns Associated graph (Perfect) matching. (At most) one edge for each row/column

14 What are Matchings Example 0 original A 1 C A Rows Columns (Perfect) matching. (At most) one edge for each row/column

15 What are Matchings Example 0 original A 1 C A Rows Columns π = (5, 3, 4, 2, 1) Associated permutation

16 What are Matchings Example 0 original A 1 C A Rows Columns π = (5, 3, 4, 2, 1) 0 permuted Π A 1 C A Permuted matrix

17 Minimum Weight Matchings Matching Permutation of the matrix to zero free representation [ MC21, Duff 77] Refined Objective: Strengthening diagonal dominance ny Find matching such that a π(i),i is maximized i=1

18 Minimum Weight Matchings Matching Permutation of the matrix to zero free representation [ MC21, Duff 77] Refined Objective: Strengthening diagonal dominance ny Find matching such that a π(i),i is maximized i=1 dense case [Olschowka, Neumeier 96], sparse case [ MC64, Duff,Koster 99]

19 Minimum Weight Matchings nx reformulation: minimize c π(i),i, where c ij = j i=1 maxj a ij log a ij a ij 0 otherwise known linear sum assignment problem

20 Minimum Weight Matchings nx reformulation: minimize c π(i),i, where c ij = j i=1 maxj a ij log a ij a ij 0 otherwise known linear sum assignment problem solution leads to two vectors (u i ) i, (v j ) j such that u i + u j c ij and if (i, j) are part of the matching u i + u j = c ij diagonal scalings D r = dgl (e u i ) i, D c = dgl (e v i / max k a jk ) j

21 Minimum Weight Matchings nx reformulation: minimize c π(i),i, where c ij = j i=1 maxj a ij log a ij a ij 0 otherwise known linear sum assignment problem solution leads to two vectors (u i ) i, (v j ) j such that u i + u j c ij and if (i, j) are part of the matching u i + u j = c ij diagonal scalings D r = dgl (e u i ) i, D c = dgl (e v i / max k a jk ) j yields permutation Π and diagonal scalings D r and D c. A Π D r AD c such that a ii = 1, a ij 1 for all i, j

22 How matchings improve iterative solvers Example 33 large sparse unstructured systems from chemical engineering Simple dual threshold ILU with pivoting from SPARSKIT [Saad 94] Restarted GMRES(30), check convergence after at most 500 steps AMF MMD RCM ND ILUTP, percentage of problems solved without matching 70 percentage e 3 1e 4 1e 5 drop tolerance

23 How matchings improve iterative solvers Example 33 large sparse unstructured systems from chemical engineering Simple dual threshold ILU with pivoting from SPARSKIT [Saad 94] Restarted GMRES(30), check convergence after at most 500 steps ILUTP, percentage of problems solved WITH MC64 AMF MMD RCM ND with matching 70 percentage e 3 1e 4 1e 5 drop tolerance

24 How matchings improve iterative solvers Minimum weight matchings significantly improve preconditioning methods [Benzi,Haws,Tuma 00] also applicable to factored sparse approximate inverses High potential also as part direct solvers (PARDISO, SuperLU,... ) Matchings allow Level 3 BLAS with static pivoting use of symmetric reordering techniques

25 How matchings improve iterative solvers BUT Minimum weight matchings significantly improve preconditioning methods [Benzi,Haws,Tuma 00] also applicable to factored sparse approximate inverses High potential also as part direct solvers (PARDISO, SuperLU,... ) Matchings allow Level 3 BLAS with static pivoting use of symmetric reordering techniques matching destroys symmetry structures leads to unsymmetric scaling in this way not directly applicable to symmetrically structured systems

26 Outline 1 Challenging application problems Circuit and device simulation Anderson Model a case study 2 Advances in Preconditioning Techniques Matchings Symmetric Matchings Symbolic reorderings techniques Inverse-based Techniques Aggressive Dropping 3 Application Problems Circuit and Device Simulation Anderson Model a case study ILUPACK

27 Basic Idea of Symmetric Matchings Symmetric matchings [Duff,Gilbert 02], [Duff,Pralet 04] Example 0 1 C A π = (5, 3, 4, 2, 1) original A

28 Basic Idea of Symmetric Matchings Symmetric matchings [Duff,Gilbert 02], [Duff,Pralet 04] Decompose permutation as product of cycles Example 0 1 C A π = (5, 3, 4, 2, 1) = (5, 1)(3, 4, 2) original A

29 Basic Idea of Symmetric Matchings Symmetric matchings [Duff,Gilbert 02], [Duff,Pralet 04] Decompose permutation as product of cycles each cycle is associated with a diagonal block, pivots stay there Example 0 1 C A π = (5, 3, 4, 2, 1) = (5, 1)(3, 4, 2) 0 1 C A original A permuted Π A

30 Basic Idea of Symmetric Matchings Symmetric matchings [Duff,Gilbert 02], [Duff,Pralet 04] Decompose permutation as product of cycles each cycle is associated with a diagonal block, pivots stay there pivots stay inside if related SYMMETRIC permutation is applied Example 0 1 C A π = (5, 3, 4, 2, 1) = (5, 1)(3, 4, 2) 0 1 C A original A permuted Π AΠ

31 Basic Idea of Symmetric Matchings break up cycles into pairs of 2 cycles and group them together various strategies of breaking up the cycles [Duff,Gilbert 02],[Duff,Pralet 04],[Schenk,Hagemann 04] long cycles rarely show up in practice [Duff,Pralet 04] Scaling replaced by symmetric scaling D = (D cd r ) 1/2 ensures that entries are still at most 1 in modulus Example 0 1 C A 0 1 C A 0 1 C A Π AΠ rearranged Π 1 AΠ 1 final Π 2 AΠ 2

32 Outline 1 Challenging application problems Circuit and device simulation Anderson Model a case study 2 Advances in Preconditioning Techniques Matchings Symmetric Matchings Symbolic reorderings techniques Inverse-based Techniques Aggressive Dropping 3 Application Problems Circuit and Device Simulation Anderson Model a case study ILUPACK

33 Symbolic reorderings techniques multilevel nested dissection MeTiS [Karypis,Kumar 95] Approximate minimum degree [Amestoy,Davis,Duff,Gilbert,Larimore,Ng] AMD, UMFPACK further old fashioned orderings (Reverse Cuthill McKee, Minimum Degree) Different approach: diagonal dominance + sparsity [Saad 03] alternative to matching + reordering

34 Outline 1 Challenging application problems Circuit and device simulation Anderson Model a case study 2 Advances in Preconditioning Techniques Matchings Symmetric Matchings Symbolic reorderings techniques Inverse-based Techniques Aggressive Dropping 3 Application Problems Circuit and Device Simulation Anderson Model a case study ILUPACK

35 Algebraic Multilevel Approach j F fine grid points Reordering (+ rescaling) the system C coarse grid points «A Π AFF A FC AΠ = A CF A CC

36 Algebraic Multilevel Approach j F fine grid points Reordering (+ rescaling) the system C coarse grid points «A Π AFF A FC AΠ = Approximate block decomposition ««««AFF A FC LFF 0 DFF 0 UFF U FC = + E A CF A CC L CF I 0 S CC 0 I {z } L A CF A CC {z } D A {z } U

37 Algebraic Multilevel Approach j F fine grid points Reordering (+ rescaling) the system C coarse grid points «A Π AFF A FC AΠ = Approximate block decomposition ««««AFF A FC LFF 0 DFF 0 UFF U FC = + E A CF A CC L CF I 0 S CC 0 I {z } L A CF A CC {z } D A {z } U S CC coarse grid system, E error matrix E represents the entries being discarded in L, U

38 solution operator Approximation B FF A 1 FF, B FC A 1 FF AFC, B CF A CFA 1 FF, e.g. via solving with LFFDFFUFF e.g. via L 1 FF LFC e.g. via UCFU 1 FF

39 solution operator Approximation B FF A 1 FF, B FC A 1 FF AFC, B CF A CFA 1 FF, e.g. via solving with LFFDFFUFF e.g. via L 1 FF LFC e.g. via UCFU 1 FF AFF A FC A CF A CC «1 BFF «BFC + I «{z } P S 1 ` CC BCF {z I } R P interpolation R restriction

40 solution operator Approximation B FF A 1 FF, B FC A 1 FF AFC, B CF A CFA 1 FF, e.g. via solving with LFFDFFUFF e.g. via L 1 FF LFC e.g. via UCFU 1 FF AFF A FC A CF A CC «1 BFF «BFC + I «{z } P S 1 ` CC BCF {z I } R P interpolation R restriction Multilevel approach: same approach recursively applied to S CC

41 Solution Operator Refinement Supplement by smoothing steps G 1, G 2 (e.g. Jacobi, Gauss Seidel) Iterations matrix for the error e = x x j «ff BFF 0 e (I + PS 1 CC R A)e 0 0 e (I G 2 A)(I PS 1 CC R A)(I G 1 A)e

42 Solution Operator Refinement Supplement by smoothing steps G 1, G 2 (e.g. Jacobi, Gauss Seidel) Iterations matrix for the error e = x x j «ff BFF 0 e (I + PS 1 CC R A)e 0 0 e (I G 2 A)(I PS 1 CC R A)(I G 1 A)e V cycle (µ = 1), W cycle (µ = 2) (I G 2 A)(I PS 1 CC R A) µ (I G 1 A)

43 What means inverse based decomposition Prescribed uniform bound κ for the inverse transformations L 1, U 1 L 1 = L 1 FF 0 L CFL 1 FF «I κ U 1 = U 1 FF U 1 FF UFC 0 I «κ

44 What means inverse based decomposition Prescribed uniform bound κ for the inverse transformations L 1, U 1 L 1 = L 1 FF 0 L CFL 1 FF I «L 1 FF 0 B CF I R «κ U 1 = U 1 FF U 1 FF UFC 0 I «B FC U 1 FF 0 I P! κ

45 Why inverse based Decompositions Generally speaking: Norm of the inverse factors drive the approximation error Inverse error F = L 1 EU 1 is amplified If the norms of inverse factors are even kept bounded: Absolute error of the coarse grid system S CC can be predicted Tight κ forces approximately sparse factors L 1, U 1

46 Approximation Error Why Inverse Based Decompositions Approximation A = LDU + E For solving Ax = b we have to apply L 1, U 1

47 Approximation Error Why Inverse Based Decompositions Approximation L 1 AU 1 = D + F Can be used directly to construct an approximate inverse decomposition

48 Approximation Error Why Inverse Based Decompositions Approximation L 1 AU 1 = D + F Can be used directly to construct an approximate inverse decomposition AINV [Benzi, Tuma 98] compute approximate inverses W L 1, Z U 1 directly without L, U A!! + F 1 + F 2 WAZ =! + F Numerical usually more expensive than ILU Significantly more robust if small entries in W, Z are discarded. ILUs as by-product from AINV inherit robustness [Benzi,Tuma 03]

49 Approximation Error Why Inverse Based Decompositions w ij z kl ff ε j wij := 0 z kl := 0 Theorem [B.,Saad 02] l im e m W u mj Ze m ) ( lim := 0 ε u mj := 0 e i I LW e m (m i)ε, e m I ZU e j (m j)ε

50 Approximation Error Why Inverse Based Decompositions w ij z kl ff ε j wij := 0 z kl := 0 Theorem [B.,Saad 02] l im e m W u mj Ze m ) ( lim := 0 ε u mj := 0 e i I LW e m (m i)ε, e m I ZU e j (m j)ε Observation Norm of the inverse factors drive the approximation error

51 The Inverse Error Why Inverse Based Decompositions LFF 0 I L CF «1 UFF U FC A 0 I «1 DFF 0 = 0 S CC «! F + FC F CF {z } F

52 The Inverse Error Why Inverse Based Decompositions LFF 0 I L CF «1 UFF U FC A 0 I «1 DFF 0 = 0 S CC «! F + FC F CF {z } F Lemma [B.,Saad 04] Denote by E L,FF, E U,FF the entries being dropped from L FF, U FF. coarse grid system S CC from ILU F FC = L 1 FF (E L,FF D FF + D FFE U,FF ) U 1 FFU FC Coarse grid system S CC = R AP (successively obtained via Galerkin) F FC = D FFE U,FF U 1 FFU FC

53 Error Propagation Why Inverse Based Decompositoins Suppose that the diagonal entries occuring during the decomposition L FFD FFU FF = A FF + E FF are uniformly bounded.

54 Error Propagation Why Inverse Based Decompositoins Suppose that the diagonal entries occuring during the decomposition L FFD FFU FF = A FF + E FF are uniformly bounded. Theorem [B.,Saad 04] Coarse grid system S CC from ILU If l im, u mj are dropped, whenever l im, u mj ε/κ 2, There exists a constant K such that s ij s ij K (κε) 2 Coarse grid system S CC = R AP (successively from Galerkin) If l im, u mj are dropped, whenever l im, u mj ε, There exists a constant K such that s ij s ij K (κε) 2

55 Consequences for inverse based multilevel methods Estimate L 1, U 1 efficiently [Cline,Moler,Stewart,Wilkinson 77] Construct a well suited initial system by a priori permutation and scaling ( A FF) Keep L 1, U 1 below κ by inverse-based pivoting factor postpone Tight κ desired, since L 1 κ P j<i (L 1 ) ij κ 1

56 Outline 1 Challenging application problems Circuit and device simulation Anderson Model a case study 2 Advances in Preconditioning Techniques Matchings Symmetric Matchings Symbolic reorderings techniques Inverse-based Techniques Aggressive Dropping 3 Application Problems Circuit and Device Simulation Anderson Model a case study ILUPACK

57 Aggressive Dropping Problem for practical problems we do not precisely know the optimal ε to be save we prefer a smaller tolerance Consequences as ε 0, the fill in significantly increases as ε 0, number of iteration steps decreases to a few number of steps memory requirement dramatically increases even for the iterative part, efficiency does not increase since the fill increases

58 Aggressive Dropping Solution Lemma We do not necessarily need a small number of iteration steps instead: L 1 AU 1 = D F should lead to small F and small perturbations in L 1, U 1 could be tolerated Denote by µ k, ν k the number of nonzeros in column k of L (resp. row k of U). Let L, Ũ be constructed from L, U by dropping entries l ik, u kj satisfying L 1 e i l ik τ µ k, u kj e j U 1 τ ν k, then where D F = (I + E L )( D F )(I + E U ) E L 1 τ, E U τ.

59 Aggressive Dropping Example. BCSSTK25. Compare time and memory WITHOUT and WITH aggressive dropping drop tolerance ε is decreased threshold τ for aggressive dropping is kept at time[sec] 10 9 nnz(l)/nnz(a) drop tolerance drop tolerance 10 6

60 Aggressive Dropping Example. BCSSTK25. Compare time and memory WITHOUT and WITH aggressive dropping drop tolerance ε is decreased threshold τ for aggressive dropping is kept at time[sec] 10 9 nnz(l)/nnz(a) drop tolerance drop tolerance 10 6

61 Outline 1 Challenging application problems Circuit and device simulation Anderson Model a case study 2 Advances in Preconditioning Techniques Matchings Symmetric Matchings Symbolic reorderings techniques Inverse-based Techniques Aggressive Dropping 3 Application Problems Circuit and Device Simulation Anderson Model a case study ILUPACK

62 Circuit Simulation (benchmark collection from Infineon) compare regular and inverse based ILU Dependence of the decomposition on drop tolerance ε (Convergence of GMRES(30) after at most 500 steps) simple ILU inverse based ILU 6 6 number of systems number of systems e 3 1e e 3 1e 4 drop tolerance ε drop tolerance ε

63 Device Simulation direct solver (PARDISO, [Schenk,Gärner 04] excellent, but causes a lot of fill inverse based ILU (ILUPACK [B.,Saad 04]): fix drop tolerance at ε = 10 3 and use κ = 10. regular ILU fails until 1e 7 for most problems both use minimum weight matching and MeTiS. direct solver inverse based ILU nnz(l+u) nnz(l+u) time[sec] nnz(a) nnz(a) time[sec] barrier e e1 barrier e e1 barrier e3 0.9 barrier e e2 barrier e e1 barrier e e1 barrier e e1 barrier e e1

64 Outline 1 Challenging application problems Circuit and device simulation Anderson Model a case study 2 Advances in Preconditioning Techniques Matchings Symmetric Matchings Symbolic reorderings techniques Inverse-based Techniques Aggressive Dropping 3 Application Problems Circuit and Device Simulation Anderson Model a case study ILUPACK

65 Challenges Symmetry requires structure preserving solver Block oriented variant (1 1, 2 2), since highly indefinite Matching based preprocessing [Duff,Koster 98], [Gilbert,Duff 02], [Duff,Pralet 04], [Schenk,Hagemann 04] classical approach: Cullum Willoughby algorithm [ 85] iterative methods ARPACK [Lehoucq,Sorensen,Yang 98], Jacobi Davidson [Sleijpen,Van der Vorst 96],[Geus 02] direct solver PARDISO [Schenk,Gärtner], preconditioner ILUPACK [B.,Saad 04],[B.,Schenk 05]

66 Numerical results [Schenk,B.,Römer 05] n = m 3, m = 100, 130, 160, 190. ω = ω c = (SGI Altix/BX2, Intel Itanium2, 1.6GHz) Time system size n = m Cullum Willoughby 71 : 04

67 Numerical results [Schenk,B.,Römer 05] n = m 3, m = 100, 130, 160, 190. ω = ω c = (SGI Altix/BX2, Intel Itanium2, 1.6GHz) Time system size n = m Cullum Willoughby 71 : 04 ARPACK + PARDISO 5 : 37

68 Numerical results [Schenk,B.,Römer 05] n = m 3, m = 100, 130, 160, 190. ω = ω c = (SGI Altix/BX2, Intel Itanium2, 1.6GHz) Time system size n = m Cullum Willoughby 71 : 04 ARPACK + PARDISO 5 : 37 ARPACK + ILUPACK 0 : 45 5 : : 58

69 Numerical results [Schenk,B.,Römer 05] n = m 3, m = 100, 130, 160, 190. ω = ω c = (SGI Altix/BX2, Intel Itanium2, 1.6GHz) Time system size n = m Cullum Willoughby 71 : 04 ARPACK + PARDISO 5 : 37 ARPACK + ILUPACK 0 : 45 5 : : 58 Jacobi Davidson + ILUPACK 0 : 18 1 : 01 5 : 03 9 : 06

70 Numerical results [Schenk,B.,Römer, 05] n = m 3, m = 100, 130, 160, 190. ω = ω c = Memory [GB] system size n = m ARPACK + PARDISO 14.3 ARPACK + ILUPACK Jacobi Davidson + ILUPACK

71 Inverse based decompositions: the key to success clear: modern iterative eigenvalue solver (Jacobi-Davidson) essential Matching quite helpful key role: bound κ for L 1 Numerical example m = 70, 100. Vary bound κ inside ILUPACK fill in nnz(l) nnz(a) κ(m = 70) κ(m = 100) Time [sec] 1.7e2 3.7e2 7.8e2 5.2e2 1.1e3

72 Outline 1 Challenging application problems Circuit and device simulation Anderson Model a case study 2 Advances in Preconditioning Techniques Matchings Symmetric Matchings Symbolic reorderings techniques Inverse-based Techniques Aggressive Dropping 3 Application Problems Circuit and Device Simulation Anderson Model a case study ILUPACK

73 ILUPACK V2.0 multilevel ILU preconditioning software package inverse based decompositions single/double, real and complex arithmetic supported classes of matrices that are supported general symmetric (Hermitian) positive definite new in V2.0 real/complex symmetric, complex Hermitian indefinite interfaces to incorporate matchings is provided (PARDISO, MC64) to be released soon...

74 Conclusions development in recent years has significantly changed preconditioning methods as well as direct solvers matchings dramatically stabilize preconditioning methods symmetric matchings nowadays allow to efficiently treat symmetrically structured indefinite systems inverse based decompositions are complementary approach analysis partially gives an explanation for this effect