Toward black-box adaptive domain decomposition methods

Transcription

1 Toward black-box adaptive domain decomposition methods Frédéric Nataf Laboratory J.L. Lions (LJLL), CNRS, Alpines Inria and Univ. Paris VI joint work with Victorita Dolean (Univ. Nice Sophia-Antipolis) Patrice Hauret (Michelin, Clermont-Ferrand) Frédéric Hecht (LJLL) Pierre Jolivet (LJLL) Clemens Pechstein (Johannes Kepler Univ., Linz) Robert Scheichl (Univ. Bath) Nicole Spillane (LJLL) JLPC Lyon 2013

2 Outline 1 Some Applications 2 An abstract 2-level Schwarz: the GenEO algorithm 3 Conclusion F. Nataf et al BB DDM 2 / 28

3 Outline 1 Some Applications 2 An abstract 2-level Schwarz: the GenEO algorithm 3 Conclusion F. Nataf et al BB DDM 3 / 28

4 Motivation Large Sparse discretized system with strongly heterogeneous coefficients (high contrast, nonlinear, multiscale) E.g. Darcy pressure equation, P 1 -finite elements: Applications: flow in heterogeneous / stochastic / layered media structural mechanics time dependent waves etc. AU = F cond(a) α max α min h 2 Goal: iterative solvers robust in size and heterogeneities F. Nataf et al BB DDM 4 / 28

5 Problem div(σ(u)) = f, where ( ) σ ij (u) = 2µε ij (u) + λδ ij div(u), ε ij (u) = 1 ui 2 x j + u j x i, f = (0, g) T = (0, 10) T, µ = E 2(1+ν), λ = Eν (1+ν)(1 2ν). After a FEM (fintie element method) discretization: AU = F where A is a large sparse highy heterogeneous matrix. F. Nataf et al BB DDM 5 / 28

6 Black box solvers (solve(mat,rhs,sol)) Direct Solvers Iterative Solvers Pros Robustness Naturally Cons Difficult to Robustness Domain Decomposition Methods (DDM): Hybrid solver should be naturally parallel and robust General form: Au = f, solved with PCG for a preconditioner M 1. What s classical: Robustness with respect to problem size (scalability) What s New here: Provable Robustness in the SPD case with respect to: Coefficients jumps System of PDEs F. Nataf et al BB DDM 6 / 28

7 Black box solvers (solve(mat,rhs,sol)) Direct Solvers Iterative Solvers Pros Robustness Naturally Cons Difficult to Robustness Domain Decomposition Methods (DDM): Hybrid solver should be naturally parallel and robust General form: Au = f, solved with PCG for a preconditioner M 1. What s classical: Robustness with respect to problem size (scalability) What s New here: Provable Robustness in the SPD case with respect to: Coefficients jumps System of PDEs F. Nataf et al BB DDM 6 / 28

8 The First Domain Decomposition Method The original Schwarz Method (H.A. Schwarz, 1870) (u) = f in Ω u = 0 on Ω. Ω 1 Ω 2 Schwarz Method : (u1 n, un 2 ) (un+1 1, u n+1 2 ) with (u n+1 1 ) = f in Ω 1 u n+1 1 = 0 on Ω 1 Ω u n+1 1 = u2 n on Ω 1 Ω 2. (u n+1 2 ) = f in Ω 2 u n+1 2 = 0 on Ω 2 Ω u n+1 2 = u n+1 1 on Ω 2 Ω 1. Parallel algorithm, converges but very slowly, overlapping subdomains only. The parallel version is called Jacobi Schwarz method (JSM). F. Nataf et al BB DDM 7 / 28

9 Jacobi and Schwarz (I): Algebraic point of view The set of indices is partitioned into two sets N 1 and N 2 : ( ) ( ) ( ) A11 A 12 x1 b1 = A 21 A 22 x 2 b 2 The block-jacobi algorithm reads: ( ) ( ) ( A11 0 x k A12 0 A 22 x k+1 = A ) ( x k 1 x k 2 ) ( b1 + b 2 It corresponds to solving a Dirichlet boundary value problem in each subdomain with Dirichlet data taken from the other one at the previous step Schwarz method with minimal overlap 1 2 ) 1 2 Figure : Domain decomposition with minimal overlap F. Nataf et al BB DDM 8 / 28

10 Jacobi and Schwarz (II): Larger overlap Let δ be a non negative integer 1 2 n s n s + δ Figure : Domain decomposition with overlap A = n s ( ) or n s + δ ( ) n s + 1 n s Figure : Matrix decomposition Without or with overlap F. Nataf et al BB DDM 9 / 28

11 Strong and Weak scalability How to evaluate the efficiency of a domain decomposition? Strong scalability (Amdahl) How the solution time varies with the number of processors for a fixed total problem size Weak scalability (Gustafson) How the solution time varies with the number of processors for a fixed problem size per processor. Not achieved with the one level method Number of subdomains ASM The iteration number increases linearly with the number of subdomains in one direction. F. Nataf et al BB DDM 10 / 28

12 How to achieve scalability Stagnation corresponds to a few very low eigenvalues in the spectrum of the preconditioned problem. They are due to the lack of a global exchange of information in the preconditioner. u = f in Ω u = 0 on Ω The mean value of the solution in domain i depends on the value of f on all subdomains. A classical remedy consists in the introduction of a coarse problem that couples all subdomains. This is closely related to deflation technique classical in linear algebra (see Nabben and Vuik s papers in SIAM J. Sci. Comp, 200X) and multigrid techniques. F. Nataf et al BB DDM 11 / 28

13 Adding a coarse space We add a coarse space correction (aka second level) Let V H be the coarse space and Z be a basis, V H = span Z, writing R 0 = Z T we define the two level preconditioner as: M 1 ASM,2 := RT 0 (R 0AR T 0 ) 1 R 0 + N i=1 R T i A 1 i R i. The Nicolaides approach is to use the kernel of the operator as a coarse space, this is the constant vectors, in local form this writes: Z := (Ri T D i R i 1) 1 i N where D i are chosen so that we have a partition of unity: N Ri T D i R i = Id. i=1 F. Nataf et al BB DDM 12 / 28

14 Theoretical convergence result Theorem (Widlund, Dryija) Let M 1 ASM,2 be the two-level additive Schwarz method: κ(m 1 ASM,2 (1 A) C + H ) δ where δ is the size of the overlap between the subdomains and H the subdomain size. This does indeed work very well Number of subdomains ASM ASM + Nicolaides F. Nataf et al BB DDM 13 / 28

15 Failure for Darcy equation with heterogeneities (α(x, y) u) = 0 in Ω R 2, u = 0 on Ω D, u n = 0 on Ω N. IsoValue Our approach Decomposition α(x, y) Jump ASM ASM + Nicolaides Fix the problem by an optimal and proven choice of a coarse space Z. F. Nataf et al BB DDM 14 / 28

16 Outline 1 Some Applications 2 An abstract 2-level Schwarz: the GenEO algorithm Choice of the coarse space Parallel implementation 3 Conclusion F. Nataf et al BB DDM 15 / 28

17 Objectives Strategy Define an appropriate coarse space V H 2 = span(z 2 ) and use the framework previously introduced, writing R 0 = Z2 T the two level preconditioner is: P 1 ASM 2 := RT 0 (R 0AR T 0 ) 1 R 0 + N i=1 R T i A 1 i R i. The coarse space must be Local (calculated on each subdomain) parallel Adaptive (calculated automatically) Easy and cheap to compute Robust (must lead to an algorithm whose convergence is proven not to depend on the partition nor the jumps in coefficients) F. Nataf et al BB DDM 16 / 28

18 Abstract eigenvalue problem Gen.EVP per subdomain: Find p j,k V h Ωj and λ j,k 0: a Ωj (p j,k, v) = λ j,k a Ω j (Ξ j p j,k, Ξ j v) v V h Ωj A j p j,k = λ j,k X j A j X j p j,k (X j... diagonal) a D... restriction of a to D In the two-level ASM: Choose first m j eigenvectors per subdomain: V 0 = span { Ξ j p j,k } j=1,...,n k=1,...,m j This automatically includes Zero Energy Modes. F. Nataf et al BB DDM 17 / 28

19 Abstract eigenvalue problem Gen.EVP per subdomain: Find p j,k V h Ωj and λ j,k 0: a Ωj (p j,k, v) = λ j,k a Ω j (Ξ j p j,k, Ξ j v) v V h Ωj A j p j,k = λ j,k X j A j X j p j,k (X j... diagonal) a D... restriction of a to D In the two-level ASM: Choose first m j eigenvectors per subdomain: V 0 = span { Ξ j p j,k } j=1,...,n k=1,...,m j This automatically includes Zero Energy Modes. F. Nataf et al BB DDM 17 / 28

20 Comparison with existing works Galvis & Efendiev (SIAM 2010): κ p j,k v dx = λ j,k κ p j,k v dx Ω j Ω j v V h Ωj Efendiev, Galvis, Lazarov & Willems (submitted): a Ωj (p j,k, v) = λ j,k a Ωj (ξ j ξ i p j,k, ξ j ξ i v) v V Ωj Our gen.evp: i neighb(j) ξ j... partition of unity, calculated adaptively (MS) a Ωj (p j,k, v) = λ j,k a Ω j (Ξ j p j,k, Ξ j v) v V h Ωj both matrices typically singular = λ j,k [0, ] F. Nataf et al BB DDM 18 / 28

21 Theory Two technical assumptions. Theorem (Spillane, Dolean, Hauret, N., Pechstein, Scheichl) If for all j: 0 < λ j,mj+1 < : [ κ(m 1 ASM,2 A) (1 + k 0) 2 + k 0 (2k 0 + 1) N max j=1 ( )] λ j,mj +1 Possible criterion for picking m j : (used in our Numerics) λ j,mj +1 < δ j H j H j... subdomain diameter, δ j... overlap F. Nataf et al BB DDM 19 / 28

22 E Eigenvalues and eigenvectors Eigenvector number 1 is e-15; exageration coefficient is: Eigenvector number 2 is e-16; exageration coefficient is: Eigenvector number 3 is e-15; exageration coefficient is: Eigenvector number 4 is e-05; exageration coefficient is: Eigenvector number 5 is e-05; exageration coefficient is: Eigenvector number 6 is e-05; exageration coefficient is: Eigenvector number 7 is ; exageration coefficient is: Eigenvector number 8 is ; exageration coefficient is: Eigenvector number 9 is ; exageration coefficient is: Eigenvector number 10 is ; exageration coefficient is: Eigenvector number 11 is ; exageration coefficient is: Eigenvector number 12 is ; exageration coefficient is: Eigenvector number 13 is ; exageration coefficient is: Eigenvector number 14 is ; exageration coefficient is: Eigenvector number 15 is ; exageration coefficient is: eigenvalue (log scale) eigenvalue number Logarithmic scale F. Nataf et al BB DDM 20 / 28

23 Convergence AS P BNN : AS + Z Nico P BNN : AS + Z D2N GMRES P BNN : AS + Z D2N Error Iteration count F. Nataf et al BB DDM 21 / 28

24 Numerical results via a Domain Specific Language FreeFem++ ( with: Metis Karypis and Kumar 1998 SCOTCH Chevalier and Pellegrini 2008 UMFPACK Davis 2004 ARPACK Lehoucq et al MPI Snir et al Intel MKL PARDISO Schenk et al MUMPS Amestoy et al PaStiX Hénon et al PETSC Why use a DS(E)L instead of C/C++/Fortran/..? performances close to low-level language implementation, hard to beat something as simple as: varf a(u, v) = int3d(mesh)([dx(u), dy(u), dz(u)]' * [dx(v), dy(v), dz(v)]) + int3d(mesh)(f * v) + on(boundary mesh)(u = 0) Much More in F. Hecht s talk tomorrow F. Nataf et al BB DDM 22 / 28

25 Strong scalability in two dimensions heterogeneous elasticity (P. Jolivet with Frefeem ++) Elasticity problem with heterogeneous coefficients Timing relative to processes Linear speedup 10 #iterations #processes Speed-up for a 1.2 billion unknowns 2D problem. Direct solvers in the subdomains. Peak performance wall-clock time: 26s. F. Nataf et al BB DDM 23 / 28

26 Strong scalability in three dimensions heterogeneous elasticity Elasticity problem with heterogeneous coefficients Timing relative to processes Linear speedup 10 #iterations #processes Speed-up for a 160 million unknowns 3D problem. Direct solvers in subdomains. Peak performance wall-clock time: 36s. F. Nataf et al BB DDM 24 / 28

27 Weak scalability in two dimensions Darcy problems with heterogeneous coefficients Weak efficiency relative to processes 100 % 80 % 60 % 40 % 20 % 0 % #d.o.f. (in millions) #processes Efficiency for a 2D problem. Direct solvers in the subdomains. Final size: 22 billion unknowns. Wall-clock time: 200s. F. Nataf et al BB DDM 25 / 28

28 Weak scalability in three dimensions Darcy problems with heterogeneous coefficients Weak e ciency relative to processes 100 % 80 % 60 % 40 % 20 % 0% #d.o.f. (in millions) #processes Efficiency for a 3D problem. Direct solvers in the subdomains. Final size: 2 billion unknowns. Wall-clock time: 200s. F. Nataf et al BB DDM 26 / 28

29 Outline 1 Some Applications 2 An abstract 2-level Schwarz: the GenEO algorithm 3 Conclusion F. Nataf et al BB DDM 27 / 28

30 Conclusion Summary Using generalized eigenvalue problems and projection preconditioning we are able to achieve a targeted convergence rate. Works for ASM, BNN and FETI methods This process can be implemented in a black box algorithm. Future work Build the coarse space on the fly. Multigrid like three (or more) level methods THANK YOU FOR YOUR ATTENTION! F. Nataf et al BB DDM 28 / 28

31 Conclusion Summary Using generalized eigenvalue problems and projection preconditioning we are able to achieve a targeted convergence rate. Works for ASM, BNN and FETI methods This process can be implemented in a black box algorithm. Future work Build the coarse space on the fly. Multigrid like three (or more) level methods THANK YOU FOR YOUR ATTENTION! F. Nataf et al BB DDM 28 / 28