Preprint Alexander Heinlein, Axel Klawonn, and Oliver Rheinbach Parallel Two-Level Overlapping Schwarz Methods in Fluid-Structure Interaction

Transcription

1 Fakultät für Mathematik und Informatik Preprint Alexander Heinlein, Axel Klawonn, and Oliver Rheinbach Parallel Two-Level Overlapping Schwarz Methods in Fluid-Structure Interaction ISSN

2 Alexander Heinlein, Axel Klawonn, and Oliver Rheinbach Parallel Two-Level Overlapping Schwarz Methods in Fluid-Structure Interaction TU Bergakademie Freiberg Fakultät für Mathematik und Informatik Prüferstraße FREIBERG

3 ISSN Herausgeber: Herstellung: Dekan der Fakultät für Mathematik und Informatik Medienzentrum der TU Bergakademie Freiberg

4 Parallel Two-Level Overlapping Schwarz Methods in Fluid-Structure Interaction Alexander Heinlein 1, Axel Klawonn 1, and Oliver Rheinbach 2 1 Mathematisches Institut, Universität zu Köln, Weyertal 86-90, Köln, Germany. {alexander.heinlein,axel.klawonn}@uni-koeln.de 2 Institut für Numerische Mathematik und Optimierung, Fakultät für Mathematik und Informatik, Technische Universität Bergakademie Freiberg, Akademiestr. 6, Freiberg, Germany. oliver.rheinbach@math.tu-freiberg.de December 2, 2015 Abstract. Parallel overlapping Schwarz preconditioners are considered and applied to the structural block in monolithic fluid-structure interaction (FSI). The two-level overlapping Schwarz method uses a coarse level based on energy minimizing functions. Linear elastic as well as nonlinear, anisotropic hyperelastic structural models are considered in an FSI problem of a pressure wave in a tube. Using our recent parallel implementation of a two-level overlapping Schwarz preconditioner based on the Trilinos library, the total computation time of our FSI benchmark problem was reduced by more than a factor of two compared to the algebraic onelevel overlapping Schwarz method used previously. Finally, also strong scalability for our FSI problem is shown for up to 512 processor cores. 1 The Two-Level Overlapping Schwarz Preconditioner The GDSW preconditioner [4] is a two-level additive Schwarz preconditioner M 1 GDSW = Φ( Φ T AΦ ) 1 Φ T + N i=1 R T i à 1 i R i, (1) with a special choice of energy minimizing coarse space functions Φ. The coarse space functions are discrete harmonic extensions of the restrictions of the null space of A to connected components (vertices, edges, and faces) of the interface Γ of a nonoverlapping domain decomposition. For the elasticity problems considered here, the null space is spanned by the three translations r 1,r 2,r 3 and three (linearized) rotations r 4,r 5,r 6. Let I = Ω\Γ be the set of degrees of freedom (d.o.f.) in the interior of a subdomain. Then the basis functions of the GDSW coarse space are defined by [ A 1 Φ = II AT ΓI Φ ] [ ] Γ ΦI =, (2) Φ Γ Φ Γ

5 2 A. Heinlein, A. Klawonn, and O. Rheinbach where Φ Γ = [ RΓ1 T G Γ1... RΓM T G ] ΓN, and RΓj is the restriction from Γ onto Γ j, the j-th interface component. The matrices G Γj are chosen such that their columns form a basis of the restriction of the matrix G to the indices corresponding to Γ j. For GDSW the condition number bound κ ( ( M 1 GDSW K) C 1+ H )( ( )) H 1+log, δ h was shown in [4]. Here, H and h are the typical subdomain and finite element diameters and δ denotes the overlap. We have implemented a parallel GDSW preconditioner based on Trilinos [7] and report on parallel scalability for model problems in [6]. Currently, we use UMFPACK to solve the problems on the subdomains and MUMPS in MPI parallel mode for the coarse problem. We use a one-to-one correspondance of subdomains to cores. 2 Monolithic Fluid-Structure Interaction We use the software environment from [1], i.e., we use the LifeV software library coupled to FEAP 8.2. As opposed to [1], where a convective explicit (CE) approach was used for the fluid, we now use a fully implicit scheme, and the linearized systems are now preconditioned using a FaCSI preconditioner applying a SIMPLE preconditioner for the fluid; see [3]. 2.1 Model Description The fluid-structure interaction (FSI) problem consists of the fluid problem ( ) u ρ f t +((u w) )u σ f (u,p) = 0 in Ω f t (0,T], X (3) u = 0 in Ω f t (0,T], which corresponds to the incompressible Navier-Stokes equations in Arbitrary Lagrangian Eulerian (ALE) formulation, cf. [5], and the structural problem ρ s 2 d s t 2 (FS) = 0 in Ωs (0,T]. (4) Here, Ω f t and Ω f are the fluid domain in actual and reference configuration, respectively, Ω s is the structural reference configuration, and Γ = Ω f Ω s is the FSI interface. In (3), ρ f denotes the density of the fluid, u and p are the velocity and pressure, respectively, w = d f t X is the velocity and d f the displacement of the fluid mesh movement, and σ f (u,p) is the Cauchy stress tensor. In (4), d s is the displacement of the structure, ρ s is the density of the structure, and FS are the first Piola-Kirchhoff stresses.

6 Two-Level Overlapping Schwarz Methods in Fluid Structure Interaction 3 The ALE mapping A t = id + d f is obtained by solving an additional geometry problem d f = 0 in Ω f, d f = d s on Γ, (5) d f n f = 0 on Ω f \Γ, i.e., by means of discrete harmonic extensions. The fluid, structural, and geometry problems are coupled by the geometric adherence(6), the continuity of the velocities (7), and the continuity of the stresses (8) on Γ, d f = d s, (6) d s = u A t, t (7) (det[f]) 1 F T σ f n f A t +(FS)n s = 0. (8) 2.2 Monolithic Coupling in FSI We use finite differences, in a fully implicit scheme, for the approximation of the time derivatives of both the fluid and the structure equations. We use piecewise quadratic (P2) finite elements for the structure and geometry problems and P2 P1 mixed finite elements for the fluid, using conforming meshes at the FSI interface. The monolithic approach leads to a single nonlinear system containing the fluid (F), the structure (S), the geometry (G) problem, and the coupling conditions, F(u n+1 f,p n+1,d n+1 f ) C1 T λ n S(d n+1 s ) + C3 T λ n C 1 u n+1 f + C 2 d n+1 s C 4 d n+1 s Hd n+1 f = b f b s C 2 d n s 0. (9) Here, λ is the vector of Lagrange multipliers. For conforming meshes, we have C 1 Γ = I Γ, C 3 Γ = I Γ, C 2 Γ = 1/ tc 3, C 4 Γ = I Γ, where I Γ is the identity matrix defined on the interface Γ. 2.3 Linearization and Parallel Monolithic Preconditioner As in [1], we solve the nonlinear monolithic FSI problem (9) using an inexact Newton method. The corresponding tangent J M, associated with (9), reads J M = D df F 0 C1 T D df F 0 D ds S C3 T 0 C 1 C C 4 0 H D df F 0 C1 T D df F 0 D ds S 0 0 C 1 C C 4 0 H =: P DN. (10)

7 4 A. Heinlein, A. Klawonn, and O. Rheinbach Inflow Outflow Mesh #1: Interior radius of the structure 0.15cm Outer radius of the structure 0.21cm Length 2.5cm Mesh #2: Interior radius of the structure 0.08cm Outer radius of the structure 0.1cm Length 5cm Mesh #3: Interior radius of the structure 0.08cm Outer radius of the structure 0.11cm Length 10cm Fig.1. Geometry of the FSI problem. The number of d.o.f. is almost identical for all geometries and well-balanced between fluid (F) and structure (S), cf. Table 1. Mesh Velocity (F) Pressure (F) Displacement (S) Displacement (G) # # # Table 1. Number of degrees of freedom of the different meshes. Here, D df F denotes the linearization of the fluid operator, D df F the shape derivatives, and D ds S the linearization of the structural operator. We solve the linearized system using a GMRES iteration with the FaCSI preconditioner [3], which is based on a factorization of the matrix P DN. The fluid block is treated further by static condensation of the interface degrees of freedom and the use of a SIMPLE preconditioner for the fluid block; see [3]. The inverses appearing in the application of the FaCSI preconditioner are replaced by algebraic (for fluid or structure, separately) and geometric onelevel overlapping Schwarz preconditioner (for the structure) or the GDSW preconditioner (for the structure). 3 Numerical Results for Fluid-Structure Interaction We consider our FSI problem while applying different preconditioners to the structural block but without changing the preconditioners for the fluid and geometry blocks. We then report on the resulting performance of the full monolithic FSI simulation. The default preconditioner for the structural block is IFPACK, a parallel algebraic overlapping Schwarz preconditioner from Trilinos [7]. Our parallel preconditioner has two potential advantages over IFPACK: it uses a geometric overlap, and it can use a coarse space, for better robustness and improved numerical scalability. We consider three different meshes; cf. Fig. 1. We apply zero-displacement Dirichlet boundary conditions to the structure at the inlet and the outlet. A pressure wave in a tube (see Fig. 5) is created by applying a constant normal stress σ n = 0.133kPa at the inflow for t 0.003s. We consider three different material models for the wall, i.e., a linear elastic (LE), a Neo-Hookean (NH), and a realistic anisotropic nonlinear material

8 Two-Level Overlapping Schwarz Methods in Fluid Structure Interaction 5 model (Ψ A ), cf. [8], which we have already considered in FSI for arterial walls in [1]. For linear elasticity, we use E = 40kPa and ν = 0.3, for Neo-Hooke, µ = 7.72kPa and κ = 383.3kPa, and for the Ψ A model, we use the parameters from [2, (Ψ A Set 2)]. Time to Solution using Different Preconditioners for the Structure Block We perform simulations of the pressure wave in a tube for a total simulation time T = 0.01s. We use a time step t = , , , or , i.e., we solve 100, 50, 25 or 20 monolithic nonlinear systems. The nonlinear problems are solved using, on average, 5.1 (LE s), 5.6 (NH s), 6.6 (Ψ A s), 6.1 (LE s), 6.3 (NH s), 7.9 (Ψ A s), 7.4 (LE s), 7.9 (NH s), 11.9 (Ψ A s), 8.4 (LE s), or 9.5 (NH s) Newton iterations, and a preconditioned GMRES iteration is used to solve the linearized monolithic systems in each Newton step. Our stopping criterion for Newton is a mixed criterion with a relative and absolute tolerance of 10 8 and for GMRES a relative tolerance of We compare the number of iterations and the computing times in Table 2; see also Fig. 2. When using our preconditioner, we consider three cases: only using the first level (OS1), using the first and coarse level but neglecting the rotations (r 4,r 5,r 6 ) when constructing the coarse level (GDSW-nr), and using the full GDSW preconditioner (GDSW), i.e., with first and coarse level. In the case where rotations are neglected (GDSW-nr) no geometric information is needed for the construction of the coarse problem. For all methods, we specify an overlap of δ = 2h. We perform the comparison using Mesh #1 and 128 cores. In Table 2, for a small time step, all preconditioners show a very similar performance with respect to the number of GMRES iteration as well as the timings. However, for a larger time step, where the weight in front of the mass matrix is small, the number of iterations and the timings for IFPACK quickly deteriorate. The other methods, which use a geometric overlap, show a better performance. The use of a coarse space gives further improvements: for the largest time steps the GDSW preconditioner is the fastest method. Neglecting the rotations in the GDSW preconditioner (GDSW-nr), which makes the preconditioner more algebraic, yields a number of iterations which falls between the one-level preconditioner and the GDSW with the full coarse space. For our experiments, it thus seems that for GDSW-nr it is not easy to amortize the cost for the coarse level compared to OS1. The use of the GDSW preconditioner with the full coarse space, however, can be recommended as a new default. For smaller time steps the performance of all preconditioners is similar and for larger time steps it is clearly the fastest option: For the most challenging structural model (Ψ A ) combined with the largest time step the monolithic FSI simulation is more than 2.5 times faster when using the GDSW preconditioner instead of IFPACK; see Table 2. This is especially remarkable since, in our monolithic preconditioner, we have only exchanged the preconditioner for the structural block whereas the timings are for the complete FSI simulation.

9 6 A. Heinlein, A. Klawonn, and O. Rheinbach Fig. 2. Total number of GMRES iterations (top) and total runtime (bottom) for the pressure wave in a tube FSI problem using Mesh #1 and 128 cores; see also Table 2. We use different preconditioners for the structure block. OS1 is the onelevel Schwarz preconditioner, GDSW-nr is the GDSW preconditioner without rotations, and OS2 is the GDSW preconditioner with full coarse space. Strong Scaling for the Fluid-Structure Interaction problem In Fig. 3 and Fig. 4, we present strong parallel scaling results for the pressure wave in a tube using t = s and t = s, respectively, for a linear elastic tube. For the structure, we have used our new default preconditioner, i.e., the GDSW preconditioner including rotations, with overlaps of δ = 1h and δ = 2h. We present the GMRES iterations per Newton step and the total runtime for a time step. The timings are for the fully coupled FSI simulation. For all cases, we observe good scalability results, whereas the scaling is slightly worse for a time step of s. This is partially a result of the number of Newton iterations, which varies from 3 to 5. We also observe a significant influence of the shape of the geometry on the performance. For Mesh #3, we observe the lowest number of iterations, the best numerical scalability, the lowest computing computing times and the best parallel scalability. Acknowledgements The authors gratefully acknowledge the Gauss Centre for Supercomputing (GCS) for providing computing time through the John von Neumann Institute for Computing (NIC) on the GCS share of the supercomputer JUQUEEN at Jülich Supercomputing Centre (JSC). The authors also gratefully acknowledge the Cray XT6m at Universität Duisburg-Essen and the financial support by the German Science Foundation (DFG), project no. KL2094/3-1 and RH122/4-1.

10 Two-Level Overlapping Schwarz Methods in Fluid Structure Interaction 7 GMRES Iterations Mesh 1, GDSW, overlap 1h Mesh 1, GDSW, overlap 2h Mesh 2, GDSW, overlap 1h Mesh 2, GDSW, overlap 2h Mesh 3, GDSW, overlap 1h Mesh 3, GDSW, overlap 2h # Cores Time in s Optimal Scaling Mesh 1, GDSW, overlap 1h Mesh 1, GDSW, overlap 2h Mesh 2, GDSW, overlap 1h Mesh 2, GDSW, overlap 2h Mesh 3, GDSW, overlap 1h Mesh 3, GDSW, overlap 2h # Cores Fig.3. Strong scaling (16 to 512 cores) for FSI using linear elasticity and t = s The computing time for one time step is shown. Always 3 Newton steps. GMRES Iterations Mesh 1, GDSW, overlap 1h Mesh 1, GDSW, overlap 2h Mesh 2, GDSW, overlap 1h Mesh 2, GDSW, overlap 2h Mesh 3, GDSW, overlap 1h Mesh 3, GDSW, overlap 2h # Cores Time in s Optimal Scaling Mesh 1, GDSW, overlap 1h Mesh 1, GDSW, overlap 2h Mesh 2, GDSW, overlap 1h Mesh 2, GDSW, overlap 2h Mesh 3, GDSW, overlap 1h Mesh 3, GDSW, overlap 2h # Cores Fig.4. Same as Fig. 3 but using t = s. Newton steps vary from 3 to 5. Fig.5. Fluid pressure (top) and structural deformation (bottom) for the linear elastic (left), the Neo-Hookean (middle), and the Ψ A (right) material model at t = 0.003s. The structural displacement is magnified by a factor of 10. The figure also illustrates the significantly different behavior for the material models.

11 8 A. Heinlein, A. Klawonn, and O. Rheinbach t Struct. IFPACK One-level Schwarz GDSW w/o rot. GDSW (OS1) (GDSW-nr) Time GMRES Time GMRES Time GMRES Time GMRES iter iter iter iter LE 5.0m m m m 50.8 NH 8.6m m m 55.m 7.0m 52.7 Ψ A 19.7m m m m LE 8.9m m m m 58.0 NH 14.2m m m m 66.0 Ψ A 33.3m m m m LE 15.3m m m m 71.9 NH 24.7m m m m 88.4 Ψ A 63.0m m m m LE 19.4m m m m 76.3 NH 33.5m m m m 96.1 Table 2. Average computing time per time step (in minutes) and average number of GMRES iterations per Newton step for the pressure wave in a tube problem; see Fig. 2 for the total runtimes. Linear elasticity (LE), Neo-Hooke (NH), and a nonlinear, anisotropic hyperelastic material law to model an arterial wall (Ψ A); also see Fig. 5. The time step is t and the final simulation time is T = 0.01s. We compare IFPACK with the one-level overlapping Schwarz preconditioner (OS1) and the GDSW preconditioner with and without rotations (OS2/OS2-nr) on 128 cores of a Cray XT6m. No convergence for Ψ A and t = Best numbers in bold face.

12 Two-Level Overlapping Schwarz Methods in Fluid Structure Interaction 9 References 1. D. Balzani, S. Deparis, S. Fausten, D. Forti, A. Heinlein, A. Klawonn, A. Quarteroni, O. Rheinbach, and J. Schröder, Numerical modeling of fluid-structure interaction in arteries with anisotropic polyconvex hyperelastic and anisotropic viscoelastic material models at finite strains, Internat. J. Numer. Meth. Biomed. Engrg. (2016), Accepted 10/ D. Brands, A. Klawonn, O. Rheinbach, and J. Schröder, Modelling and convergence in arterial wall simulations using a parallel FETI solution strategy, Comput. Methods Biomech. Biomed. Engin. 11 (2008), S. Deparis, D. Forti, G. Grandperrin, and A. Quarteroni, FaCSI : A block parallel preconditioner for fluid-structure interaction in hemodynamics, Tech. Report 13, C. R. Dohrmann, A. Klawonn, and O. B. Widlund, Domain decomposition for less regular subdomains: overlapping Schwarz in two dimensions, SIAM J. Numer. Anal. 46:4 (2008), L. Formaggia, A. Quarteroni, and A. Veneziani, Cardiovascular Mathematics, vol. 3, A. Heinlein, A. Klawonn, and O. Rheinbach, Parallel overlapping Schwarz with an energy-minimizing coarse space, 2015, To be submitted to the Proceedings of the 23rd International Conference on Domain Decomposition Methods, Springer Lect. Notes Comput. Sci. Eng. 7. M. A. Heroux, R. A. Bartlett, V. E. Howle, R. J. Hoekstra, J. J. Hu, T. G. Kolda, R. B. Lehoucq, K. R. Long, R. P. Pawlowski, E. T. Phipps, A. G. Salinger, H. K. Thornquist, R. S. Tuminaro, J. M. Willenbring, A. Williams, and K. S. Stanley, An overview of the Trilinos project, ACM Trans. Math. Softw. 31:3 (2005), G. Holzapfel, T. Gasser, and R. Ogden, A new constitutive framework for arterial wall mechanics and a comparative study of material models, Journal of Elasticity 61 (2000), 1 48.