On the Development of Implicit Solvers for Time-Dependent Systems

Transcription

1 School o something FACULTY School OF OTHER o Computing On the Development o Implicit Solvers or Time-Dependent Systems Peter Jimack School o Computing, University o Leeds In collaboration with: P.H. Gaskell, C.E. Goodyer, J.R. Green, Y.-Y. Koh, A.M. Mullis, J. Rosam, M. Sellier, H.M. Thompson & M.A. Walkley. AMMW03, October 2012

2 Introduction 1. Introduction 2. Time-dependent partial dierential equations/systems 3. Explicit versus implicit temporal discretisation 4. Multigrid solvers 5. Thin-ilm models or the spreading o viscous drops 6. Phase-ield models or simulating solidiication o a binary alloy 7. Parallel scalability 8. Summary and discussion

3 Time-dependent PDEs and systems Models o this type occur in a very wide range o applications including weather and climate modelling My experience is primarily with viscous low and phase-change problems and so I use examples rom these applications here (sorry!) We will ollow the approach o discretising separately in space and time: Spatial discretization leads to a large system o initial value ODEs (oten reerred to as method o lines) The examples used here are inite dierence but we have similar experiences with inite element/volume schemes in space I hope that I am able to present some ideas that can be o value to the discussions at this meeting!

4 Explicit versus implicit temporal discretisations Explicit schemes are simplest to implement but always come with conditional stability restrictions on the time step size As the spatial resolution grows these can be prohibitive This is especially true i ast transients are present and we do not wish to resolve them Unconditionally stable implicit schemes can allow the time step size to be selected based upon accuracy criteria alone but this comes at the expense o having to solve a very large algebraic system at each step! Semi-implicit schemes are can give best o both worlds e.g. treat nonlinear terms explicitly and linear terms implicitly to increase maximum stable step size Can also treat nonlinear terms implicitly however...

5 Multigrid solvers A quick overview Consider model equation u Discretization leads to an algebraic system N unknowns but number o non-zeros per row is independent o N So iterative scheme costs O(N) per sweep BUT the error reduces slowly with the number o sweeps... It typically takes a long time or the lowest requency components o the error to be damped however... the highest requency components o the error that can be represented on this grid are damped almost immediately...

6 Multigrid solvers linear example Using the dierential equation s structure Discretization on a ine grid gives: A ew sweeps o Jacobi gives: The algebraic error satisies: Approximate this error on a coarse grid: Interpolate this coarse grid correction back to the ine grid: Take a ew more sweeps o Jacobi: Note that it is possible to solve the coarse grid equation using this same algorithm recursively this is multigrid... u A û u A r e A ˆ u u e ˆ c c c c r I r e A c c e I u u ˆ ˆ u ~

7 Multigrid solvers Theory This gives an MG cycle: There exists a rigorous convergence theory, or this and similar cycles, showing mesh-independent convergence rates (e.g. Hackbusch, 1985): requires a smoothing property on the iteration requires an approximation property on The cost o each V-cycle is Hence the total cost is also O(N) O(N) c ( I, I c )

8 Example I: A thin ilm model or viscous droplets We consider a thin-ilm model or a viscous droplet lowing down an inclined plane. Lubrication approximation based upon H 0 L0

9 This leads to the Reynolds equation or the non-dimensional ilm thickness h and an associated pressure equation or p: A thin ilm model or viscous droplets cos 3 sin s h Bo h s h p y p y Bo x p x t h h h This is a nonlinear time-dependent system in h(x,y,t) A very ine spatial mesh is required to represent a thin precursor ilm Implicit time-stepping overcomes excessive conditional stability conditions allowing step selection based upon error control...

10 Nonlinear multigrid solver At each time step a large nonlinear algebraic system o equations must be n1 n1 solved or the new values: hij and p ij. Unless this can be done eiciently the method is worthless A ully coupled nonlinear Multigrid solver is used to achieve this: based upon the FAS (ull approximation scheme) approach to resolve the non-linearity this is a slight variation on the linear multigrid approach outlined above. a pointwise weighted nonlinear Jacobi iterative scheme is seen to be an adequate smoother. Excellent, h-independent, convergence results are obtained (see below).

11 Multigrid perormance or implicit time stepping Convergence rate is mesh independent...

12 Multigrid perormance or implicit time stepping Cost o solution is optimal O(N)...

13 Example II: Phase-ield model or alloy solidiication Basic Idea o Phase-Field Introduce an artiicial phase-ield variable to describes the state (Φ = -1 or liquid and φ = +1 or solid). At the interace φ varies smoothly between these bulk values the motion o this interace is determined rom the geometry and a concentration ield U.

14 Binary alloy phase-ield model (2-d or simplicity) Karma s Phase-ield model ) ( ) (1 ) '( ) ( ) '( ) ( ) '( ) ( 2 ) ( ) ( U Mc x A A y y A A x y y x x A A A t A ix Phase Equation Properties: highly nonlinear noise introduced by anisotropy unction A(Ψ) where arctan( x) y

15 Binary alloy phase-ield model (2-d or simplicity) Karma s Phase-ield Model t U k t y U t x U k t y t x U k U y U y x U x D t U k k y x y x ) ) (1 (1 2 1 ) (1 ) ( Concentration Equation Also highly nonlinear and coupled to the phase equation

16 Evolution o a typical solution in 3-d For a model o an isothermal alloy (concentration and phase ields) An example o the growth o a dendritic structure this animation plots the φ = 0 isosurace at dierent time intervals. Begins with a small solid seed. Here we impose preerred growth along the axis through our choice o anisotropy unction A...

17 Adaptive spatial discretization (3-d) Adaptive mesh reinement From the solution proiles there is an obvious need or adaptive mesh reinement. Here this is based upon hierarchical hexahedral meshes with local reinement and coarsening using PARAMESH. Have implemented a number o dierent Finite Dierence stencils all 2 nd order (based upon 7, 19 and 27 points). Adaptive remeshing is controlled by a simple gradient criterion. The ollowing diagram illustrates a typical mesh with reinement concentrated in the regions where the phase variable and the concentration variable have the highest gradients at a particular snapshot in time...

18 Adaptive spatial discretization (3-d) Adaptive mesh reinement Here we see local mesh reinement around the φ = 0 isosurace...

19 Implicit temporal discretization Explicit time integration methods Explicit methods are "easy" to apply but impose a time step restriction: t C h 2 2 Very ine mesh resolution is needed,so the time steps become excessively small (not viable). This plot shows the maximum stable time step or an isothermal model...

20 Implicit temporal discretisation Adaptive time step control Fully implicit BDF2 method, combined with variable time stepping (based upon a local error estimator), is used to overcome time-step restrictions... The adaption o the time step leads to a much larger time step than the maximum stable time step or the explicit Euler method!!

21 Nonlinear multigrid solver Nonlinear multigrid solver or adaptive meshes At each time step a large nonlinear algebraic system o equations must be n1 n1 solved or the new values: ijk and U ijk. Unless this can be done eiciently the method is worthless A ully coupled nonlinear Multigrid solver is used to achieve this: based upon the FAS (ull approximation scheme) approach to resolve the non-linearity and the MultiLevel AdapTive (MLAT) scheme o Brandt to handle the adaptivity a pointwise weighted nonlinear Jacobi iterative scheme is seen to be an adequate smoother. Excellent, h-independent, convergence results are obtained (see below).

22 Nonlinear multigrid solver Nonlinear multigrid solver or adaptive meshes Here we see close to an optimal convergence rate or the nonlinear multigrid solver applied to the isothermal problem in 3-d...

23 Nonlinear multigrid solver Nonlinear multigrid solver or adaptive meshes Here we see that the solver displays almost linear run time:

24 Results in 3-d For a model o an isothermal alloy Altering the run-time parameters gives rise to dierent dendrite morphologies in these two images a dierent under-cooling has been used, with all other parameters held constant...

25 Results in 3-d For a model o an isothermal alloy Altering the run-time parameters gives rise to dierent dendrite morphologies in these two images a dierent under-cooling has been used, with all other parameters held constant...

26 Parallel perormance o multigrid For very large time-dependent models eicient parallel implementations are essential... The bottlenecks or multigrid come rom: the need to update on each grid in a sequential manner; the limited amount o computational work on the coarsest grids. A geometric partition can help with the load balancing on each grid (though this is considerably more complex when local adaptivity is used) The issues o how to deal with the coarsest grids are very much open questions: How coarse? How accurate? Are all processors needed at this level?

27 Parallel perormance thin ilm example Illustration o domain decomposition approach Here we illustrate the geometric decomposition o the problem Only neighbour to neighbour communication is required... P0 P1 P2 P3... P9 P10 P11 P12 P13...

28 Parallel perormance thin ilm example Illustration o domain decomposition approach Here we illustrate the geometric decomposition o the problem on the coarsest grid Finer grid levels obey the same geometric partition Only neighbour to neighbour communication is required

29 Parallel perormance thin ilm example Illustration o additional capability due to parallel implementation This table illustrates the weak scalability o the nonlinear multigrid Cores Size o Grid Solver Time Multigrid levels x s x s x s x s 5 This is or a ixed number o time steps coarse grid solve is the bottleneck.

30 Parallel perormance phase ield example Illustration o additional capability due to parallel implementation Here we demonstrate the capability o combining parallelism and adaptivity Cores Uniorm Grid Adapted Grid Level (Cells) Level (Equiv Cells) 1 7 (262k) 8 (2.1M) 2 9 (16.8M) 8 8 (2 097k) 10 (134.2M) (1073.7M) 64 9 (16.8M) (8.59B) But what about scalability?

31 Parallel perormance phase ield example Partitioning strategy is key or good parallel perormance Here we demonstrate the importance o ensuring a good load balance at each mesh level in the MG hierarchy This plot shows the wallclock time per 1000 DoF or a parallel adaptive run with two loadbalancing strategies...

32 Parallel perormance algebraic multigrid Illustration o parallel perormance o AGMGv2.3 with MUMPS on coarsest level Here we demonstrate the parallel perormance o algebraic MG Cores DoF Iterations Setup Solve 32 31M s 40.4s 64 63M s 40.8s M s 41.5s M s 41.8s M s 42.5s M s 44.0s M s 48.2s M s 59.4s

33 Discussion 1. (Semi-)Implicit time-stepping is only practical i we have an eicient algebraic equation solver (nonlinear or linear) 2. We make use o multigrid (nonlinear/linear and geometric/algebraic) 3. Can combine with adaptivity and with parallel implementation 4. Scales well to thousands o cores: Coarse grid solver becomes an issue eventually May need to sacriice exact coarse grid solve (thereore optimal convergence) or eicient parallel implementation Adaptivity adds to the challenges o scalability 5. Current research is ocusing on scaling our (optimal complexity) implicit solvers to tens o thousands o cores...