Shock Capturing for Discontinuous Galerkin Methods using Finite Volume Sub-cells

Abstract We present a shock capturing procedure for high order Discontinuous Galerkin methods, by which shock regions are refined in sub-cells and treated by finite volume techniques Hence, our approach combines the good properties of the Discontinuous Galerkin method in smooth parts of the flow with the perfect properties of a total variation diminishing finite volume method for resolving shocks without spurious oscillations Due to the sub-cell approach the interior resolution of the Discontinuous Galerkin grid cell is nearly preserved and the number of degrees of freedom remains the same To fully benefit from the high order of the DG method we prefer rather large grid cells This is in contrast to the requirement of resolving small geometrical shapes or other local flow features With the possibilty to locally refine grids including non-conforming interfaces we close the gap between those two contrary requirements The overall implementation is designed to run very efficiently on massively parallel computers

Shock Capturing for Discontinuous Galerkin Methods using Finite Volume Sub-cells Matthias Sonntag, Claus-Dieter Munz Institute of Aerodynamics and Gas Dynamics University of Stuttgart SHARK-FV May 23, 2016

Motivation high order methods handle shocks, discontinuities, instabilities very efficiently on massive parallel systems 1.1 1 0.9 0.8 0.7 Density 0.6 0.5 0.4 0.3 0.2 0.1 0 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 x Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 2 / 39

Content 1 Discontinuous Galerkin Spectral Element Method 2 Finite Volume Sub-cell Shock Capturing 3 Mesh refinement with Mortar interfaces 4 Numerical Results 5 Conclusion & Outlook Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 3 / 39

Navier Stokes Equation Conservation Law u t + F (u, u) = 0 in Ω with the vector of conservative variables ρ u = ρv ρe and the convective and viscous fluxes F (u, u) = F c (u) + F v (u, u) Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 5 / 39

Discretization & Weak Formulation hexahedral meshes mapping to reference element J(ξ)u t + ξ F = 0 test function + integration Ju t, φ E + F, φ E = 0 partial integration weak formulation Ju t, φ E = f (u, u, n), φ E + F, φ E where f (u, u, n) is a numerical flux function (Riemann solver) Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 6 / 39

Discontinuous Galerkin Spectral Element Method (DGSEM) u i,1 ξ 2 1 ansatz functions u = N i,j,k=0 û ijk ψ ijk (ξ), with ψ ijk = l i (ξ 1 )l j (ξ 2 )l k (ξ 3 ) u 1,j where l i (ξ 1 ),... are the 1D Lagrange interpolation polynomials nodes of Gauss quadrature: {ξ i } N i=0 collocation: points of interpolation = points of integration Lagrange property l i (ξ j ) = δ ij i, j test functions = ansatz functions same approximation for fluxes: F = N i,j,k=0 ˆF ijk ψ ijk (ξ) û ij 0 1 ξ 1 Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 7 / 39

Time Derivative Integral Ju t, φ E = 1 1 ( 1 N ) J(ξ) û mno ψ mno (ξ) ψ ijk (ξ)dξ 1 dξ 2 dξ 3 dt 1 1 1 m,n,o=0 inserting Gauss quadrature + ansatz function: Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 8 / 39

Time Derivative Integral Ju t, φ E = 1 1 ( 1 N ) J(ξ) û mno ψ mno (ξ) ψ ijk (ξ)dξ 1 dξ 2 dξ 3 dt 1 1 1 m,n,o=0 inserting Gauss quadrature + ansatz function: = N N J(ξ λµν ) û mno l m (ξ 1 2 3 dt }{{ λ) l n (ξ } µ) l o (ξν) ψ ijk (ξ λµν )ω λ ω µ ω ν }{{}}{{} λ,µ,ν=0 m,n,o=0 δ mλ δ nµ δ oν Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 8 / 39

Time Derivative Integral Ju t, φ E = 1 1 ( 1 N ) J(ξ) û mno ψ mno (ξ) ψ ijk (ξ)dξ 1 dξ 2 dξ 3 dt 1 1 1 m,n,o=0 inserting Gauss quadrature + ansatz function: = N N J(ξ λµν ) û mno l m (ξ 1 2 3 dt }{{ λ) l n (ξ } µ) l o (ξν) ψ ijk (ξ λµν )ω λ ω µ ω ν }{{}}{{} λ,µ,ν=0 m,n,o=0 δ mλ δ nµ δ oν = N J(ξ λµν )û λµν l i (ξ 1 2 3 dt }{{ λ) l j (ξ } µ) l k (ξν) ω λ ω µ ω ν }{{}}{{} λ,µ,ν=0 δ iλ δ jµ δ kν Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 8 / 39

Time Derivative Integral Ju t, φ E = 1 1 ( 1 N ) J(ξ) û mno ψ mno (ξ) ψ ijk (ξ)dξ 1 dξ 2 dξ 3 dt 1 1 1 m,n,o=0 inserting Gauss quadrature + ansatz function: = N N J(ξ λµν ) û mno l m (ξ 1 2 3 dt }{{ λ) l n (ξ } µ) l o (ξν) ψ ijk (ξ λµν )ω λ ω µ ω ν }{{}}{{} λ,µ,ν=0 m,n,o=0 δ mλ δ nµ δ oν = N J(ξ λµν )û λµν l i (ξ 1 2 3 dt }{{ λ) l j (ξ } µ) l k (ξν) ω λ ω µ ω ν }{{}}{{} λ,µ,ν=0 δ iλ δ jµ δ kν = J(ξ ijk ) û ijk dt ω iω j ω k Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 8 / 39

Semidiscrete DG Formulation weak formulation Ju t, φ E = f (u, u, n), φ E + F, φ E J ijk (û ijk ) t = + + [ [ [ f,ξ1 jk f,ξ2 ik f,ξ3 ij ˆl i ( 1) f,ξ1 jk ˆl j ( 1) f,ξ2 ik ˆl k ( 1) f,ξ3 ij ˆl i (1) + ˆl j (1) + ˆl k (1) + only 1D operations due to tensor product structure Gauss integration weigths are hidden in ˆ -terms explicit Runge Kutta time integration ] N ˆD iλ ˆF λjk 1 λ=0 N λ=0 N λ=0 ˆD jλ ˆF 2 iλk ] ˆD kλ ˆF 3 ijλ ] Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 9 / 39

Algorithm u Extrapolate u u 1. MPI 2. other Send S M : u explicit Runge Kutta time integration 1. MPI Lifting-Flux 2. other Lifting Volume Operator Lifting Surface Integral Extrapolate u u Volume Operator DG Fluxes 1. MPI 2. other 1. MPI 2. other Receive M S : u + Send M S : Lifting-Flux Receive S M : Lifting-Flux Send S M : u Receive M S : u + Send M S : Flux Surface integral 1. other 2. MPI Receive S M : Flux u t Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 10 / 39

Order Of Convergence (DG) poly. deg. cells L 2 error L 2 order L error L order theor. order 16 8.14e-04 3.10e-03 32 1.12e-04 2.86 4.34e-04 2.84 N=2 64 1.49e-05 2.91 5.91e-05 2.88 3 128 1.89e-06 2.98 7.51e-06 2.97 256 2.38e-07 2.99 9.43e-07 2.99 16 3.00e-05 1.15e-04 32 1.80e-06 4.06 6.92e-06 4.06 N=3 64 9.08e-08 4.31 3.65e-07 4.25 4 128 6.36e-09 3.84 2.54e-08 3.84 256 3.75e-10 4.08 1.53e-09 4.05 16 1.74e-06 7.25e-06 32 5.98e-08 4.86 2.50e-07 4.86 N=4 64 2.27e-09 4.72 9.29e-09 4.75 5 128 6.61e-11 5.10 2.75e-10 5.08 256 2.20e-12 4.91 9.29e-12 4.89 8 3.47e-06 1.36e-05 16 4.48e-08 6.27 1.71e-07 6.31 N=5 32 6.58e-10 6.09 2.52e-09 6.09 6 64 1.02e-11 6.01 3.88e-11 6.02 128 2.00e-13 5.67 6.89e-13 5.81 Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 11 / 39

Finite Volume Sub-cells DG u i,1 ξ 2 1 0 1 ξ 1 u 1,j û ij = interface flux points (Riemann solver) = interpolation/integration points Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 13 / 39

Finite Volume Sub-cells DG u i,1 ξ 2 1 FV sub-cells 0 1 ξ 1 u 1,j û ij = interface flux points (Riemann solver) = interpolation/integration points DG reference element is splitted into (N + 1) d sub-cells = FV sub-cell borders = inner flux points (Riemann solver) Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 13 / 39

Finite Volume Method integration over sub-cell Ju t dξ + κ ijk κ ijk F dξ = 0 i, j, k = 0,..., N divergence theorem + numerical flux function Ju t dξ + f (u, u, n) ds ξ = 0 κ ijk κ ijk choose nodal value û ijk as constant value of ijk-th FV sub-cell J ijk (û ijk ) t = ω [ jω k ω i ω j ω k f,ξ1 i 1 2,j,k f,ξ1 i+ 1 2,j,k ] + ω [ ] iω k f,ξ2,ξ2 f ω i ω j ω i,j 1 k 2,k i,j+ 1 2,k + ω [ ] iω j f,ξ3 f,ξ3 ω i ω j ω i,j,k 1 k 2 i,j,k+ 1 2 Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 14 / 39

Coupling of DG and FV P interface fluxes computed on FV points solution is interpolated to FV fluxes projected back to DG conservative / free-stream preserving even on curved meshes coupling of DG and FV via numerical fluxes Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 15 / 39

2nd Order Reconstruction first order FV is very dissipative improved by 2nd order reconstruction total variation diminishing (TVD) limiters: minmod, van Leer, Sweby,... computation of slopes: Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 16 / 39

Algorithm u Extrapolate u u 1. MPI 2. other Send S M : u, Type Receive M S : u +, Type explicit Runge Kutta time integration Lifting-Flux 1. MPI 2. other Lifting Vol. Op. / Inner Reconstr. Lifting Surf Int. / Surf. Reconstr. Extrapolate u u 1. MPI 2. other Vol. Op. DG / FV inner faces Fluxes 1. MPI 2. other Send M S : Lifting-Flux Receive S M : Lifting-Flux Send S M : u Receive M S : u + Send M S : Flux Surface integral 1. other 2. MPI Receive S M : Flux u t Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 17 / 39

Indicators detect shocks Persson indicator Jameson indicator (3D sub-cell modification) Ducros sensor (with Jameson to avoid turbulence detection) upper and lower threshold indicator value upper threshold lower threshold time DG FV DG Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 18 / 39

Timestep Restriction CFL condtion DG timestep: where: CFL = t λ x t = α RK (N) x DG 2N + 1 λ αrk - constant of explicit time integration λ - max propagation velocity FV timestep of sub-cell: with 2 N+1 = size of FV sub-cell 2 t = α RK (0) N + 1 x DG λ, Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 19 / 39

Order Of Convergence (FV sub-cells) no reconstruction minmod central poly. deg. cells L 2 error L 2 order L 2 error L 2 order L 2 error L 2 order 16 3.62e-02 9.32e-03 1.60e-03 32 1.89e-02 0.93 3.23e-03 1.53 3.83e-04 2.06 N=2 64 9.68e-03 0.97 1.04e-03 1.63 9.50e-05 2.01 128 4.89e-03 0.98 3.35e-04 1.64 2.37e-05 2.00 256 2.46e-03 0.99 1.07e-04 1.64 5.94e-06 2.00 16 2.87e-02 6.46e-03 1.25e-03 32 1.48e-02 0.95 2.14e-03 1.59 3.16e-04 1.99 N=3 64 7.55e-03 0.98 6.92e-04 1.63 7.92e-05 2.00 128 3.81e-03 0.99 2.22e-04 1.64 1.98e-05 2.00 256 1.91e-03 0.99 7.10e-05 1.64 4.97e-06 2.00 16 2.40e-02 4.81e-03 8.98e-04 32 1.24e-02 0.96 1.54e-03 1.64 2.30e-04 1.97 N=4 64 6.27e-03 0.98 5.09e-04 1.60 5.73e-05 2.00 128 3.16e-03 0.99 1.63e-04 1.64 1.43e-05 2.00 256 1.59e-03 0.99 5.26e-05 1.64 3.59e-06 2.00 Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 20 / 39

Remarks same DOFs, same data structure straightforward implementation comparable performance (DG/FV sub-cells per element) conservative interpolation/projection between DG and FV sub-cells 2nd order reconstruction for FV sub-cells slope limiters, TVD Indicators decide where to use FV sub-cells, where DG Persson indicator JST scheme like indicator Ducros sensort... no additional timestep restriction Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 21 / 39

Mesh refinement with Mortar interfaces complex geometries mesh refinement especially for hex-only-meshes non-conforming interfaces Mortar approach 3 different types in 3D Type 2 Type 1 Type 3 Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 23 / 39

Mesh refinement with Mortar interfaces Type 1 is combination of types 2 and 3 Interpolation-operators M 0 1 and M 0 2: M 0 1 : big left half Projection-operators M 1 0 and M 2 0: M 0 2 : big right half M 1 0 : left half big M 2 0 : right half big operators reduce to matrix-vector-product use this operators in η or ξ direction additional operators M * * FV for FV sub-cell shock capturing Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 24 / 39

Parallel performance (strong scaling) speedup 6 elements per core 10 4 pure DG pure FV 87 95% half/half checker board 10 3 ideal 10 2 10 2 10 3 10 4 cores polynomial degree N = 5 15 10 6 DOFs Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 26 / 39

Performance index (PID) PID = wall-clock-time#cores #DOF#timesteps 24 to 12288 cores 3072 to 6 elements per core 8 7 PID (µs/dof) 6 5 4 3 10 3 10 4 10 5 10 6 Load (DOF/core) pure DG pure FV half/half checker board Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 27 / 39

Parallel performance (strong scaling) speedup 10 2 10 1 5.7 elements per core PID (µs/dof) 8 7 6 10 0 10 2 10 3 cores 5 10 4 10 5 Load (DOF/core) realistic test case (N=8, 17582 cells) strong scaling down to about 6 elements per core cache effects best performance Hazel Hen of HLRS Stuttgart (Cray XC40) Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 28 / 39

Forward Facing Step Mach 3 flow over step in wind tunnel Mortar interfaces, refinement at 1. step 6543 grid cells 2. additionally: top shear wave (red region) 8343 grid cells polynomial degree N = 5 2.3 10 5 DOF computation by M. Hoffmann Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 29 / 39

Forward Facing Step Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 30 / 39

Forward Facing Step Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 31 / 39

Shockless explosion combustion test case validation test case only 64388 grid cells N = 3 4.1 10 6 DOFs curved mesh mortar interfaces computation by N. Krais Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 32 / 39

Shockless explosion combustion test case Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 33 / 39

Towards Air-Breathing Hypersonic Vehicles (M. Atak) How effcient are DG methods for such class of problems? This will be the first DNS of a compressible supersonic turbulent boundary layer with DG Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 34 / 39

Shock Boundary Layer Interaction (M. Atak) DNS of a shock wave interacting with a flat plate turbulent boundary layer at M = 2.67 shock intensity p 2 /p 1 = 1.3 yields a steady state structured hexahedra mesh with 1,125,000 grid cells N = 5 243 10 6 DOF indicator: Ducros with Jameson up to 56,250 processors more than 10 6 CPU hours no mortar interfaces computation & visualization by M. Atak Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 35 / 39

Shock Boundary Layer Interaction (M. Atak) Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 36 / 39

Conclusion & Outlook Conclusion Discontinuous Galkerkin Spectral Element Method Shock Capturing with Finite Volume Sub-cells Mortar interfaces for mesh refinement Numerical examples Outlook adaptiv mesh refinement (AMR) dynamic load balancing References M. Sonntag, C.-D. Munz: Efficient Parallelization of a Shock Capturing for Discontinuous Galerkin Methods using Finite Volume Sub-cells, Journal of Scientific Computing, in Review. Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 38 / 39

Thank you for your attention! Visit flexi-project.org nrg.iag.uni-stuttgart.de

Sod Shock Tube Initial conditions: Mesh: 10 DG elements N = 11 Jameson indicator I Jameson = left right ρ 1.0 0.125 v 0 0 p 0.125 0.1 N i,j,k=0 p min,ijk 2p ijk + p max,ijk ω i ω j ω k p min,ijk + 2p ijk + p max,ijk Vol(Element), where p min,ijk is the minimal pressure of the surrounding nodes t end = 0.2 Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 40 / 39

Sod Shock Tube 1 Exact solution DG part of solution FV part of solution Density 0.5 0 0.2 0.4 0.6 0.8 1 x Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 41 / 39

Strong Shock-Vortex Interaction Mach Shock: 7.0 Mach Vortex: 1.7 Grid: h = 1/100 N = 5 FV distribution: equidistant Indicator: Jameson (with Persson on FV DG) Riemann: LF (HLLE at BC) Limiter: MinMod CFL: 0.5 Initial condition: numerical converged stationary 1D-shock Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 42 / 39

Strong Shock-Vortex Interaction Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 43 / 39

Strong Shock-Vortex Interaction Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 44 / 39

Double Mach Reflection send shock wave diagonally into a wall shock speed: Mach 10 Grid: h = 1/120 (480 x 120 cells) polynomial degree N = 5 2 10 6 DOF Indicator: Jameson (with Persson on FV DG) Riemann: HLLE Limiter: MinMod no mortar interfaces Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 45 / 39

Double Mach Reflection Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 46 / 39

Double Mach Reflection 1 0.75 0.5 0.25 0 1 0.75 0.5 0.25 0 0 0.5 1.0 1.5 2.0 2.5 3.0 3.2 0 0.5 1.0 1.5 2.0 2.5 3.0 3.2 22.5 20 16 12 8 4 1.5 Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 47 / 39

Double Mach Reflection 0.59 ρ 22.3 20 0.4 16 12 0.2 8 4 0 2.15 2.35 2.55 2.75 2.86 1.4 Matthias Sonntag (IAG) Shock Capturing for DG using FV Sub-cells SHARK-FV May 23, 2016 48 / 39