March 4-5, 2015 Performance comparison between hybridizable DG and classical DG methods for elastic waves simulation in harmonic domain M. Bonnasse-Gahot 1,2, H. Calandra 3, J. Diaz 1 and S. Lanteri 2 1 INRIA Bordeaux-Sud-Ouest, team-project Magique 3D 2 INRIA Sophia-Antipolis-Méditerranée, team-project Nachos 3 TOTAL Exploration-Production
Motivation Examples of seismic applications M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-2/28
Motivation Imaging method : the full wave inversion Iterative procedure Inverse problem requiring to solve a lot of forward problems M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-3/28
Motivation Imaging method : the full wave inversion Iterative procedure Inverse problem requiring to solve a lot of forward problems Seismic imaging : time-domain or harmonic-domain? Time-domain : imaging condition complicated but low computational cost Harmonic-domain : imaging condition simple but huge computational cost M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-3/28
Motivation Imaging method : the full wave inversion Iterative procedure Inverse problem requiring to solve a lot of forward problems Seismic imaging : time-domain or harmonic-domain? Time-domain : imaging condition complicated but low computational cost Harmonic-domain : imaging condition simple but huge computational cost Forward problem of the inversion process Elastic wave propagation in harmonic domain : Helmholtz equation Reduction of the size of the linear system M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-3/28
Motivation Seismic imaging in heterogeneous complex media Complex topography High heterogeneities M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-4/28
Motivation Seismic imaging in heterogeneous complex media Complex topography High heterogeneities Use of unstructured meshes with FE methods M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-4/28
Motivation Seismic imaging in heterogeneous complex media Complex topography High heterogeneities Use of unstructured meshes with FE methods DG method Flexible choice of interpolation orders (p adaptativity) Highly parallelizable method Increased computational cost as compared to classical FEM M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-4/28
Motivation Seismic imaging in heterogeneous complex media Complex topography High heterogeneities Use of unstructured meshes with FE methods DG method Flexible choice of interpolation orders (p adaptativity) Highly parallelizable method Increased computational cost as compared to classical FEM DOF of classical FEM DOF of DGM M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-4/28
Motivation Objective of this work Development of an hybridizable DG (HDG) method Comparison with a reference method : a standard nodal DG method Figure : Degrees of freedom of DGM Figure : Degrees of freedom of HDGM M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-5/28
HDG methods HDG methods B. Cockburn, J. Gopalakrishnan, R. Lazarov Unified hybridization of discontinuous Galerkin, mixed and continuous Galerkin methods for second order elliptic problems, SIAM Journal on Numerical Analysis, Vol. 47 (2009) S. Lanteri, L. Li, R. Perrussel, Numerical investigation of a high order hybridizable discontinuous Galerkin method for 2d time-harmonic Maxwell s equations, COMPEL, Vol. 32 (2013) (time-harmonic domain) N.C. Nguyen, J. Peraire, B. Cockburn, High-order implicit hybridizable discontinuous Galerkin methods for acoustics and elastodynamics, J. of Comput. Physics, Vol. 230 (2011) (time domain for seismic applications) M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-6/28
2D Helmholtz equations Contents 2D Helmholtz elastic equations Notations and definitions Hybridizable Discontinuous Galerkin method Numerical results Conclusions-Perspectives M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-6/28
2D Helmholtz equations 2D Helmholtz elastic equations First order formulation of Helmholtz wave equations x = (x, y) Ω R 2, { iωρ(x)v(x) = σ(x) + fs (x) iωσ(x) = C(x) ε(v(x)) Free surface condition : σn = 0 on Γ l Absorbing boundary condition : σn = v p (v n)n + v s (v t)t on Γ a v : velocity vector σ : stress tensor ε : strain tensor M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-7/28
2D Helmholtz equations 2D Helmholtz elastic equations First order formulation of Helmholtz wave equations x = (x, y) Ω R 2, { iωρ(x)v(x) = σ(x) + fs (x) iωσ(x) = C(x) ε(v(x)) Free surface condition : σn = 0 on Γ l Absorbing boundary condition : σn = v p (v n)n + v s (v t)t on Γ a ρ : mass density C : tensor of elasticity coefficients v p : P-wave velocity v s : S-wave velocity f s : source term, f s L 2 (Ω) M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-7/28
Definitions Contents 2D Helmholtz elastic equations Notations and definitions Hybridizable Discontinuous Galerkin method Numerical results Conclusions-Perspectives M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-7/28
Definitions Notations and definitions Notations T h mesh of Ω composed of triangles M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-8/28
Definitions Notations and definitions Notations T h mesh of Ω composed of triangles F h set of all faces F of T h F M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-8/28
Definitions Notations and definitions Notations T h mesh of Ω composed of triangles F h set of all faces F of T h n the normal outward vector of an element n M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-8/28
Definitions Notations and definitions Approximations spaces P p () set of polynomials of degree at most p on M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-9/28
Definitions Notations and definitions Approximations spaces P p () set of polynomials of degree at most p on V p h = {v ( L 2 (Ω) ) 2 : v V p () = (P p ()) 2, T h } M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-9/28
Definitions Notations and definitions Approximations spaces P p () set of polynomials of degree at most p on V p h = {v ( L 2 (Ω) ) 2 : v V p () = (P p ()) 2, T h } Σ p h = {σ ( L 2 (Ω) ) 3 : σ Σ p () = (P p ()) 3, T h } M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-9/28
Definitions Notations and definitions Approximations spaces P p () set of polynomials of degree at most p on V p h = {v ( L 2 (Ω) ) 2 : v V p () = (P p ()) 2, T h } Σ p h = {σ ( L 2 (Ω) ) 3 : σ Σ p () = (P p ()) 3, T h } M h = {η ( L 2 (F h ) ) 2 : η F (P p (F )) 2, F F h } M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-9/28
Definitions Notations and definitions Definitions Jump [[ ]] of a vector v through F : [[v]] = v + n + + v n = v + n + v n + Jump of a tensor σ through F : [[σ]] = σ + n + + σ n = σ + n + σ n + + n n + M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-10/28
HDG method Contents 2D Helmholtz elastic equations Notations and definitions Hybridizable Discontinuous Galerkin method Formulation Algorithm Numerical results Conclusions-Perspectives M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-10/28
HDG method Formulation HDG formulation of the equations Local HDG formulation { iωρv σ = 0 iωσ Cε (v) = 0 M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-11/28
HDG method Formulation HDG formulation of the equations Local HDG formulation iωρ v w + iωσ : ξ + v σ : w ( C ξ σ n w = 0 ) v C ξ n = 0 σ and v are numerical traces of σ and v respectively on M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-11/28
HDG method Formulation HDG formulation of the equations Local HDG formulation We define : iωρ v w + iωσ : ξ + v σ : w ( C ξ σ n w = 0 ) v C ξ n = 0 v F = λ F, F F h, σ n = σ n τi ( v λ ), on where τ is the stabilization parameter (τ > 0) M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-11/28
HDG method Formulation HDG formulation of the equations Local HDG formulation We replace v and ( σ n ) by their definitions into the local equations iωρ v w + σ : w σ n w ( ) + τi v λ w = 0 ( ) iωσ : ξ + v C ξ λ C ξ n = 0 M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-12/28
HDG method Formulation HDG formulation of the equations Local HDG formulation iωρ v w iωσ : ξ + v ( σ ) w + ( ) C ξ τi ( v λ ) w = 0 λ C ξ n = 0 M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-13/28
HDG method Formulation HDG formulation of the equations Local HDG formulation iωρ v w iωσ : ξ + v ( σ ) w + ( ) C ξ τi ( v λ ) w = 0 λ C ξ n = 0 We define : W = ( v x, v z, σ xx, σ zz, σ ) T xz Λ = ( ) T Λ F1, Λ F2,..., Λ Fn f, where nf = card(f h ) Discretization of the local HDG formulation A W + C Λ = 0 M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-13/28
HDG method Formulation HDG formulation of the equations Transmission condition In order to determine λ, the continuity of the normal component of σ is weakly enforced, rendering this numerical trace conservative : [[ σ n]] η = 0 F M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-14/28
HDG method Formulation HDG formulation of the equations Transmission condition In order to determine λ, the continuity of the normal component of σ is weakly enforced, rendering this numerical trace conservative : [[ σ n]] η = 0 F Replacing ( σ n ) and summing over all faces, the transmission condition becomes : ( ) σ n η ( ) τi v λ η = 0 T h T h M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-14/28
HDG method Formulation HDG formulation of the equations Transmission condition In order to determine λ, the continuity of the normal component of σ is weakly enforced, rendering this numerical trace conservative : [[ σ n]] η = 0 F Replacing ( σ n ) and summing over all faces, the transmission condition becomes : ( ) σ n η ( ) τi v λ η = 0 T h T h Discretization of the transmission condition [ B W + L Λ ] = 0 T h M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-14/28
HDG method Formulation HDG formulation of the equations Global HDG formulation iωρ v w iωσ : ξ + T h ( σ ) w + v ( σ n ) η T h ( ) C ξ τi ( v λ ) w = 0 λ C ξ n = 0 τi ( v λ ) η = 0 M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-15/28
HDG method Formulation HDG formulation of the equations Global HDG formulation iωρ v w iωσ : ξ + T h ( σ ) w + v ( σ n ) η T h ( ) C ξ τi ( v λ ) w = 0 λ C ξ n = 0 τi ( v λ ) η = 0 Global HDG discretization A W + C Λ = 0 [ B W + L Λ ] = 0 T h M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-15/28
HDG method Formulation HDG formulation of the equations Global HDG formulation iωρ v w iωσ : ξ + T h ( σ ) w + v ( σ n ) η T h ( ) C ξ τi ( v λ ) w = 0 λ C ξ n = 0 τi ( v λ ) η = 0 Global HDG discretization W = (A ) 1 C Λ [ B W + L Λ ] = 0 T h M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-15/28
HDG method Formulation HDG formulation of the equations Global HDG formulation iωρ v w iωσ : ξ + T h ( σ ) w + v ( σ n ) η T h ( ) C ξ τi ( v λ ) w = 0 λ C ξ n = 0 τi ( v λ ) η = 0 Global HDG discretization [ B (A ) 1 C + L ] Λ = 0 T h M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-15/28
HDG method Algorithm Main idea of the algorithm using the HDG formulation 1. Construction of the linear system MΛ = 0 with M = T h [ B (A ) 1 C + L ] for = 1 to Nb tri do Compute matrices B, (A ) 1, C and L Construction of the corresponding section of end for M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-16/28
HDG method Algorithm Main idea of the algorithm using the HDG formulation 1. Construction of the linear system MΛ = 0 2. Construction of the right hand side S M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-16/28
HDG method Algorithm Main idea of the algorithm using the HDG formulation 1. Construction of the linear system MΛ = 0 2. Construction of the right hand side S 3. Resolution MΛ = S M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-16/28
HDG method Algorithm Main idea of the algorithm using the HDG formulation 1. Construction of the linear system MΛ = 0 2. Construction of the right hand side S 3. Resolution MΛ = S 4. Computation of the solutions of the initial problem for = 1 to Nb tri do Compute W = (A ) 1 C Λ end for M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-16/28
Numerical results Contents 2D Helmholtz elastic equations Notations and definitions Hybridizable Discontinuous Galerkin method Numerical results Plane wave in an homogeneous medium Marmousi test-case Conclusions-Perspectives M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-16/28
Numerical results Plane wave in an homogeneous medium Plane wave 10000 m Physical parameters : ρ = 2000kg.m 3 λ = 16GPa µ = 8GPa Plane wave : i(k cos θx+k sin θy) u = e 10000 m Computational domain Ω setting where k = ω v p θ = 0 Three meshes : 3000 elements 10000 elements 45000 elements M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-17/28
Numerical results Plane wave in an homogeneous medium Plane wave 20 15 2.3 1 W a W e 10 5 3.4 4.0 5.4 1 1 1 P 1 P 2 P 3 P 4 0 5 5 5.5 6 6.5 h max Convergence order of the HDG scheme M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-18/28
Numerical results Marmousi test-case Marmousi test-case Computational domain Ω composed of 235000 triangles M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-19/28
Numerical results Marmousi test-case Parallel results for the Marmousi test-case with the HDG-P3 scheme, f = 2Hz M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-20/28
Numerical results Marmousi test-case Caracteristics of the computing processors used Plafrim platform Hardware specification : 16 nodes, 12 cores by nodes Caracteristics of computing nodes : 2 Hexa-core Westmere Intel R Xeon R X5670 Frequency : 2,93 GHz Cache L3 : 12 Mo RAM : 96 Go Infiniband DDR : 20Gb/s Ethernet : 1Gb/s M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-21/28
Numerical results Marmousi test-case Efficiency of the parallelism of the global matrix construction 1 Efficiency 0.8 0.6 0.4 HDGm IPDGm 0.2 0 1 2 4 8 16 32 64 96 Nb proc M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-22/28
Numerical results Marmousi test-case Efficiency of the parallelism of the whole algorithm 1 Efficiency 0.8 0.6 0.4 HDGm IPDGm 0.2 0 1 2 4 8 16 32 64 96 Nb proc M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-23/28
Numerical results Marmousi test-case Speed up for the global matrix construction 5 TimeIPDG/TimeHDG 4 3 2 1 0 1 2 4 8 16 32 64 96 Nb proc M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-24/28
Numerical results Marmousi test-case Speed up (Total simulation time) 10 TimeIPDG/TimeHDG 8 6 4 2 0 1 2 4 8 16 32 64 96 Nb proc M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-25/28
Numerical results Marmousi test-case Memory required (GB) for the simulation 120 Memory (GB) 100 80 60 40 20 HDGm IPDGm 1 2 4 8 16 32 64 96 Nb proc M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-26/28
Conclusion Contents 2D Helmholtz elastic equations Notations and definitions Hybridizable Discontinuous Galerkin method Numerical results Conclusions-Perspectives M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-26/28
Conclusion Conclusions-Perspectives Conclusions On a same mesh, with the HDG method : Memory gain Computational time gain M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-27/28
Conclusion Conclusions-Perspectives Conclusions On a same mesh, with the HDG method : Memory gain Computational time gain Perspectives Develop 3D HDG formulation for Helmholtz equations Solution strategy for the HDG linear system M. Bonnasse-Gahot - DG and HDG methods for Helmholtz wave equations March 5, 2015-27/28
Conclusion Thank you!