Accelerating incompressible fluid flow simulations on hybrid CPU/GPU systems

Transcription

1 Accelerating incompressible fluid flow simulations on hybrid CPU/GPU systems Yushan Wang 1, Marc Baboulin 1,2, Karl Rupp 3,4, Yann Fraigneau 1,5, Olivier Le Maître 1,5 1 Université Paris-Sud, France 2 INRIA, France 3 Argonne National Laboratory, USA 4 Vienna University of Technology, Austria 5 LIMSI-CNRS, France 26 th, Nov 2013

2 Laboratoire de Recherche en Informatique (LRI) 2 The main research areas addressed by ParSys include high-performance computing, distributed algorithms, compilation and code optimization. Founded more than 30 years ago, LRI has now over 260 members, including 105 faculty and staff and 90 Ph.D. students.

3 Outline 3 Introduction to Navier-Stokes equations Helmholtz-like equations Poisson equation Hybrid model and performance on a multicore + GPU architecture Conclusion and future work

4 Navier-Stokes equations 4 The incompressible NS equations are the fundamental bases of many CFD problems. A million dollars in cash awaits anyone who can develop a rigorous mathematical model for how fluids flow. -- Clay Mathematics Institute

5 Incompressible Navier-Stokes equations 5 ' V ) t + ( V V T ) = P + 1 Re Δ V ( ) * V = 0 Density is neglected because the problem is supposed to be with constant coefficient. Reynolds number Re indicates the fluid state. Larger demands finer mesh discretization. Non-linear convection term for incompressible fluid flow. ( V V T ) Re can be simplified as ( V ) V

6 6 Solving NS equations with a prediction-projection method Hodge-Helmholtz decomposition: ( V n, P n ) V * = V div=0 + φ V * φ ( V n+1, P n+1 ) (I 2Δt 3Re Δ)( V * V n ) = S Helmholtz-like equation Δφ = 3 2Δt V * Poisson equation V n+1 = V * 2Δt 3 φ P n+1 = P n +φ 1 Re V * u Time increments Y. Wang, M. Baboulin, J. Dongarra, J. Falcou, Y. Fraigneau, O. Le Maître A Parallel Solver for Incompressible Fluid Flows. ICCS 2013:

7 Solving Helmholtz-like equation 7 (I 2Δt 3Re Δ)( V * i V n i ) = S i i {x, y, z} Alternating Direction Implicit method: System of 3 Helmholtz-like equations: Thomas algorithm (I αδ) = (I αδ x )(I αδ y )(I αδ z )+ο(α 2 ) $ & & & % & & & ' (I 2Δt 3Re Δ x ) T # = S i (I 2Δt 3Re Δ y) T ## = T# (I 2Δt 3Re Δ z)( V * i V n i ) = T## Matrix transpose: reordering Tridiagonal systems

8 Solving Helmholtz-like equation 8 Bx = f. B is a tridiagonal block matrix. For certain cases, with multiple RHS. Bx = f Two methods available: B = [a i, b i, c i ] i=1,...,m can by considered as a smaller system Thomas algorithm: Gaussian elimination without pivoting. Explicit inverse of B è x = B 1 f. " ( 1) i+ j c i c i+1...c j 1 θ i 1 φ j+1 /θ n, i > j, $ B 1 ij = # θ i 1 φ i+1 /θ n, i = j, $ ( 1) i+ j a j+1 a j+2...a i θ j 1 φ i+1 /θ n, % $ i > j. θ 0 =1, θ 1 = b 1, θ i = b i θ i 1 c i 1 a i θ i 2, i = 2,..., m, φ m+1 =1, φ m = b m, φ i = b i φ i+1 c i a i+1 φ i+2, i = m 1,...,1,

9 Solving Helmholtz-like equation 9 #% u(x) αδu(x) = f (x), $ &% u(x) = 0, x Ω = (0,1) 3, x Ω, f (x) = (1+ 3απ 2 )u(x), α =10 7, u(x) = sin(π x 1 )sin(π x 2 )sin(π x 3 ). Using the explicit inverse of B gains a factor of 4 over the Thomas algorithm, while the accuracy is the same. However, the application of the explicit inverse is limited: only problems with no immerged body, and with same boundary conditions, etc.

10 Solving Poisson equation 10 Δφ = 3 2Δt V * = S Partial diagonalization: Δ x = Q x Λ x Q x 1 Δ y = Q y Λ y Q y 1 S $ = Q 1 x Q 1 y S φ $ = Q 1 1 x Q y φ % ' ' & ' ' (' Δ = Δ x + Δ y + Δ z (Λ x + Λ y + Δ z ) φ # = S# Tridiagonal system Thomas algorithm Matrix transpose: reordering Most time-consuming part is the matrix-matrix multiplication. Using GPU to accelerate. MAGMA: Matrix Algebra on GPU and Multicore Architectures.

11 CPU vs. GPU on Helmholtz and Poisson problems 11 Problem size = Helmholtz (with B -1 ) Poisson Transfer CPU è GPU Matrix multiplication Solution reordering Tridiagonal system solve Total GPU solver Total CPU solver * (12 MPI processes) Acceleration x4.3 x4.2 2 Inter Xeon E5645 è 12 cores in total. 2 NVIDIA Tesla C2075.

12 SUNFLUIDH 12 Navier-Stokes solver developed at LIMSI (Laboratoire d Informatique pour la Mécanique et les Sciences de l Ingénieur) 3D simulation of unsteady incompressible flow or low Mach number flow. Forced convection flow Thermal convection flow Multispecies flow Reactive flow The base frame of our current work. More information on

13 Hybrid model of our NS solver 13 Domain is divided equally into subdomains. MPI One subdomain corresponds to one MPI process. OpenMP thread Multicore processor + GPU thread thread Each process is associated to one GPU acceleration. Multi-threading techniques are applied within each subdomain.

14 Performance results 14 2 MPI processes. Each process is an hexa-core processor. Up to 6 threads per process (12 threads in total). p Problem size = p p About 50% of the computational work is done by GPU. Multithreading is not yet fully developed.

15 Conclusion and future work 15 A hybrid multi-core GPU Navier-Stokes solver which includes the solution of the Helmholtz-like and Poisson equations. Significant acceleration by taking advantage of GPU devices. More computational work to be transferred on GPU. Construction of the tridiagonal systems. Computation of convection flux, diffusion flux, etc.. Multi-threading implementation to be ameliorated. Using PETSc iterative solver when direct solver is not available. Larger scale simulations.

16 16