Scalable, performant, and resilient large-scale applications of molecular process engineering

Size: px

Start display at page:

Download "Scalable, performant, and resilient large-scale applications of molecular process engineering"

Lorena Nicholson
6 years ago
Views:

1 Scalable, performant, and resilient large-scale applications of molecular process engineering M. Horsch,1 P. Gralka,2 C. Niethammer,3 N. Tchipev,4 J. Vrabec,5 H. Hasse1 1 University of Kaiserslautern, Laboratory of Engineering Thermodynamics, University of Stuttgart, Visualization Research Centre, 3 High Performance Computing Centre Stuttgart, 4 TU München, Scientific Computing in Computer Science, 5 University of Paderborn, Thermodynamics and Energy Technology 2 Leibniz Supercomputing Centre, Garching SuperMUC Status Workshop,

Molecular Process Engineering From Physics (qualitative accuracy) Physically realistic modelling of intermolecular interactions Separate contributions due to repulsive and dispersive as well as

2 Molecular Process Engineering From Physics (qualitative accuracy) Physically realistic modelling of intermolecular interactions Separate contributions due to repulsive and dispersive as well as electrostatic interactions To Engineering (quantitative reliability) No blind fitting, but parameters of effective pair potentials are adjusted to experimental data Physical realism facilitates reliable interpolation and extrapolation 2

potential: Size and energy parameters Electrostatics Point

3 Pair potentials for low-molecular fluids Geometry Rigid bond lengths and angles Dispersion and repulsion Lennard-Jones potential: Size and energy parameters Electrostatics Point polarities (charge, dipole, quadrupole): Position and magnitude 3

4 Scalable MD simulation: SkaSim Methods for heterogeneous or fluctuating particle distributions 4

Scalable MD simulation with ls1 mardyn Methods for heterogeneous or fluctuating particle distributions Linked-cell data structure suitable for

5 Scalable MD simulation with ls1 mardyn Methods for heterogeneous or fluctuating particle distributions Linked-cell data structure suitable for spatial domain decomposition (non-blocking, overlapping MPI send/ receive operations) large systems 1 : molecular dynamics 5

Traversal of the linked cells Cells enter and leave a «sliding window» as they are touched for the first or the last time during each time step: Optionally,

6 Traversal of the linked cells Cells enter and leave a «sliding window» as they are touched for the first or the last time during each time step: Optionally, forces acting on molecules are only stored until their cell leaves the sliding window. sliding window large systems 1 : molecular dynamics 6

Traversal of the linked cells Hyperthreading and vectorization: Optionally, forces acting on molecules are only stored until their cell leaves the sliding window.

7 Traversal of the linked cells Hyperthreading and vectorization: Optionally, forces acting on molecules are only stored until their cell leaves the sliding window. hyperthreaded sliding window Efficient vectorization: Optimization by hand, using advanced vector extensions (AVX). Conversion from array of structures (AoS) to structure of arrays (SoA). 7

8 Strong scaling of ls1 mardyn on SuperMUC speedup (from 128 cores) Scaling of ls1 mardyn examined on SuperMUC up to cores homogeneous LJTS liquid with 4.8 billion molecules id 8192 r on t s l ea ing l a c gs str d e v r obse alin c s g on g number of cores 8

9 speedup Weak scaling of ls1 mardyn on SuperMUC weak scaling with 31.5 million molecules per core number of cores 9

MD world record simulation on SuperMUC speedup Up to N = 4 1012 on SuperMUC weak scaling with 31.

10 MD world record simulation on SuperMUC speedup Up to N = on SuperMUC weak scaling with 31.5 million molecules per core PRACE ISC Prize number of cores MD world record achieved from simulations of a homogeneous LJTS liquid. 10

11 Large-scale production simulations Technically / scientifically relevant simulations of large systems always deal with heterogeneous systems (e.g. at vapour-liquid interfaces). 11

12 Large-scale production simulations homogeneous cavitation CO2 (T = 280 K and ρ = 17.2 mol/l), 3CLJQ 100 million interaction sites, cores 12

13 MD simulation of homogeneous cavitation Canonical MD simulation of cavitation in carbon dioxide. homogeneous cavitation Evaluation of local density at 180 x 180 x 180 grid points: 5? (6 mol/l) Liquid phase detected for more than 5 neighbors within a radius of 6.9 Å around the grid point. liquid carbon dioxide, 220 K, 23.9 mol/l 13

14 Long-range correction for planar interfaces Example: Vapour-liquid interfaces. short range (explicit) long range (correction) Long-range correction from the density profile, following Janeček. cu f ra tof s diu Full evaluation of all pairwise interactions is too expensive instead, short-range interactions are evaluated for neighbours. 14

15 Long-range correction for planar interfaces Two-centre LJ fluid (2CLJ) Long-range correction from the density profile, following Janeček. Angle-averaging expression for multi-site models, following Lustig. surface tension / εσ -2 Example: Vapour-liquid interfaces. 1 nm T = ε Janeček-Lustig term no angle averaging no correction at all cutoff radius / σ Computationally efficient correction scheme for polarity and dispersion. 15

16 Computing project pr83ri ( ) Simulation of capillary waves and fluctuations of liquid droplets LJTS T = 0.8 ε θpl = 90 16

17 Computing project pr48te ( ) Scalable, performant, and resilient large-scale applications of molecular process engineering (Sparlampe) Approved in March 2016 Increased resiliency by data reduction and reconstruction from local radial distribution functions Efficient long-range corrections for spherical and arbitrary geometries 17

Large-scale MD simulation of heterogeneous systems with ls1 mardyn

Large-scale MD simulation of heterogeneous systems with ls1 mardyn M. T. Horsch, R. Srivastava, S. J. Werth, C. Niethammer, C. W. Glass, W. Eckhardt, A. Heinecke, N. Tchipev, H.-J. Bungartz, S. Eckelsbach,