XXL-BIOMD. Large Scale Biomolecular Dynamics Simulations. onsdag, 2009 maj 13

Transcription

1 XXL-BIOMD Large Scale Biomolecular Dynamics Simulations David van der Spoel, PI Aatto Laaksonen Peter Coveney Siewert-Jan Marrink Mikael Peräkylä Uppsala, Sweden Stockholm, Sweden London, UK Groningen, Netherlands Kuopio, Finland

2 Molecular Dynamics 1. Start with a system of particles with given coordinates 2. Compute forces on each particle due to a (classical) energy function 3. Integrate the particle positions 4. Save coordinates and energies etc. 5. Go to 2. Particles!= Atoms

3 Protein BBA5: 4000 Waters: Total: 400 atoms atoms atoms

4

5 BBA5 folding: CPU Time atoms 100 interactions per atom 50 flops / interaction 2 x 10 7 integration steps 1.24 Petaflop (NOW: 1 day on 8 cores) 10,000 copies of the simulation (in 2004 using Folding@Home with GROMACS)

6 Fraction folded structures Fold Fraction over time Folding time from slope 4-5 μs 0.2 lagtime ns 10 ns 20 ns 30 ns Rhee et al. Proc. Natl. Acad. Sci. U.S.A. 101 (2004) p. 6456

7 Simulation vs. Experiments Experiments Efficient averaging Bond vibrations torsions single ion passes through a channel Where we need to be Less detail s 10 s 10 s 10 s 10 s 10 s 10 s Simulations Extreme detail Sampling issues? Parameter quality? Where we are Fast protein folding Normal protein folding Where we want to be Interesting protein folding Biologically interesting stuff

8 GROMACS The world s fastest MD code - and it s GPL! Estimated 5,000-10,000 academic and industrial users Used in Folding@Home - 250,000 CPUs (2008) Part of SPECfp & PRACE benchmark suites PRACE project for improvements on GROMACS (CSC, Erik Lindahl)

9 DPPC & Cholesterol: 130k atoms BlueGene: 6ns/day, using 2000 CPUs GROMACS only achieves 2ns/day......on a single dual dual core Opteron node!

10 Parallel Domain Decomposition Partition space, instead of atoms, over nodes Supported in version 4 of Gromacs Good for load balancing Bad for communication bandwidth Each node imports coordinate and exports forces from neighbors within a sphere with radius=cutoff (expensive) Data must be imported from whole sphere, although it can be optimized to half

11 The Eighth-Sphere Method Smarter way to communicate Don t calculate interactions on a home node, but in general on neutral territory (David Shaw) Drastically reduced communication bandwidth needs for dom. dec. 2D example to the right In 3D, we need to import data from 1/8 sphere to the central cell Red/Yellow cells send data to central (purple) cell, where interactions are calculated Implemented in GROMACS 4

12 Efficient PME Parallelization Almost all accurate simulations today use Particle-Mesh Ewald lattice summation Small 3D Fourier Transforms scale bad - all-to-all communication Direct space & PME are mostly independent, though! Y Dedicate a subset of nodes to run a separate PME-only version of the program to improve scaling X FFT over 5 instead of 25 nodes! Original implementation with help of RZG of MPI PME nodes

13 Scaling - DHFR atoms 1 fs time step Constraints on H- Bonds PME every other step (NAMD, Desmond) resp. every step (GROMACS) Hess et al. J. Comp.Theor. Chem. 4 (2008)

14 Blue Gene/P scaling M atoms 60 Coarse grained system Cut-off 2.6 sigma steps / second M atoms steps / second #cores 0 Amdahl s law: 0.43 Matoms on 1024 cores - 28% time in global sum 3.4 Matoms on 2048 cores: 33% time in global sum

15 Deisa Extreme Computing Initiative Larger systems Longer simulation times

16 Dynamics of a Virus Capsid Daniel Larsson Lars Liljas David van der Spoel Uppsala University Sweden

17 Why study viruses? New viral diseases may be on the way - Bird flu, SARS, Mexican 40 existing drugs against viruses - 20 against AIDS Most drugs target viral reproduction cycle (reverse transcriptases, proteases) Interesting features - self assembly Packaging of RNA/DNA

18 Satellite Tobacco Necrosis Virus Discovered 1967 Icosahedral plant virus Satellite virus, TNV is the helper Transmitted by a fungus Small: 18 nm diameter Simple model system

19 5 ns simulation. Blue: protein. Green: Cl -. Red Na + Computer time estimate: 12 Petaflop

20 What can simulations contribute with? Non-averaged aspects Non-symmetrical aspects Missing pieces of the structure (Res. 1-11) Effect of structural Ca 2+ on stability RNA binding / Salt effects Dynamics the fourth dimension

21

22 RNA Secondary Structure Prediction Bringloe et al. J. Gener. Virol. 79 (1998) p. 1539

23 The goal is not merely reproducing experimental results.

24 Unfortunately, reproducing experimental results is difficult.

25 Simulation Details 1,000,000+ particles OPLS/AA force field + TIP3P water Particle mesh Ewald 5 fs timestep Dodecahedron simulation box

26 Hardware Specs Louhi (CSC, Espoo) HECToR (Edinburgh) Neolith (Linköping) Vendor Cray Cray HP CPU type AMD Opteron 2.3 GHz AMD Opteron Intel Xeon E534 (2.33 GHz, 8 MB L2 cache) #Cores Interconnect Cray Cray Infiniband ConnectX OS Cray Linux Cray Linux Centos 5 Linux Top500 (11-08) Price??? Time (2008) 400 kh (DECI) 800 kh (DECI) 3600 kh (VR)

27 Superposition of 60 monomers after 100 ns MD Superposition of 12 pentamers after 100 ns MD Main part of the protein acts like a rigid body. Residue 1-24 forms a flexible arm.

28 500 ns: ~1.2 Exaflop, 30 core-years

29 Size of the virus particle

30 Water flow analysis Is the virus capsid leaky? Can water molecules pass? Can ions pass? Is the flow concentrated to specific regions?

31 Physiological Conditions

32 Conclusions N-terminal arm is flexible Ionic strength is important for stability but may not be enough. RNA! Capsid is very leaky (interfering with capsid stability is a possibility for antiviral therapy, but not proven yet)

33 Questions & future plans... What structural changes facilitate the water flow? The RNA structure - 2D to 3D structure Affinities (ΔGbind) between the proteins - quantitative stability analysis Pathways for virus assembly (simplified models?)

34 Extreme Computing If you can not solve a small problem, try a bigger problem How about accuracy of the results & the predictive power? HPC will force many codes to reconsider the physics Amdahl s law needs to be taken care of in MD

35 Molecular Biophysics, Uppsala Daniel Larsson