Monte Carlo strategies for first-principles simulations of elemental systems

Size: px

Start display at page:

Download "Monte Carlo strategies for first-principles simulations of elemental systems"

Stella Griffith
6 years ago
Views:

1 Monte Carlo strategies for first-principles simulations of elemental systems Lev Gelb Department of Materials Science and Engineering, University of Texas at Dallas XSEDE12 Lev Gelb (UT Dallas) Monte Carlo Strategies XSEDE12 1 / 28

2 Introduction Atomistic simulations Require a potential, which gives the energy of a system as a function of atomic coordinates. Classical (empirical) V (r N ) = i V 1 (r i ) + i>j V 2 ( rij ) + i>j>k V 3 ( rij, r jk, r ik ) +... Quantum-mechanical (semi-empirical or first-principles ) Wavefunction-based methods H ψ = E ψ Density Functional Theory (DFT) E = F[ρ] only real choice for condensed-phase systems. Lev Gelb (UT Dallas) Monte Carlo Strategies XSEDE12 2 / 28

3 Introduction First-principles simulations of phase behavior First-principles energies are desirable when chemical interactions are significant, rather than physical ones: high-pressure melting of Si 1, Fe 2, Cu 3, MgO 4, Mo 5, C 6 others... High-pressure solid-solid transitions (minerology) Phosphorous LLE 7 Metallic hydrogen by QMC 8 Water (by Monte Carlo 9 and CPMD 10 ) 1 Sugino and Car, PRL 74 (1995) Alfè et al., Nature 401 (1999) Vočadlo et al., JCP 120 (2004) Alfè, PRL 94 (2005) Belonoshko et al., PRL 92 (2004) Wang et al., PRL 95 (2005) Ghiringhelli and Meijer, JCP 122 (2005) Delaney et al., PRL 97 (2006) McGrath et al., Comp. Phys. Comm 169 (2005) 289; Kuo et al. JCPB 108 (2004) 12990; McGrath et al., ChemPhysChem 6 (2005) Many. Lev Gelb (UT Dallas) Monte Carlo Strategies XSEDE12 3 / 28

Introduction Monte Carlo basics Starting at some initial state i... 1 Generate a new trial state j. For instance: Displace one atom Insert one atom Change one atom to a different element Many others!

4 Introduction Monte Carlo basics Starting at some initial state i... 1 Generate a new trial state j. For instance: Displace one atom Insert one atom Change one atom to a different element Many others! 2 Evaluate E = E j E i 3 Accept or reject the new state according to π j /π i = e β E, etc. 4 Repeat 1-3 many times. For few-body empirical potentials, evaluating E is an O(N) operation. For many-body potentials, its O(N 2 ) or greater. Lev Gelb (UT Dallas) Monte Carlo Strategies XSEDE12 4 / 28

5 Introduction Molecular dynamics vs Monte Carlo Most phase coexistence calculations with first-principles potentials are done using molecular dynamics. Direct simulation of a two-phase system Thermodynamic integration and reference systems Most phase coexistence calculations with empirical potentials are done using Monte Carlo. Open system simulations - direct access to µ Multiple-cell (Gibbs Ensemble) simulations Lev Gelb (UT Dallas) Monte Carlo Strategies XSEDE12 5 / 28

6 Introduction Molecular dynamics vs Monte Carlo Most phase coexistence calculations with first-principles potentials are done using molecular dynamics. Direct simulation of a two-phase system Thermodynamic integration and reference systems Most phase coexistence calculations with empirical potentials are done using Monte Carlo. Open system simulations - direct access to µ Multiple-cell (Gibbs Ensemble) simulations In molecular dynamics, all the atoms are moved at once. In Monte Carlo, atoms are (usually) moved a few at a time. Lev Gelb (UT Dallas) Monte Carlo Strategies XSEDE12 5 / 28

7 Introduction Molecular dynamics vs Monte Carlo Most phase coexistence calculations with first-principles potentials are done using molecular dynamics. Direct simulation of a two-phase system Thermodynamic integration and reference systems Most phase coexistence calculations with empirical potentials are done using Monte Carlo. Open system simulations - direct access to µ Multiple-cell (Gibbs Ensemble) simulations In molecular dynamics, all the atoms are moved at once. In Monte Carlo, atoms are (usually) moved a few at a time. For many-body potentials, this is extremely expensive. Lev Gelb (UT Dallas) Monte Carlo Strategies XSEDE12 5 / 28

8 Introduction Benchmark DFT calculations Scale of the problem Determining liquid-state properties and/or phase coexistence requires: 1 N particles (atoms or molecules) (as few as 30 atoms in some cases.) 2 Simulating for a sufficient duration: Molecular dynamics: 10 ps up to many ns. ( steps) Monte Carlo: O(10 4 ) trial moves per particle. These are serial processes: because equilibration may take significant time, many very short trajectories are not equivalent to one long one. Lev Gelb (UT Dallas) Monte Carlo Strategies XSEDE12 6 / 28

9 Introduction Benchmark DFT calculations Cubic scaling with system size Most of our calculations are performed using NWChem, a well-parallelized and widely-used code from PNL 11. Benchmark calculations: startup SCF cycles, revpbe XC functional with TM pseudopotentials, Γ-point only. Calculations run on crystalline Aluminum with random vacancies. 11 Valiev et al., Comp. Phys. Comm. 181 (2010) Lev Gelb (UT Dallas) Monte Carlo Strategies XSEDE12 7 / 28

10 Introduction Benchmark DFT calculations Parallel efficiency on Ranger Good scaling can be obtained up to 512 processors, but only for sufficiently large problems. Lev Gelb (UT Dallas) Monte Carlo Strategies XSEDE12 8 / 28

11 Introduction Benchmark DFT calculations Time to self-consistent solution The time required to converge the DFT energy to a specified tolerance can vary significant from one configuration to the next. This seriously impacts efficiency of any method where multiple energy evaluations are performed in parallel. Lev Gelb (UT Dallas) Monte Carlo Strategies XSEDE12 9 / 28

12 Introduction Benchmark DFT calculations Time to self-consistent solution (2) If the number of atoms fluctuates (as in an open ensemble) the variation is larger: Lev Gelb (UT Dallas) Monte Carlo Strategies XSEDE12 10 / 28

13 Introduction Benchmark DFT calculations Benchmark summary Computational cost scales as N 3. Good parallel scaling obtained, but only for relatively large N/P The total wall-clock time per calculation will not be reduced much below O(1 min). Time to SCF solution varies substantially even between configurations with similar density and N. Multiple simultaneous evaluations present a load-balancing problem. (Not shown) Computational cost can vary dramatically between different XC functionals. Lev Gelb (UT Dallas) Monte Carlo Strategies XSEDE12 11 / 28

14 Presampling Code and early work Pre-sampling with an approximate potential 12 Use an approximate potential, E a, to generate large moves with a high probability of acceptance by the correct potential E c. One cycle: 1 For I = 1, M 1 Generate displacement (or particle exchange, etc...) 2 Accept or reject based on e β Ea (for canonical ensemble) 2 Accept or reject the final state after M moves on: e β( Ec Ea) ; generally, α ij = min ( π c j πi a πi c πj a If M N = N particles, then this requires only one E c evaluation per cycle, instead of N., 1 ) 12 Akhmatskaya et al., CPL 267 (1997) 105; Liu and Chen J. Am. Stat. Assoc., 93 (1998) 1032; Iftimie et al., JCP 113 (2000) 4852; Liu, Monte Carlo Strategies for Scientific Computing (2001) Springer; Hetényi et al., JCP 117 (2002) 8203; Gelb, JCP 118 (2003) 7747; Zwin and Luo, JCP 123 (2005) Lev Gelb (UT Dallas) Monte Carlo Strategies XSEDE12 12 / 28

15 Presampling Code and early work Code design and implementation Capabilities: 1 NVT, µvt, NPT, NhT, NVE, NPH, GEMC, binary-mixture GEMC, NPH-GEMC, and three-cell NVE-GEMC simulations 2 Parallel tempering 3 Use of external electronic structure codes unmodified 4 Approximate-potential optimization Architecture choices: Written in Python and pypar Extensive use of Numpy, Scipy, Multiprocessing modules Expensive routines in C/C++ (through Weave) Built-in architecture aware processor scheduler Interface with external codes through standard I/O and text files Current code: NWChem and CPMD integrated as drivers, and all basic functionality met in lines. Lev Gelb (UT Dallas) Monte Carlo Strategies XSEDE12 13 / 28

16 Presampling Code and early work Some early work: Lithium NPT Consistently high densities Large XC functional dependence Some cutoff dependence Little system-size dependence Poor parallel tempering efficiency Gelb and Carnahan, CPL 417 (2006) Lev Gelb (UT Dallas) Monte Carlo Strategies XSEDE12 14 / 28

17 Presampling Code and early work Some early work: Lithium GEMC Lev Gelb (UT Dallas) Monte Carlo Strategies XSEDE12 15 / 28

18 Presampling Efficiency Pre-sampling efficiency analysis For cycles of M moves: Cost of N moves with the correct potential: Nw c Cost of N moves using presampling: (N/M)w c + Nw a, Speedup is then: S = 1 w a /w c + 1/M M However, this doesn t address efficiency! If E c E a grows linearly with M, S = 1 w a /w c + 1/M e σm Acceptance probability drops exponentially with presampling chain length. Lev Gelb (UT Dallas) Monte Carlo Strategies XSEDE12 16 / 28

19 Presampling Efficiency Proxy system: embedded-atom potentials Generic form (Daw and Baskes, 1983): V = G i (ρ i ) + ( ) V 2 rij i i>j ρ i = j ρ a j (r ij) G i is the embedding energy for placing atom i into the electron density provided by all the other atoms. ρ a j (r ij) is the spherically-averaged electron density from atom j at position r i. Sutton-Chen potential (Sutton and Chen, 1990): V /ɛ = i j>i ( a r ij ) n c ρi i ρ i = j i ( a ) m r ij Lev Gelb (UT Dallas) Monte Carlo Strategies XSEDE12 17 / 28

20 Presampling Efficiency Pre-sampling performance with cycle length Test system: qsc simulations of liquid copper, with different potential truncations ( correct is nm.) Speedups as predicted in this instance. Lev Gelb (UT Dallas) Monte Carlo Strategies XSEDE12 18 / 28

21 Presampling Three-stage presampling Three-stage sampling Introduce a third, middle -level energy evaluation: E a. Modified algorithm: 1 Every cycle of M moves, accept/reject the final state by: E 1 = E a E a 2 Every M cycles of M moves, accept/reject the final state by: E 2 = E c E a Application: E a : empirical potential E a : first-principles potential, low-quality E c : first-principles potential, high-quality Lev Gelb (UT Dallas) Monte Carlo Strategies XSEDE12 19 / 28

22 Presampling Three-stage presampling Low-cutoff DFT as a presampler Test: 50 random (but nonoverlapping) configurations of Al with N = 30. Densities between 0.01 Å 3 and 0.05 Å 3. Take correct potential as E = 20.0 H cutoff. Lev Gelb (UT Dallas) Monte Carlo Strategies XSEDE12 20 / 28

23 Presampling Three-stage presampling Low-cutoff DFT as a presampler Compare differences between consecutive configurations. Absolute performance is less critical for E c E a. E c will be much smaller in practice (Configurations will be strongly correlated.) Lev Gelb (UT Dallas) Monte Carlo Strategies XSEDE12 21 / 28

24 Presampling Three-stage presampling Three-stage pre-sampling scaling analysis In one overall cycle of M M moves, do: 1 E c evaluation M E a evaluations M M E a evaluations. S = If the cost w a is again negligible, we get [ 1 M M + 1 w a + M ] 1 w a M w c M w c [ ] 1 S = M w a /w c + 1/M [ ] wc M w a Overall sampling quality improved by a factor of w c /w a, ignoring acceptance rate issues. Lev Gelb (UT Dallas) Monte Carlo Strategies XSEDE12 22 / 28

25 Presampling Three-stage presampling Example application Liquid Li at 1500 K, 1 bar; LDA/TM theory. Total simulation time for each run: 140 hours. Acceptance frequency at top stage with M = 10: 95%. The three-stage sampling is twice as efficient! Lev Gelb (UT Dallas) Monte Carlo Strategies XSEDE12 23 / 28

26 Coarse-grained parallelism Tree sampling Generate X new configurations in parallel, and try them one at a time; if one is accepted, discard the rest. The probability of accepting a move is Inefficiency (states never considered): I = X j=1 P tree = 1 (1 P acc) X X(X j)p acc(1 P acc) j 1 Constant P acc Constant X P acc X P tree I/X P acc X P tree I/X For 4 work, get 3 the throughput (moves accepted per wallclock time.) Lev Gelb (UT Dallas) Monte Carlo Strategies XSEDE12 24 / 28

27 Coarse-grained parallelism Tree sampling performance This works as advertised, provided that: We use a three-stage scheme, where the middle-level potential is DFT run for a fixed number of SCF iterations, rather than to a specified convergence. Test case: 32 Al atoms at 3000 K and 1 kbar. E a : quantum-corrected Sutton-Chen potential E a : revpbe, 10.0 H, max 50 SCF cycles. E c : revpbe, 10.0 H, converged M % accepted (P acc ) 5 63% 10 57% 20 40% 40 22% Also, > 95% acceptance on E a /E c (M = 5). counts run time (seconds) Lev Gelb (UT Dallas) Monte Carlo Strategies XSEDE12 25 / 28

28 Coarse-grained parallelism Tree sampling performance (2) There are now many factors influencing simulation efficiency: M, low-level cycle-length Dynamically adjustable for target P acc. M, mid-level cycle-length Number of SCF iterations at middle level; determines P acc. X, number of tree branches Trade-off with more processors per DFT evaluation. 2nd-generation trees? Target higher P acc, because need to accept either y 1 or y 2 for good efficiency! For P acc = 0.5, accept one move 75% of the time, and two 56% of the time. y 2 y 21 x y 1 y 11 y 12 y 22 Lev Gelb (UT Dallas) Monte Carlo Strategies XSEDE12 26 / 28

29 Conclusions Summary and prognosis Quality of the empirical approximation potential is critical. On-the-fly refinement of the approximate potential? Mid-level low-quality first-principles potentials are a time-saver, but do not substantially improve sampling. (Not shown) OBMC sampling on the whole system is inefficient at least for small numbers of replicas. Tree sampling on the whole system can be very effective. Requires fixed-iteration-count middle-level potential. Multi-generation trees possible, but not yet tested. Alleviates poor scaling of DFT codes for large P. Support from NSF (# ) Lev Gelb (UT Dallas) Monte Carlo Strategies XSEDE12 27 / 28

30 Coarse-grained parallelism Exploiting parallelism: whole-system OBMC (Also called Multiple-Try Monte Carlo (Liu 2001)) Generate X new configurations {y i } in parallel, each by an approximate chain of M moves, and choose one according to P(k) = π c (y k )/ j πc (y j ). Then generate X 1 backwards chains {x j } to calculate the final acceptance probability: W (y k ) W (x 0 ) = i πc (y i ) j πc (x j ) πa (x 0 ) π a (y k ) Sequence requires wall-clock time for two consecutive first-principles calculations. Performance: so far, not very good. For X 4, no significant improvement over regular sampling. Issue: since forward move often uphill in energy, the backwards chains tend to be very uphill. Lev Gelb (UT Dallas) Monte Carlo Strategies XSEDE12 28 / 28

Accelerated Quantum Molecular Dynamics

Accelerated Quantum Molecular Dynamics Enrique Martinez, Christian Negre, Marc J. Cawkwell, Danny Perez, Arthur F. Voter and Anders M. N. Niklasson Outline Quantum MD Current approaches Challenges Extended