ADAPTIVE TWO-STAGE INTEGRATORS FOR SAMPLING ALGORITHMS BASED ON HAMILTONIAN DYNAMICS

Size: px

Start display at page:

Download "ADAPTIVE TWO-STAGE INTEGRATORS FOR SAMPLING ALGORITHMS BASED ON HAMILTONIAN DYNAMICS"

Leslie Barton
5 years ago
Views:

1 ADAPTIVE TWO-STAGE INTEGRATORS FOR SAMPLING ALGORITHMS BASED ON HAMILTONIAN DYNAMICS E. Akhmatskaya a,c, M. Fernández-Pendás a, T. Radivojević a, J. M. Sanz-Serna b a Basque Center for Applied Mathematics (BCAM), Bilbao, Spain b Departamento de Matemáticas, Universidad Carlos III de Madrid, Madrid, Spain c IKERBASQUE, Basque Foundation for Science, Bilbao, Spain

2 SAMPLING WITH HAMILTONIAN DYNAMICS Let H(θ,p)= 1 2 pt M 1 p+u(θ ) is Hamiltonian θ position, p momentum, U potential energy, M mass matrix, (θ,p) R 2D, D - dimension Equations of motion associated with H: dθ dt = M 1 p; dp dt =-U θ (θ ) The algorithms sampling with Hamiltonian dynamics: ª Can be viewed as special cases of Generalized Hybrid Monte Carlo (GHMC) [Kennedy, Pendleton, 21] ª Numerically solve Hamiltonian equations

3 GHMC AT GLANCE Initialization: positions θ, momenta p (drawn from Gaussian distribution), step size h, length of trajectories L, number of samples N, φ Proposal: numerical integration of Hamiltonian equations with the symplectic method Ψ h over L steps and step-size h: Ψ T (θ,p) = ( θ!, p!),t = Lh Sampling: Metropolis accept / reject test # % FΨ T (θ,p) with probability min(1,exp(-βδh)) % ( θ!, p!) = $ % (θ, p) otherwise % &% Momentum flip F : (θ,p) = (θ,-p) ΔH = H(Ψ T (θ,p)) H(θ,p) = H( FΨ T (θ,p)) H(θ,p) Momentum refreshment: " p! $ % " # u! & ' = $ # cosϕ sinϕ sinϕ cosϕ % " ' F $ & # p u % ', < ϕ π / 2, u ~ N(,β 1 M) &

4 METHODS & APPLICATIONS Molecular Simulations Computational Statistics GHMC GSHMC MMHMC Sample with Shadow Hamiltonians HMC Hybrid MC HMC Hamiltonian φ=π/2

5 METHODS & APPLICATIONS Molecular Simulations Computational Statistics GHMC GSHMC MMHMC Sample with Shadow Hamiltonians Shadow Hamiltonians SHMC / S2HMC HMC Hybrid MC Adaptive parameters NUTS HMC Hamiltonian φ=π/2 Delayed rejections LAHMC / ECGHMC

6 IN FOCUS Molecular simulations: ª Hybrid Monte Carlo (HMC)[Duane, et. al, 1987] ª Generalized shadow hybrid Monte Carlo (GSHMC) [Akhmatskaya, Reich, 28] HMC: GHMC with a complete momentum update GSHMC: Importance sampling GHMC

IN FOCUS Computational statistics: ª Hamiltonian Monte Carlo (HMC)[Neil, 1993] ª Mixed & Match Hamiltonian Monte Carlo (MMHMC) [Radivojevic, Akhmatskaya, 216] HMC: Hybrid Monte Carlo formulated for

7 IN FOCUS Computational statistics: ª Hamiltonian Monte Carlo (HMC)[Neil, 1993] ª Mixed & Match Hamiltonian Monte Carlo (MMHMC) [Radivojevic, Akhmatskaya, 216] HMC: Hybrid Monte Carlo formulated for statistics MMHMC: GSHMC adjusted for statistics ª For the target density π(θ) of position vector θ, momenta p conjugate to θ, with a mass matrix M (a preconditioner) Potential function: Hamiltonian: U(θ) = log(π (θ)) H(θ,p)= 1 2 pt M 1 p+u(θ )

8 WHAT INTEGRATOR TO CHOOSE? Other suggestions?

9 NUMERICAL INTEGRATORS FOR SIMULATING HAMILTONIAN DYNAMICS Wish list: ª Reversible ª Symplectic (preserve the symplectic structure of the Hamiltonian dynamics) => volume preserving ª Low integration error ª Computationally efficient ª Stable

10 VERLET INTEGRATOR Verlet or Verlet / Stormer or leapfrog: a golden standard for Hamiltonian dynamics based simulation methods ª ª ª Reversible & symplectic Second order of accuracy (global error of O (h 2 ), h time step) One force evaluation per step ª Stability interval: (,2) ª ª Easy to implement Verlet as a splitting integrator: ψ h = ϕ B h/2 ϕ A B h ϕ h/2 ϕ t X - t-flow of X={A, B} Can we beat it?

11 SPLITTING INTEGRATORS Potential competitors to Verlet are members of one-parameter family of 2-stage splitting integrators of Hamiltonian system with H(θ,p)= 1 2 pt M 1 p+u(θ ) A+B b is a parameter of the family; b=.25 for Verlet 2-stage splitting integrators: ψ h = ϕ B bh ϕ A B h/2 ϕ (1 2b)h ϕ A B h/2 ϕ bh ª Capable to provide higher accuracy than Verlet (substeps h/2) ª Can achieve bigger time steps (stability interval up to: (,4)) ª Two force evaluations per step but ª Fair comparison with Verlet: equal numbers of force evaluations h Verlet = h 2 stage / 2; L Verlet = 2L 2 stage

12 2-STAGE INTEGRATORS: SPECIAL CASE 1 Minimum-error (ME) integrator by McLachlan (1995): ª Criterion: to minimize the error εper step ε Ch 3 + O(h 5 ); h ª Resulting parameter: b.1932 ª Performance: ü outperforms Verlet at small time steps ü performance degrades for large h: stability interval (, 2.55)

13 2-STAGE INTEGRATORS: SPECIAL CASE 2 BCSS integrator by Blanes, Casas and Sanz-Serna (214): ª Criterion: to minimize expectation of the energy error,, for < h < 2 Δ = H(ψ h,l (θ,p)) H(θ,p) E(Δ) ρ(h,b) h 4 (2b 2 (1/ 2 b)h 2 + 4b 2 6b +1) 2 ρ(h,b) = 8(2 bh 2 )(2 (1/ 2 b)h 2 )(1 b(1/ 2 b)h 2 ) E(Δ) ª Resulting parameter: b.2113 ª Performance: ü Shows the best performance around h=2 ü Performance may drop for smaller or larger step sizes Any better values of b?

14 ADAPTIVE INTEGRATION APPROACH AIA Idea: to present a parameter b as a function of internal properties (frequencies) of a simulated system and a simulation time step, i.e., in such a way that: b = b(δt,ω) Given a simulation problem (ω)and a simulation time (Δt), b defines the unique integrator which guarantees the best conservation of energy for harmonic forces. ª The resulting integrator: a problem specific two-stage integrator ª Criterion: to minimize max <h<h ρ(h,b), h h(δt,ω) ª AIA uses the upper bound suggested in BCSS method

15 AIA: ALGORITHM Given a simulation problem (ω)and a simulation time (Δt): 1. Compute the non-dimensional quantity Molecular Simulation: S = 2 " D % 6 6 dh / 2 ω $ i ' # i=1 & Statistics : S = 2 ω, D Δt - highest frequencies and dimension of the system, - average energy error obtained from warm-up simulation with 2. If h>4 abort the integration 3. Find the optimal value of the parameter b as b = arg min max ρ(h,b) b (,.25) h (,h ) h = S ωδt (Schlick, 22; Skeel, 1999) dh Δt

16 AIA: REMARKS ª No computational overheads in simulations ª Extended to constrained dynamics: ü Presented the two-stage integrator as two concatenated velocity Verlet steps and combined with RATTLE

17 AIA FOR MODIFIED HAMILTONIAN METHODS ª MAIA: AIA for methods sampling with modified Hamiltonian ª The same ideas as in AIA but the expected value of modified energy Δ = H(ψ h,l (θ,p)) H(θ,p) w.r.t. to the modified density is minimised ª MAIA requires: ü Modified Hamiltonians for 2-stage splitting integrators ü Upper bound for expectation of modified energies

18 MODIFIED HAMILTONIANS FOR SPLITTING INTEGRATORS 4-th order modified Hamiltonians for 2-stage splitting integrators were derived in terms of quantities available during simulation - numerical time derivative of - parameter of a 2-stage integrator

19 MAIA: UNDERLYING ANALYSIS Model problem: Standard harmonic oscillator in non-dimensional variables (standard univariate Gaussian). The solution flow is given by Defining depends on a integrator. obtain: ü Numerical solution is exact solution of modified Hamiltonian ü Numerical solution stays on ellipse ü Maximum Δ: numerical solution starts at ellipse vertices on major semiaxis

20 UPPER BOUND FOR EXPECTATION OF MODIFIED ENERGY ERROR D-variate Gaussian target distribution (d coupled linear oscillators): D j=1 E(Δ) ρ(ω j h) ω j - the angular frequencies of the oscillators

21 INTEGRATOR S PARAMETER VS. TIME STEP.27 Sampling with Hamiltonians Sampling with shadow Hamiltonians b b h AIA BCSS ME h MAIA M-BCSS M-ME

22 EXPECTED ERROR VS. TIME STEP Expected error Sampling with Hamiltonians AIA BCSS ME Expected error Sampling with shadow Hamiltonians MAIA M-BCSS M-ME h h

23 EXPECTED ERROR VS. TIME STEP: ZOOM Expected error AIA BCSS ME MAIA M-BCSS M-ME h

24 TIME STEP vs. EXPECTED ERROR 4 Sampling with Hamiltonians 4 Sampling with shadow Hamiltonians 3 3 h 2 h 2 1 AIA BCSS ME Expected error 1 MAIA M-BCSS M-ME Expected error

25 TIME STEP vs. EXPECTED ERROR: ZOOM 4 3 h 2 1 AIA BCSS ME MAIA M-BCSS M-ME Expected error

26 (M)AIA IMPLEMENTATION ª Can be implemented in any MD/HMC software with no overheads ª Two major implementation steps: 1. Introduce AIA in a pre-processing unit to be run once before a simulation ü find b and pass to a simulator 2. Present 2-stage scheme in a kick/drift factorization form in a simulation module ª Current implementation ü MS: in multihmc-gromacs BCAM in-house modified version of GROMACS ü CS: in BCAM software package Hamiltonians in Computational Statistics (HaiCS)

A TESTING (M)AIA 18 o Molecular Simulation B

Coarse grained unconstrained system with 781

Metrics ESS time normalized effective sample

27 A TESTING (M)AIA 18 o Molecular Simulation B Spider toxin in membrane/water environment: Coarse grained unconstrained system with 781 particles Villiin: Atomistic system with 9389 atoms. Hydrogens are constrained using the SHAKE/RATTLE algorithm Computational Statistics Gaussian distribution: D=1, 2 Metrics ESS time normalized effective sample size Rel. ESS relative ESS w.r.t. ESS achieved with Verlet IACF integrated autocorrelation function

28 ACCEPTANCE RATES: TOXIN & VILLIN AR (%) AR (%) AIA BCSS HMC Δt (fs) Δt (fs) 4 5 AR (%) AR (%) MAIA M-BCSS GSHMC Δt (fs) Δt (fs) 4 5 Toxin Villin

29 IACF: TOXIN IACF x time (h) HMC AIA BCSS IACF x time (h) GSHMC MAIA M-BCSS Δt (fs) Δt (fs)

30 TOXIN-BILAYER DISTANCE 3 HMC 3 HMC Distance (nm) BCSS AIA Target distance True distribution BCSS AIA Time (ps) Distance (nm)

31 ACCEPTANCE RATES: GAUSSIAN 1 HMC MMHMC D=1 75 AR (%) 5 25 AIA BCSS ME MAIA M-BCSS M-ME HMC t MMHMC t 75 D=2 AR (%) t t

32 RELATIVE ESS: GAUSSIAN, D=1 rel. min ESS rel. medess AIA BCSS ME HMC t MAIA M-BCSS M-ME MMHMC t

33 RELATIVE ESS: GAUSSIAN, D=2 rel. min ESS rel. medess AIA BCSS ME HMC t MAIA M-BCSS M-ME MMHMC t

34 ESS: GAUSSIAN min ESS AIA BCSS ME HMC HMC h MAIA M-BCSS M-ME MMHMC MMHMC h D=1 D=2 min ESS h h

35 CONCLUSIONS ª We present an alternative to the standard Verlet integrator, (M)AIA, which offers, for any chosen simulation problem and step size, a system-specific two-stage splitting integrator to provide the best conservation of energy for harmonic forces ª The method was derived for sampling with Hamiltonians and modified Hamiltonians and extended to constrained dynamics ª Testing M(AIA) in molecular simulation and statistical applications demonstrated its superiority over velocity Verlet, BCSS and ME integrators ü never worse than, BCSS and ME by design ü leads to higher acceptance rates and better sampling ü allows for longer time steps ü no computational overheads in simulations

36 WORK IN PROGRESS ª M(AIA) for n-stage splitting integrators (n>2) ª M(AIA) for multiple-time stepping methods ª Predicting a range of optimal time steps using the (M)AIA underlying analysis

37 Guggenheim, Bilbao Modeling & Simulation in Life & Materials Sciences: MSLMS BCAM

Mix & Match Hamiltonian Monte Carlo

Mix & Match Hamiltonian Monte Carlo Elena Akhmatskaya,2 and Tijana Radivojević BCAM - Basque Center for Applied Mathematics, Bilbao, Spain 2 IKERBASQUE, Basque Foundation for Science, Bilbao, Spain MCMSki