The parallel replica method for Markov chains

Size: px

Start display at page:

Download "The parallel replica method for Markov chains"

Sheena Peters
5 years ago
Views:

1 The parallel replica method for Markov chains David Aristoff (joint work with T Lelièvre and G Simpson) Colorado State University March 2015 D Aristoff (Colorado State University) March / 29

2 Introduction Definition Let (X n ) be a time homogeneous Markov chain: P(X n+1 A X 0,, X n 1, {X n = x}) = Our goal To efficiently simulate (X n ) when it is metastable State space can be discrete: random walk in random environment, Markov State Models or continuous: time discretization of molecular dynamics (MD) (X n ) can be reversible: MCMC, time discretization of Brownian/Langevin MD or non-reversible: stochastic MD with flow/external forces We are interested in both dynamics and equilibrium! A K(x, dy) D Aristoff (Colorado State University) March / 29

and entropic (right) barriers for Brownian/Langevin dynamics Here, S is a

3 Introduction Metastability can arise from energetic barriers or entropic barriers entropic barrier S V = 0 in blue region V = in red region Example: Energy (left) and entropic (right) barriers for Brownian/Langevin dynamics Here, S is a metastable subset of state space D Aristoff (Colorado State University) March / 29

4 The QSD Definition If ν is a probability measure supported in S with ν( ) = P(X n X 0 ν, X 1,, X n S), n 1, then ν is a quasi-stationary distribution (QSD) in S If for any µ supported in S and any A S, ν(a) = lim n P(X n A X 0 µ, X 1,, X n S), then ν is the unique QSD in S Intuitively: (X n ) reaches ν after spending a long time in S without leaving Our setting: Time scale to reach ν time scale to leave S We need to understand exit events from S, starting from ν D Aristoff (Colorado State University) March / 29

5 The QSD Let ν be the QSD in S and τ the first exit time from S Theorem If X 0 ν, then τ is geometrically distributed, and τ, X τ are independent The geometric distribution is P(τ > n) = (1 p) n, p = P(X 1 / S X 0 ν) It is memoryless: P(τ > n + k τ > k) = P(τ > n) D Aristoff (Colorado State University) March / 29

6 The QSD S ν Figure: Serial simulation of an exit event from S, starting from ν D Aristoff (Colorado State University) March / 29

7 The QSD S ν ν ν Figure: Parallel simulation of an exit event from S, starting from ν D Aristoff (Colorado State University) March / 29

8 The QSD S ν ν ν Figure: Parallel simulation of an exit event from S, with a tie D Aristoff (Colorado State University) March / 29

9 The QSD S ν ν ν Figure: Parallel simulation of an exit event from S, with a tiebreaker D Aristoff (Colorado State University) March / 29

10 The Parallel Step Parallel Step Choose a polling time T poll, set M = 1 and iterate: 1 Start N iid replicas of (X n ) at independent samples of ν 2 Evolve the replicas from time (M 1)T poll to time MT poll 3 If no replica leaves S during this time, update M = M + 1 and return to 2 Else, stop and compute an exit time, τ acc, and exit point, X acc D Aristoff (Colorado State University) March / 29

11 The Parallel Step T poll 2T poll (M 1)T poll MT poll replica 1 replica 2 replica K 1 replica K replica K+1 replica N time 0 NT poll 2NT poll (M 1)NT poll τ acc D Aristoff (Colorado State University) March / 29

12 The Parallel Step T poll 2T poll (M 1)T poll MT poll replica 1 replica 2 replica K 1 replica K replica K+1 replica N time 0 NT poll 2NT poll (M 1)NT poll τ acc D Aristoff (Colorado State University) March / 29

13 The Parallel Step T poll 2T poll (M 1)T poll MT poll replica 1 replica 2 replica K 1 replica K replica K+1 replica N time 0 NT poll 2NT poll (M 1)NT poll τ acc D Aristoff (Colorado State University) March / 29

14 The Parallel Step T poll 2T poll (M 1)T poll MT poll replica 1 replica 2 replica K 1 replica K replica K+1 replica N time 0 NT poll 2NT poll (M 1)NT poll τ acc D Aristoff (Colorado State University) March / 29

15 The Parallel Step T poll 2T poll (M 1)T poll MT poll replica 1 replica 2 replica K 1 replica K replica K+1 replica N time 0 NT poll 2NT poll (M 1)NT poll τ acc D Aristoff (Colorado State University) March / 29

16 The Parallel Step T poll 2T poll (M 1)T poll MT poll replica 1 τ K replica 2 replica K 1 replica K replica K+1 replica N time 0 NT poll 2NT poll (M 1)NT poll τ acc τ acc = sum of lengths of bluelines = (M 1)NT poll +(K 1)T poll +[τ K (M 1)T poll ] X acc = positionof K-th replica at red cross (time τ K ) D Aristoff (Colorado State University) March / 29

17 The Parallel Step Parallel Step Choose a polling time T poll, set M = 1 and iterate the following 1 Start N iid replicas of (X n ) at independent samples of ν 2 Evolve the replicas from time (M 1)T poll to time MT poll 3 If no replica leaves S during this time, update M = M + 1 and return to 2 Else, record the exit event (τ acc, X acc ) Theorem The Parallel Step is exact: (τ acc, X acc ) (τ, X τ ), with τ and X τ the first exit time and exit point from S, starting at ν D Aristoff (Colorado State University) March / 29

18 The Main Algorithm Let S be a collection of disjoint subsets of state space Let T corr = T corr (S) be the time it takes to reach the QSD in S Main Algorithm 1 Evolve (X n ) until it spends time T corr in some set S S without leaving 2 Run the Parallel Step in S Then, return to 1 Let Π be the quotient map which mods out state space by S Coarse dynamics The Main Algorithm produces an approximation ( X n ) of (Π(X n )) via X n = Π(X n ) in Step 1, Xn = Π(S) in Step 2 D Aristoff (Colorado State University) March / 29

19 The Main Algorithm Main Algorithm 1 Evolve (X n ) until it spends time T corr in some set S S without leaving 2 Run the Parallel Step in S Then, return to 1 Let f be a function on state space and π the equilibrium distribution of (X n ) Equilibrium averages The Main Algorithm produces an approximation f sim T sim of f dπ via f sim = f sim + f (X n ) in Step 1, f sim = f sim + f acc in Step 2, where f acc is the sum of values of f over replicas in the Parallel Step In Step 1, T sim increases by 1 for each time step; in Step 2, T sim is increased by τ acc D Aristoff (Colorado State University) March / 29

20 The Main Algorithm T poll 2T poll (M 1)T poll MT poll replica 1 τ K replica 2 replica K 1 replica K replica K+1 replica N time τ acc = sum of lengths of bluelines f acc = sum of values of f over positions of replicas along bluelines D Aristoff (Colorado State University) March / 29

21 The Main Algorithm Idealization After spending T corr consecutive time steps in S, (X n ) exactly reaches the QSD Assumption (X n ) is uniformly ergodic: Theorem lim sup P(X n X 0 = x) π TV = 0 n x Under the Idealization, the Main Algorithm generates exact coarse dynamics: ( X n ) (Π(X n )) Under the Idealization and Assumption, the Main Algorithm generates exact equilibrium averages: f sim as lim = f dπ T sim T sim D Aristoff (Colorado State University) March / 29

22 Numerics S S 1 barrier pathways Figure: A random walk with entropic barriers At each step it moves up, down, left or right at random, unless that results in crossing a barrier D Aristoff (Colorado State University) March / 29

23 Numerics x 60 y T corr (S 1 ) T corr (S 1 ) f time speedup T corr (S 1 ) T corr (S 1 ) Figure: Equilibrium averages vs T corr when N = 100 Here f = 1 (x,y) upper half of right box D Aristoff (Colorado State University) March / 29

24 Numerics x 106 time speedup decorrelation steps N N x 107 parallel steps parallel loops N N Figure: Parallel step stats when T corr = largest value from last slide and T sim = D Aristoff (Colorado State University) March / 29

25 Numerics slope = 005 slope = S 1 S 2 S 3 Figure: A random walk with energetic barriers At each step, it moves to the left with probability 1/2 + slope, and to the right with probability 1/2 slope D Aristoff (Colorado State University) March / 29

26 Numerics x T corr (S 3 ) f T corr (S 3 ) 70 time speedup T corr (S 3 ) Figure: Equilibrium averages vs T corr when N = 100 Here, f = 1 y right half of state space D Aristoff (Colorado State University) March / 29

27 Numerics x 105 time speedup decorrelation steps N N parallel steps N parallel loops x N Figure: Parallel step stats when T corr = largest value from last slide and T sim = 10 9 D Aristoff (Colorado State University) March / 29

28 Conclusion Remarks Our algorithm applies to any Markov chain It is efficient when for S S, Time scale to reach QSD in S Time scale to leave S S need not be known a priori Sets in S can be identified on the fly Future work (X n ) only approximately reaches ν How does this error propagate? When can S be efficiently identified on the fly? D Aristoff (Colorado State University) March / 29

29 Conclusion Acknowledgements Collaborators T Lelièvre (École des Ponts ParisTech), G Simpson (Drexel University) Funding DOE Award de-sc (Thanks to Mitch Luskin for the support) D Aristoff (Colorado State University) March / 29

The Parallel Replica Method for Simulating Long Trajectories of Markov Chains

The Parallel Replica Method for Simulating Long Trajectories of Markov Chains D. Aristoff et al. (2014) ParRep for Simulating Markov Chains, Applied Mathematics Research express, Vol. 2014, No. 2, pp. 332 352 Advance Access publication August 12, 2014 doi:10.1093/amrx/abu005 The