Importance splitting for rare event simulation F. Cerou Frederic.Cerou@inria.fr Inria Rennes Bretagne Atlantique Simulation of hybrid dynamical systems and applications to molecular dynamics September 27-3 21, IHP F. Cerou (Inria) Rare event simulation Hybrid workshop 21 1 / 38
Introduction Rare event Low but non-zero probability, say < È(X A) 1 8. It is very important to know this probability, or to generate typical realizations. We assume that we know how to draw (pseudo) realizations of X. Different from extreme values: here we have a model that we can simulate, extreme values refer to some dataset used with purely statistical approaches. Why naive Monte-Carlo does not work? Let p = È(X A). ˆp = N i=1 ½ A (X i ), and the relative mean square error is var(ˆp) p 2 we need N 1/p at least... = 1 p Np. So Two main frameworks: importance sampling vs. importance splitting. Splitting can work with a simulation blackbox, and gives rare trajectories drawn with the original dynamics. F. Cerou (Inria) Rare event simulation Hybrid workshop 21 2 / 38
Introduction Some applications Particle transmission [KH51] Queueing networks Air traffic management Satellite versus debris collision Finance Food contaminant exposure Molecular dynamics F. Cerou (Inria) Rare event simulation Hybrid workshop 21 3 / 38
Introduction Naive Monte-Carlo N trajectories 1/N 1/N... 1/N 1/N F. Cerou (Inria) Rare event simulation Hybrid workshop 21 4 / 38
Introduction Trajectory splitting Splitting and weighting 1/N 1/(2N)... 1/(3N) 1/(4N) 1/(2N) F. Cerou (Inria) Rare event simulation Hybrid workshop 21 5 / 38
Introduction Trajectory splitting Splitting (time and number of offsprings) may depend on the past of the current trajectory. With the right weighting, the splitting MC estimates are unbiased. But the goal is to lower the variance, by using a "clever" way of splitting. Consider splitting those trajectories that goes "towards" the rare event. F. Cerou (Inria) Rare event simulation Hybrid workshop 21 6 / 38
Multilevel splitting Precise setting X continuous time stochastic process in Ê d, with initial distribution µ. X has almost surely right continuous, left limited trajectories (RCLL). A is a recurrent event for X, i.e. È(X returns to A i.o. ) = 1. Let τ A = inf{t >, X t A} and τ A = inf{t >, X t A }. Consider the rare event A = È(τ A < τ A ). In many cases, A has the following form: A = {x Ê d, φ(x) L} for some scalar continuous function φ, and some real L. F. Cerou (Inria) Rare event simulation Hybrid workshop 21 7 / 38
Multilevel splitting Algorithm: fixed splitting Parameters: Splitting levels L 1 < L 2 < < L k < < L for a chosen importance function φ, and mean offspring numbers R. We estimate the #particles reaching L total probability by ˆp n,fs =. NR n 1 L L2 L1 F. Cerou (Inria) Rare event simulation Hybrid workshop 21 8 / 38
Multilevel splitting Efficiency How to measure efficiency? Let τ k = inf{t >, φ(x t ) L k } and p k = È(τ k < τ A ). Assume lim k + p k =. Work normalized variance var(ˆp k )w k. Naive Monte-Carlo var(ˆp k,mc )w k = var(ˆp k,mc )N = p k (1 p k ) Definition We define the asymptotic efficiency [GW92] of an estimate ˆp k by log(var(ˆp k )w k ) lim = 2 k + log p k By Jensen inequality, we always have 2 for unbiased estimators. For Naive Monte-Carlo, we get log(var(ˆp k,mc )w k ) log p k = log(p k(1 p k )) log(p k ) = 1+ log(1 p k) log(p k ) 1. F. Cerou (Inria) Rare event simulation Hybrid workshop 21 9 / 38
Multilevel splitting Efficiency: fixed splitting log p Assume lim k k + k = logρ exists. From [GHSZ99], we have Theorem Under some mixing conditions, log(var(ˆp k,fs )w k,fs ) lim = 2 for R = 1/ρ, k + log p k and log(var(ˆp k,fs )w k,fs ) lim < 2 for R 1/ρ, k + log p k Fixed splitting efficient only in critical regime. From simulations, it proved to be very sensitive to the branching rate R. But we can choose R adaptively... F. Cerou (Inria) Rare event simulation Hybrid workshop 21 1 / 38
Interacting particle systems Fixed effort splitting At each level, we randomly branch N offsprings on the remaining paths. Let ˆp k k 1 = #particles reaching L k N, we estimate the total probability by ˆp n,fe = n k=1ˆp k k 1. L 2 L 1 L 1 x x F. Cerou (Inria) Rare event simulation Hybrid workshop 21 11 / 38
Interacting particle systems Feynman-Kac representation Following [CDLL6] let τ k = inf{t >,φ(x t ) L k }, and T k = τ k τ A. and E[f(X t,t T n ) τ A τ A ] = E[f(X t,t T n ) n k= ½ φ(xtk ) L k ] E[ n, k= ½ φ(xtk ) L k ] È(T A T A ) = E[ n ½ φ(xtk ) L k ]. k= Special case of general Feynman-Kac formula η n (f) = E[f(Z 1,...,Z n ) n k= G k(z k )] E[ n k= G k(z k )] where G k are measurable non-negative functions called potentials. The corresponding Interacting Particle System approximation is exactly the fixed effort splitting algorithm. F. Cerou (Inria) Rare event simulation Hybrid workshop 21 12 / 38
Interacting particle systems Convergence results Once we have our problem in the form of a Feynman-Kac formula, associated with an IPS, we can use directly results in the monograph [DM4]. This includes unbiasedness of ˆp n,fe, SLLN, CLT... Central Limit Theorem: Theorem with σ 2 = n k=1 n 1 + k= N ˆp n,fe p p 1 p k k 1 p k k 1 D N + N(,σ2 ), var(è(φ(x τn ) L X τk,φ(x τk ) L k )) 1 pk k 1 2 È 2. (φ(x τn ) L φ(x τk ) L k ) p k k 1 F. Cerou (Inria) Rare event simulation Hybrid workshop 21 13 / 38
Interacting particle systems Some remarks σ 2 = n k=1 n 1 + k= 1 p k k 1 p k k 1 var(è(φ(x τn ) L X τk,φ(x τk ) L k )) 1 pk k 1 2 È 2. (φ(x τn ) L φ(x τk ) L k ) p k k 1 The vaviance in the second term is zero if the importance function φ is the committor function, i.e. the levels L k define iso-committor surfaces. First term is minimal when all the p k k 1 are equal (optimization with the constraint that their product is p). F. Cerou (Inria) Rare event simulation Hybrid workshop 21 14 / 38
Interacting particle systems Efficiency of fixed effort Theorem ([CDG1]) Under mixing conditions, and provided that the levels are well chosen (in the sense that k, < ρ 1 p k k 1 ρ 2 < 1), we have for some C >. var(ˆp k,fe ) p 2 k [(1+ C (N 1)ρ 1 ) k 1] and thus, taking w k,fe = kn, Corollary lim inf k log(w k,fe var(ˆp k,fe )) log p k 2+ 1 2logρ 2 log(1+ Almost get efficiency, closer as N gets larger. Paul Dupuis and Yi Cai recently obtained a similar result. C ρ 1 (N 1) ). F. Cerou (Inria) Rare event simulation Hybrid workshop 21 15 / 38
Adaptive level splitting Adaptive splitting To each trajectory X we associate S = sup t τa τ A φ(x t ). Choose K < N, and compute the K/N quantile ˆq 1 of S 1,...,S N. This quantile will be the first level of splitting. ˆq 1 X 1 x X 4 X 3 X 2 F. Cerou (Inria) Rare event simulation Hybrid workshop 21 16 / 38
Adaptive level splitting Adaptive splitting Keep the trajectories above ˆq 1, and split (branch) them when they first corss this level. From the whole N again trajectories, all above ˆq 1, compute ˆq 2, and iterate. ˆq 2 ˆq 1 x F. Cerou (Inria) Rare event simulation Hybrid workshop 21 17 / 38
Adaptive level splitting Convergence of adaptive splitting Only in dimension 1: a.s. convergence and CLT. Theorem ([CG7]) with with p = K N. N(pn ˆp n,as ) σ 2 = p 2 n((n 1) 1 p p L n + N(,σ2 ), + 1 ρ ρ ), Minimal variance in the CLT, but no asyptotic efficiency results. Complexity in nn log N. New version with K = N 1: discard only the lower trajectory at each step. F. Cerou (Inria) Rare event simulation Hybrid workshop 21 18 / 38
Numerical simulations Application to reactive trajectories Let V : Ê d Ê the potential function, consider the overdamped Langevin dynamics: dx t = V(X t )dt + 2β 1 dw t, where β = 1/(k B T) Equilibrium canonical measure: dµ = Z 1 exp( βv(x))dx where Z = exp( βv(x))dx Ê d F. Cerou (Inria) Rare event simulation Hybrid workshop 21 19 / 38
Numerical simulations Reactive trajectories A and B two metastable (recurrent) regions in Ê d. A B Reactive trajectory: piece of equilibrium trajectory that leaves A and goes to B without going back to A in the meantime (excursion from A to B) Problem: one may wait a long time before the trajectory eventually reaches B F. Cerou (Inria) Rare event simulation Hybrid workshop 21 2 / 38
Algorithm Reaction coordinate Also called "importance function" in rare event literature Smooth one-dimensional function: such that: ξ : Ê d Ê ξ, A {x Ê d,ξ(x) < z min } and B {x Ê d,ξ(x) > z max }, where z min < z max are two given real numbers Example: ξ(x) = x x A with x A A denotes a reference configuration in A F. Cerou (Inria) Rare event simulation Hybrid workshop 21 21 / 38
Algorithm Committor function It is well known (e.g. Metzner, Schütte and Vanden-Eijnden (26), Vanden-Eijnden, Venturoli, Ciccotti and Elber (28)) that the optimal reaction coordinate/importance function is the committor function q(x) = È(τ A (x) > τ B (x)). q is solution of { V q +β 1 q = in Ê d \(A B), q = on A and q = 1 on B. F. Cerou (Inria) Rare event simulation Hybrid workshop 21 22 / 38
Algorithm Algorithm A B F. Cerou (Inria) Rare event simulation Hybrid workshop 21 23 / 38
Algorithm Algorithm A B F. Cerou (Inria) Rare event simulation Hybrid workshop 21 24 / 38
Algorithm Algorithm A B F. Cerou (Inria) Rare event simulation Hybrid workshop 21 25 / 38
Algorithm Algorithm A B F. Cerou (Inria) Rare event simulation Hybrid workshop 21 26 / 38
Algorithm Algorithm A B F. Cerou (Inria) Rare event simulation Hybrid workshop 21 27 / 38
Algorithm Algorithm A B F. Cerou (Inria) Rare event simulation Hybrid workshop 21 28 / 38
Algorithm Algorithm A B F. Cerou (Inria) Rare event simulation Hybrid workshop 21 29 / 38
Algorithm Algorithm Iterate until all the paths reach Σ max, and keep only those which actually reach B. Initialization: generate a long trajectory "around" A (wait until equilibrium), and keep (a subsample of) the upward crossing points of Σ min, and the corresponding piece of trajectory coming from A. Remark: the algorithm is a kind of adaptive Forward Flux Sampling (e.g. Allen, Valeriani and ten Wolde (29)), or more generally Importance Splitting Diffusion process approximated by Euler scheme. F. Cerou (Inria) Rare event simulation Hybrid workshop 21 3 / 38
Algorithm Convergence and efficiency Unbiased estimates ˆp k,ir. In dimension 1, almost have efficiency. In this case var(ˆp k,ir ) = p 2 k (p 1 N k 1) and w k,ir = N log N log p k. log(var(ˆp k,ir )w k,ir ) lim = 2 1 k + log p k N. (Guyader,Hengertner,Matzner-Løber, not yet published). F. Cerou (Inria) Rare event simulation Hybrid workshop 21 31 / 38
Numerical results 1D example Simple example to validate the method against DNS V(x) = x 4 2x 2. This potential has two minima at ±1 and one saddle point at. In this simple one dimensional setting, we set as metastable states A = { 1} and B = {+1}, and the reaction coordinate is taken to be simply ξ(x) = x. We computed the distribution of the duration of the reactive paths. F. Cerou (Inria) Rare event simulation Hybrid workshop 21 32 / 38
Numerical results 1D example Distribution of the length of a reactive path, beta=1 Distribution of the length of a reactive path, beta=5 2 1.8 algorithm, 1E5 paths DNS, 1E5 paths 1.9 algorithm, 1E5 paths DNS, 1E5 paths 1.6.8 1.4.7 1.2.6 probability 1.8 probability.5.4.6.3.4.2.2.1.5 1 1.5 2 2.5 3 3.5 4 4.5 1 2 3 4 5 6 time time Distribution of the length of a reactive path, beta=1 Distribution of the length of a reactive path 1.2 1 DNS, 1E4 paths algorithm, 1E5 paths 2 1.8 1.6 beta=1 beta=5 beta=1 beta=15 beta=2 beta=3 beta=4.8 1.4 1.2 probability.6 probability 1.8.4.6.2.4.2.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 1 2 3 4 5 6 time time F. Cerou (Inria) Rare event simulation Hybrid workshop 21 33 / 38
Numerical results 2D example V(x,y) = 3e x2 (y 1 3 )2 3e x2 (y 5 3 )2 5e (x 1)2 y 2 5e (x+1)2 y 2 +.2x 4 +.2 ( y 1 3) 4. as considered by Metzner, Schütte and Vanden-Eijnden (26) V(x,y) 8 6 4 2-2 -4-2 -1.5-1 -.5 x.5 1 1.5.5 1 -.5-1 2-1.5 1.5 2 2.5 Figure: The potential V. F. Cerou (Inria) Rare event simulation Hybrid workshop 21 34 / 38 y
Numerical results Density of reactive paths Density for beta = 1.67 Density for beta = 6.67.35.4.3.35.35.3.25.2.15.1 5e-5.25.2.15.1 5e-5.4.35.3.25.2.15.1.5.3.25.2.15.1.5 2. 2. 1.25 1.25-1.2 -.6 x.6 1.2-1 -.25.5 y -1.2 -.6 x.6 1.2-1 -.25.5 y F. Cerou (Inria) Rare event simulation Hybrid workshop 21 35 / 38
Numerical results Density of reactive paths Density for beta = 1.67 Density for beta = 6.67.4.4.35.35.4.35.3.25.2.15.1 5e-5.3.25.2.15.1 5e-5.4.35.3.25.2.15.1.5.3.25.2.15.1.5 2. 2. 1.25 1.25-1.2 -.6 x.6 1.2-1 -.25.5 y -1.2 -.6 x.6 1.2-1 -.25.5 y F. Cerou (Inria) Rare event simulation Hybrid workshop 21 36 / 38
y Numerical results y Flux of reactive trajectories Flux of reactive path Flux of reactive path 1 2 1 1.5.8 1.5.8 1.6 1.6.5.4.5.4 -.5.2.2 -.5-1 -1 -.5.5 1-1 -.5.5 1 x x Figure: Flux of reactive trajectories, at inverse temperature β = 1.67 on the left, and β = 6.67 on the right. The color indicates the norm of the flux. F. Cerou (Inria) Rare event simulation Hybrid workshop 21 37 / 38
y Numerical results y A few reactive trajectories Reactive paths for beta =1.67 Reactive paths for beta =6.67 2 long path middle path short path 2 long path middle path short path 1.5 1.5 1 1.5.5 -.5 -.5-1 -1.5-1 -.5.5 1 1.5 x -1-1.5-1 -.5.5 1 1.5 x Figure: A few reactive paths for β = 1.67 (left), β = 6.67 (right). F. Cerou (Inria) Rare event simulation Hybrid workshop 21 38 / 38
Numerical results F. Cerou, P. Del Moral, et al. A non asymptotic variance theorem for unnormalized feynman-kac particle models. Annales de l IHP, 21. F. Cérou, P. Del Moral, et al. Genetic genealogical models in rare event analysis. Latin American Journal of Probability and Mathematical Statistics, 1, 26. F. Cérou and A. Guyader. Adaptive multilevel splitting for rare event analysis. Stoch. Anal. Appl., 25(2):417 443, 27. P. Del Moral. Feynman-Kac formulae, Genealogical and interacting particle systems with applications. Probability and its Applications. Springer-Verlag, New York, 24. F. Cerou (Inria) Rare event simulation Hybrid workshop 21 38 / 38
Numerical results P. Glasserman, P. Heidelberger, et al. Multilevel splitting for estimating rare event probabilities. Oper. Res., 47(4):585 6, 1999. P. W. Glynn and W. Whitt. The asymptotic efficiency of simulation estimators. Oper. Res., 4(3):55 52, 1992. H. Kahn and T. Harris. Estimation of particle transmission by random sampling. National Bureau of Standards Appl. Math. Series, 12:27 3, 1951. F. Cerou (Inria) Rare event simulation Hybrid workshop 21 38 / 38