Mathematical analysis of an Adaptive Biasing Potential method for diffusions

Transcription

1 Mathematical analysis of an Adaptive Biasing Potential method for diffusions Charles-Edouard Bréhier Joint work with Michel Benaïm (Neuchâtel, Switzerland) CNRS & Université Lyon 1, Institut Camille Jordan (France) Stochastic Sampling and Accelerated Time Dynamics on Multidimensional Surfaces C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

2 Plan of the talk 1 Motivations 2 The Adaptive Biasing Potential method 3 Consistency and elements of proof Self-interacting diffusions Analysis of the ABP method 4 Extensions Examples Abstract framework C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

3 Motivations: setting Goal: estimating averages φdµ β = 1 φ(x)e βv (x) dx, Z(β) T d where V : T d R. From now on β = 1. The probability distribution µ is ergodic for the overdamped Langevin dynamics: dxt 0 = V (Xt 0 )dt + 2dW t. Difficulty: slow convergence of temporal averages (metastability) 1 t φ(x s )ds t 0 t φdµ. C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

4 Context Use of an importance sampling strategy: change of the drift coefficient in the dynamics. A nice review B. M. Dickson. Survey of adaptive biasing potentials: comparisons and outlook. Current Opinion in Structural Biology, Related to the following techniques: umbrella sampling S. Marsili, A. Barducci, R. Chelli, P. Procacci, and V. Schettino. Self-healing umbrella sampling: a non-equilibrium approach for quantitative free energy calculations. The Journal of Physical Chemistry B, G. Fort, B. Jourdain, T. Lelièvre, and G. Stoltz. Self-healing umbrella sampling: convergence and efficiency. Stat. Comput., C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

5 Context Use of an importance sampling strategy: change of the drift coefficient in the dynamics. A nice review B. M. Dickson. Survey of adaptive biasing potentials: comparisons and outlook. Current Opinion in Structural Biology, Related to the following techniques: metadynamics A. Laio and M. Parrinello. Escaping free-energy minima. Proceedings of the National Academy of Sciences, A. Barducci, G. Bussi, and M. Parrinello. Well-tempered metadynamics: a smoothly converging and tunable free-energy method. Physical review letters, C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

6 Context Use of an importance sampling strategy: change of the drift coefficient in the dynamics. A nice review B. M. Dickson. Survey of adaptive biasing potentials: comparisons and outlook. Current Opinion in Structural Biology, Related to the following techniques: Wang-Landau F. Wang and D. Landau. Determining the density of states for classical statistical models: A random walk algorithm to produce a flat histogram. Physical Review E, F. Wang and D. Landau. Efficient, multiple-range random walk algorithm to calculate the density of states. Physical review letters, G. Fort, B. Jourdain, E. Kuhn, T. Lelièvre, and G. Stoltz. Convergence of the Wang-Landau algorithm. Math. Comp., C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

7 Biasing methods for diffusions Adaptive Biasing Force (ABF) dx t = ( V (X t )+F t (X t ) ) dt + 2dW t. Adaptive Biasing Potential (ABP) References on ABF dx t = ( V (X t )+ V t (X t ) ) dt + 2dW t. J. Comer, J. C. Gumbart, J. Hénin, T. Lelièvre, A. Pohorille, and C. Chipot. The adaptive biasing force method: Everything you always wanted to know but were afraid to ask. The Journal of Physical Chemistry B, T. Lelièvre, M. Rousset, and G. Stoltz. Long-time convergence of an adaptive biasing force method. Nonlinearity, The bias F t depends on the law L(X t ). C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

8 Biasing methods for diffusions Adaptive Biasing Force (ABF) dx t = ( V (X t )+F t (X t ) ) dt + 2dW t. Adaptive Biasing Potential (ABP) dx t = ( V (X t )+ V t (X t ) ) dt + 2dW t. We study a continuous time version of the ABP method proposed in B. Dickson, F. Legoll, T. Lelièvre, G. Stoltz, and P. Fleurat-Lessard. Free energy calculations: An efficient adaptive biasing potential method. J. Phys. Chem. B, The bias V t depends on the past trajectory of the process, X r for 0 r t. C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

9 Plan of the talk 1 Motivations 2 The Adaptive Biasing Potential method 3 Consistency and elements of proof Self-interacting diffusions Analysis of the ABP method 4 Extensions Examples Abstract framework C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

10 Biasing the potential Reaction coordinate: ξ : x T d x 1 T. Biasing: for A : T R, dx A t = ( V A ξ ) (X A t )dt + 2dW t. Ergodic invariant law is modified e V (x)+a(ξ(x)). Why it is useful: weighted averages converge to φdµ (consistency) convergence is faster when A is well-chosen: free energy can be made adaptive C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

11 The free energy A : T R is given by e A (z) = Z(1) 1 e V (z,x 2,...,x d ) dx 2... dx d. T d 1 Acceleration of the sampling = removing free energy barriers. Choosing A = A : flat histogram for ξ(x A 1 t t 0 t ) δ ξ(x A s ) ds dz. t Adaptive algorithm: A t A A. t C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

12 The Adaptive Biasing Potential method Two unknowns: X t and A t. Dynamics for X t : dx t = ( V A t ξ ) (X t )dt + 2dW (t). Computation of the bias A t : convolution of a kernel function K e At(z) = K ( z, ξ(x) ) µ t (dx), T d with the weighted empirical averages µ t = µ 0 + t 0 e Ar ξ(xr ) δ Xr dr 1 + t. 0 e Ar ξ(xr ) dr C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

13 Convergence results dx t = ( V A t ξ ) (X t)dt + 2dW (t), e At (z) = µ t = µ 0 + t 0 e Ar ξ(xr ) δ Xr dr 1 + t. 0 e Ar ξ(xr ) dr K ( z, ξ(x) ) µ t (dx) Theorem (Benaïm-B.) Almost surely, µ t t µ, A t t A, with µ(dx) = e V (x) dx and e A (z) = T d K(z, ξ( ))dµ. C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

14 Convergence results dx t = ( V A t ξ ) (X t)dt + 2dW (t), e At (z) = µ t = µ 0 + t 0 e Ar ξ(xr ) δ Xr dr 1 + t. 0 e Ar ξ(xr ) dr K ( z, ξ(x) ) µ t (dx) Theorem (Benaïm-B.) Almost surely, µ t t µ, A t t A, with µ(dx) = e V (x) dx and e A (z) = T d K(z, ξ( ))dµ. consistent with non-adaptive versions (A t = A t 0) A A due to the kernel function. C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

15 Comments on the efficiency Convergence of histograms: almost surely 1 t δ t ξ(xs)ds e A (z) A (z) dz. 0 t Asymptotic variance: same as in non-adaptive version with A = A. The choice of the kernel function K is important. Assumption K : T T (0, ) is of class C and positive. K(z, )dz = 1. Examples: K(z, ζ) e z ζ 2 2δ vs. K(z, ζ) = e A(z). C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

16 Plan of the talk 1 Motivations 2 The Adaptive Biasing Potential method 3 Consistency and elements of proof Self-interacting diffusions Analysis of the ABP method 4 Extensions Examples Abstract framework C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

17 Role of the weighted empirical measures µ t dx t = ( V A t ξ ) (X t )dt + 2dW (t), µ t = µ 0 + t 0 e Ar ξ(xr ) δ Xr dr 1 + t, 0 e Ar ξ(xr ) dr exp ( A t (z) ) = K ( z, ξ(x) ) µ t (dx). Considering A t = A(µ t ), may be interpreted as a self-interacting diffusion, with unknowns X t and µ t dx t = ( V A(µ t ) ξ ) (X t )dt + 2dW (t), µ t = µ 0 + t 0 e A(µ r ) ξ(xr ) δ Xr dr 1 + t 0 e A(µ r ) ξ(xr ) dr C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

18 Self-interacting diffusions Classical formulation: dy t = V (Y t, ν t )dt + 2dW t with V (y, ν) = V (y, )dν, ν t = 1 ( t ν0 + δ Yr dr ). 1 + t 0 M. Benaïm, M. Ledoux, and O. Raimond. Self-interacting diffusions. Probab. Theory Related Fields, Main differences: type of the coupling µ t in ABP is a weighted empirical distribution. C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

19 Stochastic approximation: the ODE method dy t = V (Y t, ν t )dt + 2dW t, ν t = 1 ( t ν0 + δ Yr dr ). 1 + t 0 The empirical distribution solves the random ODE dν t dt = 1 ( ) δyt ν t. 1 + t Asymptotic time-scale separation: slow-fast system (t ). A. Benveniste, M. Métivier, and P. Priouret. Adaptive algorithms and stochastic approximations. Springer-Verlag, H. J. Kushner and G. G. Yin. Stochastic approximation and recursive algorithms and applications. Springer-Verlag, C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

20 Stochastic approximation: the ODE method dy t = V (Y t, ν t )dt + 2dW t, dν t dt = 1 ( ) δyt ν t. 1 + t Claim The asymptotic behavior of ν t is governed by the ODE dγ t dt = Π(Γ t ) Γ t, where Π(ν) is the unique invariant distribution of dy t = V (Y t, ν)dt + 2dW t C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

21 Stochastic approximation: the ODE method Claim The asymptotic behavior of ν t is governed by the ODE dγ t dt = Π(Γ t ) Γ t, where Π(ν) is the unique invariant distribution of dy t = V (Y t, ν)dt + 2dW t A precise statement: in terms of asymptotic pseudo-trajectories and of chain-recurrent sets. M. Benaïm. Dynamics of stochastic approximation algorithms. In Séminaire de Probabilités, XXXIII, C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

22 Analysis of the ABP method (1) System: dx t = ( V A t ξ ) (X t )dt + 2dW (t), µ t = µ 0 + t 0 e Ar ξ(xr ) δ Xr dr 1 + t 0 e Ar ξ(xr ) dr The weighted empirical distribution µ t solves the ODE dµ t dt = e At ξ(xt) 1 + ( ) t δxt µ 0 e Ar ξ(xr ) t dr C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

23 Analysis of the ABP method (1) The weighted empirical distribution µ t solves the ODE dµ t dt = e At ξ(xt) 1 + ( ) t δxt µ 0 e Ar ξ(xr ) t dr Observation Weights are eliminated by the random change of time variable s = t 0 e Ar ξ(xr ) dr. This may be performed thanks to stability bounds: almost surely m A t (z) M, z T, t 0. C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

24 Analysis of the ABP method (2) In the new variables the dynamics becomes Y s = X t, B s = A t, ν s = µ t, dy s = ( V B s ξ ) (Y s )e Bs(ξ(Ys)) ds + 2e 1 2 Bs(ξ(Ys)) d W (s), ν s = 1 ( s µ0 + δ Yσ dσ ), e βbs = K(z, ξ(x))ν s (dx) 1 + s Now the weights appear in the dynamics. 0 T C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

25 Analysis of the ABP method (2) In the new variables the dynamics becomes Y s = X t, B s = A t, ν s = µ t, dy s = ( V B s ξ ) (Y s )e Bs(ξ(Ys)) ds + 2e 1 2 Bs(ξ(Ys)) d W (s), ν s = 1 ( s µ0 + δ Yσ dσ ), e βbs = K(z, ξ(x))ν s (dx) 1 + s Now the weights appear in the dynamics. Asymptotic behavior: almost surely 0 µ t t µ ν s s µ. T C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

26 Analysis of the ABP method (3) Strategy: identify ν Π(ν) = invariant distribution of dy s = ( V B(ν) ξ ) (Y s )e B(ν)(ξ(Ys)) ds+ 2e 1 2 B(ν)(ξ(Ys)) d W (s), with e B(ν)(z) = K(z, ξ(x))ν(dz)? T study the asymptotic behavior of the ODE dγ s ds = Π(Γ s) Γ s, C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

27 Analysis of the ABP method (3) Strategy: identify ν Π(ν) = invariant distribution of dy s = ( V B(ν) ξ ) (Y s )e B(ν)(ξ(Ys)) ds+ 2e 1 2 B(ν)(ξ(Ys)) d W (s), with e B(ν)(z) = K(z, ξ(x))ν(dz)? T study the asymptotic behavior of the ODE dγ s ds = Π(Γ s) Γ s, Nice result: for all ν, Π(ν) = µ. Then Γ s = (1 e s µ) + e s Γ 0 µ. s C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

28 Hidden technical details (1) Generator of the diffusion with fixed B: ) L B = e ( (V B ξ B ξ), +. Invariant distribution: ( L B ) µ = ( (V B ξ), + ) ( e B ξ V dx ) = 0. C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

29 Hidden technical details (1) Generator of the diffusion with fixed B: ) L B = e ( (V B ξ B ξ), +. Invariant distribution: ( L B ) µ = ( (V B ξ), + ) ( e B ξ V dx ) = 0. Poisson equation: for smooth ϕ : T d R,! Ψ(, B) such that L B Ψ(, B) = ϕ ϕdµ, Ψ(, B) = 0. Dependence with respect to B: uniform bounds on Ψ(, B) and all its derivatives, when controlling B and all its derivatives. C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

30 Hidden technical details (2) Analysis of the error for the original system: µ t (ϕ) µ(ϕ) = = t 0 e Ar ξ(xr )[ ϕ(x r ) µ(ϕ) ] dr 1 + t 0 e Ar ξ(xr ) dr t 0 e Ar ξ(xr ) L Ar Ψ(X r, A r )dr 1 + t 0 e Ar ξ(xr ) dr + o(1) + o(1). C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

31 Hidden technical details (2) Analysis of the error for the original system: t 0 µ t (ϕ) µ(ϕ) = e Ar ξ(xr )[ ϕ(x r ) µ(ϕ) ] dr 1 + t + o(1) 0 e Ar ξ(xr ) dr t 0 = e Ar ξ(xr ) L Ar Ψ(X r, A r )dr 1 + t + o(1). 0 e Ar ξ(xr ) dr Using Itô s formula: t 0 e Ar ξ(xr ) L Ar Ψ(X r, A r )dr = t 0 ( ) (V A r ξ), + Ψ(X r, A r )dr = Ψ(X t, A t ) Ψ(X 0, A 0 ) + t 0 t 0 r Ψ(, B r )(X r )dr (V A r ξ), Ψ(X r, A r ) dw (r) C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

32 Plan of the talk 1 Motivations 2 The Adaptive Biasing Potential method 3 Consistency and elements of proof Self-interacting diffusions Analysis of the ABP method 4 Extensions Examples Abstract framework C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

33 Extensions The analysis is not restricted to dx 0 t = V (X 0 t )dt + 2dW t, overdamped Langevin dynamics on the flat d-dimensional torus T d with reaction coordinate ξ : x T d x 1 T. C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

34 Extensions The analysis is not restricted to dx 0 t = V (X 0 t )dt + 2dW t, overdamped Langevin dynamics Langevin, extended dynamics on the flat d-dimensional torus T d dynamics on R d, on L 2 (0, 1) (SPDE) with reaction coordinate ξ : x T d x 1 T. ξ is smooth with values in a compact m-dimensional manifold M m, for instance T m with arbitrary m 1. The kernel function K : M m M m (0, ) is still assumed smooth and positive. C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

35 Extension 1: Langevin dynamics State space: S = T d R d or R d R d. Reaction coordinate: ξ(q, p) = ξ(q) M m. Dynamics: { dq p = p t dt, dp t = ( V A t ξ ) (q t )dt γp t + 2γβ 1 dw (t), µ t = µ 0 + t 0 e βar ξ(qr ) δ (qr,p r )dr 1 + t 0 e βar ξ(qr ) dr, e βat (z) = K(z, ξ(q))µ t (dqdp). Consistency: almost surely µ t µ(dq) N (0, σ 2 ). t C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

36 Extension 2: extended dynamics State space: S = T d M m or R d M m. Reaction coordinate: ξ(x, z) = z. Extended potential energy function: U(x, z) = V (x) + 1 2ɛ ξ(x) z 2. Dynamics: { dx t = V (X t )dt 1 ɛ ξ(x t). ( ξ(x t ) Z t ) dt + 2β 1 dw x t dz t = 1 ɛ ( Zt ξ(x t ) ) dt + A(Z t )dt + 2β 1 dw z t. µ t = µ 0 + t 0 e βar (Zr ) δ (Xr,Z r )dr 1 + t 0 e βar (Zr ) dr, e βat (z) = K(z, ζ)µ t (dxdζ). Consistency: almost surely µ t 1 t Z(β,ɛ) e βv (x) e β(ξ(x) z) 2 2ɛ dxdz. C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

37 Extension 3: infinite dimensional dynamics (SPDE) { du 0 (t, x) = 2 u 0 (t,x) x 2 dt V(u 0 (t, x))dt + 2β 1 dw (t, x) u 0 (t, 0) = 0 = u 0 (t, 1). Example: V(x) = x4 Energy functional: x2 4 2 u 1 0 (Allen-Cahn equation). [1 u(x) 2 + V ( u(x) )] dx, 2 x C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

38 Extension 3: infinite dimensional dynamics (SPDE) State space: H = L 2 (0, 1) separable infinite dimensional Hilbert space Invariant distribution of the SPDE is du 0 t = Lu 0 t dt DV (u 0 t )dt + 2dW (t) µ(du) = e V (u) λ(du) with V (u) = 1 0 V(u( )) and λ(du) = N (0, L 1 ) Gaussian distribution on H. In this example: λ is the distribution of the Brownian Bridge. C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

39 Extension 3: infinite dimensional dynamics (SPDE) Reaction coordinate: ξ(u) = 1 u mod M. 0 Dynamics: du(t) = Lu(t)dt D ( V A t ξ ) (u(t))dt + 2β 1 dw (t) µ t = µ 0 + t 0 e βar ξ(ur ) δ ur dr 1+ t 0 e βar ξ(ur ) dr e βat(z) = K ( z, ξ(u) ) µ t (du) Consistency: almost surely µ t (ϕ) t µ(ϕ), for all smooth functions ϕ : H R. C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

40 Abstract framework Dynamics (with fixed bias A): dx A t on a state space S. = D(V, A)(X A t )dt + 2β 1 ΣdW (t) C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

41 Abstract framework Dynamics (with fixed bias A): dx A t on a state space S. = D(V, A)(X A t )dt + 2β 1 ΣdW (t) Potential energy function: V : E R, smooth. Bias: A : M m R. Reaction coordinates: ξ : E M and ξ S : S M. C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

42 Abstract framework Invariant distribution: µ A β(dx) = 1 Z A (β) e βe(v,a)(x) λ(dx), with a reference measure λ on S. Compatibility condition: E(V, A) = E(V, 0) A ξ S. C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

43 Abstract framework Invariant distribution: µ A β(dx) = 1 Z A (β) e βe(v,a)(x) λ(dx), with a reference measure λ on S. Compatibility condition: E(V, A) = E(V, 0) A ξ S. Free energy function: e βa is the Radon-Nikodym derivative: φ(z)e βa (z) π(dz) = φ ( ξ S (x) ) µ β (dx), M m S with respect to a reference probability distribution π on M m. C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

44 Abstract ABP Adaptive Biasing Potential dynamics: dx t = D ( ) V, A t (Xt )dt + 2β 1 ΣdW t, µ t = µ 0 + t 0 Fτ (ξ S(X τ ))δ Xτ dτ, with 1+ t 0 Fτ (ξ S(X τ ))dτ F t = N ( S K(, ξ S (x) ) µ t (dx) ), A t = 1 log( ) F β t, an additional variable F t (weights) a normalization operator N : e.g. N (f ) = f = f fdπ, N (f ) = f max f, N (f ) = f min f. C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

45 Conclusion and perspectives Mathematical analysis of the asymptotic behavior of an ABP method: interpretation as a self-interacting diffusion, reinforcement with the past almost sure convergence results (consistency) a general framework Some perspectives: more quantitative analysis? construction and analysis of ABF methods? C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

46 Conclusion and perspectives Mathematical analysis of the asymptotic behavior of an ABP method: interpretation as a self-interacting diffusion, reinforcement with the past almost sure convergence results (consistency) a general framework Some perspectives: more quantitative analysis? construction and analysis of ABF methods? References: M. Benaïm and C.-E. Bréhier. Convergence of adaptive biasing potential methods for diffusions. C. R. Math. Acad. Sci. Paris, M. Benaïm and C.-E. Bréhier. Convergence analysis of adaptive biasing potential methods for diffusions. Arxiv preprint, C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31

47 Conclusion and perspectives Mathematical analysis of the asymptotic behavior of an ABP method: interpretation as a self-interacting diffusion, reinforcement with the past almost sure convergence results (consistency) a general framework Some perspectives: more quantitative analysis? construction and analysis of ABF methods? References: M. Benaïm and C.-E. Bréhier. Convergence of adaptive biasing potential methods for diffusions. C. R. Math. Acad. Sci. Paris, M. Benaïm and C.-E. Bréhier. Convergence analysis of adaptive biasing potential methods for diffusions. Arxiv preprint, Thanks for your attention. C-E Bréhier (CNRS-Lyon) Convergence of ABP UCLA, / 31