A function space HMC algorithm with second order Langevin diffusion limit

Transcription

1 Bernoulli 221), 2016, DOI: /14-BEJ621 A function pace HMC algorithm with econd order Langevin diffuion limit MICHELA OTTOBRE 1, NATESH S. PILLAI 2, FRANK J. PINSKI 3 and ANDREW M. STUART 4 1 Department of Mathematic, Imperial College, 180 Queen Gate, SW7 2AZ, London, UK. michelaottobre@gmail.com 2 Department of Statitic, Harvard Univerity, 1 Oxford Street, Cambridge, MA 02138, USA. pillai@fa.harvard.edu 3 Department of Phyic, Univerity of Cincinnati, P.O. Box , Cincinnati, OH , USA. pinkifj@ucmail.uc.edu 4 Mathematic Intitute, Warwick Univerity, CV4 7AL, UK. a.m.tuart@warwick.ac.uk We decribe a new MCMC method optimized for the ampling of probability meaure on Hilbert pace which have a denity with repect to a Gauian; uch meaure arie in the Bayeian approach to invere problem, and in conditioned diffuion. Our algorithm i baed on two key deign principle: i) algorithm which are well defined in infinite dimenion reult in method which do not uffer from the cure of dimenionality when they are applied to approximation of the infinite dimenional target meaure on R N ; ii) nonreverible algorithm can have better mixing propertie compared to their reverible counterpart. The method we introduce i baed on the hybrid Monte Carlo algorithm, tailored to incorporate thee two deign principle. The main reult of thi paper tate that the new algorithm, appropriately recaled, converge weakly to a econd order Langevin diffuion on Hilbert pace; a a conequence the algorithm explore the approximate target meaure on R N in a number of tep which i independent of N. Wealo preent the underlying theory for the limiting nonreverible diffuion on Hilbert pace, including characterization of the invariant meaure, and we decribe numerical imulation demontrating that the propoed method ha favourable mixing propertie a an MCMC algorithm. Keyword: diffuion limit; function pace Markov chain Monte Carlo; hybrid Monte Carlo algorithm; econd order Langevin diffuion 1. Introduction Markov chain Monte Carlo MCMC) algorithm for ampling from high dimenional probability ditribution contitute an important part of Bayeian tatitical inference. Thi paper i focued on the deign and analyi of uch algorithm to ample a probability ditribution on an infinite dimenional Hilbert pace H defined via a denity with repect to a Gauian; uch problem arie in the Bayeian approach to invere problem or Bayeian nonparametric) [21] and in the theory of conditioned diffuion procee [9]. Metropoli Hating algorithm [10] contitute a popular cla of MCMC method for ampling an arbitrary probability meaure. They proceed by contructing an irreducible, reverible Markov chain by firt propoing a candidate move and then accepting it with a certain probability. The acceptance probability i choen o a to preerve the detailed balance condition enuring reveribility. In thi work, we build on the generalized ISI/BS

2 A function pace HMC algorithm with econd order Langevin diffuion limit 61 Hybrid Monte Carlo HMC) method of [11] to contruct a new nonreverible MCMC method appropriate for ampling meaure defined via denity with repect to a Gauian meaure on a Hilbert pace. We alo demontrate that, for a particular et of parameter value in the algorithm, there i a natural diffuion limit to the econd order Langevin SOL) equation with invariant meaure given by the target. We thu name the new method the SOL-HMC algorithm. Our contruction i motivated by the following two key deign principle: 1. deigning propoal which are well-defined on the Hilbert pace reult in MCMC method which do not uffer from the cure of dimenionality when applied to equence of approximating finite dimenional meaure on R N ; 2. nonreverible MCMC algorithm, which are hence not from the Metropoli Hating cla, can have better ampling propertie in comparion with their reverible counterpart. The idea behind the firt principle i explained in [6] which urvey a range of algorithm deigned pecifically to ample meaure defined via a denity with repect to a Gauian; the unifying theme i that the propoal i reverible with repect to the underlying Gauian o that the accept reject mechanim depend only on the likelihood function and not the prior ditribution. The econd principle above i alo well-documented: nonreverible Markov chain, often contructed by performing individual time-reverible 1 tep ucceively [12,13], or by building on Hamiltonian mechanic [7,11,15], may have better mixing propertie. Since the target ditribution ha upport on an infinite dimenional pace, practical implementation of MCMC involve dicretizing the parameter pace, reulting in a target meaure on R N, with N 1. It i well known that uch dicretization cheme can uffer from the cure of dimenionality: the efficiency of the algorithm decreae a the dimenion N of the dicretized pace grow large. One way of undertanding thi i through diffuion limit of the algorithm. In the context of meaure defined via denity with repect to Gauian, thi approach i taken in the paper [14,18] which how that the random walk Metropoli and Langevin algorithm require ON) and ON 1/3 ) tep repectively, to ample the approximating target meaure in R N. If, however, the algorithm i defined on Hilbert pace then it i poible to explore the target in O1) tep and thi may alo be demontrated by mean of a diffuion limit. The paper [17] ue thi idea to tudy a Metropoli Hating algorithm which i defined on Hilbert pace and i a mall modification of the random walk Metropoli method; the diffuion limit i a firt order reverible Langevin diffuion. Moreover the diffuion limit in [14,18] are derived under tationarity wherea the reult in [18] hold for any initial condition. The above dicuion ha important practical conequence: a implied by the above diffuion limit, algorithm which are well defined on the function pace how an order of magnitude improvement in the mixing time in thee high dimenional ampling problem. Here we employ imilar technique a that of [17] to tudy our new nonreverible MCMC method, and how that, after appropriate recaling, it converge to a econd order nonreverible Langevin diffuion. Our new algorithm i inpired by imilar algorithm in finite dimenion, tarting with the work of [11], who howed how the momentum update could be correlated in the original HMC method of [8], and the more recent work [5] which made the explicit connection to econd order Langevin diffuion; a helpful overview and dicuion may be found in [15]. 1 For a definition of time-reveribility, ee Section 2.3.

3 62 Ottobre, Pillai, Pinki and Stuart Diffuion limit reult imilar to our are proved in [4,5] for finite dimenional problem. In thoe paper, an accept reject mechanim i appended to variou tandard integrator for the firt and econd order Langevin equation, and hown not to detroy the trong pathwie convergence of the underlying method. The reaon for thi i that rejection are rare when mall time-tep are ued. The ame reaoning underlie the reult we preent here, although we conider an infinite dimenional etting and ue only weak convergence. Another exiting work underpinning that preented here i the paper [2] which generalize the hybrid Monte Carlo method for meaure defined via denity with repect to a Gauian o that it applie on Hilbert pace. Indeed the algorithm we introduce in thi paper include the one from [2] a a pecial cae and ue the plit-tep non-verlet) integrator firt ued there. The key idea of the plitting employed i to plit according to linear and nonlinear dynamic within the numerical Hamiltonian integration tep of the algorithm, rather than according to poition and momentum. Thi allow for an algorithm which exactly preerve the underlying Gauian reference meaure, without rejection, and i key to the fact that the method are defined on Hilbert pace even in the non-gauian cae. We now define the cla of model to which our main reult are applicable. Let π 0 and π be two meaure on a Hilbert pace H,,, ) and aume that π 0 i Gauian o that π 0 = N0,C), with C a covariance operator. The target meaure π i aumed to be abolutely continuou with repect to π 0 and given by the identity dπ dπ 0 x) = M exp x) ), x H 1.1) for a real valued functional which denote the negative log-likelihood in the cae of Bayeian inference) and M a normalizing contant. Although the above formulation may appear quite abtract, we emphaize that thi point to the wide-ranging applicability of our theory: the etting encompae a large cla of model ariing in practice, including nonparametric regreion uing Gauian random field and tatitical inference for diffuion procee and bridge ampling [9, 21]. In Section 2, we introduce our new algorithm. We tart in a finite dimenional context and then explain parametric choice made with reference to the high or infinite dimenional etting. We demontrate that variou other algorithm defined on Hilbert pace, uch a the function pace MALA [3] and function pace HMC algorithm [2], are pecial cae. In Section 3, we decribe the infinite dimenional etting in full and, in particular, detail the relationhip between the change of meaure, encapulated in, and the propertie of the Gauian prior π 0. Section 4 contain the theory of the SPDE which both motivate our cla of algorithm, and act a a limiting proce for a pecific intance of our algorithm applied on a equence of pace of increaing dimenion N. We prove exitence and uniquene of olution to the SPDE and characterize it invariant meaure. Section 5 contain tatement of the key diffuion limit Theorem 5.1. Whilt the tructure of the proof i outlined in ome detail, variou technical etimate are left for Appendice A and B. Section 6 contain ome numeric illutrating the new algorithm in the context of a problem from the theory of conditioned diffuion. We make ome brief concluding remark in Section 7. The new algorithm propoed and analyzed in thi paper i of interet for two primary reaon. Firtly, it contain a number of exiting function pace algorithm a pecial cae and hence play a ueful conceptual role in unifying thee method. Secondly, numerical evidence

4 A function pace HMC algorithm with econd order Langevin diffuion limit 63 demontrate that the method i comparable in efficiency to the function pace HMC method introduced in [2] for a tet problem ariing in conditioned diffuion; until now, the function pace HMC method wa the clear bet choice a demontrated numerically in [2]. Furthermore, our numerical reult indicate that for certain parameter choice in the SOL-HMC algorithm, and for certain target meaure, we are able to improve upon the performance of the function pace HMC algorithm, corroborating a imilar obervation made in [11] for the finite dimenional ampler that form motivation for the new family of algorithm that we propoe here. From a technical point of view, the diffuion limit proved in thi paper i imilar to that proved for the function pace MALA in [18], extending to the nonreverible cae; however ignificant technical iue arie which are not preent in the reverible cae and, in particular, incorporating momentum flip into the analyi, which occur for every rejected tep, require new idea. 2. The SOL-HMC algorithm In thi ection, we introduce the SOL-HMC algorithm tudied in thi paper. We firt decribe the baic idea from tochatic dynamic underlying thi work, doing o in the finite dimenional etting of H = R N, i.e., when the target meaure πq) i a probability meaure on R N of the form dπ q) exp q) ), dπ 0 where π 0 i a mean zero Gauian with covariance matrix C and q) i a function defined on R N. A key idea i to work with an extended phae pace in which the original variable are viewed a poition and then momenta are added to complement each poition. We then explain the advantage of working with velocitie rather than momenta, in the large dimenion limit. And then finally we introduce our propoed algorithm, which i built on the meaure preerving propertie of the econd order Langevin equation. A already mentioned, our algorithm will build on ome baic fact about Hamiltonian mechanic. For a ynopy about the Hamiltonian formalim ee Appendix C Meaure preerving dynamic in an extended phae pace Introduce the auxiliary variable p momentum ) and M a uer-pecified, ymmetric poitive definite ma matrix. Let 0 denote the Gauian on R2N defined a the independent product of Gauian N0, C) and N0, M) on the q and p coordinate, repectively, and define by d q, p) exp q) ). d 0 A key point to notice i that the marginal of q, p) with repect to q i the target meaure πq). Define the Hamiltonian in H : R 2N R given by Hq, p) = 1 2 p,m 1 p q,lq + q),

5 64 Ottobre, Pillai, Pinki and Stuart where L = C 1. The correponding canonical Hamiltonian differential equation i given by dq dt = H p = M 1 p, dp dt = H = Lq D q). 2.1) q Thi equation preerve any mooth function of Hq, p) and, a a conequence, the Liouville equation correponding to 2.1) preerve the probability denity of q, p), which i proportional to exp Hq, p)). Thi fact i the bai for HMC method [8] which randomly ample momentum from the Gauian N0, M) and then run the Hamiltonian flow for T time unit; the reulting Markov chain on q i πq) invariant. In practice, the Hamiltonian flow mut be integrated numerically, but if a uitable integrator i ued volume-preerving and time-reverible) then a imple accept reject compenation correct for numerical error. Define ) q z = p and J = Then the Hamiltonian ytem can be written a dz dt ) 0 I. I 0 = JDHz), 2.2) where, abuing notation, Hz) := Hq, p). The equation 2.2) preerve the meaure. Now define the matrix ) K1 0 K =, 0 K 2 where both K 1 and K 2 are ymmetric. The following SDE alo preerve the meaure : dz dt = KDHz) + 2K dw dt. Here W = W 1,W 2 ) denote a tandard Brownian motion on R 2N. Thi SDE decouple into two independent equation for q and p; the equation for q i what tatitician term the Langevin equation [18], namely dq dt = K ) dw 1 1 Lq + D q) + 2K1, dt whilt the equation for p i imply the Orntein Uhlenbeck proce: dp dt = K 2 M 1 p + dw 2 2K 2. dt

6 A function pace HMC algorithm with econd order Langevin diffuion limit 65 Dicretizing the Langevin equation rep., the random walk found by ignoring the drift) and adding an accept reject mechanim, lead to the Metropoli Adjuted Langevin MALA) rep., the Random Walk Metropoli RWM) algorithm). A natural idea i to try and combine benefit of the HMC algorithm, which couple the poition and momentum coordinate, with the MALA and RWM method. Thi thought experiment ugget conidering the econd order Langevin equation 2 dz dt = JDHz) KDHz) + 2K dw dt, 2.3) which alo preerve a a traightforward calculation with the Fokker Planck equation how Velocity rather than momentum Our paper i concerned with uing the equation 2.3) to motivate propoal for MCMC. In particular, we will be intereted in choice of the matrice M, K 1 and K 2 which lead to well-behaved algorithm in the limit of large N. To thi end, we write the equation 2.3) in poition and momentum coordinate a dq dt = ) dw 1 M 1 p K 1 Lq + D q) + 2K1, dt dp = Lq + D q) ) K 2 M 1 p + dw 2 2K 2. dt dt In our ubequent analyi, which concern the large N limit, it turn out to be ueful to work with velocity rather than momentum coordinate; thi i becaue the optimal algorithm in thi limit are baed on enuring that the velocity and poition coordinate all vary on the ame cale. For thi reaon, we introduce v = M 1 p and rewrite the equation a dq dt = v K ) dw 1 1 Lq + D q) + 2K1, dt M dv dt = Lq + D q) ) K 2 v + dw 2 2K 2. dt In the infinite dimenional etting, i.e., when H i an infinite dimenional Hilbert pace, thi equation i till well poed ee 2.5) below and Theorem 4.1). However in thi cae W 1 and W 2 are cylindrical Wiener procee on H ee Section 3.1) and L = C 1 i necearily an unbounded operator on H becaue the covariance operator C i trace cla on H. The unbounded operator introduce undeirable behaviour in the large N limit when we approximate them; thu we chooe M and the K i to remove the appearance of unbounded operator. To thi end, we 2 Phyicit often refer to thi a the Langevin equation for the choice K 1 0 which lead to noie only appearing in the momentum equation.

7 66 Ottobre, Pillai, Pinki and Stuart et M = L = C 1, K 1 = Ɣ 1 C and K 2 = Ɣ 2 C 1 and aume that Ɣ 1 and Ɣ 2 commute with C to obtain the equation or imply dq dt = v Ɣ ) 1 q + CD q) + 2Ɣ1 C dw 1, dt 2.4a) dv dt = q + CD q) ) Ɣ 2 v + 2Ɣ 2 C dw 2, dt 2.4b) dq dt = v Ɣ ) db 1 1 q + CD q) + 2Ɣ1, dt 2.5a) dv dt = q + CD q) ) Ɣ 2 v + db 2 2Ɣ 2. dt 2.5b) In the above, B 1 and B 2 are H-valued Brownian motion with covariance operator C. Thi equation i well-behaved in infinite dimenion provided that the Ɣ i are bounded operator, and under natural aumption relating the reference meaure, via it covariance C, and the log denity, which i a real valued functional defined on an appropriate ubpace of H. Detailed definition and aumption regarding 2.5) are contained in the next Section 3. Under uch aumption, the function Fq):= q + CD q) 2.6) ha deirable propertie ee Lemma 3.4), making the exitence theory for 2.5) traightforward. We develop uch theory in Section 4 ee Theorem 4.1. Furthermore, in Theorem 4.2 we will alo prove that equation 2.5) preerve the meaure dq,dv) defined by d d 0 q, v) exp q) ), 2.7) where 0 i the independent product of N0, C) with itelf. The meaure rep., 0 )iimply the meaure rep., 0 ) in the cae M = C 1 and rewritten in q, v) coordinate intead of q, p). In finite dimenion, the invariance of follow from the dicuion concerning the invariance of Function pace algorithm We note that the choice Ɣ 1 0 give the tandard phyicit) Langevin equation d 2 q dt dq + Ɣ 2 dt + q + CD q) ) = 2Ɣ 2 C dw ) dt In thi ection, we decribe an MCMC method deigned to ample the meaure given by 2.7) and hence, by marginalization, the meaure π given by 1.1). The method i baed on dicretization of the econd order Langevin equation 2.8), written a the hypo-elliptic firt order

8 A function pace HMC algorithm with econd order Langevin diffuion limit 67 equation 2.9) below. In the finite dimenional etting, a method cloely related to the one that we introduce wa propoed in [11]; however we will introduce different Hamiltonian olver which are tuned to the pecific tructure of our meaure, in particular to the fact that it i defined via denity with repect to a Gauian. We will be particularly intereted in choice of parameter in the algorithm which enure that the output uitability interpolated to continuou time) behave like 2.8) whilt, a i natural for MCMC method, exactly preerving the invariant meaure. Thi perpective on dicretization of the phyicit) Langevin equation in finite dimenion wa introduced in [4,5]. In poition/velocity coordinate, and uing 2.6), 2.5) become dq dt = v, 2.9) dv dt = Fq) Ɣ 2v + 2Ɣ 2 C dw 2. dt The algorithm we ue i baed on plitting 2.9) into an Orntein Uhlenbeck OU) proce and a Hamiltonian ODE. The OU proce i and the Hamiltonian ODE i given by dq dt = 0, 2.10) dv dt = Ɣ 2v + 2Ɣ 2 C dw 2, dt dq dt = v, 2.11) dv dt = Fq). The olution of the OU proce 2.10) i denoted by qt), vt)) = 0 q0), v0); ξ t ); here ξ t i a mean zero Gauian random variable with covariance operator CI exp 2tƔ 2 )). Notice that the dynamic given by both 2.10) and by 2.11) preerve the target meaure given in 2.7). Thi naturally ugget contructing an algorithm baed on alternating the above two dynamic. However, note that whilt 2.10) can be olved exactly, 2.11) require a further numerical approximation. If the numerical approximation i baed on a volume-preerving and time-reverible numerical integrator, then the accept reject criterion for the reulting MCMC algorithm can be eaily expreed in term of the energy difference in H. Aflowϕ t on R 2N i aid to be timereverible if ϕ t q0), v0)) = qt), vt)) implie ϕ t qt), vt)) = q0), v0)). Defintion of time-reverible and dicuion of the role of time-reverible and volume-preerving integrator may be found in [20]. To contruct volume-preerving and time-reverible integrator, the Hamiltonian integration will be performed by a further plitting of 2.11). The uual plitting for the widely ued Verlet method i via the velocity and the poition coordinate [11]. Motivated by our infinite dimenional etting, we replace the Verlet integration by the plitting method propoed in [2]; thi

9 68 Ottobre, Pillai, Pinki and Stuart lead to an algorithm which i exact no rejection) in the purely Gauian cae where 0. The plitting method propoed in [2] i via the linear and nonlinear part of the problem, leading u to conider the two equation dq dt = v, dv dt with olution denoted a qt), vt)) = R t q0), v0)); and dq dt = 0, dv dt = q, 2.12) = CD q), 2.13) with olution denoted a qt), vt)) = t 1 q0), v0)). We note that the map χ t = t/2 1 R t t/2 1 i a volume-preerving and time-reverible econd order accurate approximation of the Hamiltonian ODE 2.11). We introduce the notation χτ t = χ t χ t) τ, time t to denote integration, uing thi method, up to time τ. Thi integrator can be made to preerve the meaure if appended with a uitable accept reject rule a detailed below. On the other hand the tochatic map t 0 preerve ince it leave q invariant and ince the OU proce, which i olved exactly, preerve 0. We now take thi idea to define our MCMC method. The infinite dimenional Hilbert pace H H in which the chain i contructed will be properly defined in the next ection. Here we focu on the algorithm, which will be explained in more detail and analyzed in Section 5. Define the operation o that v i the velocity component of δ 0 q, v). The preceding conideration ugget that from point q 0,v 0 ) H H we make the propoal q 1,v 1 ) = χ h τ δ 0 q 0,v 0) and that the acceptance probability i given by where α x 0,ξ δ) := 1 exp H q 0, v 0) ) H q 1,v 1 )), Hq, v) = 1 2 q,c 1 q v,c 1 v + q), 2.14), denoting calar product in H. One tep of the reulting MCMC method i then defined by etting q 1,v 1) = q 1,v1 ) with probability α x 0,ξ δ) 2.15) = q 0, v 0) ) otherwie.

10 A function pace HMC algorithm with econd order Langevin diffuion limit 69 We will make further comment on thi algorithm and on the expreion 2.14) for the Hamiltonian in Section 5, eeremark5.6. Here it uffice imply to note that whilt H will be almot urely infinite, the energy difference i well-defined for the algorithm we employ. We tre that when the propoal i rejected the chain doe not remain in q 0,v 0 ) butitmovetoq 0, v 0 ) ); i.e., the poition coordinate tay the ame while the velocity coordinate i firt evolved according to 2.10) and then the ign i flipped. Thi flipping of the ign, needed to preerve reveribility, lead to ome of the main technical difference with repect to [17]; ee Remark Forthe finite dimenional cae with Verlet integration the form of the accept reject mechanim and, in particular, the ign-reveral in the velocity, wa firt derived in [11] and i dicued in further detail in Section 5.3 of [15]. The algorithm 2.15) preerve and we refer to it a the SOL-HMC algorithm. Recalling that v denote the velocity component of δ 0 q, v), we can equivalently ue the notation αx,ξ δ ) and αq,v ),forx = q, v) indeed, by the definition of δ 0, v depend on ξ δ ). With thi in mind, the peudo-code for the SOL-HMC i a follow. SOL-HMC in H : 1. Pick q 0,v 0 ) H H and et k = 0; 2. given q k,v k ), define v k ) to be the v-component of δ 0 qk,v k ) and calculate the propoal q k+1,v k+1 ) = χ h τ q k, v k) ) ; 3. define the acceptance probability αq k,v k ) ); 4. et q k+1,v k+1 ) = q k+1,v k+1 ) with probability αq k,v k ) ); otherwie et q k+1,v k+1 ) = q k, v k ) ); 5. et k k + 1 and go to 2). Theorem 2.1. Let Aumption 3.1 hold. For any δ,h,τ > 0, the Markov chain defined by 2.15) i invariant with repect to given by 2.7). Proof. See Appendix A. Remark 2.2. We firt note that if δ then the algorithm 2.15) i that introduced in the paper [2]. From thi it follow that, if δ = and τ = h, then the algorithm i imply the function-pace Langevin introduced in [3]. Secondly, we mention that, in the numerical experiment reported later, we will chooe Ɣ 2 = I. The olution of the OU proce 2.10)forv i thu given a vδ) = 1 ι 2) 1/2 v0) + ιw, 2.16) where w N0,C) and e 2δ = 1 ι 2 ). The numerical experiment will be decribed in term of the parameter ι rather than δ.

11 70 Ottobre, Pillai, Pinki and Stuart 3. Preliminarie In thi ection, we detail the notation and the aumption Section 3.1 and Section 3.2, rep.) that we will ue in the ret of the paper Notation Let H,,, ) denote a eparable Hilbert pace of real valued function with the canonical norm derived from the inner-product. Let C be a poitive, trace cla operator on H and {ϕ j,λ 2 j } j 1 be the eigenfunction and eigenvalue of C, repectively, o that Cϕ j = λ 2 j ϕ j for j N. We aume a normalization under which {ϕ j } j 1 form a complete orthonormal bai in H. For every x H we have the repreentation x = j x j ϕ j, where x j = x,ϕ j. Uing thi notation, we define Sobolev-like pace H r,r R, with the inner product and norm defined by x,y r = j 2r x j y j and x 2 r = j 2r xj 2. j=1 Notice that H 0 = H. Furthermore, H r H H r for any r>0. The Hilbert Schmidt norm C i defined a x 2 C = C 1/2 x 2 = j=1 j=1 λ 2 j x 2 j. For r R, letq r : H H denote the operator which i diagonal in the bai {ϕ j } j 1 with diagonal entrie j 2r, i.e., Q r ϕ j = j 2r ϕ j o that Q 1/2 r ϕ j = j r ϕ j. The operator Q r let u alternate between the Hilbert pace H and the interpolation pace H r via the identitie: x,y r = Q 1/2 r x,q 1/2 r y and x 2 r = Q 1/2 r x 2. Since Q 1/2 r ϕ k r = ϕ k =1, we deduce that {Q 1/2 r ϕ k } k 1 form an orthonormal bai for H r. A function y N0, C) can be expreed a y = D λ j ρ j ϕ j with ρ j N0, 1) i.i.d; 3.1) j=1

12 A function pace HMC algorithm with econd order Langevin diffuion limit 71 if j λ2 j j 2r < then y can be equivalently written a y = λj j r) 1/2 ) ρ j Q r ϕ j j=1 with ρ j D N0, 1) i.i.d. 3.2) For a poitive, elf-adjoint operator D : H H, it trace in H i defined a Trace H D) def = ϕ j,dϕ j. We tre that in the above {ϕ j } j N i an orthonormal bai for H,, ). Therefore if D : H r H r, it trace in H r i Trace H r D) def 1/2 = Q r ϕ j, DQ 1/2 r ϕ j r. j=1 Since Trace H r D) doe not depend on the orthonormal bai, the operator D i aid to be trace cla in H r if Trace H r D) < for ome, and hence any, orthonormal bai of H r. Becaue C i defined on H, the covariance operator j=1 C r = Q 1/2 r CQ 1/2 r 3.3) i defined on H r. With thi definition, for all the value of r uch that Trace H r C r ) = j λ2 j j 2r <, we can think of y a a mean zero Gauian random variable with covariance operator C in H and C r in H r ee 3.1) and 3.2)). In the ame way, if Trace H r C r )< then B 2 t) = λ j β j t)ϕ j = λ j j r β j t) ˆϕ j, j=1 with {β j t)} j N a collection of i.i.d. tandard Brownian motion on R, can be equivalently undertood a an H-valued C-Brownian motion or a an H r -valued C r -Brownian motion. In the next ection, we will need the cylindrical Wiener proce Wt) which i defined via the um Wt):= j=1 β j t)ϕ j. j=1 Thi proce i H r -valued for any r< 1 2. Oberve now that if {ˆϕ j } j N i an orthonormal bai of H r then, denoting H r H r ˆϕ 1 j = ˆϕ j, 0) and H r H r ˆϕ 2 j = 0, ˆϕ j ), F ={ˆϕ 1 j, ˆϕ2 j } j N i an orthonormal bai for H r H r.letc r : H r H r H r H r be the diagonal operator uch that C r ˆϕ 1 j = 0, 0), C r ˆϕ 2 j = j 2r λ 2 j ˆϕ2 j = 0, C r ˆϕ j ) j N

13 72 Ottobre, Pillai, Pinki and Stuart and C r : H r H r H r H r be the diagonal operator uch that C r ˆϕ 1 j = j 2r λ 2 j ˆϕ1 j = C r ˆϕ j, 0), C r ˆϕ 2 j = j 2r λ 2 j ˆϕ2 j = 0, C r ˆϕ j ) j N. 3.4) Conitently, Bt) := 0,B 2 t)) will denote an H r H r valued Brownian motion with covariance operator C r and Bt) := B 1 t), B 2 t)) will denote a H r H r valued Brownian motion with covariance operator C r. In other word, B 1 t) and B 2 t) are independent H r -valued C r -Brownian motion. Throughout, we ue the following notation. Two equence of nonnegative real number {α n } n 0 and {β n } n 0 atify α n β n if there exit a contant K>0 atifying α n Kβ n for all n 0. The notation α n β n mean that α n β n and β n α n. Two equence of nonnegative real function {f n } n 0 and {g n } n 0 defined on the ame et atify f n g n if there exit a contant K>0 atifying f n x) Kg n x) for all n 0 and all x. The notation f n g n mean that f n g n and g n f n. The notation E x [fx,ξ)] denote expectation with variable x fixed, while the randomne preent in ξ i averaged out. Alo, let H r denote the outer product operator in H r defined by x H r y)z def = y,z r x x,y,z H r. For an operator A : H r H l, we denote it operator norm by LH r,h l ) defined by A LH r,h l ) def = up Ax l. x r =1 For elf-adjoint A and r = l = 0 thi i, of coure, the pectral radiu of A. Finally, in the following we will conider the product pace H r H r. The norm of w = w 1,w 2 ) H r H r i w 2 r r := w 1 2 r + w 2 2 r Aumption In thi ection, we decribe the aumption on the covariance operator C of the Gauian meaure D π 0 N0, C) and the functional. We fix a ditinguihed exponent >0 and aume that : H R and Trace H C )<. For each x H the derivative D x) i an element of the dual H ) of H dual with repect to the topology induced by the norm in H), compriing the linear functional on H. However, we may identify H ) = H and view D x) a an element of H for each x H. With thi identification, the following identity hold: D x) LH,R) = D x) ;

14 A function pace HMC algorithm with econd order Langevin diffuion limit 73 furthermore, the econd derivative 2 x) can be identified with an element of LH, H ).To avoid technicalitie, we aume that x) i quadratically bounded, with firt derivative linearly bounded and econd derivative globally bounded. Weaker aumption could be dealt with by ue of topping time argument. Aumption 3.1. The functional, covariance operator C and the operator Ɣ 1,Ɣ 2 atify the following aumption. 1. Decay of Eigenvalue λ 2 j of C: there exit a contant κ>1 2 uch that λ j j κ. 2. Domain of : there exit an exponent [0,κ 1/2) uch that i defined everywhere on H. 3. Size of : the functional : H R atifie the growth condition 0 x) 1 + x Derivative of : The derivative of atify D x) 1 + x and 2 x) LH,H ) ) 5. Propertie of the Ɣ i : The operator Ɣ 1,Ɣ 2 commute with C and are bounded linear operator from H into itelf. Remark 3.2. The condition κ> 1 2 enure that Trace H r C r)< for any r<κ 1 2 : thi implie that π 0 H r ) = 1 for any τ>0and r<κ 1 2. Remark 3.3. The functional x) = 1 2 x 2 i defined on H and it derivative at x H i given by D x) = j 0 j 2 x j ϕ j H with D x) = x. The econd derivative 2 x) LH, H ) i the linear operator that map u H to j 0 j 2 u, ϕ j ϕ j H : it norm atifie 2 x) LH,H ) = 1 for any x H. The Aumption 3.1 enure that the functional behave well in a ene made precie in the following lemma. Lemma 3.4. Let Aumption 3.1 hold. 1. The function Fx)given by 2.6) i globally Lipchitz on H : Fx) Fy) x y x,y H. 2. The econd order remainder term in the Taylor expanion of atifie y) x) D x), y x y x 2 x,y H. 3.6) Proof. See [14,17].

15 74 Ottobre, Pillai, Pinki and Stuart 4. SPDE theory In thi ection, we tudy the SDE 2.5) in the infinite dimenional Hilbert pace etting; we work under the aumption pecified in the previou ection. Recall that our goal i to ample the meaure π in 1.1), but that we have extended our tate pace to obtain the meaure given by 2.7), with q marginal given by π. Here 0 i the independent product of π 0 = N0, C) with itelf in the q and p coordinate. The finite dimenional argument in Section 2 how that the equation 2.5) preerve 0. The aim of thi ection i to how that thee tep all make ene in the infinite dimenional context, under the aumption laid out in the previou ection. Theorem 4.1. Let Aumption 3.1 hold. Then, for any initial condition q0), v0)) H H, any T>0 and almot every H H -valued C -Brownian motion Bt) = B 1 t), B 2 t)), there exit a unique olution of the SDE 2.5) in the pace C[0,T], H H ). Furthermore, the Itô map B 1,B 2 ) C[0,T]; H H ) q, v) C[0,T]; H H ) i Lipchitz. Proof. If we define ) q x =, v together with the operator ) Ɣ1 0 Ɣ =, 0 Ɣ 2 then equation 2.5) take the form dx dt = Gx) + 2Ɣ d B dt, 4.1) where ) v Ɣ1 Fq) Gx) =. 4.2) Fq) Ɣ 2 v A olution of 2.5) atifie the integral equation t xt) = x 0 + G x) ) d + 2Ɣ Bt), 0 where x0) = x 0. By virtue of Lemma 3.4, we ee that G : H H H H i globally Lipchitz. Furthermore, Remark 3.2 how that B C[0,T]; H H ) almot urely. To prove exitence and uniquene of a olution, we conider the map : C[0,T]; H H ) C[0,T]; H H ) defined by t x)t) := x 0 + G x) ) d + 2Ɣ Bt). 0

16 A function pace HMC algorithm with econd order Langevin diffuion limit 75 Since F i globally Lipchitz from H into itelf, it follow that G i globally Lipchitz from H H into itelf. Thi in turn implie that i Lipchitz and that, furthermore, the Lipchitz contant may be made le than one, by chooing t ufficiently mall. From thi exitence and uniquene of a olution follow by the contraction mapping principle, on time-interval ufficiently mall. The argument may then be repeated on ucceive time-interval to prove the reult on any time-interval [0,T]. Now let ϒ : x 0, B) H H C [0,T]; H H ) x C [0,T]; H H ). 4.3) The argument ued in Lemma 3.7 of [14] how that ϒ i Lipchitz continuou and hence the deired propertie of the Itô map follow. For N N, leth N denote the linear pan of the firt N eigenfunction of C, P N : H H N denote the projection map and N = P N. Define Q N = I P N. Recall equation 2.4). Let ϒ N denote the Ito map obtained by replacing D by P N D N in 2.4). The following i the key reult of thi ection. Our choice of meaure 2.7) and dynamic 2.4) have been coordinated to enure that the reulting tochatic dynamic preerve : Theorem 4.2. For any initial condition q0), v0)) and any T>0, the equation 2.4) preerve : qt ), vt )). Proof. The proof follow along the line of Theorem 3.1 of [2]. The key idea i to exploit the fact that for finite dimenional H, the invariance of under the dynamic qt ), vt )) follow eaily. From thi, the invariance for an infinite dimenional H follow from an application of the dominated convergence theorem which we outline below. We let W denote the Weiner meaure on X = C[0,T]; H H ) induced by Brownian motion with covariance the ame a that of 0. For any continuou, bounded function g : H H R and T>0, we need to how that g ϒq,v,W) ) exp q) ) d 0 q, v) dww ) H X 4.4) = gq,v)exp q) ) d 0 q, v). H Firt, we claim that for any N N, g ϒ N q,v,w) ) exp N q) ) d 0 q, v) dww ) H X = gq,v)exp N q) ) d 0 q, v). H Thi follow from the fact that the flow ϒ N preerve the invariant meaure proportional to exp N ) 0 a obtained below in Lemma )

17 76 Ottobre, Pillai, Pinki and Stuart In Lemma 4.4 below, we will how that ϒ N converge pointwie to ϒ. Thu by the continuity of g, gϒ N q,v,w)) converge pointwie to gϒq,v,w)). Clearly, exp N q)) converge to exp q)) pointwie. Since g i bounded and, N are poitive, by the dominated convergence theorem the right, repectively, left-hand ide of 4.5) converge to the right, repectively, left-hand ide of 4.4) and the claim follow. Lemma 4.3. Let Aumption 3.1 hold. The meaure N exp N ) 0 factor a the product of two meaure on P N H and Q N H. The meaure N exp N ) 0 i preerved by ϒ N. Proof. By contruction, the meaure 0 factor a the product of two meaure μ 0 = N0,P N CP N ) and μ 0 = N0,QN CQ N ). Since N i 0 on Q N, it follow that N factor into μ 1 exp N )μ 0 on P N H and μ 1 = μ 0 on QN H. Now, a explained in Section 2 for any N, μ 1 i invariant for P N ϒ N. Alo etting = 0in 2.4) reult in an OU flow on H for which 0 i invariant. Thu if D i replaced by P N D N in 2.4), the reulting flow on Q N i an Ortein Uhlenbeck proce with invariant meaure μ 1. Since N i a product of μ 1 and μ 1, the reult follow. The following reult how the pointwie convergence of ϒ N to ϒ. Lemma 4.4. Let Aumption 3.1 hold. A N, ϒ N x 0, B) converge to ϒx 0, B) for every x 0, B) H H C[0,T]; H H ). Proof. Proceeding imilarly a in Theorem 4.1,et where dx N dt = G N x) + 2Ɣ d B dt, G N v Ɣ1 F x) = N ) q) F N q) Ɣ 2 v 4.6) with F N q) = q + CP N D N q).letxt) denote the olution of 2.4) and x N above atifie t x N t) = x 0 + G N x N ) ) d + 2Ɣ Bt), 4.7) 0 where x0) = x 0. Set e = x x N. The pointwie convergence of ϒ N to ϒ i etablihed by howing that e 0 in the path pace C[0,T]; H H ). We firt decompoe: Gx) G N x N) = Gx) G N x) ) + G N x) G N x N )). 4.8) Next, it can be hown that G N i globally Lipchitz with the Lipchitz contant L independent of N ee [18], Lemma 4.1). Thu we have G N xt)) G N x N t)) L et). Combining

18 A function pace HMC algorithm with econd order Langevin diffuion limit 77 thi bound with 4.7) and 4.8), et) t L eu) du + 0 t Thu by Gronwall inequality, it uffice to how that a N. To thi end, write 0 G xu) ) G N xu) ) du. ) up G xt) G N xt) ) 0 0 t T Fx) F N x) = CD x) CP N D x) ) + CP N D x) CP N D N x) ). Since CD i globally Lipchitz, G xt) ) G N xt) ) I P N ) CD xt) ) + I P N) xt). 4.9) From the exitence of a global olution for 2.4) a hown in Theorem 4.1, it follow that up 0 t T xt) <. Thu from 4.9) we infer that up 0 t T Gxt)) G N xt)) 0, and the claim follow. 5. Diffuion limit of algorithm The main reult of thi ection i the diffuion limit Theorem 5.1: uing the precription 2.15) and etting δ = h = τ, we contruct a equence of Markov chain x k,δ i.e., for every fixed delta, {x k,δ } k i a Markov chain) and conider the proce z δ t) which i the continuou time interpolant of the chain x k,δ. Then z δ t) converge to the olution of the SDE 5.9), which i a pecific intance of 4.1), when Ɣ 1 = 0. By Theorem 4.2, the flow 5.9) preerve the meaure defined in 2.7). More preciely, for q,v H,letx H H denote the pair x = q, v); we recall that the norm of x i then x 2 := q 2 + v 2. With the algorithm decribed in Section 2.3, taking δ = h = τ we contruct the Markov chain x k+1,δ := q k+1,δ,v k+1,δ ) a follow q k+1,δ,v k+1,δ) = q k+1,δ,v k+1,δ ) = q k,δ, v k,δ) ) with probability α k,δ 5.1) otherwie, where α k,δ = α x k,δ,ξ δ) := 1 exp H q k,δ, v k,δ) ) H q k+1,δ,v k+1,δ )).

19 78 Ottobre, Pillai, Pinki and Stuart We pecify that in the above q k+1,δ,v k+1,δ ) = χ δ δ 0 q k,δ,v k,δ) and q k,δ ), v k,δ ) ) = q k,δ, P v δ 0 q k,δ,v k,δ))), where for x H H, we denote by P q x) and P v x) the projection of x on the q and v component, repectively. Notice that introducing γ k,δ Bernoulliα k,δ ), the algorithm 5.1) can be alo written a q k+1,δ,v k+1,δ) = γ k,δ q k+1,δ,v k+1,δ ) + q k,δ, v k,δ) ). Following [17], we conider the piecewie linear and the piecewie contant interpolant of the chain x k,δ, z δ t) and z δ t), repectively: z δ t) := 1 δ t t k)x k+1,δ + 1 δ t k+1 t)x k,δ, t k t<t k+1,t k = kδ, 5.2) z δ t) := x k,δ, t k t<t k+1,t k = kδ. 5.3) Decompoe the chain x k,δ into it drift and martingale part: x k+1,δ = x k,δ + δg δ x k,δ) + 2δSM k,δ, where [ ] Id 0 S =, 0 Ɣ 2 G δ x) := 1 δ E x[ x k+1,δ x k,δ x k,δ = x ], 5.4) M k,δ := S 1/2 2δ x k+1,δ x k,δ δg δ x k,δ)), 5.5) M δ x) := E [ M k,δ x k,δ = x ]. 5.6) Notice that with thi definition, if F k,δ i the filtration generated by {x j,δ,γ j,δ,ξ δ,j = 0,...,k}, we have E[M k,δ F k,δ ]=0. Alo, let u introduce the recaled noie proce B δ t) := k 1 2Sδ j=0 M j,δ + A imple calculation, which we preent in Appendix A, how that 2S δ t t k)m k,δ, t k t<t k ) z δ t) = ϒ x 0, ˆB δ), 5.8)

20 A function pace HMC algorithm with econd order Langevin diffuion limit 79 where ϒ i the map defined in 4.3) and ˆB δ i the recaled noie proce B δ plu a term which we will how to be mall: ˆB δ t) := B δ t) + t 0 [ G δ z δ u) ) G z δ u) )] du; we tre that in the above and throughout thi ection the map Gx) i a in 4.2) with Ɣ 1 = 0. Let B 2 t) be an H -valued C -Brownian motion we recall that the covariance operator C ha been defined in 3.3)) and H H Bt) = 0,B 2 t)). Recall the SPDE 2.5) written in the form 4.1). The main reult of thi ection i the following diffuion limit of the Markov chain 5.1) to4.1). Theorem 5.1 Diffuion limit). Let Aumption 3.1 hold and let H,, ) be a eparable Hilbert pace, x k,δ be the Markov chain 5.1) tarting at x 0,δ = x 0 H H and let z δ t) be the proce defined by 5.2). If Aumption 3.1 hold, then z δ t) converge weakly in C[0,T]; H H ) to the olution zt) H H of the tochatic differential equation dzt) = Gz) dt + 2Ɣ dbt), z0) = x ) The diffuion limit can be proven a a conequence of [17], Lemma 3.5. Propoition 5.4 below i a lightly more general verion of [17], Lemma 3.5. Proof of Theorem 5.1. Theorem5.1 followa a conequenceofpropoition5.4 andlemma5.5 below. Conider the following condition: Condition 5.2. The Markov chain x k,δ H H defined in 5.1) atifie Convergence of the approximate drift. There exit a globally Liphitz function G : H H H H, a real number a>0 and an integer p 1 uch that G δ x) Gx) δ a 1 + x p ). 5.10) Size of the increment. There exit a real number r>0 and an integer n 1 uch that E [ x k+1,δ x k,δ x k,δ = x ] δ r 1 + x n ). 5.11) A priori bound. There exit a real number ε uch that 1 ε + a r) > 0with a and r a in 5.10) and 5.11), rep.) and the following bound hold: { up δ ε E δ 0,1/2) [ kδ T x k,δ p n ]} <. 5.12)

21 80 Ottobre, Pillai, Pinki and Stuart Invariance principle. A δ tend to zero the equence of procee B δ defined in 5.7) converge weakly in C[0,T]; H H ) to the Brownian motion H H B = 0,B 2 ) where B 2 i a H -valued, C -Brownian motion. Remark 5.3. Notice that if 5.10) hold for ome a>0 and p 1, then [ E x k+1,δ x k,δ x k,δ] δ 1 + x p ) 5.13) and Indeed E [ x k+1,δ x k,δ x k,δ = x ] G δ x) 1 + x p. 5.14) = δ G δ x) δ G δ x) Gx) + δ Gx) δ 1 + x p ), having ued the Liphitzianity of the map Gx). Analogouly one can obtain 5.14) a well. Propoition 5.4. Let Aumption 3.1 hold and let H,, ) be a eparable Hilbert pace and x k,δ a equence of H H valued Markov chain with x 0,δ = x 0. Suppoe the drift martingale decompoition 5.4) 5.5) of x k,δ atifie Condition 5.2. Then the equence of interpolant z δ t) defined in 5.2) converge weakly in C[0,T]; H H ) to the olution zt) H H of the tochatic differential equation 5.9). Proof. Thank to the Liphitzianity of the map ϒ in 5.8) ee Theorem 4.1), the proof i analogou to the proof of [17], Lemma 3.5. We ketch it in Appendix A. Lemma 5.5. Let Aumption 3.1 hold and let x k,δ be the Markov chain 5.1) tarting at x 0,δ = x 0 H H. Under Aumption 3.1 the drift martingale decompoition of x k,δ,5.4) 5.5), atifie Condition 5.2. The remainder of thi ection i devoted to proving Lemma 5.5, which i needed to prove Theorem 5.1. Firt, in Section 5.1 we lit and explain everal preliminary technical lemmata, which will be proved in Appendix B. The main one i Lemma 5.7, where we tudy the acceptance probability. Then, in Section 5.2 and Section 5.3, we prove Lemma 5.5; in order to prove uch a lemma we need to how that if Aumption 3.1 hold, the four condition lited in Condition 5.2 are atified by the chain x k,δ. To thi end, Lemma 5.10 prove that 5.10) hold with a = 1 and p = 6; Lemma 5.11 how that 5.11) i atified with r = 1/2 and n = 6;theapriori bound 5.12) i proved to hold for ε = 1 and for any power of x k,δ in Lemma 5.12; finally, Lemma 5.18 i the invariance principle. Proof of Lemma 5.5. Lemma 5.5 follow a a conequence of Lemma 5.10, Lemma 5.11, Lemma 5.12 and Lemma 5.18.

22 A function pace HMC algorithm with econd order Langevin diffuion limit Preliminary etimate We firt analye the acceptance probability. Given the current tate of the chain x k,δ = x = q, v), the acceptance probability of the propoal q,v ) i α δ := α 0,δ x,ξ δ) = 1 exp H q,v ) Hq,v ) ) = 1 exp H q,v )). 5.15) Similarly, we denote γ δ := γ 0,δ Bernoulli α δ). For an infinite dimenional Hilbert pace etting, the matter of the well-poedne of the expreion for the acceptance probability i not obviou; we comment on thi below. Remark 5.6. Before proceeding to the analyi, let u make a few obervation about the expreion 5.15) for the acceptance probability. A we have already mentioned, the flip of the ign of the velocity in cae of rejection of the propoal move guarantee time-reveribility. A a conequence, the propoal move are ymmetric and the acceptance probability can be defined only in term of the energy difference. We are lightly abuing notation in going from the original Hq,p) to Hq,v). However notice that Hq, v) i preerved by the flow 2.11). The relevant energy difference here i Hq, v ) Hq,v ) rather than Hq, v) Hq,v )); indeed the firt tep in the definition of the propoal q,v ), namely the OU proce δ 0 q, v), i baed on an exact integration and preerve the deired invariant meaure. Therefore, the accept reject mechanim which i here only to preerve the overall reveribility of the chain by accounting for the numerical error made by the integrator χτ h) doen t need to include alo the energy difference Hq, v) Hq, v ). The Hamiltonian Hq, v), defined in 2.14), i almot urely infinite in an infinite dimenional context; thi can be een by jut applying a zero one law to the erie repreentation of the calar product q,c 1 q. However, in order for the acceptance probability to be well defined, all we need i for the difference Hq, v ) Hq,v ) to be almot urely finite, i.e., for Hq, v ) to be a bounded operator. Thi i here the cae thank to the choice of the Verlet algorithm. Indeed from [2], page 2212, we know that H q,v ) = q) q ) δ D q), v + ) D q ), v 2 + δ2 C 1/2 D q ) 2 C 1/2 D q) 2 ). 8 More detail on thi fact can be found in [2], page 2210, 2212, Lemma 5.7. Let Aumption 3.1 hold. Then, for any p 1, E x 1 α δ p δ 2p 1 + q 4p + v 4p ). 5.16)

23 82 Ottobre, Pillai, Pinki and Stuart Proof. The proof of Lemma 5.7 can be found in Appendix B. The above 5.16) quantifie the intuition that the acceptance rate i very high, i.e., the propoal i rejected very rarely. Therefore, the analyi of Section 5.2 and Section 5.3 i done by bearing in mind that everything goe a if α δ were equal to one. We now tate a few technical reult, gathered in Lemma 5.8 and Lemma 5.9, that will be frequently ued in the following. Lemma 5.8. Let Aumption 3.1 hold. Then, for any q, q,v,ṽ H, CD q) CD q) q q and D q), v ) 1 + q v ; CD q) 1 + q ) ; 5.17) D q), v D q),ṽ v q q q ) v ṽ ; 5.18) C 1/2 D q) 1 + q ; C 1/2 D q) C 1/2 D q) q q. 5.19) Proof. See [2], Lemma 4.1. Recall that B 2 t) i an H -valued C -Brownian motion and that ξ δ i the noie component of the OU proce δ 0, i.e., v = e δɣ 2 v + δ By integrating χ δ and δ 0, the propoal move at tep k, xk+1,δ q k+1,δ v k+1,δ 0 e δ u)ɣ 2 2Ɣ2 db 2 u) =: e δɣ 2 v + ξ δ. 5.20) = q k+1,δ,v k+1,δ ), i given by = co δq k,δ + in δ v k,δ) δ 2 in δcd q k,δ), 5.21) = in δq k,δ + co δ v k,δ) δ 2 co δcd q k,δ) δ 2 CD q k+1,δ ). 5.22) If γ k := γ k,δ Bernoulliα k,δ ), then the k + 1)th tep of the Markov chain i q k+1,δ = γ k q k+1,δ + 1 γ k) q k,δ, v k+1,δ = γ k v k+1,δ 1 γ k) v k,δ). 5.23) Lemma 5.9. Let Aumption 3.1 hold. Then, for any p 1, we have E ξ δ p δp/2 ; 5.24) E v k,δ) x k,δ = x p 1 + v p ; 5.25) E [ q k+1,δ q k,δ p xk,δ = x ] δ p 1 + q p + v p ). 5.26)

24 A function pace HMC algorithm with econd order Langevin diffuion limit 83 Proof. See Appendix B Analyi of the drift Let Gx) be the map in 4.2) with Ɣ 1 = 0, i.e., [ Gx) = Gq, v) = v q CD q) Ɣ 2 v and G i x) and G δ i,i = 1, 2, be the ith component of G and Gδ, repectively. Lemma Let Aumption 3.1 hold. Then, for any x = q, v) H H, G δ 1 x) G 1x) δ 1 + q 6 ) + v 6, 5.27) G δ 2 x) G 2 x) δ 1 + q 6 ) + v 6. Proof. By 5.23), ], q k+1,δ q k,δ = γ k q k+1,δ q k,δ), v k+1,δ v k,δ = γ k v k+1,δ + γ k 1 ) v k,δ) v k,δ. 5.28) So if we define A 1 := 1 [ E x γ δ co δ 1)q ] δ, A 2 := E x γ δ in δ ) δ e δɣ 2 v, v, [ A 3 := E x γ δ in δ δ ξ δ γ δ in δ CD q)] 2 and E 1 := q E x γ δ in δ q), δ E 2 := CD q) + E x γ δ 2 E 3 := E x γ δ co δ e δɣ 2 v δ ) co δcd q) γ δ E 4 := 1 [ E x γ δ co δξ δ + γ δ 1 ) ξ δ] δ, 2 CD q k+1,δ ) ), 1 [ γ δ δ v + Ɣ 1 2v + E x e δɣ 2 v], δ

25 84 Ottobre, Pillai, Pinki and Stuart by the definition of G δ equation 5.4)) and uing 5.20) and 5.21), we obtain G δ 1 x) G 1 x) A 1 + A 2 + A 3 and G δ 2 x) G 2 x) E 1 + E 2 + E 3 + E 4. We will bound the A i and the E i one by one. To thi end, we will repeatedly ue the following imple bound: γ δ,γ k {0, 1} and 0 α δ 1; 5.29) E [ ξ δ] = 0; 5.30) E [ α δ 1 ) ξ δ] [ E α δ 1 ) 2] 1/2 [ E ξ δ 2 ] 1/2 δ 5/2 1 + q 4 ) + v ) 5.31) follow from uing Bochner inequality 3 and Cauchy Schwartz firt and then 5.24) and 5.16). Uing 5.29), it i traightforward to ee that A 1 δ q. A for A 2, A 2 = I E x α δ ) ) in δ δ e δɣ 2 v 1 E x α δ ) v + E x α δ ) 1 in δ ) δ e δɣ 2 v δ q 4 + v 4 ) v + δ v δ 1 + q 6 + v 6 ), having ued, in the econd inequality, 5.16) and 5.29). A 3 i bounded by uing 5.30), 5.31) and 5.17): A 3 in δ δ E x[ α δ 1 ) ξ δ + ξ δ] + E x δcd q) δ 5/2 1 + q 4 + v 4 ) + δ 1 + q ) δ 1 + q 4 + v 4 ). Hence 5.27) ha been proven. We now come to etimating the E i. Proceeding a in the bound for A 2 above we obtain: E 1 q E x α δ ) q + E x α δ ) 1 in δ ) q δ δ q 4 + v 4 ) q + δ 2 q δ q 6 + v 6 ). 3 Let X, ) be a Banach pace and f L 1, F,μ); X). Then f dμ f dμ. For a proof of the Bochner inequality ee [19].

26 A function pace HMC algorithm with econd order Langevin diffuion limit 85 Alo, E 2 CD q) E x α δ ) co δcd q) E x [ α δ co δcd q) α δ CD q k+1,δ )] 1 E x α δ )) CD q) + co δ 1)CD q) + )) E x CD q) CD q k+1,δ δ q 6 ) + v 6 + δex q k+1,δ q 5.26) δ 1 + q 6 ) + v 6, where the penultimate inequality i obtained by uing 5.16) and 5.17). For the lat two term: E 3 1 Ex α δ ) co δ 1)e δɣ 2 v δ + 1 Ex α δ 1 ) e δɣ 2 v δ Finally, from 5.30) and 5.31), + 1 ) E x e δɣ δɣ 2 v δ 5.29) δ v + 1 δ E α δ 1 v 5.16) δ v + δ 1 + q 4 + v 4 ) v δ 1 + q 6 + v 6 ). E 4 1 E x α δ co δξ δ) δ + 1 E x α δ 1 ) ξ δ δ 1 [ E x α δ 1 ) co δξ δ + co δξ δ] δ + 1 [ E x α δ 1 ) ξ δ] δ δ 3/2 1 + q 4 + v 4 ). Thi conclude the proof. Let u now how that condition 5.11) i atified a well. Lemma Under Aumption 3.1, the chain x k,δ H H defined in 5.1) atifie E [ q k+1,δ q k,δ x k,δ = x ] δ ) 1 + q + v, 5.32) E [ v k+1,δ v k,δ x k,δ = x ] δ 1/2 1 + q 6 ) + v ) In particular,5.11) hold with r = 1/2 and n = 6.

27 86 Ottobre, Pillai, Pinki and Stuart Proof. 5.32) i a traightforward conequence of 5.28), 5.29) and 5.26). In order to prove 5.33) we tart from 5.28) and we write E [ v k+1,δ v k,δ x k,δ = x ] = E γ δ v + γ δ 1 ) v v 5.34) By uing 5.29), 5.17) and 5.26), we get Notice that E γ δ in δq + δcd q) + δcd q ) ) 5.35) + E γ δ co δv 1 γ δ) v v. 5.36) E γ δ in δq + δcd q) + δcd q ) ) δ 1 + q ). 5.37) E γ δ 1 l = 1 E α δ), l ) Therefore by 5.20) and 5.29) and repeatedly uing 5.38), E γ δ co δv 1 γ δ) v v E [ γ δ co δ 1 γ δ)] ξ δ + E 1 γ δ) e δɣ 2 v + E γ δ co δe δɣ 2 v v 5.24) δ 1/2 + E 1 α δ v + E 1 γ δ co δ e δɣ 2 v + e δɣ 2 v v 5.39) 5.16) δ 1/2 + δ q 4 + v 4 ) v + E γ δ 1 e δɣ 2 v + E γ δ co δ 1) e δɣ 2 v + δ v δ 1/2 1 + q 6 + v 6 ). Now 5.34), 5.37) and 5.39)imply5.33). Finally, the a priori bound 5.12) hold. Lemma Let Aumption 3.1 hold. Then the chain 5.1) atifie { [ δe x k,δ l up δ 0,1/2) kδ<t ]} < for any integer l ) In particular, the bound 5.12) hold with ε = 1 and for any moment of x k,δ ). Remark Before proving the above lemma, let u make ome comment. Firt of all, the etimate of condition 5.12) i needed mainly becaue the proce ha not been tarted in tationarity and hence it i not tationary. For the ame reaon an analogou etimate wa needed in