Lecture 10: Forward and Backward equations for SDEs

Transcription

1 Miranda Holme-Cerfon Applied Stochatic Analyi, Spring 205 Lecture 0: Forward and Backward equation for SDE Reading Recommended: Pavlioti [204] , 3.4, Gardiner [2009] Other ection are recommended too thi i a nice book to read (and own), and it i trongly uggeted to tart looking through it. Optional: Okendal [2005] 7.3, 8., Koralov and Sinai [200] 2.3, 2.4 Conider the SDE dx t = b(x t,t)dt + σ(x t,t)dw t, X t R d. () We have looked at how to olve for the actual olution trajectorie themelve. For the next few lecture we will conider another approach to tudy propertie of the olution, via partial differential equation. Recall (from Lecture 8) that the olution to () are Markov (with ufficient moothne and growth condition on the coefficient.) Therefore we expect to be able to decribe them via their tranition denitie p(x,t y,) = P(X t [x,x + dx) X = y). (2) Thi ha exactly the ame interpretation a the tranition denity for a continuou-time Markov chain, but now the tate pace i continuou, not dicrete. In Lecture 3, we howed we could decribe the joint evolution of (x,t) or (y,) by linear equation: the Kolmogorov forward and backward equation. From thee we can obtain equation for the evolution of probability denity ρ(x), or the evolution of obervable E x f (X t ). Our goal will be to find uch equation for the olution to the SDE above. 0. Forward and backward equation The forward and backward equation come nearly directly from Itô formula. Conider a function f C c (R d ). Itô formula give Integrate from to t: f (X t ) f (X ) = d f (X t ) = (b f + 2 σσt : 2 f )dt + f σ dw t. t Take the expectation, conditional on X = y: (b f + t 2 σσt : 2 f )(X τ,τ)dτ + f σ(x τ,τ)dw τ. t E y, f (X t ) f (y) = E y, L f (X τ,τ)dτ, (3)

2 where L f = b(x,t) f (x) + a(x,t) : 2 f i a linear operator, E y, f (X t ) E[ f (X t ) X = y], and a = 2 σσt i the diffuion matrix. From thi we can immediately calculate the generator of a time-homogeneou diffuion proce. Let E x f (X t ) = E[ f (X t ) X 0 = x]. We have that E x f (X t ) f (x) t lim = lim t 0 + t E L f (X τ )dτ = L f (x). t 0 + t 0 The lat tep follow from the Dominated Convergence Theorem, ince f and all it derivative are bounded (ee Koralov and Sinai [200], p.323.) Definition. The generator of a time-homogeneou diffuion proce () i L f = b(x) f (x) + 2 a(x) : 2 f (x). Indeed, from (3) we have when f C 2 c. E x f (X t ) f (x) A f (x) = lim = L f (4) t 0 + t The domain of the generator i D(A ) = { f C 0 (R d ).t. the limit above exit}. In general, thi can include function not in C 2 c (in which cae the expreion L f would not apply directly.) The coefficient of the operator L can alo be obtained from the law of the proce, without conidering the SDE it atifie. We howed on HW5 that E x [(X t ) i x i ] b i (x) = lim, t 0 t 2 a E[((X t ) i x i )((X t ) j x j )] i j(x) = lim. t 0 t Thi i an approach that i often followed in phyic textbook, uch a Gardiner [2009]. Now we can write down the forward and backward equation. Backward Kolmogov Equation. (Verion ) Let X t olve a time-homogeneou SDE (). Let u(x,t) = E x f (X t ) = E[ f (X t ) X 0 = x], where f C 2 c (R d ). Then t u = L u, u(x,0) = f (x), t > 0. (5) (Verion 2) Let X t olve a (poibly time-inhomogeneou) SDE (). Let u(y,) = E y, f (X t ), where f Cc 2 (R d ). Then u(y,) + L u(y,) = 0, u(y,t) = f (y), < t. (6) (Verion 3) Let p(x,t y,) be the tranition denity of X t olving (time-inhomogeneou) (). Then Note p + L y p = 0, p(x,t y,t) = δ(x y), < t. (7) Equation (6) hould be olved backward in time, ubject to the terminal condition p(x,t y,t) = δ(x y). Thi i the origin of the name backward equation. Equation (7) hould alo be olved backward in time. 2

3 Verion can alo be derived from verion 2 for a time-homogeneou proce, by relabelling time to be t = t and conidering the function u(x,t ). Proof, Verion. Thi baically follow formally from (3), if we aume that E x,l commute (they do by Leibniz if f ha continuou econd derivative, or ee other argument below.) Formally, we have E x f (X t+h ) E x f (X t ) t u(x,t) = lim = lim E x h 0 h h 0 h t+h = lim L E x f (X τ )dτ = L u(x,t). h 0 h t t+h t L f (X τ )dτ Here i a proof (from Okendal [2005], ection 8.), that ue our previou calculation of the generator, that doen t require formally interchanging E,L. Let calculate: t u (x,t) = lim (u(x,t + ) u(x,t)) = lim 0 + (Ex f (X t+ ) E x f (X t )) ( = lim E x E X f (X t ) E x f (X t ) ) (Markov property) 0 + = lim 0 + (Ex u(x,t) u(x,t)) = L u 0 + Another way to write Okendal proof i a follow. Let define an operator T t f (x) E x f (X t ), which act on the et of bounded, meaurable function. The Chapman-Kolmogorov equation imply that T (T t f ) = T +t f. Then the time-derivative of the generator of a C 2 -function f D(A ) i t u = lim h 0 T t+h f T t f h T h (T t f ) T 0 (T t f ) = lim = L T t f. (8) h 0 h To ee formally why E,L commute, we could have factored the other way in (8), and found that (auming that we can interchange lim and T t ) T t (T h f ) T t (T 0 f ) t u = lim = T t L f. h 0 h Proof, Verion 2. (from E et al. [204]) From Itô formula, we have du(x τ,τ) = ( τ u + L u)(x τ,τ)dτ + u σ dw τ. Integrate from to t and take the expectation conditional on X = y to get lim t t (Ey, u(x t,t) u(y,)) = lim t t P(B, x,t) = R P(B,x y,u)p(dy,u x,t), t u. See Pavlioti [204], ection 2.2. t E y, ( τ u + L u)(x τ,τ)dτ = u(y,) + L u(y,). 3

4 However, we alo have that o the LHS i zero. Therefore (6) hold. E y, u(x t,t) = E y, f (X t ) = u(y,), Proof, Verion 3. We can write u(y,) = f (x)p(t,x,y)dx. Apply the operator + L to u(y,), and ue Verion 2. Auming we can interchange derivative and integral, 2 we have 0 = u(y,) + L u(y,) = f (x)( p(x,t y,) + L y p(x,t y,))dx. Thi hold for all tet function f, o the equation alo hold for p. Forward Kolmogov Equation. (Verion ) The tranition probabilty denity p(x,t y,) olve t = L x p, p(x, y,) = δ(x y). (9) Here L f = x (b(x,t) f (x)) + 2 x : (a(x,t) f (x)) i the formal adjoint of L, i.e. it i the operator that atifie L f,g L 2 = f,l g L 2. (Verion 2) Let ρ 0 (x) be the initial probability denity, and ρ(x,t) the denity at time t. Then ρ olve Thi i alo called the Fokker-Planck equation. Proof, Verion. Write (3) a t ρ = L ρ, ρ(x,0) = ρ 0 (x). (0) f (x)p(x,t y,)dx f (y) = Rd t Take t to find R d t pdx = (L f )pdx. Rd Thi hold for all tet function f, o p i a weak olution to (9). R d L f (x,τ)p(x,τ y,)dxdτ. Proof, Verion 2. We have ρ(x,t) = p(x,t y,0)ρ 0 (y)dy. Integrate both ize of (9) over ρ 0 (y) to get (0). Remark. We ee that the forward equation i weaker than the backward equation it i only expected to only in a weak ene, in general. Indeed, the forward equation require taking derivative of b,σ, but thee are not required to be C 2 in general, o the derivative may only exit in a weak ene. Another example where it hold weakly but not trongly i when the probability denity contain ingular meaure, uch a delta function we will ee an example of thi in ection 0.4 below. Remark. The Fokker-Planck equation for a Stratonovich SDE ha a nice form: dx t = σ dw t = ( ( t ρ = i σik j σ jk ρ )) i, j,k 2 If not, e.g. at t = 0 when derivative of p blow up, then ue the trick from the Lecture 5 note, of writing p( y,) = p( y,u)du, and working with integral intead. 4

5 0.2 Probability flux It i ueful to think about the Fokker-Planck equation (0) uing concept of flux, borrowing idea from fluid dynamic and mechanic. Write (0) a t ρ = ( bρ + (aρ)) t ρ + j = 0, j = bρ (aρ). Thi i a conervation law / continuity equation for ρ. i the probability current, or probability flux. The integral j S ˆn give the total probability that croe a urface S, per unit time. j To ee why: Conider a cloed urface S bounding domain Ω. The change in total probability in Ω per unit time i Ω tρ = Ω = j S ˆn, where ˆn i the unit outward normal, by the Divergence j Theorem. There i a trong connection to fluid dynamic. Suppoe ρ = concentration of omething (tracer, heat,...) b = velocity of fluid a = diffuion tenor (or twice the diffuion tenor) Then the FP equation ay that the tracer i advected with velocity b, and diffue with diffuion tenor a. Probability behave exactly like a paive tracer in a fluid! (Note that the more familiar equation for a paive tracer i t c + u c = [diffuion], becaue in many cae u = 0.) In many cae diffuion i contant an iotropic, in which cae the diffuion term ha the form A ρ for ome contant A thi i the more familiar vicoity term in the fluid equation. The total probability change a dt d D ρ = D j = D j ˆn, where D i our domain (bounded or not.) Thi will be conerved if j ˆn D = Boundary condition for the forward and backward equation 0.3. Boundary condition for the Fokker-Planck equation The Fokker-Planck equation i a PDE, o it need to come with boundary condition to be well-poed. What hould thee be? Thi depend on what happen to the proce when it hit the boundary. Let the proce live in domain D, with boundary D. Here are ome poible boundary condition. (If D i unbounded, thee are decay condition at.) Reflecting boundary j ˆn = 0 on D. Thi correpond to trajectorie being reflected at the boundary. There i no net flux of probability acro the boundary, o the total probability i conerved. 5

6 Aborbing boundary ρ = 0 on D. Thi correpond to trajectorie being aborbed at the boundary and taken out of the ytem immediately. The total probability i not conerved. Periodic boundary (on interval [a,b]) j b = j a +, ρ b = ρ a +. Trajectorie that leave one ide, immediately re-enter on the other. Total probability i conerved. Sticky boundary j ˆn = κl ρ on D. Thi correpond to trajectorie that can tick, or pend finite time, at the boundary. The total probability i conerved, provided it include a ingular part on the boundary. Here κ i a contant that meaure how ticky the boundary i. Notice that when κ 0 we recover the reflecting boundary condition, and when κ we have L ρ = 0 ρ t = 0 ρ = 0, the aborbing boundary condition. At a dicontinuity What happen if the coefficient b(x, t), a(x, t) are dicontinuou on ome urface S, but particle can till cro it? Then j ˆn S + = j ˆn S, ρ S + = ρ S. Both the probability, and the normal component of the probability current, are continuou acro S. Note that the derivative of ρ may not be continuou. Thi alo conerve total probability. One way to derive the condition i to mooth the coefficient near the dicontinuity, e.g. with an appropriate mollifier, and then conider the limiting equation a the moothed equation approache the dicontinuou one. Note alo that when the coefficient are dicontinuou, we really have eparate equation, one on each part of the domain where the coefficient are continuou, o thi i really a matching condition that ay how thee different olution are related. 6

7 0.3.2 Boundary condition for the backward equation Thee can be derived from the boundary condition for the FP equation through integration by part: L ρ, f = f = ( f j ) j f D j D = (bρ (aρ)) f f j ˆn D D = ρb f (ρa f ) + ρa : 2 f f j ˆn Term A i the generator L f. = D D ρ ( b f + a : 2 f ) }{{} = ρ,l f A D D f j ˆn + (ρa f ) ˆn }{{} B D f j ˆn + (ρa f ) ˆn }{{} B Term B tell u the boundary condition. We need thi term to vanih, o that L ρ, f = ρ,l f. Given boundary condition on ρ from L, we hould chooe boundary condition on f (for L ) to make B=0 everywhere. Here are ome example: Reflecting boundary In component, thi i i, j n i a i j j f = 0. Aborbing boundary (a f ) ˆn = 0 on D. f = 0 on D. The other are ELFS. See alo paper by Feller, e.g. Feller [952], which dicue the mot general cla of boundary condition for a econd-order parabolic equation. 0.4 Stationary ditribution Suppoe the SDE () i time-homogeneou: b = b(x), σ = σ(x). I there a probability denity that doen t change with time? If o, it mut atify t ρ = 0, which implie L ρ = 0 j = 0 (bρ (aρ )) = 0. () Therefore, in order to undertand if a tationary olution exit, we mut undertand whether () ha a olution. If o, i it unique? And doe an arbitrary olution to (0) approach it at long time? Thee are the ame quetion we aked for Markov chain, only now they have turned into quetion about an elliptic PDE. There are many different reult. A particularly tractable cae i when a i bounded and 7

8 uniformly elliptic. To be uniformly elliptic mean that c ξ 2 a i j (x)ξ i ξ j c 2 ξ 2 i, j ξ,x, where c,c 2 are contant. Another way of aying thi i that a i uniformly poitive-definite, or that the eigenvalue of a are uniformly bounded away from 0. Theorem. If a i bounded and uniformly elliptic, then there i a unique olution in a bounded domain with periodic or reflecting boundary condition. Remark. If the domain i unbounded, then one alo ha uniquene reult for the Fokker-Planck equation (tationary or time-dependent), provided the coefficient of the SDE atify certain moothne condition (Lipchitz continuity, growth condition.) In thi cae, one look for a olution that atifie certain growth condition at one option i ρ(x,t) L (0,T ) ce α x 2. See Pavlioti [204], ection 4. for a tatement of one uch reult. Remark. Weaker reult are poible, for example if the operator i emi-elliptic but it i hypoelliptic, then there i a unique olution to (). In mot application you will work with () a a definition for a tationary ditribution. However, in general a tationary ditribution need not have a denity (be abolutely continuou) with repect to the Lebegue meaure, and even if it doe, it need not be in C 2. In thee cae, we will not be able to find the tationary ditribution by olving (), but rather we will have to work with it weak formulation. Here i a more general definition of a tationary ditribution, often called an invariant meaure. Definition. An invariant meaure i a meaure µ uch that where P t P t µ = µ, (2) i the emi-group acting on probability meaure a (P t µ)(c) = P(X t C X 0 = x)dµ(x). Remark. Notice that T t, Pt Sinai [200], p One can how that (2) i equivalent to are adjoint of each other: T t f (x)dµ(x) = f (x)d(p t µ)(x). See Koralov and for all tet function f. Thi imply ay that µ olve () in the weak ene. L f dµ = 0 (3) To ee why (3) i true, notice that (T t f f )dµ(x) = f d(pt µ µ)(x) = 0, o (T t f f )(x) (Tt f f )(x) L f (x)dµ(x) = lim dµ(x) = lim dµ(x) = 0. t 0 t t 0 t We can interchange the limit and expectation by the Dominated Convergence Theorem, ince (T t f f )/t i uniformly bounded for t > 0 provided the derivative of f decay quickly enough at infinity. Thi will be true if f i in the Schwartz pace S(R d ), for example (Koralov and Sinai [200], p.333.) The convere i alo true: (3) implie (2). Definition. Informally, a Markov proce i ergodic i 0 i a imple eigenvalue of L, or equivalently the equation L u = 0 ha only contant olution. 8

9 Remark. If a proce i ergodic, then there i a unique invariant meaure µ that i preerved by the emigroup Pt, a Pt µ = µ. If µ ha a denity ρ with repect to Lebegue meaure, then thi implie (), L ρ = 0. Therefore another way of defining ergodicity (informally) i that there i a unique probability denity atifying the Fokker-Planck equation. Remark. One can how that for an ergodic proce, lim t P t µ 0 = µ for any initial ditribution µ 0. In addition, the long-time average of an obervable f converge to E µ f (X) at long time. Thi i the phyic way of defining ergodic that the phae-pace average with repect to the unique invariant meaure, equal the long-time average. See Pavlioti [204], 2.4 for a ummary of thee idea. Example (Orntein-Uhlenbeck proce). Conider dx t = αx t + σdw t. The equation to olve for the tationary ditribution i α x (xρ) + 2 σ 2 2 ρ x 2 = 0. Let look for a olution with = 0. Then we mut olve j αxρ + 2 σ 2 ρ x = 0 where C i a contant. The tationary ditribution i ρ x ρ = 2αx σ 2 lnρ = α σ 2 x2 +C ρ = 2πσ 2 /(2α) e 2 x 2 σ 2 /(2α). Thi how the tationary ditribution i Gauian, with mean 0, variance σ 2 /2α. Example (Brownian dynamic). The equation for a ytem of particle interacting with potential energy U(x) and forced with white noie i, when the mae of the particle are mall enough that inertia can be neglected: dx t = U(X t) 2β dt + dw t, γ γ where γ i the damping parameter, and β = (k B T ) i the invere temperature. Here X t,w t R d. Thi i alo known a the overdamped Langevin equation The correponding FPE i ( ) t ρ = γ U(x)ρ + β γ ρ. To find the tationary ditrbution, let look for a olution with j = 0: β U(x)ρ + γ γ ρ = 0 ρ ρ = β U lnρ = βu(x) +C. 9

10 The tationary ditribution i therefore ρ = Z e βu(x), Z = Thi i the Boltzmann ditribution, or Gibb meaure. Note R d e βu(x) dx. In thee example we looked for a olution with = 0. Thi doen t alway have to be the cae, but j ince we found a olution that atifie thi extra condition, we are done, ince the olution i unique. We will ee in a couple of clae that we can tell, from the coefficient of the SDE, when uch a olution will be poible. Equation where thi i the cae are phyically very important. We will ee they correpond to equilibrium ytem, with no flux in the teady-tate. Thi will be a verion of detailed balance for an SDE. Example (Sticky Brownian Motion). Conider a Brownian motion on [0,], with a ticky boundary at x = 0 and a reflecting boundary at x =. Contructing the trajectorie explicitly require more mathematical tool, uch a the concept of local time ee e.g. Ikeda and Watanabe [98]. However, we can undertand how the tranition probabilitie behave, from the correponding forward and backward equation. The FP equation i 2 ρ = 0, ρ x = κρ xx x=0, ρ x = 0 x=. Let how the tationary ditribution i ρ (x) = +κ ( + κδ(0)). Notice that we can t compute L ρ directly, ince ρ contain a delta-function. Therefore we mut ue the weak formulation of the FP equation, and how that L f,ρ = 0 for all tet function f. We compute (leaving out the normalization factor): L f,ρ = f xx ( + κδ(0))dx = f x 0 + κ f xx (0) = 0 f x (0) + κ f xx (0) (boundary condition at ) = 0. (boundary condition at 0) Reference W. E, T. Li, and E. Vanden-Eijnden. Applied Stochatic Analyi. In preparation., 204. W. Feller. The parabolic differential equation and the aociated emi-group of tranformation. The Annal of Mathematic, 55:468 59, 952. C. Gardiner. Stochatic method: A handbook for the natural cience. Springer, 4th edition, N. Ikeda and S. Watanabe. Stochatic Differential Equation and Diffuion Procee. Elevier, 98. L. B. Koralov and Y. G. Sinai. Theory of Probability and Random Procee. Springer,

11 B. Okendal. Stochatic Differential Equation. Springer, 6 edition, G. A. Pavlioti. Stochatic Procee and Application. Springer, 204.