Lecure 6: Wiener Process Eric Vanden-Eijnden Chapers 6, 7 and 8 offer a (very) brief inroducion o sochasic analysis. These lecures are based in par on a book projec wih Weinan E. A sandard reference for he maerial presened hereafer is he book by R. Dure, Sochasic Calculus: A Pracical Inroducion (CRC 1998). For a discussion of he Wiener measure and is link wih pah inegrals see e.g. he book by M. Kac, Probabiliy and Relaed Topics in Physical Sciences (AMS, 1991). 1 The Wiener process as a scaled random walk Consider a simple random walk {X n } n N on he laice of inegers Z: X n n ξ k, (1) k1 where {ξ k } k N is a collecion of independen, idenically disribued (i.i.d) random variables wih P(ξ k ±1) 1. The Cenral Limi Theorem (see he Addendum a he end of his chaper) assers ha X N N N(,1) ( Gaussian variable wih mean and variance 1) in disribuion as N. This suggess o define he piecewise consan random funcion on [, ) by leing W N W N X N N, () where N denoes he larges ineger less han N and in accordance wih sandard noaions for sochasic processes, we have wrien as a subscrip, i.e. W N W N (). I can be shown ha as N, W N converges in disribuion o a sochasic process W, ermed he Wiener process or Brownian moion 1, wih he following properies: (a) Independence. W W s is independen of {W τ } τ s for any s. 1 The Brownian moion is ermed afer he biologis Rober Brown who observed in 187 he irregular moion of pollen paricles floaing in waer. I should be noed, however, ha a similar observaion had been made earlier in 1765 by he physiologis Jan Ingenhousz abou carbon dus in alcohol. Somehow Brown s name became associaed o he phenomenon, probably because Ingenhouszian moion does no sound very good. Some of us wih complicaed names are moved by his sory. 5
3.5 1.5 1.5.5 1 1.5.1..3.4.5.6.7.8.9 1 Figure 1: Realizaions of W N for N 1 (blue), N 4 (red), and N 1 (green). (b) Saionariy. The saisical disribuion of W +s W s is independen of s (and so idenical in disribuion o W ). (c) Gaussianiy. W is a Gaussian process wih mean and covariance EW, EW W s min(,s). (d) Coninuiy. Wih probabiliy 1, W viewed as a funcion of is coninuous. To show independence and saionariy, noice ha for 1 m n X n X m n km+1 is independen of X m and is disribue idenically as X n m. I follows ha for any s, W W s is independen of W s and saisfies ξ k W W s d W s, (3) where d means ha he random processes on boh sides of he equaliy have he same disribuion. To show Gaussianiy, observe ha a fixed ime, W N converges as N o Gaussian variable wih mean zero and variance since W N X N X N N N(,1) d N(,). N N N In oher words, P(W [x 1,x ]) x x 1 ρ(x,)dx (4) 51
where ρ(x,) e x / π. (5) In fac, given any pariion 1 n, he vecor (W N 1,...,W N n ) converges in disribuion o a n-dimensional Gaussian random variable. Indeed, using (3) recursively ogeher wih (4),(5) and he independence propery (a), i is easy o see ha he probabiliy densiy ha (W 1,...,W n ) (x 1,...,x n ) is simply given by ρ(x n x n 1, n n 1 ) ρ(x x 1, 1 )ρ(x 1, 1 ) (6) A simple calculaion using EW xρ(x,)dx, EW W s yxρ(y x, s)ρ(x,s)dxdy. R R for s and similarly for < s gives he mean and covariance specified in (b). Noice ha he covariance can also be specified via E(W W s ) s, and his equaion suggess ha W is no a smooh funcion of. In fac, i can be showed ha even hough W is coninuous almos everywhere (in fac Hölder coninuous wih exponen γ < 1/), i is differeniable almos nowhere. This is consisen wih he following propery of self-similariy: for λ > W d λ 1/ W λ, which is easily esablished upon verifying ha boh W and λ 1/ W λ are Gaussian processes wih he same (zero) mean and covariance. More abou he lack of regulariy of he Wiener process can be undersood from firs passage imes. For given a > define he firs passage ime by T a inf{ : W a}. Now, observe ha P(W > a) P(T a < & W > a) 1 P(T a < ). (7) The firs equaliy is obvious by coninuiy, he second follows from he symmery of he Wiener process; once he sysem has crossed a i is equally likely o sep upwards as downwards. Inroducing he random variable M sup s W s, we can wrie his ideniy as: e z / P(M > a) P(T a < ) P(W > a) dz, (8) a π where we have invoked he known form of he probabiliy densiy funcion for W in he las equaliy. Similarly, if m inf s W s, P(m < a) P(M > a). (9) Bu his shows ha he even W crosses a is no so idy as i may a firs appear since i follows from (8) and (9) ha for all ε > : P(M ε > ) > and P(m ε < ) >. (1) In paricular, is an accumulaion poin of zeros: wih probabiliy 1 he firs reurn ime o (and hus, in fac, o any poin, once aained) is arbirarily small. 5
Two alernaive consrucions of he Wiener process Since W is a Gaussian process, i is compleely specified by i mean and covariance, EW EW W s min(,s). (11) in he sense ha any process wih he same saisics is also a Wiener process. This observaion can be used o make oher consrucions of he Wiener process. In his secion, we recall wo of hem. The firs consrucion is useful in simulaions. Define a se of independen Gaussian random variables {η k } k N, each wih mean zero and variance uniy, and le {φ k ()} k N be any orhonormal basis for L [,1] (ha is, he space of square inegral funcions on he uni inerval). Thus any funcion f() in his se can be decomposed as f() k N α kφ k () where (assuming ha he φ k s are real) α k 1 f()φ k()d. Then, he sochasic process defined by: W η k φ k ( )d, (1) k N is a Wiener process in he inerval [,1]. To show his, i suffices o check ha i has he correc pairwise covariance since W is a linear combinaion of zero mean Gaussian random variables, i mus iself be a Gaussian random variable wih zero mean. Now, EW W s s Eη k η l φ k ( )d φ l (s )ds k,l N k N s φ k ( )d φ k (s )ds, where we have invoked he independence of he random variables {η k }. To inerpre he summands, sar by defining an indicaor funcion of he inerval [,τ] and argumen { 1 if [,τ] χ τ () oherwise. If τ [,1], hen his funcion furher admis he series expansion (13) χ τ () k φ k () τ φ k ( )d. (14) Using he orhogonaliy properies of he {φ k ()}, he equaion (13) may be recas as: EW W s 1 ( ) ( s ) φ k ( )d φ k (u) φ l (s )ds φ l (u) du k,l N 1 1 χ (u)χ s (u)du χ min(,s) (u)du min(,s) 53 (15)
as required. One sandard choice for he se of funcions {φ k ()} is he Haar basis. The firs funcion in his basis is equal o 1 on he half inerval < < 1/ and o -1 on 1/ < < 1, he second funcion is equal o on < < 1/4 and o - on 1/4 < < 1/ and so on. The uiliy of hese funcions is ha i is very easy o consruc a Brownian bridge: ha is a Wiener process on [,1] for which he iniial and final values are specified: W W 1. This may be defined by: Ŵ W W 1, (16) if using he above consrucion hen i suffices o omi he funcion φ 1 () from he basis. The second consrucion of he Wiener process (or, raher, of he Brownian bridge), is empirical. I comes under he name of Kolmogorov-Smirnov saisics. Given a random variable X uniformly disribued in he uni inerval (i.e. P( X < x) x), and daa {X 1,X,... X n }, define a sample-esimae for he probabiliy disribuion of X: F n (x) 1 n (number of X k < x, k 1,...,n) 1 n χ n (,x) (X k ), (17) equal o he relaive number of daa poins ha lie in he inerval x k < x. For fixed x F n (x) x as n by he Law of Large Numbers ells us ha, whereas n( ˆFn (x) x) d N(,x(1 x)). (18) by he Cenral Limi Theorem. This resul can be generalized o he funcion ˆF n : [,1] [,1] (i.e. when x is no fixed): as n n(fn (x) x) d W x xw 1 Ŵx. (19) k1 3 The Feynman-Kac formula Given a funcion f(x), define u(x,) Ef(x + W ) () This is he Feynman-Kac formula for he soluion of he diffusion equaion: To show his noe firs ha: u 1 u x u(x,) f(x). (1) u(x, + s) Ef(x + W +s ) Ef(x + (W +s W ) + W ) Eu(x + W +s W,) Eu(x + W s,) where we have used he independence of W +s W and W. Now, observe ha u 1 (x,) lim (u(x, + s) u(x,)) s + s 1 lim s + s E(u(x + W s,) u(x,)) ( 1 u lim s + s x (x,)ew s + 1 ) u x (x,)ew s + o(s), 54
where we have Taylor-series expanded o obain he final equaliy. The resul follows by noing ha EW s and EWs s. The formula admis many generalizaions. For insance: If v(x,) Ef(x + W ) + E g(x + W s )ds, () hen he funcion v(x, ) saisfies he diffusion equaion wih source-erm he arbirary funcion g(x): v 1 v + g(x) v(x,) f(x). (3) x Or: If ( ( w(x,) E f(x + W )exp c(x + W s )ds) ) (4) hen w(x,) saisfies diffusive equaion wih an exponenial growh erm: w 1 w + c(x)w w(x,) f(x). (5) x Addendum: The Law of Large Numbers and he Cenral Limi Theorem Le {X j } j N be a sequence of i.i.d. (independen, idenically disribued) random variables, le η EX 1 σ var(x 1 ) E(Z 1 η) and define S n The (weak) Law of Large Numbers saes ha if E X j <, hen S n n η n j1 X j in probabiliy. The Cenral Limi Theorem saes ha if EX j < hen S n nη nσ N(, 1) in disribuion. We firs give a proof of he Law of Large Numbers under he sronger assumpion ha E X j <. Wihou loss of generaliy we can assume ha η. The proof is based he Chebychev inequaliy: Suppose X is a random variable wih disribuion funcion F(x) P(X < x). Then, for any λ >, provided only ha E X p <. Indeed: λ p P( X λ) λ p df(x) x λ P( X λ) 1 λ pe X p, (6) x λ 55 x p df(x) R x p df(x) E X p.
Using Chebychev s inequaliy, we have P { S n n } > ε for any ε >. Using he i.i.d. propery, his gives Hence 1 ε E S n n E S n E X 1 + X +... + X n ne X 1. P { S n n } > ε 1 nε E X 1, as n, and his proves he law of large numbers. Nex we prove he Cenral Limi Theorem. Le f be he characerisic funcion of X 1, i.e. f(k) Ee ikx 1, k R. (7) and similarly le g n be he characerisic funcion of S n / nσ. Then g n (ξ) Ee iξsn/ nσ n Ee iξx j/ nσ (Ee iξx j/ nσ ) n j1 ( 1 + ik EX 1 k nσ nσ EX 1 + o(n 1 ) ) n (1 k n + o(n 1 ) e k / as n. This shows ha he characerisic funcion of S n / nσ converges o he he characerisic funcion of N(,1) as n and erminaes he proof. I is insrucive o noe ha he only propery of X 1 ha we have required in he cenral limi heorem is ha EX1 <. In paricular, he heorem holds even if he higher momens of X 1 are infinie! For one illusraion of his, consider a random variable having probabiliy densiy funcion ρ(x) ) n π(1 + x ), (8) for which all momens of order higher han are infinie. Neverheless, we have: f(k) e ikx ρ(x)dx (1 + k ) e k R 1 1 k + o(k ), and hence he Cenral Limi Theorem applies. Inuiively, he reason is ha he fa ails of he densiy ρ(x) disappear in he limi owing o he rescaling of he parial sum by 1/sqrn. Noes by Marcus Roper and Ravi Srinivasan. 56