Markov Processes and Stochastic Calculus

Transcription

1 Markov Processes and Sochasic Calculus René Caldeney In his noes we revise he basic noions of Brownian moions, coninuous ime Markov processes and sochasic differenial equaions in he Iô sense. 1 Inroducion A sample space (Ω, F) consiss on: Ω: An absrac space of poins ω Ω. F: a σ-field (or σ-algebra) on Ω. Tha is, a collecion of subses of Ω saisfying: 1. Ω F. 2. Le A Ω such ha A F hen A c = Ω A F. 3. Le A 1, A 2, A 3, F hen A 1 A 2 A 3 F. The elemens of F are called evens. Condiion (1) above simply saes ha he space Ω is necessarily an even. Condiions (2) and (3) sae ha he collecion of evens is closed under he se operaions of complemen and counable union. The space (Ω, F) saisfying hese condiions is called a measurable space. For a fixed poin ω Ω, we called he sample pah of he process X associaed wih ω he funcion X (ω) :. Definiion 1 Le (Ω, F) be a measurable space. A se funcion µ on F is called a measure if i saisfies he following condiions: 1. µ( ) =, 2. A F implies µ(a),

2 3. A 1, A 2, F and {A n } are pairwise disjoin (A i A j = for i j), hen ( ) µ A n = µ(a n ). n=1 n=1 For example, he Lebesgue measure commonly denoed by λ and defined on he class of Borel ses R 1 of he real line, is given by λ(a, b] = b a. We call a measure µ a probabiliy measure if µ(ω) = 1, in his case insead of µ, we use he noaion P. If P is a probabiliy measure hen (Ω, F, P ) is called a probabiliy space. Definiion 2 Le (Ω, F) and (Ω, F ) wo measurable spaces. F )-measurable if for each A F A mapping Y : Ω Ω is (F, Y 1 (A ) = {ω Ω : Y (ω) A } F. According o his definiion, he mapping Y is measurable if for any well-defined even A F he pre-image of A by Y (denoed by Y 1 (A )) is also a well-defined even in F. If (Ω, F ) = (R, B(R)) hen Y is called a real-valued random variable. The probabiliy disribuion of a random variable Y, is a mapping P Y : B(R) [, 1] defined as follows P Y (B) = P ({ω Ω such ha Y (ω) B}) := P (Y B). Definiion 3 A sochasic process is a collecion of random variables X = {X() : } defined on a probabiliy space (Ω, F, P ). For simpliciy, we will assume on his noes ha he index above represens ime. We can view a sochasic process as mapping ha a each ime associaed he occurrence of a random phenomenon represened by X(). The funcion X(, ω) for a fixed ω Ω is called a sample pah or sample funcion. The sochasic process X is called coninuous if he P -almos every sample pah is coninuous. An imporan consideraion is relaed o he way we collec and use informaion over ime. In paricular, we would like o be able o isolae pas and presen from fuure. For example, le us consider wo evens A, B F such ha X s (ω 1 ) = X s (ω 2 ); ω 1 A, ω 2 B, s, s. Then during he period [, ] he evens A and B canno be disinguished based on he knowledge of X. For his reason, we complemen our sample space (Ω, F) wih a filraion, i.e., a nondecreasing family {F : } of sub-σ-fields of F such ha F s F F for s. The idea is ha each F conains he informaion available up o ime. Definiion 4 A sochasic process X is adaped o he filraion F if X() is F -measurable for all. 2

3 Given a sochasic process X we define he filraion F X such ha X() F X for all. generaed by X as he smalles filraion Definiion 5 A sopping ime is a measurable funcion T from (Ω, F) o [, ) such ha {ω Ω : T (ω) } F, for all. 2 Brownian Moion The name Brownian moion (BM) comes from he sudies done by he boanis Rober Brown in 1828 on he irregular movemen (Brownian movemen) of pollen suspended in waer. A mahemaical definiion a Brownian moion is he following. Definiion 6 A sandard, one-dimensional Brownian moion (or Wiener process) is a coninuous, adaped sochasic process X = {X, F : < }, defined on some probabiliy space (Ω, F, P ) wih he properies ha X = almos surely and for s <, he incremen X X s is independen of F s and is normally disribued wih mean zero and variance s. A few poins abou he previous definiion: (a) We say ha X = almos surely (a.s.) in (Ω, F,P ) if he se A = {ω Ω : X (ω) } has probabiliy, i.e., P (A) =. (b) The sochasic process X is adaped o he filraion {F } if for each, X is an F - measurable random variable. Two exremely imporan feaures ha characerizes a Wiener process presened in he following propery: Propery 1 If X is a Wiener process hen X has independen incremens, ha is, for any posiive ineger n and any sequences of imes < 1 < < n < he random variables Y i = X( i ) X( i 1 ), i = 1, 2,..., n are independens. In addiion, X has saionary incremens, ha is, for any s < < he disribuion of X X s depends only on s. These wo properies are so aached o he Wiener process ha hey can be used as an alernaive definiion of a sandard Brownian moion (ogeher wih he requiremen of coninuous sample pahs). We noice here ha for discree ime sochasic processes he wo properies above characerize he Poisson process. 3

4 Once we have defined he Wiener process, we can exend is definiion and define he general (µ, σ) Brownian moion process Y as follows: Y () = Y () + µ + σx() where X is a Wiener process and Y () (he iniial value) is independen of X. We call µ he drif and σ 2 he variance or diffusion of Y. I follows direcly from he definiion of X ha Y ( + s) Y () is normally disribued wih mean µs and variance σ 2 s. Finally, we say ha Z is a geomeric Brownian moion if Z = e Y, where Y is a Brownian moion. Exercise 1: Prove ha if Z is a process wih saionary independen incremen, such ha σ 2 := Var(Z(1)), hen Var(Z()) = σ 2 for all. 3 Properies of Brownian Moions In his secion we presen he main properies ha make Brownian moions a very aracive modelling ool. However, we sar ironically presening some resuls showing he exremely erraic behavior of Brownian moion processes. 3.1 Basic Properies Propery 2 Le X be a Brownian moion in (Ω, F,P ). Then excep for a se of probabiliy, he sample pah X (ω) is nowhere differeniable. Even hough he variaion of X over ime is paricular unsable, some measure of is variabiliy can be compued. In fac, le define he random variable (quadraic variaion) Q as follows: Q lim n Then, we have he following resul. 2 n 1 k= [ X ( (k + 1) 2 n ) X ( )] k 2 2 n. (1) Propery 3 For almos every ω Ω we have Q (ω) = σ 2 for all. This las resul implies ha Brownian moion have infinie ordinary variaion almos surely. In addiion, as we will see laer, propery 3 conains he essence of he Iô s formula. Propery 4 If X is a (µ, σ) Brownian moion hen: 4

5 - E(X ) = X + µ, - V ar(x ) = σ 2, - Cov(X, X s ) = σ 2 ( s) = σ 2 min{, s}. The following heorem is a very imporan resul ha reflec he memoryless propery ha characerizes Brownian moion processes. Theorem 1 (Srong Markov Propery) Le X be a (µ, σ) Brownian moion and T be a finie sopping ime. Then Y = X T + X T (µ, σ) Brownian moion saring a and i is independen of F T. is a Propery 5 (Brownian maringales) Le X be a (µ, σ) Brownian moion hen: (a) If µ = hen X is a maringale, i.e., E(X X s F s ) =. (b) If µ = hen X 2 σ 2 is maringale. (c) Le q(β) = µβ σ2 β 2 and V β () = e βx q(β). Then, V β is a maringale. 3.2 Wiener Measure and Donsker s Theorem In his subsecion we explore he naure of he Wiener process as a ype of cenral limi heorem for sochasic processes. The noaion and resuls are based on he exbook Convergence of Probabiliy Measures by P. Billingsley (1999). We sar by inroducing he Wiener measure, W, which is a probabiliy measure on (C, C) having wo properies. Firs, each X is normally disribued under W wih mean and variance, ha is: W [X α] = 1 2π α e u2 2 du. For =, we have W [X = ] = 1. The second propery is ha he sochasic process X has independen incremens under W. In order o sae he main resul of his secion (Donsker s heorem), we inroduce he sequences {X n : n =, 1,... } of sochasic processes as follows. Le Ξ = {ξ 1, ξ 2,... } be a sequence of IID Where C C[, ) is he space of all coninuous funcions x : [, ) R and C is he Borel σ-algebra on C. 5

6 4 3 n=2 2 1 X n=5 1 2 n= Figure 1: Behavior of {X n } for n = 1, 5, and 2. random variables having mean and finie variance σ 2. Le S n = ξ ξ n (S = ) be he parial sums of Ξ. We define X n as follows: X n (ω) = 1 σ n S 1 n (ω) + (n n ) σ n ξ n +1(ω) (2) Figure 1 shows he behavior of he process X n for hree values of n. We can see ha as n increases, he behavior of X n resembles a Wiener process. This resul is in fac Donsker s heorem. We can see he non differeniabiliy of X n () as n increases. Theorem 2 (Donsker s Theorem) If ξ 1, ξ 2,... are independen and idenically disribued random variables wih mean and variance σ 2, and if X n is he random process defined by (2), hen X n = n W, a Wiener process. (Where he symbol = n sands for convergence in disribuion as n.) This resul can be undersood as a generalizaion of he sandard cenral limi heorem for random variables. The previous resul is inuiive in he sense ha S n -being he sum of IID random variablesconverges in disribuion o a N(, nσ 2 ). Anoher ineresing propery of he Wiener process, and more generally of any (, σ) Brownian moion is heir scale invariance ha can parially be observed in (2). In he consrucion of he hree sample pahs he ξ n are uniformly disribued in [ 1, 1]. 6

7 Propery 6 (Scale Invariance) Le X be a (, σ) Brownian moion, hen for any c > : { } X(c) D : = {X() : }. (3) c (Where D = sands for equaliy in disribuion.) This scaling propery, ha is of course relaed o he normal disribuion, is specially imporan (ogeher wih Donsker s heorem) on he use of heavy raffic approximaions for queueing sysems. We can now use Donsker s heorem o find he disribuion of M sup W, however, we need before an addiional resul. Theorem 3 (Mapping Theorem) Le {X n } be a sequence of processes such ha X n X. Le h be a measurable funcion and le D h be he se of is disconinuiies. If D h has probabiliy, hen h(x n ) h(x). Since h(x) := sup X is a coninuous funcion on C, hen from he mapping heorem and he fac ha X n W, we have ha: sup X n sup W. Le M n = max i n S i, hen i is no hard o show ha sup X n M n σ n sup = Mn σ n. Thus, W. (4) Since we can peak any sequence {ξ n } such ha E(ξ n ) = and E(ξ 2 n) <, le assume ha ξ n akes he values ±1 wih probabiliy 1 2. Therefore, S, S 1,... represens a symmeric random walk saring a. We firs prove ha P (M n a, S n < a) = P (M n a, S n > a) a. This should be clear from he fac ha he behavior of he random walk is independen of is hisory and i is symmeric, hus if he random walk reach a a ime ˆn < n hen he value of S n is symmeric wih respec o Sˆn = a. In oher words, for each pah of he random walk (S, S 1,..., S n ) such ha M n a, S n = a k < a here exiss anoher pah such ha M n a, S n = a + k > a. This symmery is an example of he reflecion principle. Given his resul, we have ha: P (M n a) = P (M n a, S n < a) + P (M n a, S n = a) + P (M n a, S n > a) = 2P (M n a, S n > a) + P (M n a, S n = a) = 2P (S n > a) + P (S n = a) 7

8 By he cenral limi heorem P (S n > a n) P (N > a) and P (S n = a n), where N is a sandard (, 1) normally disribued random variable. In addiion 2P (N > a) = P ( N > a). Thus, combining his resuls we have ha M = sup W has he same disribuion of N and 3.3 Reflecion Principle P (M a) = 2 a 2π e u2 2 du. (5) In his subsecion, we look wih more deail a he disribuion of M = sup s X s, where X is a general (µ, σ) Brownian moion. We firs sar he analysis for he special case of µ =, σ = 1. In his case, we can apply a similar argumen ha he one used in he previous subsecion based on he reflecion principle o show ha P (M x) = 2P (X x) = P ( X x). We can also compue he join disribuion for (X, M ), ha is F (x, y) = P (X x, M y). Since X = and M X w.p.1, we can focus our aenion o he case x y and y. Firs of all, we noice ha F (x, y) = P (X x) P (X x, M > y) = Φ(x 1 2 ) P (X x, M > y), where Φ( ) is he N(, 1) disribuion funcion. From he reflecion principle P (X x, M > y) = P (X 2y x) = P (X x 2y). Thus, we have he following resul. Propery 7 If µ = and σ = 1, hen P (X x, M y) = Φ(x 1 2 ) Φ((x 2y) 1 2 ). The previous resul depends heavily on he assumpion µ = or in oher words on he reflecion principle. In order o exend he resul o general Brownian moion, i is required firs o undersand how making a change of measure can lead o a change of drif. Le P and Q be wo probabiliy measures on he same space (Ω, F) wih he imporan propery ha P is dominaed by Q. Tha is, Q(A) = = P (A) =. Then, here exiss a non-negaive random variable ξ (also denoed by dp dq ) such ha P (A) = ξ dq, A F. A 8

9 An imporan implicaion of he above relaion is ha if Y is a random variable and E Q ( ξy ) < hen E P (Y ) exiss and E P (Y ) = E Q (ξy ). The random variable ξ is usually called he densiy or Radon-Nikodym derivaive (or likelihood raio) of P wih respec o Q. In order o find he densiy ξ associaed o wo Brownian moion measures of differen drif, we use an heurisical approach. Le consider now a (µ, σ) Brownian moion and a sequence of insans = < 1 < < n = such ha i i 1 = δ, i = 1,..., n. The densiy associaed o ha paricular sequence of insance is given by: 1 (σ 2πδ) n n i=1 e (X i X i 1 µδ) 2 2σ 2 δ. If he drif were insead µ+θ hen densiy is obained replacing µ by µ+θ above. Thus, he densiy is given by: e 1 2σ 2 δ P n i=1 (X i X i 1 µδ)2 (X i X i 1 (µ+θ)δ) 2. Afer some algebra, we have ha he densiy is given by: ξ() = e θ θ σ2 (X µ 2 ) (6) = V θ σ 2 (). Where V β () is Wald maringale defined in propery (5). We can compue he disribuion of M for he case of µ as follows (we sar wih σ = 1): P µ (M x) = E (V µ (); M x) ( ) ( ) x µ x µ = 1 Φ + e 2µx Φ. Finally, for he general case (µ, σ), we can rescale he probabiliy measure o obain: ( ) ( ) x µ P (M x) = Φ σ e 2µx x µ σ2 Φ σ, (7) which is called he inverse Gaussian disribuion. 3.4 Forward and Backward Equaions An imporan exension o he sandard Brownian moion is relaed o he iniial condiion X. Previously, we have imposed he resricion ha X = w.p.1. We now urn o he general case X = x w.p.1, where x is any real number. In order o make explici his new value of he iniial sae, we inroduce he noaion P x o refer o he probabiliy measure ha saisfies P x (X = x) = 1 (he same is valid for E x, he expeced value operaor under P x ). A firs imporan resul is relaed o he way we represen Brownian moions (BM). Of course, we have already given a concree definiion of a BM, however, le look a an alernaive represenaion. 9

10 We know ha X +s X has a N(µs, σ 2 s) disribuion. Thus, he ransiion densiy p(, x, y)dy P x (X dy) = 1 ( ) y x µ σ φ σ dy saisfies he following differenial equaion: wih iniial condiion p(, x, y) = ( 1 2 σ2 2 x 2 + µ x p(, x, y) = δ(x y) = { ) p(, x, y), 1 if y = x, oherwise. The differenial equaion above characerizes BM and is called Kolmogorov s backward equaion. If insead of differeniaing wih respec o he iniial sae x, we differeniae wih respec y, he final sae, we ge he Kolmogorov s forward equaion p(, x, y) = ( 1 2 σ2 2 y 2 µ y ) p(, x, y). In he special case when µ =, he previous equaion reduces o he radiional hea equaion (or diffusion equaion), for his reason Brownian moion are usually called diffusion processes. 3.5 Hiing Time Problem An imporan applicaion of BM is he Hiing Time problem. Tha is, he problem of deermining he firs ime when he process reaches a predefined sae. Le define T (y) = inf{ : X = y}, i.e., he firs ime a which X reaches he value y. Suppose ha he process sar a x and le < x b. Then, we are ineresed in finding he disribuion of T T () T (b). A firs sep is he following resul: Propery 8 E x (T ) <, x b. The proof is based on he maringale sopping heorem, ha is Theorem 4 Maringale Sopping Theorem Le T be a sopping ime and X a maringale (wih righ-coninuous sample pahs) on cerain filered probabiliy space. Then he sopped process {X( T ), } is also a maringale. 1

11 Thus, if we apply his resul o M = X µ, which is clearly a maringale, we have ha: E x (M(T )) = E x (M()) = x. Bu E x (M(T )) = E x (X(T )) µe x (T ). Thus, for µ, he resul in (8) follows direcly. For he case, µ =, we have o apply he maringale sopping heorem o he maringale X 2 σ 2. Le us now recall Wald maringale inroduced in propery (5), ha is, V β () = e βx q(β) where he funcion q( ) is given by q(β) = µβ + σ2 β 2 2. Now, i can be shown ha Therefore, we have he following decomposiion: E x (V β (T )) = E x (V β ()) = e βx, x b. e βx = E x (V β (T ); X T = ) + E x (V β (T ); X T = b), = ψ (x q(β)) + e βb ψ (x q(β)), (8) where ψ (x λ) E x (e λt ; X T = ) and ψ (X λ) E x (e λt ; X T = b). Solving he equaion q(β) = λ, we ge: β (λ) = µ + µ 2 + 2σ 2 λ σ 2 > ; β (λ) = µ µ 2 + 2σ 2 λ σ 2 <. Thus, combining his resul and (8), we ge he following sysem of equaion: e β (λ)x e β (λ)x = ψ (x λ) + e β (λ)b ψ (x λ), = ψ (x λ) + e β (λ)b ψ (x λ). The soluion of his sysem gives he following resul. Propery 9 Le λ > be fixed. For x b, ψ (x λ) = θ (x, λ) θ (x, λ)θ (, λ) 1 θ (b, λ)θ, (, λ) ψ (x λ) = θ (x, λ) θ (x, λ)θ (b, λ) 1 θ (b, λ)θ, (, λ) θ (x, λ) = e β (λ)x, θ (x, λ) = e β (λ)(b x). Finally, from he previous resul, we can obain he disribuion (or more precisely he Laplace ransform) of T. Proposiion 1 Le θ and θ be defined as above. Then, E x (e λt () ; T () < ) = θ (x, λ) ; E x (e λt (b) ; T (b) < ) = θ (x, λ), x b. In addiion, if µ = hen P x (X T = b) = x b. Oherwise, P x (X T = b) = 1 ξ(x) 1 ξ(b) ; ξ(z) e 2µz σ 2, x b. 11

12 4 Sochasic Calculus The main goals of his secion is o presen Iô s lemma and he use of sochasic differenial equaions as imporan ools for modelling sochasic processes. We sar he analysis inroducing heurisically Iô s Sochasic differenial equaion. 4.1 Moivaion I is a common pracice when modelling physical sysems o express he dynamics of he sysem, i.e., is evoluion over ime hrough a difference or differenial equaion. For example, when describing he posiion (y()) of a cerain objec a ime, we migh use he relaion dy() = v(), d where v() is he insananeous velociy of he objec a ime. In general, differenial equaions have been used exensively in science, and probably one of heir bigges advanages is ha hey are able o capure he essence of he physical sysem wihou incorporaing he naural and necessary difficulies ha are imposed by border condiions. Le us now look a he general (deerminisic) differenial equaion: dx() = f(, x()). d Solving his equaion is an old problem in mahemaics, and i is no he purpose of his noe o go ino he deails of how o solve i. We would like, however, o inroduce some ype of uncerainy ino he model. One easy way of doing his is o use he radiional rick used by economericians, ha is, o simply add an sochasic erm o he above relaion. In order o do ha, we proceed as follows. We firs approximae he dynamics by: x( + ) x() = f(, x()) + o( ), where o() is funcion such ha 1 o() as. If we assume now ha uncerainy can be model by an sochasic process v() ha we simply add in o he dynamics of he sysem, we have: x( + ) x() = f(, x()) + v( + ) v() + o( ). In paricular, we migh hink on his uncerainy as being he sum of independen and small perurbaions. Thus, a reasonable model is o suppose ha v( + ) v() = σ(, x())(z( + ) z()), where z() is Wiener process and σ accouns for he variance of v. We can hen rewrie he dynamics of he sysem as follows: dx = f(, x)d + σ(, x)dz, which is called Iô s sochasic differenial equaion. Noice ha we can noe divide by d above since z is nowhere differeniable. 12

13 4.2 Sochasic Inegraion Since he Wiener process is nowhere differeniable, Iô s differenial equaion does no have a clear meaning perse. In his subsecion, we will give i one, which is based on he noion of sochasic inegral. The idea is he following, we use he noion: as a shorhand for x() = x() + dx = f(, x)d + σ(, x)dz, f(s, x(s))ds + σ(s, x(s))dz(s). The firs inegral in he righ-hand side is undersood in he usual Riemann sense. The second inegral, however, does no have a clear meaning for he reasons ha we have already menioned. Le us hen focus in he following sochasic process: I (X) = X s dw s,, (9) ha we called he sochasic inegral. Here X is any sochasic process and W is a wiener process. Sochasic inegraion was firs presened by Iô (1944) and exended laer by Doob (1953). Here, we will no go ino he formal deails behind he heory of sochasic inegraion. We will raher give a more simpler and inuiive analysis. From radiional calculus, we know ha if a funcion is relaively well-behaved in he inerval [, ] (i.e., i is inegrable), hen we can approximae he value of I = f(s) ds, as follows. We firs inroduce a sequence of pariions {P n : n 1} where P n = { i : i n} is a pariion of he inerval [, ], i.e., = < 1 < < n =. We denoe by P n = max{ i i 1 : 1 i n}. Then, if lim n P n =, we have ha I = lim n i P n f(ξ i )( i+1 i ), where ξ i [ i, i+1 ]. In paricular, ξ i = i or ξ i = i+1 does no make much difference for deerminisic real-valued funcion. This analysis is exacly he one ha we will apply o compue sochasic inegral, however, he analysis requires some exra aenion. Le H 2 he space of F -adaped sochasic processes X such ha E[ T X2 (s) ds] <. We inroduce a special class of sochasic processes ha we call simple. A process X H 2 is simple if here exis a sequence of imes { k } such ha = < 1 < < k 13

14 Simple Process X (ω) Figure 2: Simple process. and X(, ω) = X( k 1, ω), [ k 1, k ) k = 1, 2,.... We noice ha he sequence { k } is independen of ω. Given he special form of simple processes, i is possible o give a clear definiion of he sochasic inegral in his case. In fac, if X is simple, hen I(X) = n 1 X dw = X( k )[W ( k+1 ) W ( k )], k= where = and n =. The imporance of simple processes is no only ha we are able o compue easily heir inegrals bu also ha hey can be use o approximae oher more complex processes. Tha is, Propery 1 Le consider he sochasic process X H 2, i.e., [ ] E X 2 (s) ds <,. Then, here exiss a sequence of simple processes {X n } H 2 such ha X n n X. Moreover, he value of I (X) can be obained from he fac ha I (X n ) n I (X). 14

15 Le define he norm in H 2 as follows: Then we have he following resul [ ] 1 X = E X 2 2 (s) ds. Proposiion 2 Le X H 2, hen E[I (X)] = and I (X) = X. Example 1: Le consider he case when X = W, i is no hard o show ha W H 2, moreover, W = Now, in order o compue I(W ) we inroduce he simple processes ( ) [ k k X n (s) = W 2 n, s (k + 1), 2n 2 n Le define k = k 2 n. Then, for hese simple processes we have: I (X n ) = 2 n 1 k= k= W ( k )[W ( k+1 ) W ( k )] ). = 1 2 n 1 [W 2 ( k+1 ) W 2 ( k )] 1 2 n 1 [W ( k+1 ) W ( k )] = 1 2 W 2 () n 1 k= [W ( k+1 ) W ( k )] 2. k= Bu in equaion (1), we saw ha he summaion above converges o. Therefore, we conclude ha: I (W ) 2. W dw = 1 2 W 2 () 2. (1) 4.3 Iô s Lemma In his secion, we define he noion of sochasic differenial and sae and prove Iô s lemma which is he fundamenal rule for compuing sochasic differenials. We sar by defining some noaion. As usual, we consider a probabiliy space (Ω, F,P ), a Wiener process W (, ω), a process Y (, ω) ha is joinly measurable in and ω wih respec o F, is adaped and saisfies T Y (, ω) d < w.p.1. We also consider a process X ha is non-anicipaing on [, T ]. We say ha Z is an Iô process if i has he following funcional form: Z(, ω) = Z(, ω) + X(s, ω)dw (s, ω) + Y (s, ω)ds. (11) The firs inegral in he righ-hand side is call he Brownian componen of Z and i has o be compued according o he analysis ha we did in he previous secion. The second inegral is 15

16 called he drif componen (or VF componen) of Z and i is evaluaed in he usual Reimann sense. Insead of using (11) o represen Z, we say ha Z has an Iô differenial (or sochasic differenial) dz given by: Proposiion 3 (Iô s Lemma) dz = XdW + Y d. Le u(, x) be a coninuous non-random funcion wih coninuous parial derivaes u, u x, and u xx. Suppose ha Z is a process wih sochasic differenial dz = XdW + Y d. Le define he process V () = u(, Z()), hen V has a sochasic differenial given by: dv = [ u(, Z) + Z u(, Z)Y u(, Z)X2 Z2 ] d + u(, Z)XdW. (12) Z (The proof of he Lemma uses a second order Taylor expansion of he funcion u(, x).) Le us ake a look a Iô s lemma in a paricular case. Le suppose ha u(, Z) = f(z), for some wice coninuously differeniable funcion f. Then (12) implies: df(z) = [f (Z)Y + 12 ] f (Z)X 2 d + f (Z)XdW. Rearranging erms we ge: df(z) = f (Z) [Y d + XdW ] f (Z)X 2 d = f (Z)dZ f (Z)(dZ) 2. (13) Relaion (13) is a simplify way of expressing Iô s lemma and uses he convenion (dz) 2 = (Y d + XdW ) 2 = X 2 d. The idea is ha in differenial erms only (dw ) 2, his is consisen wih our finding in (1) abou he quadraic variaion of Wiener processes. Le noice ha for ordinary differenials df(z) = f (Z)dZ, hus he second erm in (13) is he main difference for sochasic differenial ha, as we have already menioned, reflecs he infinie variaion of Brownian pahs. Example 2: Solve he SDE dv = α V d + β V dw. In order o solve he sochasic differenial above, we inroduce he following change of variable: A = ln(v ). Then, using Iô s lemma we ge: da = 1 V dv 1 }{{} 2V [1] }{{} [2] 2 (dv )2. 16

17 Thus, replacing [1] by α d + βdw and [2] by β 2 V 2 d we ge: da = (α 12 ) β2 d + β dw. This linear differenial implies A = (α 1 2 β2 ) + βw. ransformaion A = ln(v ) we conclude: Finally, combining his resul and he V () = e (α 1 2 β2 )+βw (). Tha is, V is a geomeric Brownian moion. Exercise 2: Compue E[V ] and E[V 2 ]. Theorem 5 (Inegraion by Pars) Suppose ha f(s) is a deerminisic coninuous funcion of bounded variaion in [, ]. Then f(s) dw s = f() W W s df(s). We finish his secion wih an imporan resul abou he exisence of soluions for sochasic differenial. Theorem 6 Le consider he Iô process Z defined hrough he following sochasic differenial: dz() = a(, Z())d + σ(, Z())dW (), wih iniial condiion Z(, ω) = c(ω) = c. If 1. a and σ are boh measurable wih respec o all heir argumens, 2. There exiss a consan K > such ha a(, x) a(, y) + σ(, x) σ(, y) K x y, a(, x) 2 + σ(, x) 2 K 2 (1 + x 2 ), 3. The iniial condiion Z(, ω) does no depend on W () and E[Z(, ω) 2 ] <. Then, here exiss a soluion Z() saisfying he iniial condiion which is unique w.p.1, has coninuous pahs and sup E[Z() 2 ] <. 5 Coninuous Time Markov Processes In his secion, we define and characerize Markov diffusion processes in coninuous ime and esablish heir connecion o SDE s and PDE s. 17

18 5.1 Definiions Definiion 7 (Markov Process) A sochasic process X is a Markov process if 1. For any sequence of imes 1 < 2 < < m < and B B(R) P (X() B X( 1 ), X( 2 ),..., X( m )) = P (X() B X( m )). 2. The funcion ˆP (s, y,, B) defined as ˆP (s, y,, B) = P (X() B X(s) = y) is B(R)-measurable for fixed s,, B and a probabiliy measure on B(R) for fixed s, y,. 3. The Chapman-Kolmogorov equaion holds for s < r <. ˆP (s, y,, B) = R ˆP (r, x,, B) ˆP (s, y, r, dx) The funcion ˆP (s, y,, B) is called he ransiion probabiliy of X. Le X be a one-dimensional Markov process wih ransiion probabiliy ˆP (s, x,, A). Definiion 8 A Markov process X() is called a diffusion process if he following condiions hold. 1. For every x and ɛ > uniformly over s <. x y >ɛ ˆP (s, x,, dy) = o( s) 2. There exis funcions a(, x) and b(, x) such ha for every x and ɛ > (y x) ˆP (s, x, s, dy) = a(s, x)( s) + o( s), uniformly over s <. x y ɛ x y ɛ (y x) 2 ˆP (s, x, s, dy) = b(s, x)( s) + o( s), The funcion a(, x) is called he local drif and he funcion b(, x) := σ 2 (, x) is called local diffusion. 18

19 The following resuls esablishes he relaionship beween diffusion processes and sochasic differenial equaions. Theorem 7 Le a(, x) and σ(, x) denoe wo funcions ha saisfy he assumpions of Theorem 6 and le X(s) denoe a process defined for s [, T ] ha is a soluion of he sochasic differenial equaion X(s) = X() + s a(τ, X(τ)) dτ + s σ(τ, X(τ)) db(u). Then he process X(s) is a diffusion process whose ransiion probabiliies are given by he relaion ˆP (, x, s, A) = P (X(s) A X() = x) = P (X,x (s) A), where he process X,x (s) saisfies he SDE X,x (s) = x + s a(τ, X,x (τ)) dτ + s σ(τ, X,x (τ)) db(u). 5.2 Connecion beween Diffusion Processes and PDE s We will now esablish he connecion beween diffusion processes and PDE s. As before, we will resric he exposiion o one-dimensional diffusions. The exension o higher dimensions does no involve new ideas. As before le a and σ 2 be he drif and diffusion coefficien of diffusion process X wih ransiion probabiliy ˆP (s, x,, dy). Given a s <, we define he linear operaor S s, associaed o a Markov process X as follows. For all bounded, real valued, measurable funcion f, S s, (f(y)) = We noe ha he Chapman-Kolmogorov condiion implies Noe ha S, (f) = f. S s, = S s,r S r,, for s < r <. R f(x) ˆP (s, y,, dx) = E sy [f(x())]. Definiion 9 For a given Markov process X, we define is generaor A() as follows: For all bounded, real valued, measurable funcion f, S,+h (f) f A() f = lim. h h Consider a funcion ψ(, x). Then under suiable resricions on ψ we can wrie d d S E sy [ψ( + h, X( + h)) ψ(, X())] s, ψ(, y) = lim h h = lim h E sy [ψ (, X( + h)) h] + E sy [ψ(, X( + h)) ψ(, X())] h = E sy [ψ(, X()) + A()ψ(, X())] 19

20 So we can rewrie his las equaliy as follows d d S s, ψ = S s, [ψ + A()ψ]. Inegraing his las expression over and inerchanging inegral and expecaion (no formal prove of his inerchange is aemped here) we ge E sy [ψ(, X())] ψ(s, y) = E sy [ s ] (ψ (τ, X(τ)) + A(τ)ψ(τ, X(τ))) dτ. Proposiion 4 Le X() = X s,x () be a diffusion process of he form X s,x () = x + s u(τ, X(τ)) dτ + s v(τ, X(τ)) db(τ). where u and v are bounded funcions in L 2. Then for a sopping ime T > s and a wice-coninuously differeniable funcion f we have E sx [f(x(t )] = f(x) + E sx [ T Corollary 1 For a diffusion process he infiniesimal generaor is given by s ( u(τ, X(τ)) f(x(τ)) + 1 ) ] x 2 v2 (τ, X(τ)) 2 f(x(τ)) x 2 dτ. dx() = a(, X()) d + σ(, X()) db() A = a(, x) x σ2 (, x) 2 x 2. The following resuls, due o Kolmogorov, is similar o hose resuls presened on secion (3.4). Theorem 8 (Kolmogorov Backward Equaion) Le ϕ(x) denoe a coninuous bounded funcion such ha he funcion u(, x) = ϕ(y) ˆP (s, x,, dy) = S s ϕ(x()) has bounded coninuous firs and second derivaives wih respec o x and le he funcion a(, x) and b(, x) be coninuous. Then u(, x) has a derivaive u/ which saisfies he equaion u u = a(, x) x b(, x) 2 u x 2 and u(, x) saisfies he boundary condiion lim s u(, x) = ϕ(x). 2