1. Stochastic Process

Transcription

1 HETERGENEITY IN QUANTITATIVE TSE OCTOBER 17, 216 STOCHASTIC CALCULUS BASICS SANG YOON (TIM) LEE Very simple notes (need to add references). It is NOT meant to be a substitute for a real course in stochastic calculus, just listing heuristic derivations of the stuff most oftenly used in economics. Ito calculus is a lot more than only dealing with Poisson jumps and Wiener processes. Some abuses of notation included without clarification. 1. Stochastic Process A stochastic process is a collection of random variables (measurable functions) {X t : t T}, X t : Ω S, ordered in t (time), along with a measurable space (S, Σ). The probability space (Ω, F, P) denotes, respectively, the state space" (set of all possible histories), the σ-algebra that contains all possible sets (Borel sets) of histories induced by Ω, and the probability measure over F. The space (S, Σ) contains the range of the function X t : Ω S and its corresponding σ-algebra. For example, for most of our applications X t R or R +. If X t is measurable, any process induces a measure P t that we can construct using the original probability space. This calls for the notion of a filtration: a weakly increasing collection of Borel sets on Σ, {F t, t T}, s.t. for all s < t T, F s F t F. The process X is adapted to the filtration {F t } t T if X t is F t -measurable. This just means that for any X t, I can compute the probability only using F t and not all of F. Hence, a well defined stochastic process is always adapted to its natural filtration ({ F t = σ X 1 s }) (A) : s t, A Σ. This just means that for any history of X t up to time t, all possibly realizable trajectories can be mapped backed into a subset of F t, so that I can compute its probability for all points up to time t. This generates an induced probability measure over X. EXAMPLE 1 Let Ω = [, 1. Then any ω Ω is just a coordinate on the infinite dimensional unit cube. If we let X t : Ω S denote the t-th coordinate, S is just the unit interval [, 1. If we construct, say, the probability measure so that P = P 1 P, where each P t is the uniform distribution, X t is i.i.d. uniform. 2. Poisson (Jump) Process Let N t be the random variable equal to the number of hits" up to time t. The (adapted) state space is R [,t and range is all right-continuous paths that increase by 1. Now define 1

2 the probability measure over ω as Poisson: P {N t N s = n} = (λ(t s)n n! exp( λ(t s) where λ is the rate of arrival. This is what is usually called the Poisson process. (Not to be confused with what we use more often in economics: X t is a Compound Poisson Process (CPP) if if it changes to some value at rate λ t, studied below. In fact this is a new random variable in which X t changes to some value if N t > N s for all s < t, and you could redefine the probability space to the histories of N t rather than R [,t. This is the set of all right-continuous paths that increase by 1.) More typically, the Poisson process is defined as a counting process: DEFINITION 1 A continuous stochastic process N t is Poisson if 1. N t is a counting process: (a) N t lives in (Z +, 2 Z +), for all t, (b) N s N t for all s t, (c) lim s t N s lim s t N s for all t ; that is, no hit can happen simultaneously. 2. N = a.s., 3. N is a stochastic process with stationary, independent increments The two definitions are equivalent; there are many other definitions as well but I refer you to the internet. It is easier to show that the earlier definition implies the counting process; by definition, increments are independent. The probability of getting, 1, or 2 or more hits in a time interval dt > is P(N t+dt N t = ) = exp( λdt) P(N t+dt N t = 1) = λdt exp( λdt) P(N t+dt N t 2) = (λdt) 2 e λdt /2 + o(dt) = o(dt). 1 λdt + o(dt) λdt λ 2 dt 2 + o(dt) λdt Clearly, the actual probability that something happens in any interval dt (and (t, t + dt, since the increments are independent) is. Conversely, one way to make sense of the counting process is to realize that stationarity implies E[N(T)/T = lim T N(T)/T = λ and instead of sending T to infinity, send the number of intervals dt in (, T to infinity to get that the expected number of hits in any given time interval is λ: E[dN t = E[N(dt) = λdt = P(N(dt) = ) + 1 P(N(dt) = 1) + 2 n P(N(dt) = n) n=2

3 = P(N(dt) = 1) since two hits cannot occur at the same time. This is important later when we derive the stochastic HJB equation. 2.1 Compound Poisson Process Now define a jump process over the underlying Poisson process: Let X t be a r.v. that is γ a if N t is even and γ b if N t is odd. Heuristically, E[dX t = exp( λdt) + (γ b γ a ) λdt exp( λdt) + (λdt) 2 exp λdt /2 E[Ẋ t = lim dt (γ b γ a ) λ exp( λdt) = λ(γ b γ a ) More generally, let {Z k } k 1 be an i.i.d. ordered sequence of random variables with measure G z (z), independent of the Poisson process N t. Let X t be a continuous stochastic process that is a function of (N t, Z k ), and define Then and X t = EX t = N t Z k. k=1 { [( n ) } E Z k N t = n P(N t = n) = n=1 k=1 [( n ) = E Z k = λtµ Z n=1 k=1 dx t = Z Nt dn t = assuming X =. Z Nt dn t 2.2 Stochastic Integral with Poisson e λt (λt) n 1 (n 1)! e λt (λt) n n! n=1 = µ Z λt First consider a function f (N t ). The integral is easy to write as f (N t ) f () = = N t k=1 [ f (k) f (k 1) = [ f (N s ) f (N s 1) dn s = [ f (1 + N s ) f (N s ) dn s [ f (N s ) f (N s ) dn s, 3

4 where N s is the left limit of the Poisson process, and only one jump occurs in an dt by definition (or construction) of the Poisson process. For the compound process, recall that the waiting time for the kth hit of the Poisson process, T k, is also a random variable s.t. that the event {T k > t} {N t k 1}; in particular this means that T k T k 1 is an i.i.d. process by definition. For k = 1, the waiting time follows an exponential distribution. For k > 1, since P(T k > t) = λ t e λs (λs) n 1 ds, (1) (n 1)! P(T k > t) = P(T k > t T k 1 ) + P(T k 1 > t) = P(N t = n 1) + λ = e λt (λt) n 1 (n 1)! + λ t t e λs (λs) n 2 ds (n 2)! e λs (λs) n 2 ds (n 2)! and integration by parts leads to (1). Using waiting times, the stochastic integral of a function of a compound Poisson process can be written f (Y t ) f () = = = N t k=1 [ f (Y T k + Z k ) f (Y T ) = [ f (Y s + Z Ns ) f (Y s ) dn s k [ f (Y s ) f (Y s ) dn s [ f (Y s ) f (Y s ) (dn s λds) + λ [ f (Y s ) f (Y s ) ds. 3. Wiener Process (Brownian Motion) DEFINITION 2 A Wiener process is defined by four properties: 1. W = a.s. 2. Independent increments: W t W s is independent of F s for all s t 3. Normality: W t W s N (, t s) 4. W t is continuous a.s. We could spend the whole semester just talking about this, which we won t. Basically, think of Brownian motion as a random walk in continuous time: the best predictor of dx t is, with Gaussian errors. So clearly, W t is a particular type of a martingale (E[W t F s = W s a.s., for all s < t < ). 4

5 Most commonly you will encounter a Brownian motion with drift, a geometric Brownian motion, or a generic (Ito) diffusion process: dx t = µdt + σdw t, dx t = µx t dt + σx t dw t, dx t = µ(x t )dt + σ(x t )dw t the geometric Brownian motion simply gives dx t /X t = d log X t = µdt + σdw t, so it is the just a Brownian motion with drift in percentage points (or log-points, to be exact). In the Ito process, the instantaneous drift and variance depend on the current value of X t and is related to the version of Ito s Lemma that we will look at below. Before we move along, note that both the Poisson process and Brownian motion are Markov processes, but while the Brownian motion has a continuous time path a.s., the Poisson process has a discontinuous time path a.s. Also, Poisson was not a martingale, but dn t λdt was. It will be useful to know the quadratic variation of the Brownian motion: we will use a particular formulation that exploits the CLT in discrete time: W t E[Wt 2 2 n 1 = lim n i W t W t n i+1 W t n i i=1 [ i W t 2 where t n i it/2 n. This makes the difference in adjacent W equal t/2 n, so Z i 2 n [ i W t N (, t). That is, all Z i are normal with variance t. Since 2 n 1 i W t = i=1 2 n 1 Zi 2 /2n, t=1 the term converges to t a.s. by SLLN: W t E[W 2 t = t. Hence the quadratic variation of a Brownian motion is equal to t. This is an important notion that will help us understand Ito. Conversely, suppose Z i is a random walk s.t. Z equals ± h with probability (p, 1 p). That is, Z is Bernoulli. So E Z = h(2p 1) and V Z = 4p(1 p)( h) 2. Now we repeat this process n times; this is a Bernoulli process and we can write E[ n Z = n h(2p 1) = T h(2p 1)/ t E[ T Z V[ n Z = n( h) 2 4p(1 p) = T( h) 2 4p(1 p)/ t V[ T Z, 5

6 where all we have done is to consider that the n trials happened in a time interval T with time t = T/n increments. If we want this process to converge to a Wiener process as t, n, we just choose h and p so that h(2p 1)/ t = µ, ( h) 2 4p(1 p)/ t = σ 2, h = σ [ 1 t + (µ/σ) t = σ t as t [ p = 1 ± (µ/σ) t / 1 / + (µ/σ) t 2 For the standard BM W t, µ = and σ = 1, so h = t, p = 1/2. So BM can be viewed as the limit of the sum of Bernoulli i.i.d. r.v. s with 1/2 probabilities ±1: W T = lim t [ T Z = lim n [ n Z which converges to N (, T) since t = T/n. 4. Stochastic Integral with BM and CPP The Ito stochastic integral is defined on a semi-martingale (basically, a random walk plus a process with finite variation), where the underlying martingale component has finite quadratic variation. That is, X t = X + B t + M t dx t = db t + dm t, where E[M t F t =, B t is adapted to F t, and M t <. For example, if M t is BM, M t = t. Similarly if M t is the compensated Poisson process N t λt, M t = N t. THEOREM 1 (ITO S LEMMA FOR CONTINUOUS MARTINGALES) If X t is continuous and f is a three-times continuously differentiable function, the stochastic integral of f (X t ) is f (X t ) = f (X ) + f (X s )db s + f (X s )dm s or d f (X t ) = f (X t )db t + f (X t )dm t f (M t )d M t. f (M s )d M s Note that this version of Ito does not apply to Poisson. No proof given, but the intuition is that Ito extends Riemann integrals to stochastic increments: f (X t )dx t = lim n n i=1 f (X ti 1 ) [ X ti X ti 1 where Π n is an n-partition of [, t. It is important that the point of approximation for each interval is taken from the left. Also importantly, the stochastic integral itself is not a deterministic concept: it is the martingale such that its quadratic variation equals the expectation of the square of all realized paths integrated over X t. 6

7 Formally, note that any function of M t is simply a process Y t that is adapted to F t, the filtration of the martingale. Let M t be square-integrable in the sense that M t = EM 2 t <. Let Y (k) be simple process, that is for an infinitely fine partition {t i } i= on [, t, and countably infinite sequence of random variables {ζ (k) i } i=, Y (k) t = ζ (k) 1() + ζ (k) i 1 1(t i 1 t i. i=1 One definition of the stochastic integral of Y t over M t is the (unique) square integrable martingale I t (Y) s.t. DEFINITION 3 (HEURISTIC DEFINITION OF STOCHASTIC INTEGRAL) For all sequences of simple processes lim k [Y (k) Y = (in quadratic variation): lim Y(k) Y 2 = lim E[Y (k) Y 2 k k I(Y) is the martingale s.t. [ 2 lim I(Y(k) ) I(Y) 2 = lim E I(Y (k) ) I(Y) =, k k where for each Y (n), I t (Y (k) ) = ζ (k) [ i 1 Mti M ti 1. i=1 This is just a complicated way of saying the stochastic integral is a Riemann-Stieltjes integral where the measure of integration is stochastic (so need Lebegue). When and when it doesn t work, and why it s unique, we won t worry about. Perhaps the most important property of the stochastic integral defined as such is that [ E[I t (Y) F s = I s (Y) and E[I t (Y) 2 = E Ys 2 d M s where the first part just means it is a martingale, and the second that the square can be taken inside the integral. The problem is when M ti M ti 1 goes to, but Ito tells us that as long as M t is bounded we can define an integral. [Graphical Representation of Riemann-Stieltjes and Ito Although the above doesn t apply to Poisson, we already know how to write the integral for CPP. Consider the general jump diffusion process (which is all we re going to 7

8 deal with, really) X t X = µ(t, X t ) + σ(t, X t )W t + Y t dx t = µ(t, X t )dt + σ(t, X t )dw t + dy t, (2) }{{}}{{}}{{} db t dm t jumps where W t and Y t are independent Wiener and CPP. Then since W t = t, we have f (X t ) f (X ) = or + µ(s, X s ) f (X s )ds + [ f (X s ) f (X s )dn s σ(s, X s ) f (X s )dw s d f (X t ) =µ(s, X s ) f (X t )dt + σ(s, X s ) f (X t )dw t σ2 (s, X s ) f (M t )dt + [ f (X t ) f (X t )dn t. σ 2 (s, X s ) f (X s )ds Note of caution: you cannot just write λ[ f (X t ) f (X t )dt instead of [ f (X t ) f (X t )dn t there, since dn t λdt is only in expectation. Without proving (again!) we can also let λ vary with time and state, of which the non-homogeneous Poisson process satisfies [ E N t λ(s, X s )ds = = E t [dn t λ(t, X t )dt. Intuitively, we can always reset the underlying Poisson process following any hit until the next hit arrives, during which the process remains homogeneous." 5. Stochastic HJB Let x t denote realizations from the stochastic process X t that follows the jump diffusion process (2). Now consider the stochastic control problem { T } v(t, x t, a t ) = max E t U(s, X s, a s, c s )ds (c s ) t s.t. da s = f (s, x s, a s, c s )ds where T can be finite or infinite, and I have suppressed the scrap value Z(T, a T, X T ) which may or may not be there. Except for x t, all that has changed from the deterministic control problem is that we added an expectation operator over the objective. But let us generalize the jumps a bit. Let Y t, be associated with a non-homogeneous Poission process with time- and state-dependent arrival rates λ(t, x t ), and also have jumps Z k that are drawn from a time- and state-dependent measure G z (z; t, x t ). For example, X t can be a random wage or dividend process, or interest rate process (in which case things can be simplified, since it would multiply the state a s ; likewise if we wanted a stochastic discount rate in which case it would show up multiplicatively in U). 8

9 Without going into the details, we can use similar methods as in deterministic control (stochastic versions of Taylor expansion, verification theorem) to show that the following heuristic method works: U(t, a t, c t )dt + E t [dv(t, x t, a t ) where by Ito we have [ dv = V t + V a f (t, x t, a t, c t ) + V xx 2 σ2 (t, x t, a t ) dt + crap + λ(t, x t ) [E t V(t, x t + Z k, a t ) V(t, x t, a t ) dt where we have set X t = x t a.s., since it is already realized, and have allowed µ, σ to also depend on a. The crap term is E t [crap = = E t [µ(t, x t, a t )V x dw t + E t {[V(t, x t + Z k, a) V(t, x t, a) [dn t λ(t, x t )} the latter since Z k is independent of N t. So we get the HJB equation V t (t, x, a) =H (t, x, a, V a (t, x, a)) + V xx 2 σ2 (t, x, a) [ + λ(t, x) V(t, x + z, a)dg z (z; t, x) V(t, x, a) so when U = e ρt u, we can multiply the whole system by e ρt and define v(t, x, a) e ρt V(t, x, a) to obtain ρv(t, x, a) =v t (t, x, a) + Ĥ (t, x, a, v a (t, x, a)) + v xx 2 σ2 (t, x, a) [ + λ(t, x) v(t, x + z)dg z (z; t, x) v(t, x, a). Note that the only time the expectation operator comes in is for the (possibly) stochastic r.v. Z k ; everything else is adapted to F t. That is, for the HJB, there is no longer any expectations taken over X t ; all of that is washed out in continuous time with martingales. Since the Hamiltonians are the same as in the deterministic case, it follows that the f.o.c. holds deterministically in continuous time, that is, u c (t, x, a, c) + v a f (t, x, a, c) = (no expectations over v or v a!) 6. Fokker-Planck (Kolmogorov Forward) Equation The last tool that will be relevant for our purposes is the KFE. Given a solution to v(t, x, a), we want to understand the evolution of p(t, x, a), the population p.d.f. over (x, a) at time t. KFE gives us a (partial) differential equation that does exactly this. Formally, KFE tells 9

10 you: suppose at time t, you know P{(x, a) B} for all B F t. How is P(B) evolving going forward in the filtration (for the same set B)? For example, in the savings problem a solution to v admits optimal policy functions c (t, x, a) and associated (change in) assets ȧ (t, x, a). Now suppose at time t, the p.d.f. is represented as p(t, x, a). KFE tells us how the distribution evolves going forward. (Conversely, Feynman-Kac, or the Kolmogorov Backward Equation, tells us how you would have got to p(t, x, a) going backward; but we are not so interested in this). To compare with discrete time, it is as if we are simulating a distribution of individuals starting from some given initial distribution. The following is a version of Fokker-Planck: THEOREM 2 (FOKKER-PLANCK-KOLMOGOROV) Let X t be a stochastic process as in (2), where Y t is a CPP with jumps Z k G z (t, X t ) associated with a non-homogeneous Poisson process with rate λ(t, X t ). Let p(t, x) denote the p.d.f. of x at time t. Then for all x (x min, x max ) (the interior of possibly realizable states), p(t, x) = [µ(t, x)p(t, x) + 2 [ t x 2 x 2 σ 2 (t, x)p(t, x) λ(t, x)p(t, x) + g z (x x ; t, x )λ(t, x )p(t, x )dx. Proof (heuristic): The trick is to use a function that is differentiable, so we can apply Ito. For any x S (the range of X t ), approximate the probability of the event by the expectation of a smooth function: x P(X t x) = p(t, x )dx = E[χ(X t x) = χ(x x)dx. With some abuse of notation, we will assume that the indicator is already smoothed (it does not matter how it is smoothed; convolving it with any mollifer will do). Using Ito, we obtain (the derivative is w.r.t. time): dp(x t x) =E[χ (X t x)µ(t, X t ) σ2 (t, X t )dt + crap (3) + E {λ(t, X t )[χ(x t + Z k x) χ(x t x)} where crap is again in expectations. Note that the derivatives of χ are w.r.t X t, not x. First look at the Poisson part. To compute the expectation over X t, we denote the variable of integration by X t = x, and remember that X t = X t = x a.s.: = [ λ(t, x ) χ(x + z x)dg(z; t, x ) χ(x x) p(t, x )dx z λ(t, x )G z (x x ; t, x )p(t, x )dx x λ(t, x )p(t, x )dx. Note that this is as if" we were looking backward," not forward like we did in the HJB. This is because we are looking at all the X t when they are still random variables, not 1

11 a realized point like in the HJB. (The density functions are deterministic in x, not X t = x ). For the diffusion part, we can integrate by parts: χ (x x)µ(t, x )p(t, x )dx = B 1 (x) x where B 1 is a term determined by boundary conditions: x [µ(t, x )p(t, x )dx, B 1 (x) χ(x max x)µ(t, x max )p(t, x max ) µ(t, x min )p(t, x min ). And likewise { χ (x x)σ 2 (t, x )p(t, x )dx =B 2 (x) χ (x x) } x [σ2 (t, x )p(t, x ) dx x { 2 } =B 2 (x) B 3 (x) + x 2 [σ2 (t, x )p(t, x ) dx where B j are determined by boundary conditions: B 2 (x) χ (x max x)σ 2 (t, x max )p(t, x max ) χ (x min x)σ 2 (t, x min )p(t, x min ) B 3 (x) χ(x max x) [ σ 2 (t, x max )p(t, x max ) [ σ 2 (t, x min )p(t, x min ). x x So we have obtained that (3) becomes x [ dp(t, x) = + x µ(t, x ) x 2 σ2 (t, x ) λ(t, x ) p(t, x )dx (4) λ(t, x )G z (x x ; t, x )p(t, x )dx + B 1 (x) + B 2 (x) B 3 (x) Note that B j (x) = except at the boundaries. So taking the derivative w.r.t. x on both sides of (4) we obtain the formula in the theorem. Of course in our economics problems, typically we want p(t, x, a) not p(t, x). But most problems will assume a law of motion s.t. x is subsumed in a, as we will see later. 11