Lecture 4: Stochastic Processes (I)

Size: px

Start display at page:

Download "Lecture 4: Stochastic Processes (I)"

Emory Johnson
6 years ago
Views:

Miranda Holmes-Cerfon Applied Stochastic Analysis, Spring 215 Lecture 4: Stochastic Processes (I) Readings Recommended: Pavliotis [214], sections 1.1, 1.2 Grimmett and Stirzaker [21], section 9.

1 Miranda Holmes-Cerfon Applied Stochastic Analysis, Spring 215 Lecture 4: Stochastic Processes (I) Readings Recommended: Pavliotis [214], sections 1.1, 1.2 Grimmett and Stirzaker [21], section 9.4 (spectral representation and stochastic integration) Optional: Grimmett and Stirzaker [21] 8.1, 8.2, 8.6, 9.1, 9.3, 9.5, 9.6. Koralov and Sinai [21], Ch. 12, 15.3, 15.7, other selections from Ch. 15, 16. Yaglom [1962], Ch. 1, 2 is a nice short book with many details about stationary random functions We want to be able to describe families of random functions : These functions could represent, for example: velocity of a particle in a smooth chaotic flow # of molecules in the reactant state of a chemical reaction bond angles of a folding protein randomly perturbed harmonic oscillator (swingset) size of a self-assembling viral capsid etc. Such random functions are called stochastic processes. They are written as X(t,ω) or X t (ω) or X(t)(ω), where t T is a parameter taking values an ordered parameter set T, such as T = R,Z +,[a,b], etc, and ω is an element in the sample space Ω. For each fixed t T, X t is a random variable defined on some probability space (Ω,F,P). We can think of a stochastic process in two ways: (1) A parameterized family of random variables. Consider the function t X t (ω) (2) A probability distribution on path space. Consider the function ω X t (ω) Each realization is a function Events are sets of paths, e.g. all paths that pass through [x,x + h] at time t, all paths whose maximum value is bigger than M, etc. From measure theory, we know we cannot construct a probability density on this space, because it is infinite-dimensional

2 4.1 Finite-dimensional distributions How can we construct / characterize stochastic processes? We can t characterize the joint distribution of all values X t, t T, at once. However, we can look at a finite number of points (t 1,t 2,...,t k ) and consider the distribution of (X t1,x t2,...x tk ). Definition. The finite-dimensional distributions (fdds) of a stochastic process X t is the set of measures µ t1,t 2,...,t k given by µ t1,t 2,...,t k (F 1 F 2 F k ) = P(X t1 F 1,X t2 F 2,...X tk F k ), where (t 1,...,t k ) is any point in T k, k is any integer, and F i are measurable events in R. Examples Here are some examples of processes and their fdds. 1. A sequence of i.i.d. random variables. t Z +, P t1,...,t k (F 1 F k ) = k i=1 P(F i). 2. Markov chains. Let X t be a continuous-time, homogeneous Markov chain with generator Q, and let = t < t 1 < < t k. Then the fdd is given by P(X t F,X t1 F 1,...,X tk F k ) = i F µ () (i ) i 1 F 1 P i i 1 (t 1 t ) i k F k P ik 1 i k (t k t k 1 ). 3. Poisson process. For any t 1... t k, let η 1,η 2,...,η k be independent Poisson random variables with rates λt 1,λ(t 2 t 1 ),...,λ(t k t k 1 ). Let P t1,...,t k be the joint distribution of η 1,η 1 + η 2,...,η 1 + η η k ). Suppose we are given a set of finite-dimensional distributions. Is there a corresponding stochastic process X t? Is it unique? In order for the first to be true, we must at least require the fdds to be consistent. Two criteria that must be satisfied are: (1) Invariant under permutations. For any permutation σ of {1, 2,..., k}, we have µ tσ(1),...t σ(k) (F 1 F k ) = µ t1,...,t k (F σ(1) F σ(k) ). (2) Marginal distributions are consistent. For any m N, µ t1,...,t k (F 1 F k ) = µ t1,...,t k,t k+1,...,t k+m (F 1 F k R R) Kolmogorov Extension Theorem. If the family of measures {µ t1,...,t k } satisfies the consistency conditions (1), (2) above, then there exists a stochastic process with the corresponding finite-dimensional distributions. Proof. See Koralov and Sinai [21], p Unfortunately, the construction of a stochastic process via fdds does not let us ask many questions that we may be interested in, such as continuity, differentiability, boundedness, first-passage times, etc none of these properties can be asked about using sets that are measurable in the σ-algebra generated by the fdds. Therefore, in what follows we will usually restrict our attention to a smaller class of functions, such as those that are continuous, or right-continuous, and ask what can be said about these. 2

3 4.2 Gaussian processes A very important class of processes that can be defined by their fdds are Gaussian processes. Definition. X t is Gaussian iff all its fdds are Gaussian, i.e. if (X t1,...,x tk ) is a multivariate Gaussian for any (t 1,...,t k ). Note. Recall that a multivariate Gaussian (X 1,...,X k ) T is defined by its mean µ = (µ 1,..., µ k ) T = (EX 1,...,EX k ) T and covariance matrix C, which has entries C i j = E(X i EX i )(X j EX j ). The density function is where x = (x 1,...,x k ) T. p(x 1,...,x k ) = 1 (2π) k/2 1 detc 1/2 e 1 2 (x µ)t C 1 (x µ), (1) This means that given functions m(t), B(s,t), such that B is positive semi-definite (see below), we can construct a Gaussian process with mean EX t = m(t) and whose finite-dimensional distributions have covariance matrix (B(t i,t j )) k i, j=1. This gives us a lot of flexibility in constructing a Gaussian process. We will investigate properties of them later, when we study covariance matrices and when we consider the Karhonen-Loeve decomposition. 4.3 Stationary processes Another important class of processes are those for which the distribution is invariant with time. For example, one might assume this is the case for waves in the ocean fluctuations in annual mean temperature (prior to 18...) the bond angles in a protein that is in thermodynamic equilibrium any Markov chain that has been run for a long time turbulent velocities in the atmosphere etc It would not be the case for a process that still retains memory of its initial condition, for example, (such as a protein shortly after we observe it in a particular configuration), or a process that can wander far away without bound (such as a random walker on a line or infinite lattice.) Definition. A stochastic process is strongly stationary, or strictly stationary, if all the fdds are invariant with shifts in time, i.e. µ t1 +h,t 2 +h,...,t k +h = µ t1,...,t k (t 1,...,t k ), h R. This definition is usually too hard to work with in practice: it is hard to check it may be too strong in many cases, when the information we are interested in depends only on the one-point and two-point fdds (mean and covariance) A weaker definition is more commonly used in practice. To introduce this we need the following: 3

4 Definition. The covariance function of a real-valued stochastic process X t is where m(t) = EX t is the mean of the process. B(s,t) E[(X s EX s )(X t EX t )] = EX s X t m(s)m(t), Definition. A function B(s,t) is positive (semi-)definite if if the matrix (B(t i,t j )) k i, j=1 is positive (semi- )definite for all finite slices {t i } k i=1. Lemma. The covariance function B(s,t) of a stochastic process is positive semi-definite. Proof. Left for reader. Now consider what happens to the mean and covariance function of a strongly stationary process when we shift the process in time. Mean: we have EX t = EX t+h, so m(t) = m(t + h) for all h, so the mean must be constant. Covariance: we have EX s+h X t+h = EX s X t, so B(s + h,t + h) = B(s,t) for all h. Therefore B(s,t) = f (s t) for some function f. This motivates the weaker definition of stationarity: Definition. A stochastic process is weakly stationary, second-order stationary, or wide-sense stationary if m(t) m, B(s,t) = C(s t). In other words, a process is weakly stationary if its mean and covariance function do not change with time. The covariance function function C(t) = EX t X = EX s+t X s for all s. Notes A weakly stationary process is not strongly stationary in general. If X t is Gaussian and it is weakly stationary, then it is strongly stationary. This is because the mean and covariance completely characterize a Gaussian process Covariance function and spectral measure for weakly stationary processes The covariance function is a very useful way to analyze properties of weakly stationary processes. Let s look at some of its properties. (Without loss of generality, we can assume m =, since we can consider Y t = X t m which has mean zero.) The variance of the process is C() = EX 2 t. C() > C(h) for all h (except for a trivial process which is identically one constant number.) To see why, calculate: C(t) = EX t+s X s (EX 2 t+sex 2 s ) 1/2, by Cauchy-Schwartz. But the RHS is C(). By definition, C(t) = C( t) (for a complex-valued processes X t = X (1) t +ix (2) t we have C(t) = C( t), see section ) 4

5 C(t) is a positive semi-definite function: the matrix (C(t j t i )) k i, j=1 is positive semi-definite for all finite slices {t i } k i=1. You can show this yourself as an exercise, or see Pavliotis [214], p.7. Here s another nice property: Lemma. If the covariance function C(t) of a weakly stationary stochastic process X t is continuous at t =, then it is uniformly continuous for all t R. Furthermore, C(t) is continuous at if and only if X t is continuous in the L 2 sense. Proof. (Mostly from Pavliotis [214], p.6): To show that C(t) is continuous at t = implies it is uniformly continuous: Fix t R, and suppose EX t =. Then C(t + h) C(t) 2 = EX t+h X EX t X 2 = E (X t+h X t )X ) 2 EX 2 E(X t+h X t ) 2 = C() ( EX 2 t+h + EX 2 t 2E(X t X t+h ) ) = 2C()(C() C(h)) as h (Cauchy-Schwartz) Now, Suppose C(t) is continuous. The above showed that E X t+h X t 2 = 2(C() C(h)), (2) which converges to as h. Conversely, if X t is L 2 -continuous, then the above implies lim h C(h) = C(). Note that (2) implies that C() > C(h) (unless X t+h = X t a.s..) Therefore the maximum of a covariance function is always at. There is a very beautiful result in Fourier analysis, which turns out to provide a highly practical way to infer and analyze covariance functions in practice. The main result is given in the following theorem: Bochner s Theorem. There is a one-to-one correspondence between the set of continuous positive semidefinite functions and the set of finite measures on the Borel σ-algebra of R: if ρ is a finite measure then f (x) = e iλx dρ(λ) (3) R is positive semi-definite. Conversely, any continuous positive semi-definite function can be represented in this form. 1 Proof. See Koralov and Sinai [21] and references within. Remark. This theorem was discovered independently by Khinchin slightly after its publication by Bochner, so it is sometimes called the Bochner-Khinchin theorem. 1 cited from Koralov and Sinai [21], p.213 5

6 What this means is that if C(t) is the covariance function of a weakly stationary stochastic process X(t), and C(t) is continuous at, then there exists a measure F such that C(t) = R eiλx df(λ). The measure F is called the spectral measure of the process X t. It measures the amount of energy, or power, in different frequencies of X t : the total amount of power in an interval of frequencies [λ 1,λ 2 ] is F(λ 2 ) F(λ 1 ). Therefore, F(λ) can be thought of as a distribution function in frequency space. If it is absolutely continuous with respect to the Lebesgue measure on R, then we can write df(λ) = S(λ)dλ where S(λ) is called the spectral density of the process. The functions S(λ), C(t) are Fourier transforms of each other: C(t) = e iλt S(λ)dλ, S(λ) = 1 e itλ C(t)dt. (4) R 2π The spectral density S(λ) is also a way to measure the amount of energy in different frequencies of the stochastic process. It is actually somewhat remarkable that it exists: we know, from Fourier theory, that X t itself does not have a Fourier transform indeed, it cannot be L 1 -integrable since the fluctuations are statistically constant for all time, so it never decays. Yet, the spectral measure still lets us talk about the frequencies that make up X t (and we will see later that actually, X t has a sort of generalized spectral representation.) Example. Consider a process with covariance function C(t) = Ae αt, where A,α > are parameters. We can calculate the Fourier transform to be (see Pavliotis [214], p.8) S(λ) = A π α λ 2 + α 2. If the process it represents is Gaussian then it is a stationary Ornstein-Uhlenbeck process. We will return to this example throughout the course, because it is the only example of a stationary, Gaussian, Markov process, and it is used extremely frequently in modelling. Example. How can we make up covariance functions, e.g. for testing code or theory? It is hard to do this directly, because it is hard to quickly tell whether a function is positive semi-definite. However, it is easy to make up functions (or measures) that are non-negative, and provided these are integrable we can take the Fourier transform and obtain an example of a positive semi-definite covariance function. For example, all of these are spectral densities of some covariance function: S(λ) = 1 1 λ 2, S(λ) = (1 λ 2 )1 λ 1, S(λ) = Ae Bλ 2 /2, S(λ) = Ae Bλ 2 /2 1 λ λ 12. Example. Turbulent fluids are frequently characterized by their spectrum, which says what are the energycontaining scales. This is calculated by looking at one component of the velocity field, computing its covariance function, and finding the spectral density of this. Here is an example that shows a typical velocity field, and schematic of a typical spectrum, for turbulence in the atmosphere: 2 2 from Barbara Ryden s notes on Basic Turbulence, see ryden/ast825/ch7.pdf 6

Here is another example, that shows measurements of the three components of magnetic field and corresponding velocity field in the sun (left), and a typical computed spectrum for magnetic field

4.3.2 Ergodic properties of weakly stationary processes If a process is stationary, then we may hope to be able to extract all the statistical information we need from a single trajectory of the

7 Here is another example, that shows measurements of the three components of magnetic field and corresponding velocity field in the sun (left), and a typical computed spectrum for magnetic field turbulence (right): Ergodic properties of weakly stationary processes If a process is stationary, then we may hope to be able to extract all the statistical information we need from a single trajectory of the process. This would be useful, for example, if we only have access to one trajectory: perhaps it is the temperature fluctuations in New York, the velocity field at a single location in the ocean, etc; or if it is hard to obtain enough representative samples, such as when running a molecular dynamics or monte-carlo simulation, which require a large equilibration time before one can start collecting statistics. However, it is not obvious that the time average of a function of of a single trajectory should equal the average over an ensemble of trajectories, and in fact this is only true when the covariance function decays quickly enough. Theorem (Ergodic Theorem for stationary processes). 4 Let X t be a weakly stationary process with mean µ and covariance C(t), and suppose that C(t) L 1 (, ). Then lim E 1 T T 3 from cairns/teaching/lecture12/node2.html 4 Theorem, Lemma, and proof as quoted in Pavliotis [214] 2 X s ds µ =. (5) 7

8 Notes This says that the time average of X t converges in mean-square to its mean µ. A sequence of m.s. random variables X t converges in mean-square to another random variable Y, written X t Y, if lim t E X t Y 2 =. This theorem generalizes trivially to weakly stationary sequences X,X 1,X 2,... If X t is strongly stationary then the convergence in the Ergodic Theorem is almost surely see Grimmett and Stirzaker [21], section 9.5. For a weaker version of the theorem, see Grimmett and Stirzaker [21], section 9.5. This shows that, with no restrictions on the covariance function, there exists a random variable Y such that X t dt m.s. Y. 1 T To prove this ergodic theorem, we need the following result. Lemma. Let C(t) be an integrable symmetric function. Then C(t s)dtds = 2 (T s)c(s)ds. Proof. Change variables to u = t s, v = t +s. The domain of integration in (u,v) is [ T,T ] [ u,2t u ], and the Jacobian is (t,s) (u,v) = 2 1. The integral becomes 2T u 1 T C(t s)dtds = T u 2 C(u)dvdu = (T u )C(u)du = 2 (T u)c(u)du, T where the last step follows because C(u) is symmetric. Proof of Ergodic theorem. Calculate: E 1 T 2 X s ds µ T = 1 T 2 E 2 (X s µ) ds = 1 T 2 E(X t µ)(x s µ)dtds Fubini s Theorem to interchange E, = 1 T 2 C(t s)dtds = 2 (T u)c(u)du using Lemma 2 T 2 T T 2 which, since C L 1 (, ). ( 1 u ) C(u) du T C(u)du Dominated Convergence Theorem Remark. We see from the proof that we can actually weaken the condition that C L 1 it is enough to know 1 that lim T T T C(u)(1 T u )du =. 8

9 Notes The ergodic theorem shows that if the correlation function decays quickly enough, then time-averaging is like computing an expectation. We can replace one with the other: either we can simulate (or observe) a system for a very long time, or we can average over many short simulations. A similar result holds for the empirical calculation of covariances. Here, we need 4th moments to decay sufficiently rapidly to get convergence to the ensemble covariances. Suppose we want to calculate the average value of a function of a process, E f (X t ). Can we do this by taking the time-average? We can, if we know that Y t = f (X t ) is weakly stationary with a covariance function that decays quickly enough. In general this will not hold if X t is only weakly stationary, but if X t is strongly stationary then Y t will also be strongly stationary, so we can apply the ergodic theorem provided C f (t) = E[( f (X t ) E f (X )( f (X ) E f (X ))] is L 1 -integrable. Notice that the Ergodic Theorem implies the Law of Large Numbers for i.i.d. random variables you can think about why Spectral representation of stationary processes It is actually possible to use the spectral measure obtained from the covariance function, to represent X t in a spectral form. This is possible despite the fact that it has no Fourier transform, as discussed before. The spectral representation of X t is like a random Fourier transform, and is possible because fluctuations in nearby mode numbers nearly cancel, to give a function that is finite in the end. Definition. A complex-valued stochastic process has the form X t = X (r) t + ix (i) t, where X (r) t,x (i) t are realvalued stochastic processes. The covariance function of a mean-zero complex-valued stochastic process X t is C(s,t) = EX s X t, and for a stationary process, it is C(t) = EX s+t X s. Theorem (Spectral Theorem). Given a stationary, mean-zero stochastic process X t with spectral distribution function F(λ), there exists a complex-valued process Z(λ) such that Z has the following properties: X t = e iλt dz(λ). (6) (i) Orthogonal increments: if the intervals [λ 1,λ 2 ], [λ 3,λ 4 ] are disjoint, then E(Z(λ 2 ) Z(λ 1 ))(Z(λ 4 ) Z(λ 3 )) =. (ii) Spectral weight: if λ 1 λ 2, then E Z(λ 2 ) Z(λ 1 ) 2 = F(λ 2 ) F(λ 1 ). Proof. See Grimmett and Stirzaker [21], p , or Yaglom [1962], p Notes The process Z is called the spectral process of X. 9

10 If X t is Gaussian, then Z(λ) is also Gaussian. A discrete-time stationary process X =...,X 1,X,X 1,... also has a spectral representation; the only difference is that the domain of the spectral process can be taken to be ( π,π]. The spectral process is found by Z(λ) = lim T T T e iλt 1 it X t dt, exactly as for the Fourier transform or for characteristic function inversion. The integral in (6) is defined to be lim Λ Λ Λ eiλt dz(λ), with Λ Λ e iλt dz(λ) = lim max k k e iλ k t (Z(λ k ) Z(λ k 1 )), where λ k k = [λ k 1,λ k ]. It can be verified that this definition is consistent; see for e.g. Grimmett and Stirzaker [21], sec Another form of the Spectral Theorem is X t = e iλt Z(dλ), where Z( λ) is a random interval function. This means it associates a random variable to each interval λ. It has the following properties: (i) E Z( λ) = (ii) Z( ) = Z( 1 ) + Z( 2 ) if 1, 2 are disjoint. (iii) E Z( 1 ) Z( 2 ) = if 1, 2 are disjoint. (iv) E Z([λ 1,λ 2 ]) 2 = F = F(λ 2 ) F(λ 1 ) if λ 1 λ 2. This is related to the previous formulation by Z(λ) = Z([,λ]). Example. Suppose the spectral measure is concentrated on a finite set of points λ 1,λ 2,...,λ N, i.e. df(λ) = N i=1 a i δ(λ λ i )dλ, where a i are positive weights. Then dz(λ) will also be concentrated on these points, so we can write dz(λ) = if λ / {λ i }, and dz(λ) = ξ i δ(λ λ i )dλ, i where {ξ i } N i=1 are uncorrelated random variables with E ξ i 2 = a i. 5 This means that Z(λ b ) Z(λ a ) = λi [λ a,λ b ] ξ i. Then, we can write X t = N i=1 ξ i e iλ it, i.e. the process is a random sum of Fourier components. This sum makes perfect sense and converges pointwise, however notice that it doesn t have a Fourier transform in the traditional sense since it is not in 5 Let s check we have all the δ-functions in the right places: suppose df(λ) = a i δ(λ λ i )dλ, dz(λ) = ξ i δ(λ λ i )dλ. Then EdZ(λ)dZ(λ ) = a i δ(λ λ i )δ(λ λ i )dλdλ. We need this to equal df(λ)δ(λ λ )dλ = a i δ(λ λ i )δ(λ λ )dλdλ. But δ(λ λ i )δ(λ λ i )dλdλ = δ(λ λ i )δ(λ λ)dλdλ, so this works. 1

11 L 1, and it may not have a discrete Fourier transform either if the frequencies {λ i } are not rational multiples of some common increment λ. Example (Covariance function). Let s check that the covariance function computed from the spectral representation is consistent: EX s+t X s = e iλ(s+t) e i λs E(dZ(λ)dZ( λ)) = = e is(λ λ) e iλt δ(λ λ)df(λ)d λ e iλt df(λ) = C(t). We used the formal result that E(dZ(λ)dZ( λ)) = δ(λ λ)df(λ)d λ, which is a consequence of the orthogonal increments property. The spectral representation is a very useful way to determine relationships between a process and its derivatives, its integrals, its convolution with various functions, and linear functionals of the process in general. For example, once we know the spectral representation of X t, we can very easily calculate the spectral representation of X t and hence its covariance function. We will explore some of these relationships on the homework. References G. Grimmett and D. Stirzaker. Probability and Random Processes. Oxford University Press, 21. L. B. Koralov and Y. G. Sinai. Theory of Probability and Random Processes. Springer, 21. G. A. Pavliotis. Stochastic Processes and Applications. Springer, 214. A. M. Yaglom. An Introduction to the Theory of Stationary Random Functions. Dover,

PROBABILITY: LIMIT THEOREMS II, SPRING HOMEWORK PROBLEMS

PROBABILITY: LIMIT THEOREMS II, SPRING 218. HOMEWORK PROBLEMS PROF. YURI BAKHTIN Instructions. You are allowed to work on solutions in groups, but you are required to write up solutions on your own. Please