Convergence of Markov Processes. Amanda Turner University of Cambridge

Transcription

1 Convergence of Markov Processes Amanda Turner University of Cambridge 1

2 Contents 1 Introduction 2 2 The Space D E [, The Skorohod Topology Convergence of Probability Measures The Prohorov Metric Examples The Skorohod Representation Convergence of Finite Dimensional Distributions 18 5 Relative Compactness in D E [, Prohorov s Theorem Compact Sets in D E [, Some Useful Criteria A Law of Large umbers Preliminaries The Fluid Limit A Brief Look at the Exit Time A Central Limit Theorem Relative Compactness Convergence of the Finite Dimensional Distributions Applications Epidemics Logistic Growth Conclusion 53 1

3 1 Introduction This essay aims to give an account of the theory and applications of the convergence of stochastic processes, and in particular Markov processes. This is developed as a generalisation of the convergence of real-valued random variables using ideas mainly due to Prohorov and Skorohod. Sections 2 to 5 cover the general theory, which is applied in Sections 6 to 8. For random variables taking values in R, there are a number of types of convergence including almost sure convergence, convergence in probability and convergence in distribution. The first two depend on the random variables being defined on the same probability space and are consequently not sufficiently general. Convergence in distribution is weaker than the other two types of convergence in the sense that if X n X almost surely or in probability, then X n converges to X in distribution. However, if X n converges to X in distribution, then there exists a probability space on which are defined random variables Y n and Y with the same distributions as X n and X such that Y n Y almost surely, and hence in probability. In this way convergence in distribution incorporates the other types of convergence. The notion of convergence for stochastic processes, that is random variables taking values in some space of functions on [,, is even less straightforward. Once again, there is almost sure convergence and convergence in probability, but one may also say that X n converges to X if the finite-dimensional distributions converge, that is if for any choice t 1,..., t k of times, then X n t 1,..., X n t k converges in distribution to X t1,..., X tk. It turns out that a direct generalisation of convergence in distribution of random variables in R to stochastic processes in some sense incorporates all these types of convergence. We restrict our attention to stochastic processes whose sample paths are right continuous functions with left limits at every time point t, also known as cadlag functions. The reason for this is that most processes which arise in applications have this property, and these functions are reasonably well behaved. In order to be able to talk about almost sure convergence and convergence in probability, we need a topology on the space of cadlag functions. It turns out to be extremely difficult to construct a topology with useful properties on this space and Section 2 contains a very technical discussion of the construction and properties of such a topology, the Skorohod topology. For each stochastic process X, there is a unique probability measure on the space of cadlag functions that characterises the distribution of X. As this probability measure is independent of the probability space on which X is defined, it is easier to work with the probability measures to obtain results on convergence. In Section 3 we construct a metric on the space of probability measures that induces a topology equivalent to that generated by convergence in distribution of the related stochastic processes. Using this we establish an equivalence between convergence in distribution, almost sure convergence and convergence in probability. Section 4 looks at the convergence of finite dimensional distributions. This viewpoint is used to establish a key result, due to Prohorov, that states that a sequence of stochastic processes converges if and only if it is relatively compact and the corresponding finite dimensional distributions converge. This gives the required equivalence between convergence in distribution and convergence of finite dimensional distributions. In order to apply the result from Section 4 in any practical situations, we need to have an understanding of what it means for a sequence of stochastic processes to be relatively compact. Prohorov s Theorem, discussed in Section 5, establishes equivalent conditions in terms of the compact sets on the space of cadlag functions. Characterising these compact sets in a way that can be easily applied and putting these results together gives some useful necessary and sufficient conditions for a family of stochastic processes to be relatively compact. The remainder of the essay applies the general theory that has been built up so far to various cases of interest. In Section 6 the Law of Large umbers is generalised by showing that under certain conditions a sequence of Markov jump processes can converge to the solution of an 2

4 ordinary differential equation. The sense in which this is a generalisation of the Law of Large umbers is explained at the beginning of the section. This idea is developed further in Section 7, by showing that the fluctuations about this limit converge in distribution to the solution of a stochastic differential equation, which generalises the central limit theorem. Here the results from Section 4 and the characterisation of relative compactness from Section 5 are applied to prove the convergence in distribution. Finally, the applications of the large number and central limit results to some practical situations are discussed. Particular mention is given to the application of these limit theorems to population processes in biology. The material in Sections 2 to 5 is broadly based on the approach of Ethier and Kurtz [4]. Sections 6 and 7 cover material from the paper of Darling and orris [3], although the application of the theorems from Section 4 and Section 5 is an extension of the results in this paper. 2 The Space D E [, Most stochastic processes arising in applications have right and left limits at each time point for almost all sample paths. By convention we assume that the sample paths are in fact right continuous where this can be done without changing the finite-dimensional distributions. For this reason, the space of right continuous functions with left limits is of great importance and in this section we explore its various properties and define a suitable metric on it. We conclude the section by investigating the Borel σ-algebra that results from this metric. Although in the applications to be discussed, the stochastic processes have sample paths taking values in some subset of R d,, where possible we establish results for processes with sample paths taking values in a general metric space. Throughout this essay we shall denote this metric space by E, r, and define q to be the metric q = r 1. Definition 2.1. D E [, is the space of all right continuous functions x : [, E with left limits i.e. for each t, lim s t xs = xt, and lim s t xs = xt exists. We begin with a result that shows that functions in D E [, are fairly well behaved. Lemma 2.2. If x D E [,, then x has at most countably many points of discontinuity. Proof. The set of discontinuities of x is given by n=1 A n where A n = {t > : rxt, xt > 1 n }, so it is enough to show that each A n is countable. Suppose we have distinct points t 1, t 2,... A n with t m t for some t, as m. By restricting to a subsequence if necessary, we may assume that either t m t or t m t. Then lim m xt m = xt = lim m xt m or lim m xt m = xt = lim m xt m and so rxt m, xt m < 1 n for large enough m, contradicting x m A n. Therefore A n cannot contain any limiting sequences. But for each T >, every sequence in the interval [, T] has a convergent subsequence and so there are only a finite number of points of A n in [, T]. Hence A n is countable, as required. 2.1 The Skorohod Topology The results on convergence of probability measures that we shall prove in subsequent sections are most applicable to complete separable metric spaces. For this reason we define a metric on D E [, under which it is separable and complete if E, r is separable and complete. In particular D R d[, will be separable and complete. Definition 2.3. Let Λ be the collection of strictly increasing functions λ mapping [, onto [, in particular, λ =, lim t λt =, and λ is continuous Let Λ be the set of 3

5 Lipschitz continuous functions λ Λ such that γλ = λt λs log t s < s<t For x, y D E [,, define [ dx, y = inf γλ λ Λ where ] e u dx, y, λ, udu dx, y, λ, u = qxt u, yλt u. t The Skorohod Topology is the topology induced on D E [, by the metric d. Proposition 2.4. The function d, defined above, is a metric on D E [,. Proof. Suppose x n n 1, y n n 1 are sequences in D E [,. Then lim dx n, y n = if and only if there exists a sequence λ n n 1 in Λ such that and lim γλ n = 2.1 lim µ{u [, u ] : dx n, y n, λ n, u ε} = 2.2 for every ε > and u >, where µ is Lebesgue measure. ow for all T >, T e γλ 1 = T = T s<t e s<t log λt λs t s 1 T <tt = T <tt λt λs e log t s 1 λt log e t 1 { λt t max, t max {λt t, t λt}, <tt } t λt λt where the first inequality follows from setting s = and bounding t by T and the next line follows by considering the cases log λt t and < separately. This gives us tt λt t T e γλ 1, 2.3 by which 2.1 implies for all T >. lim tt λ n t t = 2.4 ow pose dx, y =. Then setting x n = x and y n = y for all n, 2.2 and 2.4 imply that xt = yt for almost all continuity points t of y. But by Lemma 2.2, y has at most countably many points of discontinuity and so xt = yt for almost all points t and, as x and y are right continuous, x = y. Let x, y D E [,. Then since λ is bijective on [, t qxt u, yλt u = qxλ 1 t u, yt u t 4

6 for all λ Λ and u, and so dx, y, λ, u = dy, x, λ 1, u. Also γλ = λt λs log s<t t s = log λλ 1 t λλ 1 s s<t λ 1 t λ 1 s = log t s s<t λ 1 t λ 1 s = log λ 1 t λ 1 s t s s<t = γλ 1 for every λ Λ and so dx, y = dy, x. To show that d is a metric it only remains to check the triangle inequality. Let x, y, z D E [,, λ 1, λ 2 Λ, and u. Then qxt u, zλ 2 λ 1 t u t qxt u, yλ 1 t u t + qyλ 1 t u, zλ 2 λ 1 t u t = qxt u, yλ 1 t u t + qyt u, zλ 2 t u, t i.e. dx, z, λ 2 λ 1, u dx, y, λ 1, u + dy, z, λ 2, u. But since λ 2 λ 1 Λ and γλ 2 λ 1 = log λ 2λ 1 t λ 2 λ 1 s s<t t s = log λ 2λ 1 t λ 2 λ 1 s + log λ 1t λ 1 s s<t λ 1 t λ 1 s t s log λ 2λ 1 t λ 2 λ 1 s λ 1 t λ 1 s + log λ 1t λ 1 s t s s<t = γλ 2 + γλ 1, we obtain dx, z dx, y + dy, z as required. <st It is not very clear from the definition of the Skorohod topology under which conditions sequences in D E [, converge. The following two propositions establish some necessary and sufficient conditions for convergence in D E [, which are slightly easier to get an intuitive grasp of. Proposition 2.5. Let x n n 1 and x be in D E [,. Then lim dx n, x = if and only if there exists a sequence λ n n 1 in Λ such that 2.1 holds and lim dx n, x, λ n, u = for all continuity points u of x. 2.5 In particular, lim dx n, x = implies that lim x n u = lim x n u = xu for all continuity points u of x. Proof. By Lemma 2.2, x has only countably many discontinuity points and so, by the Reverse 5

7 Fatou Lemma and 2.5, lim µ{u [, u ] : dx n, x, λ n, u ε} = lim 1 {u [,u]:dx n,x,λ n,u ε}dµ 1 {u [,u]:lim dx n,x,λ n,u ε}dµ µ{u [, u ] : u is a discontinuity point of x} =, i.e 2.2 holds and so the conditions are sufficient. Conversely, pose that lim dx n, x = and that u is a continuity point of x. Then there exists a sequence λ n n 1 in Λ such that 2.1 holds, and 2.2 holds with y n = x for all n. In particular, there exists an increasing sequence m m 1 such that for all n m { µ v u, u + 1] : dx n, x, λ n, v < 1 } >. 2.6 m Hence, for each m n < m+1, there exists a u n u, u + 1] such that dx n, x, λ n, u n < 1 m. By picking arbitrary values of u n u, u + 1] for n < 1, we obtain ow lim qx n t u n, xλ n t u n = lim dx n, x, λ n, u n =. 2.7 t dx n, x, λ n, u = qx n t u, xλ n t u t qx n t u, xλ n t u u n t + qxλ n t u u n, xλ n t u t But qxλ n t u u n, xλ n t u = t qxλ n t u u n, xλ n t u tu qxλ n t u u n, xλ n t u t>u = qxλ n t u n, xλ n t u tu qxλ n u u n, xλ n t u t>u = usλ nu u qxs, xu λ nu u<su qxλ n u u n, xs where the third equality is obtained by setting s = λ n t u n in the first term and s = λ n t u in the second term. Hence dx n, x, λ n, u qx n t u n, xλ n t u n tu + usλ nu u qxs, xu λ nu u<su qxλ n u u n, xs 2.8 for each n. Thus lim dx n, x, λ n, u = by 2.7, 2.4, and the continuity of x at u. 6

8 Proposition 2.6. Let x n n 1 and x be in D E [,. Then lim dx n, x = if and only if there exists a sequence λ n n 1 in Λ such that 2.1 holds and for all T >. lim tt rx n t, xλ n t = 2.9 Remark 2.7. The above proposition is equivalent to one with 2.9 replaced by lim tt rx n λ n t, xt = 2.1 Proof. Suppose lim dx n, x =. Then there exists a sequence λ n n 1 in Λ such that 2.1 holds and 2.2 holds with y n = x for all n. In particular, by 2.6 with u = m, there exists a sequence u n n 1, with u n, and dx n, x, λ n, u n i.e. lim rx n t u n, xλ n t u n =. t But given T >, u n T λ n T for sufficiently large n using 2.4 and so the above equation implies 2.9. Conversely, pose there exists a sequence λ n n 1 in Λ such that 2.1 and 2.9 hold. Then for every continuity point u of x, qx n t u n, xλ n t u n = qx n t, xλ n t rx n t, xλ n t tu tu tu for all n large enough that u n > λ n u u. And so, by 2.8, and the right continuity of x 2.5 holds. The result follows by Proposition 2.5. Two simple examples of sequences that converge in D E [,, d may give some insight into why the metric d is defined as above: Example 2.8. Suppose x n n 1 is a sequence in D E [,, d such that x n x locally uniformly for some x D E [,, d. Then lim tt rx n t, xt = and so, taking λ n t = t for all n in Proposition 2.6, x n x with respect to d. Hence the Skorohod topology is weaker than the locally uniform topology. Example 2.9. Define a sequence x n n 1 in D E [,, d, by x n s = α n 1 {tns}, where α n E and t n [,. Suppose that t n t and α n α for some t [, and α E. Intuitively one would expect x n x where xs = α1 {ts}. However, in the locally uniform topology this is not always the case; for example if t t n for all n and α, then x n t = for all n, whereas xt = α. In other words, the locally uniform topology is too strong. The functions λ n are introduced to allow small perturbations around the points of discontinuity of x. In this example, if t, then t n > for sufficiently large n and so we may set λ n s = t t n s. Then log t lim γλ n = lim t n = and rx n s, xλ n s = rα n 1 tns, α1 tns. st st By Proposition 2.6, x n x, as intuitively expected. We are now ready to prove that D E [,, d has the required properties. ote that while separability is a topological property, completeness is a property of the metric. Theorem 2.1. If E is separable, then D E [, is separable. If the metric space E, r is complete, then D E [,, d is complete. 7

9 Proof. Since E is separable, there exists a countable dense subset {α 1, α 2,...} E. Let Γ be the countable collection of elements of D E [, of the form { α ik, t k 1 t < t k, k = 1,...,n, yt = 2.11 α in, t t n, where = t < t 1 < < t n are rationals, i 1,...i n are positive integers and n 1. We shall show that Γ is dense in D E [,. Given ε >, let x D E [, and let T be sufficiently large that e T < ε 2. By Lemma 2.2, x has only finitely many points of discontinuity in the interval, T, at s 1,...s m say, where = s < s 1 < < s m < s m+1 = T. Since x has left limits, for each j =,...,m, x is uniformly continuous on the interval [s j, s j+1 and so there exists some < δ j < s j+1 s j such that if s, t [s j, s j+1, and s t < δ j, then xs xt < ε 4. 3T Let n δ δ m and let t k = kt n for k =,...,n. For each k =,...,n 1, there exists some positive integer i k such that α ik xt k < ε 4. Define y as in For each j = 1,...m with ns j /, there exists some k j such that s j t kj, t kj+1. Let t k j = sj e ε t kj +1 1 e ε t kj 1, s j, and t k j+1 = sj eε t kj +1 1 e t kj+1, t kj+2. ote that, by the definition of n, k ε j k i 3 for all i j and so the t k j are strictly increasing. Let λ Λ be the strictly increasing piecewise linear function joining t k j, t k j to t kj+1, s j to t k, j+1 t k for those j = 1,...m, for which ns j+1 j /, with gradient 1 otherwise. Then γλ min j t k log s j+1 j t k t j+1 k j+1 log s j t k j t kj+1 t kj = ε and, if t < T, then rxt, yλt = rxt, xt k + rxt k, α ik < ε 2, for some k with t t k < δ. Hence, by the definition of d, and so Γ is dense in D E [,. dx, y ε ε 2 + e T ε, To prove completeness, pose that x n n 1 is a Cauchy sequence in D E [,. There exist 1 1 < 2 < such that m, n k implies dx n, x m 2 k 1 e k. Then if y k = x k for k = 1, 2,..., there exist u k > k and λ k Λ such that γλ k dy k, y k+1, λ k, u k 2 k. Let µ n,k = λ k+n λ k. Then γµ n,k k+n j=k γλ j 2 k+1. By a similar argument to that used to prove 2.3, for any µ, λ Λ, e γµ t λt µλt µt λt t, and so Hence, if n m, then µλt t λt t e γµ. tt tt µ n,k µ m,k µ n m 1,k+m+1 t t e γµ m,k tt tt Te γµ n m 1,k+m+1 1e 2 k+1 Te 2 k m 1e 2 k+1 8

10 as m, where the second inequality follows by 2.3. Hence, µ n,k converges uniformly on compact intervals to a strictly increasing, Lipschitz continuous function µ k, with γµ k 2 k+1, i.e. µ k Λ. ow qy k µ 1 k t u k, y k+1 µ 1 t k+1 t u k = t qy k µ 1 k t u k, y k+1 λ k µ 1 k t u k = qy k t u k, y k+1 λ k t u k t = dy k, y k+1, λ k, u k for k = 1, 2,.... Since E, r is complete, z k = y k µ 1 k D E [,, converges uniformly on bounded intervals to a function y : [, E. As each z k D E [,, y D E [, taking locally uniform limits preserves right continuity and the existence of left limits. ow lim k γµ 1 k = lim k γµ k = and lim k tt 2 k ry k µ 1 k t, yt = lim k tt rz k t, yt =, for all T > and so, by Proposition 2.6 using Remark 2.7, lim k dy k, y =. Hence D E [,, d is complete. In order to study Borel probability measures on D E [, it is important to know more about S E, the Borel σ-algebra of D E [,. The following result states that S E is just the σ-algebra generated by the coordinate random variables. Proposition For each t, define π t : D E [, E by π t x = xt. Then If E is separable, then S E = S E. S E S E = σπ t : t <. Proof. For each ε >, t, and f CE, the space of real valued bounded continuous functions, define f ε t x = 1 ε t+ε t fπ s xds. ow pose x n n 1 is a sequence in D E [, converging to x. Then there exists a sequence λ n n 1 in Λ such that 2.1 and 2.1 hold. Then ft ε x n = 1 fx n λ n s1 {λntsλ ε nt+ε}λ nsds 1 fxs1 {tst+ε} ds ε = f ε t x, by dominated convergence, since x n λ n s xs uniformly on bounded intervals, f is bounded and continuous, and λ n s s uniformly on bounded intervals, implying λ ns 1 almost everywhere, uniformly on bounded intervals. Hence f ε t is a continuous function on D E [, and so is Borel measurable. As lim ε f ε t x = fπ tx for every x D E [,, f π t is Borel measurable for every f CE and hence every bounded measurable function f. In particular, πt 1 Γ = {x D E [, : 1 Γ π t x = 1} S E for all Borel subsets Γ E, and hence S E S E. Assume now that E is separable. Let n 1, let = t < t 1 < < t n < t n+1 =, and for α, α 1,..., α n E define ηα, α 1,..., α n D E [, by ηα, α 1,..., α n t = α i, t i t < t i+1, i =, 1,..., n. 9

11 ow dηα, α 1,..., α n, ηα, α 1,..., α n max in rα i, α i, and so η is a continuous function from E n+1 into D E [,. Since E is separable, E n+1 is separable and so there exists a countable dense subset A E n+1. For fixed z D E [, and ε >, by the continuity of η, Γ = {a E n+1 : dz, ηa < ε} = {a A:dz,ηa<ε} n=1 B a, 1, n is a measurable subset of E n+1 with respect to the Borel product σ-algebra and so, since each π t is S E measurable, dz, η π t, π t1,..., π tn is an S E -measurable function from D E[, into R. For m = 1, 2,..., define η m as η but with n = m 2 and t i = i m, i =, 1,.... By an identical argument to that in the proof of the separability of D E [, in Theorem 2.1, dx, η m π t x, π t1 x,... π tn x as m and hence lim dz, η mπ t x, π t1 x,... π tn x dz, x lim dx, η mπ t x, π t1 x,... π tn x m m = for every x D E [,. Therefore dz, x = lim m dz, η m π t x, π t1 x,... π tn x is S E - measurable in x for fixed z D E [, and in particular, every open ball Bz, ε = {x D E [, : dz, x < ε} belongs to S E. Since E and, by Theorem 2.1, D E[, is separable, S E contains all the open sets in D E[, and hence contains S E. 3 Convergence of Probability Measures I order to study the convergence of the distributions of stochastic processes, it is necessary to understand the probability measures that characterise these. In this section we construct a metric on the space of probability measures corresponding to the convergence, in distribution, of the stochastic processes. Using this, we establish a relationship between convergence in distribution and convergence in probability of processes defined on a common probability space. This result is applied to sequences of Markov chains and Diffusion processes to obtain some simple conditions for convergence. Where possible, results are proved for probability measures on a general metric space S, d. However, in practice, we generally take S = D E [,, and in particular D R d[,, with the metric d defined in the previous section. otation 3.1. For a metric space S, d BS is the σ-algebra of Borel subsets of S PS is the family of Borel probability measures on S CS is the space of real-valued bounded continuous functions on S, d with norm f = x S fx C is the collection of closed subsets of S F ε = {x S : inf y F dx, y < ε} where F S Definition 3.2. A sequence P n n 1 in PS is said to converge weakly to P PS denoted P n P if lim fdp n = fdp for all f CS. The distribution of an S-valued random variable X, denoted by PX 1, is the element of PS given by PX 1 B = PX B where P is the probability measure on the probability space underlying X. 1

12 A sequence X n n 1 of S-valued random variables is said to converge in distribution to the S-valued random variable X if PXn 1 PX 1, or equivalently, if lim EfX n = EfX for all f CS. This is denoted by X n X. Remark 3.3. ote that this is a direct generalisation of the definition of convergence in distribution of a sequence of real-valued random variables X n n 1, where we say that X n X if lim EfX n = EfX for all f CR. 3.1 The Prohorov Metric We now define a metric ρ on PS with the property that a sequence of probability measures converges with respect to ρ if and only if it converges weakly. Definition 3.4. For P and Q PS the Prohorov metric is defined by using the notation defined in 3.1. ρp, Q = inf{ε > : PF QF ε + ε for all F C}, In order to prove that ρ is a metric, the following lemma is needed. Lemma 3.5. Let P, Q PS and α, β >. If PF QF α + β 3.1 for all F C, then for all F C. QF PF α + β 3.2 Proof. Suppose F C. F α is open since if x F α, then there exists y F such that dx, y < α. Then dz, y dz, x + dx, y < α for all z Bx, α dx, y and so Bx, α dx, y F α. Hence, if G = S \ F α, then G C. F S \ G α since if x G α, then there exists some y / F α such that dx, y < α. Then dy, z α for all z F and so dx, z dy, z dx, y > for all z F i.e. x / F. Substituting G into 3.1 gives PF α = 1 PG 1 QG α β QF β. Proposition 3.6. The function ρ, defined above, is a metric on PS. Proof. By the above lemma, ρp, Q = inf{ε > : PF QF ε + ε for all F C} = inf{ε > : QF PF ε + ε for all F C} = ρq, P. If ρp, Q =, then there exists a sequence ε n n 1 with ε n such that PF QF εn +ε n for all n. Letting n and using the continuity of probability measures, gives PF QF for all F C. By the above symmetry between P and Q, PF = QF for all F C and hence for all F BS. Therefore ρp, Q = if and only if P = Q. 11

13 Finally, if P, Q, R PS, and δ >, ε > satisfy ρp, Q < δ and ρq, R < ε, then PF QF δ + δ QF δ + δ RF δ ε + δ + ε RF δ+ε + δ + ε for all F C, so ρp, R δ + ε and hence ρp, R inf δ>ρp,q δ + inf ε>ρq,r ε = ρp, Q + ρq, R as required. Theorem 3.7. Let P n n 1 be a sequence in PS and P PS. If S is separable, then lim ρp n, P = if and only if P n P. Proof. Suppose lim ρp n, P =. For each, let ε n = ρp n, P+ 1 n. Given f CS with f, f f fdp n = P n f tdt P{f t} εn dt + ε n f for every n and so, by dominated convergence, Hence, for all f CS, Therefore lim lim lim fdp n lim = = f f fdp. P{f t} εn dt Pf tdt f + fdp n f + fdp, f fdp n f fdp. fdp f lim f fdp n = lim inf fdp n lim fdp n = lim f + fdp n f fdp. and so we must have equality throughout. Thus lim fdpn = fdp for all f CS i.e. P n P. Conversely, pose P n P. We first establish some preliminary results for open and closed subsets of S. Let F C and for each ε >, define f ε CS by dx, F f ε x = 1, ε and 12

14 where dx, F = inf y F dx, y. Then f ε 1 F for all ε > and so lim P n F lim f ε dp n = f ε dp, for each ε >. Therefore, by dominated convergence, lim P n F lim f ε dp = PF. ε If G S is open, then lim inf P ng = 1 lim P n G c 1 PG c = PG. ow let ε >. Since S is separable note: this is the only point in the proof where we use separability, there exists a countable dense subset {x 1, x 2,...} S. Let E i = Bx i, ε 4 for i = 1, 2,.... Then P i=1 E i = PS = 1 and so there exists some smallest integer such that P i=1 E i > 1 ε 2. ow let G be the collection of open sets of the form i I E ε 2 i, where I {1,...,}. Since G is finite, by the above result on open sets, there exists some such that PG P n G + ε 2 for all G G and n. Given F C, let F = {E i : 1 i, E i F }. Then F ε 2 G and so PF PF ε 2 + P i=+1 E i PF ε 2 + ε 2 P n F ε 2 + ε P n F ε + ε for all n, where the first inequality is by F F i=+1 E i, the second by the definition of, the third by the definition of and the fourth by the diameter of the E i being ε 2. Hence ρp n, P ε for each n, i.e. lim ρp n, P =. Definition 3.8. Let P, Q PS. Define MP, Q to be the set of all µ PS S with marginals P and Q i.e. µa S = PA and µs A = QA for all A BS. The following lemma provides a probabilistic interpretation of the Prohorov metric: Lemma 3.9. ρp, Q inf inf {ε > : µx, y : dx, y ε ε}. µ MP,Q Proof. If for some ε > and µ MP, Q we have µx, y : dx, y ε ε, then PF = µf S µ F S {x, y : dx, y < ε} + µx, y : dx, y ε µs F ε + ε = QF ε + ε, for all F C and so ρp, Q ε. The result follows. 13

15 In fact, in the case when S is separable, the inequality in the above lemma can be replaced by an equality. The proof is an immediate consequence of Lemma Proposition 3.1. Let S, d be separable. Suppose that X n, n = 1, 2,..., and X are S-valued random variables defined on the same probability space with distributions P n, n = 1, 2,..., and P respectively. If dx n, X in probability as n, then P n P. Proof. For n = 1, 2,..., let µ n be the joint distribution of X n and X. Then lim µ nx, y : dx, y ε = =, lim PdX n, X ε where P is the probability measure on the probability space underlying X n and X. By Lemma 3.9, lim ρx n, X =, and since S is separable, the result follows by Theorem Examples Proposition 3.1 suggests a method of proving that a sequence of probability measures converges weakly by constructing random variables with the required distributions on a common probability space and showing that they converge in probability. We illustrate this method by looking at three examples: discrete time Markov chains, continuous time Markov chains and diffusion processes Discrete Time Markov Chains Suppose that E is countable and that X 1 and X are discrete time Markov Chains with initial distributions λ 1 and λ, and transition matrices P 1 and P respectively. We will show that if P P and λ λ uniformly, then X X. We construct random variables Y 1 and Y with the required distributions on the probability space [, 1, B, µ where B is the Borel σ-algebra and µ is Lebesgue measure. Without loss of generality we may assume E = set the relevant probabilities to zero if E is finite. For each ω [, 1, construct a sequence aω = i n n of elements of inductively as follows: Since i= λ i = 1, there exists a smallest i such that ω > i i= λ i. Set aω = i, and let µ i = i 1 i= λ i. Since µ i ω < µ i + λ i, and i= p i i = 1, there exists some smallest i 1 such that i1 ω < µ i + λ i i= p i1 1 i i. Set aω 1 = i 1, and let µ i,i 1 = µ i + λ i i= p i i. Suppose we have constructed aω m = i m and µ i,...,i m for all m < n, such that µ i,...,i m ω < µ i,...,i m + λ i p i i 1 p im 1 i m. Since i= p i n 1 i = 1, there exists some smallest i n such that ω < µ i,...,i n 1 + λ i p i i 1 p in in 2 i n 1 i= p in 1 i. Set aω n = i n, and let µ i,...,i n = µ i,...,i n 1 + λ i p i i 1 p in 1 in 2 i n 1 i= p i n 1 i. Define a discrete time process Y n n on [, 1, B, µ by setting Y n ω = aω n. Then P Y = i, Y 1 = i 1,...,Y n = i n = µω : µ i,...,i n ω < µ i,...,i n + λ i p i i 1 p in 1 i n = λ i p i i 1 p in 1 i n, and so Y n n is a Markov chain with initial distribution λ and transition matrix P. For each, construct Y n n with initial distribution λ and transition matrix P similarly. µ i,...,i n is a continuous function in a finite number of the entries of λ and P, and λ λ and P P, uniformly in. Hence µ i,...,i n µ i,...,i n. Therefore, if µ i,...,i n < ω < µ i,...,i n+1, then there exists some such that implies that µ i,...,i n < ω < µ i,...,i n+1 14

16 and hence Y m ω = Y mω for all m n. In other words, provided ω µ i,...,i n for any i,..., i n, Y ω Y ω. But only a countable number of elements of [, 1 are equal to µ i,...,i n for some i,...,i n and so lim Y = Y almost surely. By Proposition 3.1, X X as required Continuous Time Markov Chains Suppose that E is countable and that X 1 and X are continuous time Markov Chains with initial distributions λ 1 and λ, and generator matrices Q 1 and Q respectively. We will show that if Q Q and λ λ uniformly, then X X. We construct random variables Z 1 and Z with the required distributions on a common probability space Ω, F, P, using a construction due to orris [9]. As in the discrete case we shall assume that E =. Let Π 1 and Π be the jump matrices corresponding to the generator matrices Q 1 and Q respectively. Since Q Q uniformly, the corresponding jump matrices Π Π uniformly and, by the discrete case above, there exist discrete time Markov chains Y 1 and Y with initial distributions λ 1 and λ, and transition matrices Π 1 and Π respectively, such that lim Y = Y almost surely. By discarding a set of measure zero if necessary, we may assume that Y ω Y ω for all ω Ω. Let T 1, T 2,... be independent exponential random variables of parameter 1, independent of Y 1 and Y. Defining qi = q ii, set S n = Tn qy, J n 1 n = S S n, and { Y n if J n t < J n+1 for some n Z t = otherwise. Then the S n are independent exponential random variables with parameters qy n and so Z has the required distribution. Define Sn, J n and Z similarly for 1. Since Y ω Y ω for all ω Ω, given ω Ω, for each n, there exists some n such that n implies that Ym ω = Y mω for all m n. Then since Q Q, if n, then Sm ω = T mω q Y m 1ω T mω q Y m 1 ω = Tmω qy m 1ω = S mω for all m n + 1 and hence J m J m for all m n + 1. By the same argument as that used to prove that D E [, is separable in Theorem 2.1, it follows that dz ω, Zω as, and so lim Z = Z almost surely. By Proposition 3.1, X X, as required Diffusion Processes Suppose that a n n 1 and a are bounded symmetric uniformly positive definite Lipschitz functions R d R d R d, and that b n n 1 and b are bounded Lipschitz functions R d R d. Let X n n 1 and X be diffusion processes in R d with diffusivities a n n 1 and a, and drifts b n n 1 and b respectively, starting from x R d. We shall show that if a n a and b n b uniformly, then X n X. Let B t t be a Brownian Motion in R d. We shall construct diffusions Z n n 1 and Z, with the required distributions, on the probability space underlying B t t. Since ax is symmetric positive definite for all x R d, there exists a unique symmetric positive definite map σ : R d R d R d such that σxσx = ax. Furthermore, since a is bounded and Lipschitz, σ is bounded and Lipschitz. Similarly, there exist unique symmetric positive definite bounded Lipschitz maps σ n n 1 and, since a n a uniformly, σ n σ uniformly. Assume that σ, σ n, b and b n have Lipschitz constant K, independent of n. Since σ, σ n, b and b n are Lipschitz, there exist continuous processes Z and Z n for n = 1, 2,..., adapted to the filtration generated by B t t satisfying dz t = σz t db t + bz t dt, Z = x, dz n t = σ n Z n t db t + b n Z n t dt, Zn = x. 15

17 Z n and Z have the required distributions. Using x + y 2 = 2x 2 + 2y 2, Zt n Z t 2 = 2 t t 2 σ n Zs n σz s db s + b n Zs n bz s ds 2 t 2 σ n Zs n σz s db s + 2 b n Zs n bz s ds. t By Doob s L 2 inequality, t 2 t E σ n Zs n σz sdb s 4E st and by the Cauchy-Schwartz inequality, t E b n Zs n bz 2 t sds te st ow since K is a Lipschitz constant for σ n, σ n Zs n σz s 2 ds, b n Zs n bz s 2 ds. σ n Z n r σz r σ n Z n r σ n Z r + σ n Z r σz r K Z n r Z r + σ n σ, and similarly Hence E Zs n Z s 2 st b n Z n r bz r K Z n r Z r + b n b. t 8E t σ n Zs n σz s 2 ds + 2tE t 16E K 2 Zs n Z s 2 + σ n σ 2 ds t + 4tE K 2 Zs n Z s 2 + b n b 2 ds 16t σ n σ 2 + 4t 2 b n b 2 t tK 2 E Zr n Z r 2 ds. rs b n Zs n bz s 2 ds Given ε > and T >, set c = TK 2 and ε = εe ct. Since σ n σ and b n b ε uniformly, there exists such that n implies σ n σ < 32T and b ε n b < 8T. 2 If f n t = E st Zs n Z s 2, then f n t ε + c t f n sds, for all n and t T. By Gronwall s Inequality Lemma 6.9, f n t ε e ct = ε and so E st Z n s Z s 2 as n. In particular, st Z n s Z s in probability. Taking λ n t = t for all t, Proposition 2.6 implies that dz n, Z in probability. By Proposition 3.1, X X, as required. 3.3 The Skorohod Representation A converse to Proposition 3.1 exists, in the form of the Skorohod Representation. Before we can prove this, we need the following lemma. 16

18 Lemma Let S be separable. Let P, Q PS, ε > satisfy ρp, Q < ε, and δ >. Suppose that E 1,...,E BS are disjoint with diameters less than δ and that PE δ, where E = S \ i=1 E i. Then there exist constants c 1,... c [, 1] and independent random variables X, Y,..., Y S-valued and ξ [, 1]-valued on some probability space Ω, F, P such that X has distribution P, ξ is uniformly distributed on [, 1], { Y i on {X E i, ξ c i }, i = 1,...,, Y = Y on {X E } i=1 {X E i, ξ < c i } has distribution Q, {dx, Y δ + ε} {X E } { { }} ε ξ < max PE i : PE i >, and PdX, Y δ + ε δ + ε. Proof. This lemma is not proved here as the proof is long and not very illuminating. The interested reader is referred to pp97-11 of Ethier and Kurtz [4]. Theorem 3.12 The Skorohod Representation. Let S, d be separable. Suppose P n, n = 1, 2,... and P in PS satisfy P n P. Then there exists a probability space Ω, F, P on which are defined S-valued random variables X n, n = 1, 2,... and X with distributions P n, n = 1, 2,..., and P respectively, such that lim X n = X almost surely. Proof. Let {x 1, x 2,...} be a dense subset of S. Then P i=1 B x i, 2 k = 1 for each k and so, given ε >, there exist integers 1, 2,... such that k P B x i, 2 k 1 2 k for k = 1, 2,.... Set E k i i=1 generality that ε k = min 1ik PE k i X n = = Bx i, 2 k and E k = S \ k i=1 Ek i. Assume without loss of >. Define the sequence k n n 1 by { k n = 1 max k 1 : ρp n, P < ε } k. k Apply Lemma 3.11 with Q = P n, ε = ε kn k n if k n > 1 and ε = ρp n, P + 1 n if k n = 1, δ = 2 kn, E i = E kn i, and = kn for n = 1, 2,.... Then there exists a probability space Ω, F, P on which are defined S-valued random variables Y n,...,y n kn, n = 1, 2,..., a random variable ξ, uniformly distributed on [, 1], and an S-valued random variable X with distribution P, all of which are independent, such that if the constants c n 1,..., cn kn [, 1], n = 1, 2,... are appropriately chosen, then the random variable { Y n i on {X E kn i, ξ c n i }, i = 1,..., kn, Y n on {X E kn } kn i=1 {X Ekn i, ξ < c n i } has distribution P n and { dx n, X 2 kn + ε } { k n {X E kn } ξ < 1 } k n k n if k n > 1, for n = 1, 2,

19 Since P n P, by Theorem 3.7 ρp n, P. Hence, for each k, ρp n, P < ε k k for sufficiently large n, and so k n k for sufficiently large n. If K n = min m n k m, then lim K n =. However, for K n > 1, P m=n { dx m, X 2 km + ε } k m k m So lim X n = X almost surely. PX E k + P ξ < 1 K n k=k n 2 Kn K n. We conclude this section by giving an application of this theorem. Corollary 3.13 The Continuous Mapping Theorem. Let S, d and S, d be separable metric spaces and let h : S S be Borel measurable. Suppose that P n, n = 1, 2,... and P in PS satisfy P n P, and define Q n, n = 1, 2,... and Q in PS by Q n = P n h 1, Q = Ph 1. By definition, Ph 1 B = Ps S : hs B. Let C h be the set of points of S at which h is continuous. If PC h = 1, then Q n Q on S. Proof. By Theorem 3.12, there exists a probability space Ω, F, P on which are defined S- valued random variables X n, n = 1, 2,..., and X with distributions P n, n = 1, 2,..., and P respectively, such that lim X n = X almost surely. Since PX C h = PC h = 1, we have lim hx n = hx almost surely, and so, by Proposition 3.1, Q n Q in S. 4 Convergence of Finite Dimensional Distributions We now get to a key result, due to Prohorov, on characterizing convergent processes. This states that a sequence of stochastic processes converges if and only if it is relatively compact and the finite dimensional distributions converge. This is a particularly useful method for checking the convergence of processes where the limit has independent increments, as in this case computing the finite dimensional distributions is relatively straightforward. See Example 4.4 at the end of the section for an illustration. Definition 4.1. Let {X α } where α ranges over some index set be a family of stochastic processes with sample paths in D E [,, and let {P α } PD E [, be the family of associated probability distributions i.e. P α B = PX α B, where P is the probability measure on the probability space underlying X α, for all B S E, S E being the Borel σ-algebra of D E [,. We say that {X α } is relatively compact if {P α } is i.e. if the closure of {P α } in PD E [, is compact. Lemma 4.2. If X is a process with sample paths in D E [,, then the complement in [, of DX = {t : PXt = Xt = 1} is at most countable. Proof. For each, ε >, δ >, and T >, let Aε, δ, T = { t T : P rxt, Xt ε δ}. 18

20 ow if Aε, δ, T contains a sequence t n n 1 of distinct points, then P rxt n, Xt n ε infinitely often = P {rxt m, Xt m ε} = lim P n=1 m=n {rxt m, Xt m ε} m=n lim P rx nt, Xt n ε δ >. contradicting the fact that for each x D E [,, rxt, xt ε for at most finitely many t [, T] see the proof of Lemma 2.2. Hence Aε, δ, T is finite and so DX c = {t : PrXt, Xt > > } = is at most countable. n=1 n=1 =1 1 A n, 1 m,, Theorem 4.3. Let E be separable and let X n, n = 1, 2,..., and X be processes with sample paths in D E [,. a If X n X, then X n t 1,...,X n t k Xt 1,..., Xt k 4.1 for every finite set {t 1,..., t k } DX. Moreover, for each finite set {t 1,...,t k } [,, there exist sequences t n 1 n 1 in [t 1,,..., t n k n 1 in [t k, converging to t 1,...,t k, respectively, such that X n t n 1,..., X n t n k Xt 1,..., Xt k. b If {X n } is relatively compact and there exists a dense set D [, such that 4.1 holds for every finite set {t 1,..., t k } D, then X n X. Proof. a Suppose that X n X. Using the Skorohod Representation Theorem 3.12, there exists a probability space on which are defined processes Y n, n = 1, 2,..., and Y with sample paths in D E [, and with the same distributions as X n, n = 1, 2,..., and X, such that dy n, Y = almost surely. If {t 1,...,t k } DX = DY, then using the notation of Proposition 2.11, π t1,..., π tk : D E [, E k is continuous almost surely with respect to the distribution of Y and so, by the Continuous Mapping Theorem Corollary 3.13, lim Y nt 1,..., Y n t k = lim π t 1,..., π tk Y n The first conclusion follows by Proposition 3.1. = π t1,...,π tk Y = Y t 1,..., Y t k almost surely. For the second conclusion, we observe that, by Lemma 4.2, for each finite set {t 1,... t k } [,, there exist sequences t n 1 n 1 in [t 1, DX,..., t n k n 1 in [t k, DX converging to t 1,..., t k, respectively. Then, by the above result, X n t m 1,..., X n t m k Xt m 1,..., Xtm k for each m as n. Since the process X is right continuous, Xt m 1,..., Xtm k Xt 1,..., Xt k almost surely as m and so X n t n 1,..., X n t n k Xt 1,..., Xt k. b Since {X n } is relatively compact, the closure of {P n } is compact and hence every subsequence of {P n } has a convergent subsequence. It follows that every subsequence of {X n } has a convergent in distribution subsequence and so it is enough to show that every convergent subsequence of {X n } converges in distribution to X. Restricting to a subsequence if necessary, pose that X n Y. We must show that X and Y have the same distribution. 19

21 Let {t 1,...,t k } DY and f 1,...f k CE, and choose sequences t n 1 n 1 in D [t 1,,..., t n k n 1 in D [t k, converging to t 1,..., t k, respectively. The map x 1,...,x k k i=1 f ix i CE k k and so 4.1 implies E i=1 f ix n t m i k E i=1 f ixt m i as n for each m 1. Therefore there exist integers n 1 < n 2 < n 3 <... such that k E k f i Xt m i E f i X nm t m i < 1 m. 4.2 i=1 ow k E k k f i Xt i E f i Y t i E k f i Xt i E f i Xt m i i=1 i= i=1 i=1 i=1 k E k f i Xt m i E f i X nm t m i i=1 i=1 k E k f i X nm t m i E f i X nm t i i=1 i=1 k E k f i X nm t i E f i Y t i for each m 1. All four terms on the right tend to zero as m, the first by the right continuity of X, the second by 4.2, the third by the right continuity of X nm, and the fourth by a, using the facts that X nm Y and {t 1,..., t k } DY. Hence k k E f i Xt i = E f i Y t i 4.3 i=1 for all f 1,...f k CE and all {t 1,..., t k } DY and hence for all {t 1,..., t k } [, Lemma 4.2 and right continuity of X and Y. ow let i=1 i=1 D = {A S E : E1 X A = E1 Y A }. This is clearly a d-system. Also, since 1 A is in the closure of CE for all open sets A E, 4.3, together with the dominated convergence theorem, implies that D contains the π- system consisting of all finite intersections of {πt 1 A : t [, and A E is open}. By Dynkin s π-system Lemma, D contains the σ-algebra generated by the coordinate random variables π t, and hence, by Proposition 2.11, S E itself. Thus X and Y have the same distribution. i=1 Example 4.4. Suppose X n n 1 is a relatively compact sequence of processes with sample paths in D R d[, having independent increments, and let X be a process in D R d[, having independent increments. For simplicity assume that X = X n = for all n. ow X n t 1,...,X n t k Xt 1,..., Xt k for every finite set {t 1,..., t k } DX if and only if k k E exp i θ j, X n t j E exp i θ j, Xt j j=1 for every k-tuple θ 1,..., θ k R d k. Since X n has independent increments, k k E exp i θ j, X n t j = Eexp i θ j, X nt j X n t j 1 j=1 j=1 j=1 2

22 where θ j = k m=j θ m for j = 1,...,k. The same result holds for X and so, if X n t X n s Xt Xs for all s, t, then the finite dimensional distributions of X n converge in distribution to those of X and hence X n X. Since this condition is clearly necessary, X n X if and only if X n t X n s Xt Xs for all s, t. 5 Relative Compactness in D E [, In order to apply Theorem 4.3 in any practical cases, it is necessary to have an understanding of the conditions a family of stochastic processes, or equivalently probability measures, must satisfy to be relatively compact. In this section we establish some necessary and sufficient conditions for relative compactness in D E [, which will be useful in later sections. 5.1 Prohorov s Theorem Prohorov s Theorem gives a characterisation of the compact subsets of PS, where S, d is the metric space of Section 3, by relating compactness to the notion of tightness. Definition 5.1. A probability measure P PS is said to be tight if for each ε > there exists a compact set K S such that PK 1 ε. A family of probability measures M PS is tight if for each ε > there exists a compact set K S such that inf P M PK 1 ε. Theorem 5.2 Prohorov s Theorem. Let S, d be complete and separable, and let M PS. Then the following are equivalent: a M is tight. b For each ε >, there exists a compact set K S such that where K ε is defined in 3.1. c M is relatively compact. inf P M PKε 1 ε, Before we can prove this, we need two intermediate results. Theorem 5.3. If S is separable, then PS is separable. If in addition S, d is complete, then PS, ρ is complete. Proof. Since S is separable, there exists a countable dense subset {x 1, x 2,...} S. Let δ x denote the element of PS with unit mass at x S. Fix x S. We shall show that the countable set of probability measures of the form i= a iδ xi with finite, a i rational and i= a i = 1 is dense in PS. Given ε >, let P PS. Since P i=1 Bx i, ε/2 = 1, there exists some < such that P i=1 Bx i, ε/2 1 ε 2. Set A i = Bx i, ε/2 \ i 1 j=1 Bx j, ε/2 for each i. Pick some m with m 2 ε. Let a i = mpai m < PA i for i = 1,..., and define a = 1 i=1 a i. Then the a i are rational and so Q = i= a iδ xi is of the required form. If F C, then PF P c + P A i F A i F A i QF ε + ε. A i mpa i m i=1 + m + ε 2 21

23 Therefore ρp, Q < ε and so PS is separable. To prove completeness, pose P n n 1 is a Cauchy sequence in PS. By restricting to a subsequence if necessary, we may assume that ρp n 1, P n < 2 n for each n 2. As in the proof of separability, for each n = 2, 3,... there exists some n < and disjoint sets E n 1,...,E n n BS with diameters less than 2 n such that P n 1 E n 2 n where E n = S \ n i=1 En i. By applying Lemma 3.11 successively for n = 2, 3,..., with P = P n 1, Q = P n, ε = δ = 2 n and = n, there exists a probability space Ω, F, P on which are defined S-values random variables Y n,..., Y n n, for n = 2, 3,..., [, 1]-valued random variables ξ n, n = 2, 3,..., and an S-valued random variable X 1 with distribution P 1, such that if the constants c n 1,...,cn n [, 1] are appropriately chosen, then the random variable X n = { Y n i has distribution P n, and on {X n 1 E n i, ξ n c n Y n on {X n 1 E n } n PdX n 1, X n 2 n+1 2 n+1. i }, i = 1,..., n, i=1 {X n 1 E n i, ξ n < c n Then n=2 PdX n 1, X n 2 n+1 < and, by the Borel-Cantelli Lemma, PdX n 1, X n 2 n+1 infinitely often =. Hence P dx n 1, X n < = 1. n=2 Since S, d is complete, lim X n exists on this set. Setting X to be the value of the limit where it exists, and otherwise, lim X n = X almost surely and so, by Proposition 3.1, lim ρp n, P =, where P is the distribution of X. Lemma 5.4. If S, d is complete and separable, then each P PSis tight. Proof. Let {x 1, x 2,...} be a dense subset of S, and let P PS. Then P k=1 B x k, 1 n = 1 for each n and so, given ε >, there exist integers 1, 2,... such that P n k=1 B x k, 1 1 ε n 2 n for n = 1, 2,.... Let K be the closure of n n 1 k=1 B x k, n 1. Then for each δ >, K can be covered by n balls of radius δ where n > 1 δ. Therefore K is totally bounded and hence compact, and PK 1 1 = 1 ε. [ 1 P n=1 n=1 ε 2 n n k=1 B x k, 1 ] n i } Proof of Theorem 5.2. a b Immediate. 22

24 b c By Theorem 5.3, PS, ρ is complete and hence the closure of M is complete. So it suffices to show that M is totally bounded i.e. given δ >, there exists a finite set PS such that M P {Q : ρp, Q < δ}. Let < ε < δ 2. Then there exists a compact set K S such that b holds. By the compactness of K, there exists a finite set {x 1,...,x n } K such that K ε n i=1 B i, where B i = Bx i, 2ε. Fix x S and an integer m n ε and let be the finite collection of probability measures of the form n ki P = δ xi, 5.1 m i= where k i are integers with k i m and n i= k i = m. Given Q M, let k i = mqe i for i = 1, 2,...,n, where E i = B i \ i 1 j=1 B j, and let k = m n i=1 k i. Then, defining P by 5.1, we have QF Q + QK ε c F E i F E i PF 2ε + 2ε E i mqe i m for all closed sets F S. So ρp, Q 2ε < δ as required. + n m + ε c a Let ε >. Since M is relatively compact, it is totally bounded and hence, for each n, there exists a finite subset n M such that M P n {Q : ρp, Q < ε 2 }. Since n+1 n is finite, by Lemma 5.4, for each n we can choose a compact set K n S such that PK n 1 ε 2 for all P n+1 n. Given Q M, for each n, there exists P n n such that QKn ε/2n+1 P n K n ε 2 n+1 1 ε 2 n. Letting K be the closure of n 1 Kε/2n+1 n, K is compact and QK 1 n=1 ε 2 n = 1 ε. 5.2 Compact Sets in D E [, To apply Prohorov s Theorem Theorem 5.2 to PD E [,, it is necessary to have a characterisation of the compact sets of D E [,. We first give conditions under which a collection of step functions is compact. Definition 5.5. Given a step function x D E [,, define s x = and, for k = 1, 2,..., define s k x = inf{t > s k 1 x : xt xt } if s k 1 x <, and s k x = if s k 1 x =. Lemma 5.6. For Γ E, compact, and δ >, define AΓ, δ to be the set of step functions x D E [, such that xt Γ for all t, and s k x s k 1 x > δ for each k 1 for which s k 1 <. Then the closure of AΓ, δ is compact. Proof. It is enough to show that every sequence in AΓ, δ has a convergent subsequence. Suppose x n n 1 is a sequence in AΓ, δ. Either there exists a subsequence x 1a n n 1 such that s 1 x 1a n < for all n, or there exists a subsequence x1 n n 1 such that s 1 x 1 n = for all n. In 23

25 the first case, there exists a subsequence x 1b n n 1 of x 1a n n 1 such that, for some t 1 [δ, ], lim s 1 x 1b n = t t 1. If t 1 <, we can insist further that log 1 s 1x < 1 1b n n. Since Γ is compact, the sequence x 1b n n 1 has a subsequence x 1 n n 1 such that lim x 1 ns 1 x 1 n = α 1 for some α 1. In this way, we can construct a sequence of subsequences x 1 n n 1 x 2 n n 1 such that, for k = 1, 2,..., either a s k x k n < for all n, there exists some t k [kδ, ] such that lim s k x k n = t k, and if t t k <, log k t k 1 s k x k n s k 1x < 1 k n n, and lim x k n s kx k n = α k for some α k, or b s k x k n = for all n. Let y m m 1 be the subsequence of x n n 1 defined by y m = x m m and define y D E[, by yt = α k, t k t < t k+1, for k =, 1,... where we take t =. Since s k y m s k 1 y m > δ for each k 1 and each m for which s k 1 y m <, we may define λ m Λ to be the piecewise linear function that joins s k 1 y m, t k 1 to s k y m, t k if t k < and which has gradient 1 log t otherwise. Then γλ m = k t k 1 k s k y m s k 1 y m < 1 m, and, for each t there exists a k such that ry m t, yλ m t = ry m s k y m, yt k. Thus y m y by Proposition 2.6. The conditions for compactness will be stated in terms of the following modulus of continuity. Definition 5.7. For x D E [,, δ >, and T >, define w x, δ, T = inf max {t i} i s,t [t i 1,t i rxs, xt, where {t i } ranges over all partitions of the form = t < t 1 < < t n 1 < T t n with min 1in t i t i 1 > δ and n 1. The initially strange looking condition t n 1 < T t n allows us to not have to worry about the length of the final interval, in a partition of [, T], being smaller than δ. For example, partitions where each interval is the same length, but this length does not divide T, are admissible. ote that w x, δ, T is non-decreasing in δ and in T, and that w x, δ, T w y, δ, T + 2 s<t +δ rxs, ys. Before we can characterise the compact sets, we need to establish a few properties of w x, δ, T. Lemma 5.8. a For each x D E [, and T >, w x, δ, T is right continuous in δ and lim δ w x, δ, T =. b If x n n 1 is a sequence in D E [,, and lim dx n, x =, then for every δ >, T >, and ε >. lim w x, δ, T w x, δ, T + ε c For each δ > and T >, w x, δ, T is Borel measurable in x. Proof. a The right continuity follows from the fact that any partition = t < t 1 < < t n 1 < T t n with min 1in t i t i 1 > δ and n 1 also satisfies min 1in t i t i 1 > δ for δ = 1 2 δ + min 1int i t i 1 > δ. 24