ECON 2530b: International Finance

Transcription

1 ECON 2530b: International Finance Compiled by Vu T. Chau 1 Non-neutrality of Nominal Exchange Rate 1.1 Mussa s Puzzle(1986) RER can be defined as the relative price of a country s consumption basket in terms of another country s consumption basket: or, in logs: RER f /d = NER f /d P d P f rer f /d = ner f /d + p where p is the log ratio of domestic price index to foreign price index. 1 An appreciation (depreciation) in the RER can therefore be caused by either (1) appreciation (depreciation) of the nominal exchange rate, or (2) inflation (deflation). Relative PPP: RER is constant, hence movements in NER must be offset by movement of P d /P f. Furthermore, exchange rate regimes (fixed or floating) should not matter. Empirical findings contradict these implications of relative PPP: Systemic differences in movements of RER under fixed and floating regime (RER is more volatile under floating NER regime). Under floating regime, RER co-moves closely with NER, hence contradicting the neutrality of NER. RER is very persistent with long half-lives. 1.2 Cavallo, Neiman, and Rigobon (2013) Law of One Price (LOP) holds well for thousands of goods in currency Union, but not well outside. LOP holds well for dollarized countries, but not among countries that peg to the dollar. Therefore, having the same currency is important in getting LOP! Variable markups matter! (IPIs). 1 This price index can be calculated using consumer price indices (CPIs), producer price indices (PPIs), or import price indices 1

2 2 Lebesgue Integration and Convergence Theorem Theorem 2.1 (Monotone Convergence Theorem). Let g, f 1, f 2,... be measurable functions such that g L 1, g f 1 a.e. and f n f a.e. Then: f n dµ f dµ Lemma 2.2 (Fatou s Lemma). Let g, f 1, f 2,... be measurable functions such that g L 1, f n g a.e. for n. Then: lim inf f n dµ lim inf f n dµ n n Theorem 2.3 (Lebesgue s Dominated Convergence Theorem). Let g, f 1, f 2,... be measurable functions such that g L 1, f n g n a.e. n, and f n f a.e. Then, we have f n dµ = f dµ and lim n lim n f f n = 0 If µ is a probability measure, then µ f is a probability measure too, and called the distribution of f. Proposition 2.4 (Push-forward measure 2 ). Let (Ω, F, µ) be a measure space, and (Ω, F ) a measurable space. f : (Ω, F) (Ω, F ) a measurable function. Then: µ f (B) = µ[ f 1 (B)] defines a measure on (Ω, F ), and g f dµ = gdµ Ω Ω f Definition (Expectation, Variance, and Covariance). We define: EX = Ω XdP for X L1. Var(X) = Ω (X EX)2 dp = E[X 2 ] (EX) 2 Cov(X, Y) = E[(X EX)(Y EY)] = E[XY] (EX)(EY) So far, we are saying that given a measure space and a random variable, we can characterize the distribution of that variable by the cdf F X (x) = (P X 1 )[X x]. Now we define the inverse function of F X : Definition (Right-quantile function). Define q X : (0, 1) R by q X (u) = sup{x : F X (x) u}. We note that q X = d X. So far we have linked the relationship between pointwise convergence and convergence in L 1 using the two convergence theorems. The following section explores more types of convergence, and the notion of uniform integrable. 3 Types of Convergence and Uniform Integrability Definition. Let X 1, X 2,... be randome variables on a probability space (Ω, F, P). X n X a.s. N F such that P[N] = 0 and lim n X n (Ω) = X(Ω) for ω Ω \ N. 3 X n L p X X n X p 0 as n. 4. X n P X P[ Xn X ] > ɛ 0 as n for all ɛ > 0. X n D X E[ f (Xn )] E[ f (X)] as n for all bounded continuous functions f : R R. Definition (Uniform Integrable). A family of random variables (X i ) i I on a probability space is uniformly integrable if sup i I X i >c X i dp 0 as c. 2 Take g(x) = x, we end up with EX = Ω XdP = R xd(p X 1 ) 3 Almost surely convergence is pointwise convergence at all points except for a set of measure zero. 4 Recall that f p = (E[ f p ]) 1/p. Note that when p = 1 we get convergence in L 1, which we discussed extensively earlier. 2

3 It turns out there is a very simple condition for uniform integrability: if the sequence of random variables is bounded by an integrable RV X. This is stated clearly below: Lemma 3.1 (Conditions for Uniform Integrability). If (X i ) i I is a family of random variables on a probability space (Ω, F, P) such that X i X for some X L 1 (Ω, F, P) and i I, then (X i ) i I is uniformly integrable. It follows that finitely many random variables X 1, X 2,..., X n L 1 (Ω, F, P) are uniformly integrable. Now we turn to the necessary and sufficient condition for uniform integrability. In simple words, a family of RVs is uniformly integrable if (1) each RV in the family is L 1 bounded, and (2) the L 1 norm of each RV vanishes on any small enough set. Particularly: Theorem 3.2 (Necessary and Sufficient Conditions for Uniform Integrability). Let (X i ) i I be a family of RVs. This family is uniformly integrable if: (L 1 boundedness) sup i I X i L 1 < ; and, (vanishes with small domain) For any ɛ > 0, δ > 0 s.t. E[ X i 1 A ] < ɛ for all i and set A F with P[A] < δ. Below is a major result: Theorem 3.3 (Uniform Integrability + Convergence in Probability = Convergence in L 1 ). Let X 1, X 2,... P be RVs on (Ω, F, P), X n X, and {Xn } n 1 uniformly integrable. Then X is integrable (EX < ), X n X L 1 0, and, as a result, EX n EX. Recall that we have a sufficient condition for uniform integrability before, which is (1) (X i ) has to be L 1 bounded, and (2) a really long and complicated condition. The following lemma and its corollary helps simplify the second condition: Lemma 3.4 (de la Vallee - Poussin). A family of RVs (X i ) i I on (Ω, F, P) is uniformly integrable if and only if there exists ϕ : R + R + such that lim x ϕ(x) x = and sup i I E[ϕ( X i )] <. ϕ can always be chosen to be convex and non-decreasing. Corollary 3.5 (L p bounded (p > 1) implies uniform integrability). A family of random variables (X i ) i I defined on (Ω, F, P) is uniformly integrable if sup i I X i L p < for some p (1, ]. Using Jensen s inequality 5, we can prove the following result: Lemma 3.6. Let X be a RV. Then X L p X L q for all 1 p q. 4 Weak Law of Large Numbers The law of large number states that the sample mean of n uncorrelated variables with the same mean and finite second moment will converge to the population mean in L 2 and in probability. Theorem 4.1 (Weak Law of Large Number). Let X 1, X 2,..., X n be uncorrelated RVs in L 2 s.t. E[X n ] = m n, and sup n E[Xn] 2 <. Then n 1 n X i m in L 2 (and hence, in probability). i=1 5 Distributions and Densities 5.1 Discrete distributions Definition (Dirac measure). Let Ω be a set, ω Ω. The Dirac measure δ ω : 2 Ω {0, 1} is given by: { 1 if ω A, δ ω 0 else. 5 Let X be a RV and ϕ : R R a convex function such that X, ϕ L 1. Then: E[ϕ(X)] ϕ(e[x]) 3

4 Let µ be a measure on R d. Definition. µ is called discrete if µ = n 1 p n δ xn for {x n } n 1 and p 1, p 2, Some examples of discrete distributions on R: (1) Discrete uniform distribution: µ(n) = N 1, n = 1, 2,..., N. (2) Bernoulli: µ(0) = 1 p, µ(1) = p for p [0, 1]. (3) Binomial: µ(n) = ( N n )pn (1 p) N n for n = 0, 1,..., N. (4) Poisson: µ(n) = e λ λn n!, n = 0, 1,... (5) Geometric: µ(n) = (1 p) n p, n = 1, 2, Continuous Distributions Definition. µ is called continuous if µ(a) is continuous in R d, for A = (a 1, b 1 ] (a2, b2]... (a d, b d ]. Definition. µ is called absolutely continuous if there exists a Borel measurable function f : R d R + such that: µ ((a 1, b 1 ]... (a d, b d ]) = b1 a 1... bd a d f (y 1, y 2,..., y d )dy d...dy 1 f is called a density and µ[a] = A f (x)dx A B(R) d. Also, for any g measurable, we have Examples: Rd gdµ(x) = (1) Normal distribution (or, Gaussian distribution): f (x) = g(x) f (x)dx Rd 1 (x µ)2 exp{ 2πσ 2 2σ 2 } (2) Uniform on [a, b]: f (x) = b a 1 for x [a, b] and zero outside. (3) Exponential: f (x) = λe λx, for x 0, λ > 0. 6 (memoryless) (4) Bilateral exponential: f (x) = 1 2 λe λ x for x R, λ > 0. (5) Cauchy distribution: It is the distribution of a ratio of two standard normal variables. The density is: θ f (x) = π(x 2 + θ 2 ) This distribution is pathological in the sense that it has no moment (everything, mean, variance, etc. is undefined). 6 Characteristic Function Definition (Characteristic Function). Let X be a d dimensional RV with F(x 1,..., x d ) = P(X 1 x 1,..., X d x d ). The characteristic function ψ X : R d C of X is given by: [ ] ψ X (u) = E e iut X = eiutx df(x) R d Some properties of characteristic function: Lemma 6.1. Let X be a d dimensional RV. Then: 6 This is the continuous analogue of geometric distribution. It describes the time between events in a Poisson process. 4

5 (1) ψ X (u) 1 (2) ψ X ( u) = ψ X (u) = ψ X (u) (3) If ψ X is real-valued, then X and X has the same distribution. (4) ψ X (u) is uniformly continuous in R d. 7 (5) If Y = ax + b, then the characteristic function of Y is ψ Y = e iutb ψ X (au). 8 (6) The characteristic function of Z = X + Y where X, Y are independent RVs is ψ Z (u) = ψ X (u)ψ Y (u). 6.1 Common Characteristic Function (1) Dirac delta measure P[X = x] = 1: ψ X (u) = exp(iu T x). (2) Bernoulli: P[X = 0] = 1 p, P[X = 1] = p. Then: ψ X (u) = E[e iux ] = (1 p)e iu 0 + pe iu 1 = (1 p) + pe iu. (3) Poisson distribution ( f (n) = e λ λn n! ):9 ψ X (u) = n=0 f (n)eiun = e λ [λe iu ] n n=0 n! = e λ(1 eiu ) (4) Gaussian distribution X N (µ, σ 2 ): ψ X (u) = exp(iuµ 1 2 σ2 u 2 ). Standard normal (µ = 0, σ = 1), then µ X (u) = e 1 2 u Inversion Formula One way to check if a probability measure admits a density is to see if the characteristic function is integrable. If that s the case, the theorem below helps uncover the original density: Theorem 6.2 (Inversion Formula). Let ψ X (u) = e iux dµ such that R ψ X(u) du <, then a density f (x) exists and satisfies f (x) = 1 e iux ψ(u)du 2π R 6.3 Levy s Continuity Theorem We have that the distribution of a random variable X is uniquely determined by the characteristic function ψ x due to the inversion vormula. This applies to a vector of RVs as well. Theorem 6.3 (Levy s Continuity Theorem). Let X n be RVs defined on probability spaces (Ω n, F n, P n ) with characteristic function ψ n (u), n 1. Then convergence in distribution of (X n ) n 1 implies pointwise convergence in the characteristic functions, and vice versa. More rigorously: (1) If X n d X for some RV X, then ψ n (u) ψ(u) for all u R. (2) If ψ n (u) ψ(u) for all u R, and ψ(u) is continuous at u = 0, then ψ is the characteristic function of some RV X, and X n d X. 6.4 Central Limit Theorem Theorem 6.4 (CLT). Let X 1, X 2,..., X n be RVs on (Ω, F, P) such that E[X 2 ] < and σ n = Var(X n ) > 0 1 for n. Then, n i=1 n X i E[X i ] σ(x i ) d Z where Z N (0, 1). 7 Quick proof: Fix ɛ > 0. ψ X (u + t) ψ X (u) = E[exp(iu T X)(exp(it T X) 1)] E[exp(it T X) 1] 0 as t 0 8 ψ Y (u) = E[exp(iu T Y)] = E[[exp(i(au) T X)]] = exp(iu T b)ψ X (au) 9 Recall that e x = n=0 xn n! 5

6 7 Dynamical System ( ) Definition. A random vector X R d is normal (or Gaussian) if and only ψ X (u) = exp iu T µ 1 2 ut Cu for some µ R d and C symmetric positive semidefinite. Notation: X N d (µ, C). Note that when d = 1 we get the characteristic function ψ X (u) = exp(iuµ 1 2 σ2 u 2 ) of a normal distribution mean µ and variance σ 2. Importantly, if C is not invertible, then the density doesn t exist. Proposition 7.1. Let µ R d and C R d d be a symmetric positive semidedinite matrix. Then: (1) There exists A such that A 2 = C If Z N d (0, I d ), then X = µ + AZ N d (µ, C). (2) The components of X are independent if and only if they are uncorrelated. 12 (3) If C is invertible, then X has a density: { } 1 1 f (x) = (2π) d/2 det(c) exp 2 (X µ)t C 1 (X µ) If C is not-invertible, then A is not invertible, and AR d is a strict subspace of R d, and X cannot have a density. (4) For all V R k and M R k d, Y = V + MX N d (V + Mµ, MCM T ) Proposition 7.2. A d dimensional random vector X is normal if and only if v T X is normal for all v R d. 7.1 Gaussian Process A Gaussian process is a process in which any finite set samples in time (or space) is normally distributed. Formally: Definition (Gaussian Process). Let I be a non-empty set. A family (X i ) i I of RVs on a probability space is called a Gaussian process if (X i1, X i2,..., X id ) is a finite dimensional Gaussian for all d and i 1, i 2,..., i d I. As we move to a (possibly) continuous index set I (for example, time [0, T]), we need to extend the notion of covariance matrix C from the last session. Definition (Symmetric, positive semidefiniteness of function). A function C : I 2 R is symmetric if C(i, j) = C(j, i) for all i, j I. It is positive semidefinite if [C(i k, i l )] d k,l=1 is positive semidefinite for all d and (i 1,..., i d ) I d. For a Gaussian process (X i ) i I, denote µ(i) = E[X i ] and C X (i, j) = cov(x i, X j ) for i, j I. Then, C X is symmetric and positive semidefinite by definition (since any finite sample follows multivariate normal, its covariance matrix must be positive semidefinite). However, provided a mean function µ X (i) and covariance function C X (i, j), a Gaussian process with that mean and covariance function is guaranteed to exist: Theorem 7.3 (Existence of Gaussian process). Let I be a non-empty set, µ : I R an arbitrary function, and C : I 2 R an arbitrary symmetric and positive semidefinite function. Then there exists a Gaussian process (X i ) i I with µ X = µ and C X = C. Examples of Gaussian processes: 10 Linear algebra note: C is symmetric, then the followings are equivalent: (1) C is positive semidefinite; (2) C has all nonnegative eigenvalues; (3) all principal minors of C are nonnegative; and (4) There exists A such that A T A = C. 11 Connection between definiteness and principal minors: Let D k and k denotes leading principal minor and principal minor of order k respectively. Positive definiteness = D k > 0 for all D k ; negative definiteness = ( 1) k D k > 0 for all D k ; positive semidefiniteness = Delta k 0 for all k ; negative semidefiniteness = ( 1) k k 0 for all k. 12 Generally, zero correlation does not imply independence. However, if two variables are jointly normal, then correlation does imply independence. Note however that two variables which are separately normal may still not have this property, since there is no guarantee that the joint distribution is normal. 6

7 (1) White noise: Gaussian process with µ X (i) = 0 for i I and C X (i, j) = 1 i=j for i, j I. 13 (2) Brownian motion: µ X (i) = 0 for i, and C X (i, j) = (i j) for i, j I. Properties of a Brownian motion: E[X 2 0 ] = 0 0 = 0 X 0 = 0 a.s. Increments are normally distributed: For t > s, X t X s = ( 1 1 ) ( ) X s N (0, t s). X t ( Xs X t ) (( ) ( )) 0 s s N 2,, so 0 s t Increments are independent: For s < t < u < v:cov(x v X u, X t X s ) = cov(x v, X t ) + cov(x u, X s ) cov(x v, X s ) cov(x u, X t ) = t + s s t = 0. Since increments are distributed jointly Gaussian, this also implies they are independent. 8 Martingales 8.1 Conditional Expectation Definition (Absolute Continuous Measure). Let µ 1, µ 2 be two measures on (Ω, F). µ 2 is said to be absolutely continuous wrt µ 1 if µ 1 [A] = 0 implies µ 2 [A] = 0 for all A F. 14 Notation: µ 1 >> µ 2. Note that µ 2 = A f dµ 1 = E[1 A f ] defines a measure that is absolutely continuous wrt µ 1. This means that given a function, then we can generate a corresponding absolutely continuous measure. The other direction is guaranteed by the Radon-Nikodyn theorem, which posits that given a measure, we can find such a function f : Theorem 8.1 (Radon-Nikodym). Let µ 1, µ 2 be 2 measures on (Ω, F) such that µ 2 << µ 1 and µ 1 is σ finite. Then, there exists f : Ω R + { } such that µ 2 [A] = A f dµ 1 = E[1 A f ] for all A F. We use the remark above and the R-N theorem to guarantee the existence of a conditional expectation which is defined below: Definition (Conditional Expectation). Let (Ω, F, P) be a probability space, and G a sub-algebra of F. The conditional expectation of X L 1 (Ω, F, P) given G is a RV Y which satisfies: Y is G measurable, Y L 1 (Ω, G, P). E[1 A Y] = E[1 A X] for all A G. Notation: Y = E[X G]. We also define expectation wrt another RV: E[X Z] = E[X σ(z)] Proposition 8.2. Such a conditional expectation always exists uniquely (up to a.e. equal). The idea is that: (1) given F, P, and X, we can generate another measure P 2 = E[1 A f ] << P for A G (and P 2 [A] = 0 for A G). This measure, therefore, is limited onto G only. Then, applies Radon-Nikodym on (Ω, G) guarantees the existence of Y L 1 (Ω, G) such that P 2 [A] = E[1 A Y] for all A G. It follows that E[1 A Y] = E[1 A X] for all A G. Properties of conditional expectation: Proposition 8.3 (Properties of Conditional Expectation). We have: (1) (monotonicity) E[X G] E[Y G] if X Y a.e. (2) (Pulling out what s known) E[XY G] = YE[X G] if Y is G measurable. 13 This means that the variance of the noise at each point in time is var(x i ) = 1, and the noises are independent: cov(x i, X j ) = 0 (note: they are distributed normally, so zero correlation implies independence). 14 We can understand that µ 2 is a more narrowly defined measure, and the expectation on mu 2 will be the conditional expectation viewed from µ 1. 7

8 (3) (Irrelevant information) E[X G] = E[X] if X is independent of G. (4) (Jensen s Inequality) E[ϕ(X) G] ϕ(e[x G]) for ϕ : R R, ϕ L 1 (Ω, F, P) convex. (5) (Conditional MCT) If Y X 1 X 2... for Y, X 1, X 2,... L 1 (Ω, F, P) then lim n E[X n G] = E[lim n X n G] a.s. (6) (Conditional Fatou) If X n Y for some Y L 1 (Ω, F, P), then E[lim inf n X n G] lim inf n E[X n G]. (7) (Conditional DCT) If X n Y for some Y L 1 (Ω, F, P), then lim n E[X n G] = E[lim n X n G]. 8.2 Martingales Definition. (X n ) n 0 is a martingale wrt (F n ) n 0 if X n L 1 (Ω, F n, P) and E[X n+1 F n ] = X n. Definition. (X n ) n 0 is a supermartingale wrt (F n ) n 0 if X n L 1 (Ω, F n, P) and E[X n+1 F n ] X n. Definition. (X n ) n 0 is a submartingale wrt (F n ) n 0 if X n L 1 (Ω, F n, P) and E[X n+1 F n ] X n. Proposition 8.4. X n = E[X F n ] defines a uniformly integrable martingale. Definition (Martingale Transform). Let (X n ) n 0 and (V n ) n 0 { be 2 stochastic processes on (Ω, F, P). Define i=1 n X n = X n X n 1 for n 1. Define a new process (V X) n = V i X i if n 1 0 if n = 0. A process (V X) n such as above is called a Martingale transform. If (X n ) is a martingale, and (V n ) is predictable, then the transform is also a martingale. More rigorously: Theorem 8.5. Let (X n ) n 0 is a martingale wrt (F n ) n 0, (V n ) n 0 is predictable wrt (F n ) n 0. If (V X) n is L 1 bounded (i.e. sup n E (V X) n < ), then (V X n ) n 0 is a (F n ) martingale. Definition (Stopping Time 15 ). A stopping time wrt (F n ) n 0 on (Ω, F, P) is a RV τ : Ω N {+ } such that {τ = n} = {ω Ω : τ(ω) = n} F n Definition (Stopping Algebra). Given a stopping time τ, the algebra of that stopping time is the collection of events such that for each event, the decision to stop for each state of that event depends solely on the information available up to then: (F τ ) = {A F : A {τ n} F n }. It is easy to verify that this is a σ algebra. Lemma 8.6. If τ, σ are stopping times, then so are τ + σ, τ σ, τ σ. Also, F τ σ = F τ F σ. If τ σ, then F τ F σ. Below we define X τ n as a stopped process: it takes value of X n as normal until the stopping time, then stay constant thereafter. This process will be shown to be a martingale also. Lemma 8.7. (X n ) n 0 a stochastic process adapted to (F n ) n 0, and τ is a (F n ) stopping time. Then: the random variable X τ 1 {τ< } is F τ measurable. X τ n = X n τ is (F n ) adapted. Corollary 8.8. Let X n be (F n )-martingale, τ a (F n ) stopping time. (F n ) n 0. In particular, if τ is bounded, then E[X τ ] = E[X 0 ]. Then (X τ n) n 0 is a martingale wrt This implies that the expected value of any stopped process under a bounded stopping time is the same as the expected value of the process at time 0. All this result boils down to a major theorem: Doob s Optional Stopping theorem. This theorem posits that given a martingale process, a bounded stopping time in which the decision to stop in every state depends only the information up to that time. Then, that stopping time is optional in the sense that it will not alter the expected value at stopping time. More formally: 15 We can think of τ(ω) as a stopping decision: the decision to stop at any time τ is based solely on the information up to τ. This definition is equivalent to {τ n} F n for all n. 8

9 Theorem 8.9 (Doob s Optional Stopping). Let (X n ) n 0 be a (F n ) martingale, and σ < τ N two bounded stopping times wrt (F n ) n 0. Then E[X τ F σ ] = X σ. Taking σ(ω) = 0 for ω, we have E[X τ ] = X 0. Theorem 8.10 (Doob s Decomposition Theorem). Let (X n ) n 0 be a sub-martingale wrt (F n ) n 0. Then there exists a unique decomposition X n = M n + A n in which (M n ) n 0 is a martingale and (A n ) n 0 is a non-decreasing predictable process. Before, we have that transforming a martingale using a predictable process yields another martingale. We now transform submartingale: Corollary 8.11 (Submartingale Transformation). Let (X n ) n 0 be a submartingale, (V n ) n 0 be a nonnegative predictable process wrt (F n ) n 0, and E[ (V X) n ] < for n. Then (V X n ) n 0 is a submartingale. Proposition Let (X n ) n 0 be a submartingale wrt (F n ) n 0, and ϕ : R R convex, ϕ(x n ) L 1 for all n. If X n is a martingale or ϕ is non-decreasing, then ϕ(x n ) is a martingale. Let (X n ) n 0 be a submartingale wrt (F n ) n 0 and an interval [a, b] Let β n (a, b) denotes the number of times (X n ) n crosses b from below before n. It is easy to show that the time in which (X n ) n crosses from below is a stopping time. We also have an upper bound on the β n (a, b): Theorem 8.13 (Upcrossing Inequality). If (X n ) n 0 is a submartingale, then E[β n (a, b)] E[(X n a) + ] b a. This upcrossing inequality is used to prove the following convergence theorem of sub-martingale: Theorem 8.14 (Sub/super-martingale Convergence). (X n ) n 0 is L 1 bounded submartingale. Then there exists X such that X n X a.s., and X L 1 sup n X n L 1 Similarly, an L 1 bounded supermartingale converges almost surely. Corollary If (X n ) n 0 is a uniformly integrable submartingale, then X n X a.s. and in L 1. More over, E[X F n ] X n for all n. Similarly, a uniformly integrable supermartingale converges a.s. and in L 1. Corollary A submartingale (X n ) n 0 that is bounded above converges a.s. Similarly, a supermartingale that is bounded from below converges a.s. Corollary Similarly, an L p bounded super/sub-martingale (p > 1) converges a.s. and in L Corollary 8.18 (Convergence of Conditional Expectation Martingale). Let X L 1 (Ω, F, P) and (F n ) n 0 a filtration. Denote X n = E[X F n ], and X = E[X F ] where F = n F n = σ( n F n ). Then X n X a.s. and in L 1. Theorem 8.19 (Reverse Martingale Convergence). Let G 0 G 1... be a decreasing sequence of σ algebras on a probability space (Ω, F, P) and X L 1 (Ω, F, P). Then E[X G n ] E[X G] a.s. and in L 1, where G = n 0 G n. The Reverse Martingale Convergence theorem is useful to prove the strong law of large number. Lemma Let X 1, X 2,... be i.i.d. RV in L 1 and define S n n i=1 X i. Then E[X i S n ] = S n n. Theorem 8.21 (STRONG LAW OR LARGE NUMBER). Let X 1, X 2,... be i.i.d. RV in L 1 (Ω, F, P). Then 1 n n i=1 X i E[X i ] a.s. and in L 1. Theorem 8.22 (Doob s Maximal Inequality for Probabilities). Let (X n ) n 0 be a sub-martingale and λ > 0. Then: λp[max k n X k λ] E[X n 1 {maxk n X k λ}] E[X + n ] Corollary If (X n ) n 0 is a martingale, then P[max k n X k λ] E X n p λ p. Theorem 8.24 (Doob s L p maximal inequality). Let (X n ) n 0 be a non-negative submartingale. max k n X k L p p p 1 X n L p and max k n X k L 1 e 1 e (1 + X n log + X n L 1). 16 This is straightforward, because L p bounded for p > 1 implies uniform integrability. Then: 9

10 9 Stochastic Kernel and Regular Conditional Probabilities Definition. Let (Ω, F) and (Ω, F ) be two measurable spaces. A mapping K : Ω Ω R + {± } is a stochastic kernel from (Ω, F) to (Ω, F ) if: ω K(ω, A) is F measurable for A F. A K(ω, A is a measure on (Ω, F ) for all ω Ω. 10 Markov Chains Let I be a countable state space. A vector (λ i ) i I is a distribution if λ i 0 and i λ i = 1. A matrix (P ij ) i,j I is called stochastic if all its rows are distributions. Definition. A stochastic process (X n ) n 0 on a probability space (Ω, F, P) is a Markov chain with initial distribution λ and transition matrix p if: (i) P[X 0 = i] = λ i (ii) P[X n+1 = i n+1 X n = i n,..., X 0 = i 0 ] = p in i n+1 Notation: (X n ) n 0 is Markov(λ, p). Theorem 10.1 (Equivalent definition of Markov Chain). A discrete time stochastic process (X n ) n 0 is Markov(λ, p) if and only if P[X n+1 = i n+1 X n = i n,..., X 0 = i 0 ] = λ i0 p i0 i 1...p in 1 i n for all n 0 and i 0, i 1,..., i n I. Theorem 10.2 (Markov Property). Let (X n ) n 0 be the Markov(λ, p). If P[X m = i] > 0, then conditional on X m = i, (X m+n ) n 0 is Markov(δ i, p) and is independent of (X 0,..., X m ) Note that the state space I decomposes into communicating classes. A communicating class C I is called closed if i j for i C implies j C. i I is called absorbing if {i} is a closed communicating class. A Markov chain or transition matrix is called irreducible if I is a communicating class. Definition. Let A I. Let H A = inf{n : X n A} be the hitting time of the subset A. h A = P i [H A < ] the probability that A will be reached in finite time, and k A = E i [H A ] the expected hitting time given that we start at i. Theorem The vector of hitting probabilities (h A i ) i I is the minimal non-negative solution to the system { 1 if i A j I p ij h A j if i A Theorem The vector of hitting time expectation (k A i ) i I is the minimal non-negative solution to the system of linear equations: { 1 if 0 A 1 + j I p ij k A j if i A Theorem 10.5 (Strong Markov Property). Let (X n ) n 0 be Markov(λ, p) and a stopping time τ wrt (F X n ) n 0. Let i I be such that P[τ < and X τ = i] > 0. Then, conditional on τ < and X τ = i, (X τ+n ) is Markov(δ i, p) and independent of F X τ. Definition (Recurrent and Transient State). A state is recurrent if P i [X n = i for infinitely many n] = 1. A state is transient if P i [X n = i for infinitely many n = 0. Define the hitting times τi 0 = 0, τi 1 = τ i = inf{n 1 : X n = i}, τi 2 = inf{n > τi 1 : X n = i}, etc. Also, define the time difference between two hittings Si k = τi k τ k 1 i if τ k 1 i < and 0 otherwise. Lemma For k = 2, 3,..., conditional on τ k 1 i ] = P i [τ i = n]. <, S k i is independent of F X τ k 1 i and P[S k i = n τ k 1 i < 10

11 This lemma is a direct consequence of the Strong Markov Property. Define V i = n 0 1 Xn =i (number of visits to i), and f i = P i [τ i < ]. Lemma E i [V i ] = n 0 p (n) ii. Also, P i [V i > k] = fi k, for k = 1, 2,... Theorem 10.8 (Determining Recurrent or Transient). If P i [τ i < ] = 1, then i is recurrent and n 0 p n ii =. If P i [τ i < ] < 1, then i is transient and n 0 p n ii <. In particular, every state is either transient or recurrent. Theorem The states of a given communicating class are either all recurrent or transient. Therefore, one can speak of a recurrent/transient class. Theorem Every recurrent class is closed. Theorem Every finite closed class is recurrent. Theorem If p is an irreducible and recurrent. Then, for every initial distribution λ, P[τ j < ] = 1 for all j I Equilibrium Distribution Definition. A measure λ on I is said to be invariant wrt a transition matrix p if λp = λ. Theorem If (X n ) n 0 is Markov(λ, p) and λ is invariant wrt p, then for every m 1, (X m+n ) n 0 is again Markov(λ, p). Theorem Let I be finite. Assume there exists i I such that p n ij Π j as n for all j. Then (Π j ) j I is an invariant distribution. Definition. For i, k I, set γ k i E k [ τ k 1 n=0 1 X n =i] (starting at k, number of visits to i before visiting k again). Theorem Let p be an irreducible recurrent transition matrix. Then: (1) γ k k = 1. (2) (γ k i ) i I solves γ k p = γ k (3) 0 < γ k i < for all i I. Theorem Let p be an irreducible transition matrix and λ an invariant measure such that λ k = 1. Then λ γ k. In addition, if p is recurrent, then λ = γ k. 11