Lecture 2 1 Martingales We now introduce some fundamental tools in martingale theory, which are useful in controlling the fluctuation of martingales. 1.1 Doob s inequality We have the following maximal inequality of Doob, which allows us to control the fluctuation of the whole path (X i ) 0 i n by controlling the fluctuation of the end point X n, when X is a (sub-)martingale. Theorem 1.1 [Doob s maximal inequality] Let (X i ) i N be a sub-martingale w.r.t. a filtration (F i ) i N. running maximum of X i. Then for any l > 0, Let S n max 1 i n X i be the P(S n l) 1 l E[ X n + ] 1 1 {Sn l} l E[X+ n ], (1.1) where X + n X n 0. In particular, if X i is a martingale and M n max 1 i n X i, then Proof. Let τ l inf{i 1 : X i l}. Then For each 1 i n, P(M n l) 1 l E[ ] 1 X n 1 {Mn l} l E[ X n ]. (1.2) P(S n l) n P(τ l i). (1.3) P(τ l i) E[1 {Xi l}1 {τl i}] 1 l E[X+ i 1 {τ l i}]. (1.4) Note that {τ l i} F i, and X i + is a sub-martingale because X i itself is a sub-martingale while φ(x) x + is an increasing convex function. Therefore and hence E[X + n 1 {τl i} F i ] 1 {τl i}e[x + n F i ] 1 {τl i}e[x n F i ] + 1 {τl i}x + i, E[X + i 1 {τ l i}] E[X + n 1 {τl i}]. Substituting this inequality into (1.4) and then summing over 1 i n then yields (1.1). (1.2) follows by applying (1.1) to the sub-martingale X i. Corollary 1.2 Let X, S and M be as in Theorem 1.1. Then for any p 1, we have P(S n l) 1 l p E[ (X n + ) p ] 1 1 {Sn l} l p E[(X+ n ) p ], (1.5) P(M n l) 1 l p E[ X n p ] 1 1 {Mn l} l p E[ X n p ]. (1.6) 1
Proof. Apply Theorem 1.1 respectively to the sub-martingales (X + n ) p and X n p. The bound we have on P(S n l) allows us to obtain bounds on the moments of S + n. Corollary 1.3 [Doob s L p maximal inequality] Let X i and S i be as in Theorem 1.1. Then for p > 1, we have E[(S n + ) p ] ( p ) p E[(X + p 1 n ) p ]. (1.7) Proof. Note that by the layer-cake representation of an integral, we have E[(S + n ) p ] p 0 0 P((S + n ) p > t)dt p [ l p 2 E[X n + 1 {Sn l}] dl p E p p 1 E[ X + n (S + n ) p 1] 0 l p 1 P(S n l)dl X + n 0 ] l p 2 1 {Sn l} dl p p 1 E[(X+ n ) p ] 1 p E[(S + n ) p ] p 1 p. If E[(S + n ) p ] <, then (1.7) follows immediately. Otherwise, we can first replace S + n by S + n N and repeat the above estimates, and then send N and apply the monotone convergence theorem. Example 1.4 Let X n n ξ i be a random walk with X 0 0, where (ξ i ) i N are i.i.d. random variables with E[ξ 1 ] 0 and E[ξ 2 1 ] σ2 <. Then X n is a square-integrable martingale. Let S n max 0 i n X i. By Doob s inequalities, we have ( Sn ) P a E[X2 n] n na 2 σ2 [( Sn ) 2 ] a 2 and E 4 n n E[X2 n] 4σ 2. (1.8) Exercise 1.5 Show that when ξ i are bounded, the tail bound for S n in (1.8) can be improved to a Gaussian tail bound as in the Azuma-Hoeffding inequality. Exercise 1.6 Doob s L p maximal inequality fails for p 1. Indeed, try to construct a nonnegative martingale X n with E[X n ] 1, and yet sup n N E[S n ]. To get a bound on E[S + n ], we need a bit more than E[X + n ] <. Exercise 1.7 Let X n be a sub-martingale and let S n max 1 i n X i. By mimicking the proof of Doob s L p maximal inequality and by using a log b a log a + b/e for a, b > 0, show that where log + x max{0, log x}. E[S + n ] 1 + E[X+ n log + (X + n )] 1 e 1, 1.2 Stopping times It is often useful to stop a stochastic process at a time which is determined from past observations of the process. Such times are called stopping times. Definition 1.8 [Stopping time] Given (Ω, F, P), let (F n ) n N be a filtration in F. A {0} N { }-valued random variable τ is called a stopping time w.r.t. (F n ) n N, if for every n 0, the event {ω Ω : τ(ω) n} F n. 2
Remark 1.9 Intuitively, a stopping time τ is a decision on when to stop the stochastic process (X n ) n N, using only information up to that time. Examples of stopping times include: τ : k for some fixed k 0; τ : inf{n 0 : X n a} for some a R, i.e., the first passage time of level a for a process (X n ) n N adapted to (F n ) n N ; τ : τ 1 τ 2 or τ : τ 1 τ 2, the minimum or maximum of two stopping times τ 1 and τ 2. Definition 1.10 [Stopped σ-field] Let τ be a stopping time w.r.t. the filtration (F n ) n N. The stopped σ-field F τ associated with τ is defined to be F τ : {A F : A {ω : τ(ω) n} F n for all n 0}. (1.9) Remark 1.11 F τ can be interpreted as the information available up to the stopping time τ. For each event A F τ, we can determine whether it has occurred or not based on what we have observed about the process up to (and including) time τ. Exercise 1.12 Verify that F τ is indeed a σ-algebra, and show that the definition of stopping times and stopped σ-fields are unchanged if we replace τ(ω) n by τ(ω) n, but not by τ(ω) < n. Theorem 1.13 [Stopped martingales are martingales] Let (X n ) n N be a martingale, and τ a stopping time, then the stopped martingale (X n τ ) n N is also a martingale. More generally, if θ is another stopping time and θ τ, then X n τ X n θ is a martingale. If X is a sub/super-martingale, then X n τ X n θ is also a sub/supermartingale. Proof. We will verify that X n τ X n θ is a super-martingale when X is a super-martingale. The rest then follows. Note that X n τ X n θ n 1 {θ<i τ} (X i X i 1 ), where {θ < i τ} F i 1 since {i τ} {τ i 1} c F i 1. Therefore, X n τ X n θ is a martingale transform of X n, and E[X n τ X n θ F n 1 ] X (n 1) τ X (n 1) θ + 1 {θ<n τ} E[X n X n 1 F n 1 ] X (n 1) τ X (n 1) θ. Therefore X n τ X n θ is a super-martingale when X n is a super-martingale. If the price of a stock evolves in time as a martingale, then Theorem 1.13 tells us that no matter when do we decide to buy and sell the stock, as long as our strategy is only based on past observations, the expected payoff will be zero. 1.3 Upcrossing inequality, almost sure Martingale Convergence, and Polya s urn As an application of the notion of stopped martingales, we prove the upcrossing inequality. Let (X i ) 0 i n be a super-martingale adapted to the filtration (F i ) 0 i n on a probability space (Ω, F, P). Let a < b. An upcrossing by X over the interval (a, b) is a pair of indices 3
0 k < l n with X k a and X l b. We are interested in the number U n of complete upcrossings X makes before time n. Define recursively τ 1 : inf{i : X i a}, τ 2 : inf{i τ 1 : X i b},. τ 2k+1 : inf{i τ 2k : X i a}, τ 2k+2 : inf{i τ 2k+1 : X i b},., where the infimum of an empty set is taken to be. Note that τ i are all stopping times, and the number of completed upcrossings before time n is given by U n max{k : τ 2k n}. Theorem 1.14 [Upcrossing inequality] Let (X i ) 0 i n be a (super-)martingale and let U n be the number of complete upcrossings over (a, b) defined as above. Then E[U n ] E[(a X n) + ] b a a + E[ X n ]. (1.10) b a Proof. By Theorem (1.13), (X i τ2k X i τ2k 1 ) 0 i n is a super-martingale for each 1 k n. Therefore [ n ( ) ] E Xn τ2k X n τ2k 1 0. (1.11) k1 On the other hand, note that X τ2k X τ2k 1 b a if 1 k U n, X n X τ2un+1 X n a if k U n + 1 and τ 2Un+1 n, X n τ2k X n τ2k 1 X n X n 0 if k U n + 1 and τ 2Un+1 > n, X n X n 0 if k U n + 2. Therefore, n ( ) Xn τ2k X n τ2k 1 (b a)un + 1 {τ2un+1 n}(x n a). k1 Taking expectation and combined with (1.11) then yields (b a)e[u n ] E[(a X n )1 {τ2un+1 n}] E[(a X n ) + ], (1.12) and hence (1.10). To summarize, the number of upcrossings has to be balanced out by Xn X is a super-martingale, which explains why E[U n ] is controlled by E[ Xn ]. if Remark 1.15 Note that the number of upcrossings and downcrossings differ by at most 1. When X n is a sub-martingale, the expected number of upcrossings can be bounded in terms of E[(X n a) + ]. See Theorem 4.(2.9) in Durrett [1]. A consequence of the upcrossing inequality is the almost sure martingale convergence theorem for L 1 -bounded martingales. 4
Theorem 1.16 [Martingale convergence theorem] Let (X n ) n N be a (super)-martingale with sup n E[ Xn ] <. Then X lim n X n exists almost surely, and E[ X ] <. If X n is a sub-martingale, then the condition becomes sup n E[X n + ] <. Proof. By the upcrossing inequality, for any a < b, a + E[ Xn ] sup E[U n ] sup a + sup n N E[ Xn ] <. (1.13) n N n N b a b a Since U n almost surely increases to a limit U(a, b), which is the total number of upcrossings by X over (a, b), by Fatou s Lemma, E[U(a, b)] < and hence U(a, b) < almost surely. Thus, almost surely, U(a, b) < for all a < b Q, which implies that lim sup X n lim inf X n, n n and hence X lim n X n exists in [, ] almost surely. By Fatou s Lemma, E[ X ] lim inf n E[ X n ] < by assumption. Similarly, by Fatou and the fact that X n is a super-martingale, E[X + ] lim inf n E[X+ n ] lim inf n (E[X n] + E[ X n ]) lim inf n (E[X 1] + E[ X n ]) <. Therefore E[ X ] <. Corollary 1.17 [Almost sure convergence of a non-negative supermartingale] If (X n ) n N is a non-negative super-martingale, then X lim n X n exists a.s., and E[X] E[X 1 ]. Remark 1.18 In Corollary 1.17, we could have E[X] < E[X 1 ] since part of the measure X n dp could escape to. For example, let X n X 0 + n ξ i be a symmetric simple random walk on Z starting from X 0 1, where ξ i are i.i.d. with P(ξ i 1) P(ξ i 1) 1 2. Let τ inf{n : X n 0} be the first hitting time of the origin. Then X n τ is a non-negative martingale which converges to a limit X τ almost surely. The only possible limit for X n τ is either 0 or. Since E[X τ ] E[X 1 ] 1 by Corollary 1.17, we must have X τ 0 almost surely. Note that this also implies that τ < almost surely, so that the symmetric simple random walk always visits 0, which is a property called recurrence. Exercise 1.19 Show by example that a non-negative sub-martingale need not converge almost surely. As an application of Corollary 1.17, we have the following dichotomy between convergence and unbounded oscillation for martingales with bounded increments. Theorem 1.20 Let (X n ) n N be a martingale with X n+1 X n M < a.s. for all n 0. Then almost surely, either lim n X n exists and is finite, or lim sup n X n and lim inf n X n. 5
Proof. For L < 0, let τ L inf{n : X n L}, which is a stopping time. By assumption, X τl L M. Therefore X n τl L + M is a non-negative martingale, which converges a.s. to a finite limit. In particular, on the event {τ L }, lim n X n exists and is finite. Letting L, it follows that on the event {lim inf n X n > }, lim n X n exists and is finite. Applying the same argument to X n implies that lim n X n exists and is finite on the event that {lim sup n X n < }. The theorem then follows. Exercise 1.21 Construct an example where the conclusion in Theorem 1.20 fails if the bounded increment assumption is removed. (Hint: Let X n n ξ i, where ξ i are independent with mean 0 but not i.i.d., such that X n almost surely.) Recall that the Borel-Cantelli lemma states: If (A n ) n N F satisfy P(A n) <, then almost surely, (A n ) n N occurs only finitely many times. If (A n ) n N are all independent, then P(A n) guarantees that almost surely, (A n ) n N occurs infinitely often. Using Theorem 1.20, we give a proof of the second Borel-Cantelli lemma which allows dependence of events. Corollary 1.22 [Second Borel-Cantelli Lemma] Let (F n ) n 0 be a filtration on the probability space (Ω, F, P) with F 0 {, Ω}. Let A n F n for n 1. Then { ω : n1 } { 1 An (ω) ω : n1 } P(A n F n 1 ) (1.14) modulo a set of probability 0. When A n are independent, we retrieve the classic Borel-Cantelli lemma. Proof. Let X n n ( 1Ai (ω) P(A i F i 1 ) ). Note that X n is a martingale with bounded increments. Therefore by Theorem 1.20, either X n converges to a finite limit, in which case n1 1 A n (ω) if and only if n1 P(A n F n 1 ) ; or X n oscillates between ±, in which case we have n1 1 A n (ω) n1 P(A n F n 1 ). We now study the example of Polya s urn using martingale techniques. Example 1.23 [Polya s urn] Let an urn initially contain b black balls and w white balls. Each time, we pick a ball from the urn with uniform probability, and we put back in the urn 2 balls of the same color. Obviously the number of balls in the urn increase by one each time. A natural question is what is the fraction X n of black balls in the urn after time step n? What is the asymptotic distribution of X n as n? Note that X 0 b b+w. It turns out that X n is a martingale. To check this, assume that out of the b + w + n balls at time n, balls are black and k b + w + n balls are white. Then X n, and Then +k ] E [X n+1 X 1,, X n + k + k X n. + 1 + k + 1 + k + k + k + 1 Since X n is non-negative, by the martingale convergence theorem, almost surely, X n converges to a limit X [0, 1]. 6
Polya s urn has the special property of exchangeability. If Y n denotes the indicator event that the ball drawn at the n-th time step is black, then the distribution of (Y n ) n N is invariant under finite permutation of the indices. In particular, if (Y i ) 1 i n and (Ỹi) 1 i n are two different realizations of Polya s urn up to time n with n Y i n Ỹi m, then observe that P((Y i ) 1 i n ) P((Ỹi) 1 i n ) n 1 b + w + i 1 m n m (b + i 1) (w + i 1). Assume b w 1 for simplicity, then after the n-th time step, for each 0 n, ( P X n + 1 ) ( ) n!(n )! 1 n + 2 (n + 1)! n + 1. Therefore X is uniformly distributed on [0, 1]. Check that for general b, w > 0, X follows the beta distribution with parameters b, w and density Γ(b + w) Γ(b)Γ(w) (1 x)b 1 (1 x) w 1, (1.15) where Γ(x) is the gamma function. References [1] R. Durrett, Probability: Theory and Examples, 2nd edition, Duxbury Press, Belmont, California, 1996. [2] S.R.S. Varadhan, Probability Theory, Courant Lecture Notes 7, American Mathematical Society, Providence, Rhode Island, 2001. 7