Statistics 310B, Lecture Notes, Wednesday, February 25, 2015

Transcription

1 CHAPTER 1 Statistics 310B, Lecture Notes, Wedesday, February 25, 2015 Let (X, A, µ) be a fiite measure space Itroductio Defiitio T : X X is measure preservig with respect to measure µ if µ(t 1 (A)) = µ(a) for all A A. Defiitio A measurable set A A is T-ivariat if T 1 (A) = A. The σ-field of all T-ivariat sets is deoted by I. Defiitio T measure-preservig is ergodic if A I = µ(a) = 0 or µ(x A) = 0 For f L 1 (X, A, µ), let S (f) = 1 j=0 f T j ad S (f) = S(f). Theorem (Birkoff (Idividual) Ergodic Theorem). (X, A, µ) σ-fiite, T measure-preservig ad f L 1 (X, A, µ). The, there exists φ L 1 (X, I, µ) s.t. S φ µ a.e., ad φ = φ dµ f dµ = f < Corollary If µ(x) < ad f L p (X, A, µ), p 1 = S φ i L p (X, A, µ) Proof. Cosider the simple case of f beig bouded. If f k a.e., the S (f) k a.e. ad S φ i L p by domiated covergece theorem. Otherwise ay f L p (X, A, µ) is approximated by a sequece of bouded fuctios f ɛ. Let f ɛ k ɛ < k be such that ( 1/p f f ɛ dµ) p < ɛ. The, S (f ɛ ) φ ɛ i L p ad also S (f ɛ ) a.s. φ ɛ, ad we already have S a.s. φ. Further, ( S (f ɛ ) S (f) p (f ɛ f) p ɛ = (φ φ ɛ ) p ɛ by Fatou, ad lim sup ( S (f) φ) p 2ɛ. Take ɛ 0 to coclude the corollary. 1

2 2 1. STATISTICS 310B, LECTURE NOTES, WEDNESDAY, FEBRUARY 25, 2015 Remark. Note that T measure preservig = (f ɛ f) T 1 p = (f ɛ f) p ad ( S (f ɛ ) S (f) p 1/ j=0 (f ɛ f) T j p = (f ɛ f) p ɛ. The fial coslusio follows from the triagle iequality ( S (f) φ) p ( S (f) S (f ɛ )) p + ( S (f ɛ ) φ ɛ ) p + (φ ɛ φ) p By lettig i the above iequality, we have lim sup ( S (f) φ) p 2ɛ as the secod term o the R.H.S goes to 0 ad the first ad third terms are bouded above by ɛ each. Corollary If T is ergodic, φ = cµ a.e. Proof. I is µ-trivial = every φ L 1 (X, A, µ) is µ a.e. costat. c = sup{y : µ(φ > y) > 0}. The µ(φ > c + 1/) = 0 = φ c a.e., also {φ c 1/} I = µ(φ c 1/) = 0, φ = cµ 1.2. Applicatio X = S, A = S c, µ = P ν where P ν defies the law of the homogeeous Markov Chai iitiated at a ivariat measure ν. T = Θ is the shift trasformatio o S. The T is measure preservig from the defiitio of ivariat measure ν. If A I, the {s : s A} = {s : Θ j s A} σ(s j+1, s j+2,...) j <. Hece, A T, the tail σ-algebra of {X }. I particular, I T ad T is ergodic wheever T is µ-trivial. Remark. I particular, if we take f(s) = s 1, the projectio oto the first coordiate, the we recover the Strog Law of Large Numbers (SLLN). Example Oe such example is the product measure, i.e. we have a i.i.d. sequece {X i } with product measure µ = P N o Ω a product space with appropriate σ-field. The shift operator T (x 1, x 2,...) = (x 2, x 3,...) is the measure preservig ad the fuctio f(s) = x 1, the projectio oto the first coordiate gives us S (f) = 1/ j=0 X i = S. By Kolmogorov 0-1 Law, the tail σ-field is µ-trivial ad so I T = T ergodic. The proof of Birkoff Theorem is doe below Proof of Birkoff Ergodic Theorem Proof. Let Γ(a, b) = {x : lim if S (f) < a < b < lim sup S (f)} for a < b; a, b Q. It suffices to show that µ(γ(a, b)) = 0 a, b fixed. This would imply that S (f) a.s. φ [, ]. This also implies: φ 1 f 1. The proof of this follows by otig j 0, f T j dµ f T j dµ = f dµ T j = f dµ.

3 1.3. PROOF OF BIRKOFF ERGODIC THEOREM 3 By averagig, S (f)dµ f dµ. By Fatou, φ dµ f dµ <. Hece, φ < a.e. ad w.l.og. defie φ = 0 whe lim sup S (f) =. Fially, φ L 1 (X, I, µ) is proved by observig S +1 (f) = + 1 S (f) T f. Note that S +1 (f) a.s. φ; S (f) a.s. φ; / ad 1/ as. This implies φ = φ T by takig limits. This proves that φ is measurable (by Mootoe Class). The proof is complete by showig that µ(γ(a, b)) is ideed 0. This follows with the observatio that Γ(a, b) I. Assume that µ(γ(a, b)) <. Defie g = (f b)1 Γ L 1 ad S + (g) = max 0 j {S j (g)}, ad A = {x : S + (g) > 0}. Let A A = k=1 A k. By observig S j (g) = ( S j (f) b)1 Γ ad Γ T 1 = Γ as Γ I, it follows that A = {x : x Γ, sup{ S j (f)} > b} = Γ j 1 Applyig the Maximal Ergodic Lemma, we have 0 gdµ gdµ = A A Γ gdµ = (f b)dµ. Applyig the same argumet to g = (a f)1 Γ, we derive 0 (a f)dµ. Addig up the two Γ iequalities gives us (a b)µ(γ) 0 ad a < b = µ(γ) = 0. Lemma Maximal Ergodic Lemma: If g L 1 (X, A, µ) ad A = {x : S + (g) > 0}, the A gdµ 0 Γ

4

5 CHAPTER 2 Statistics 310B, Lecture Notes, Moday, March 2, 2015 Let (X, X, µ) be a σ-fiite measure space ad T a measure-preservig trasformatio (i.e., µ T 1 = µ). Our motivatig example is statioary probability laws o the space of sequeces: X = S, X = S c the cylidrical σ-algebra, ad µ shift-ivariat; this icludes Markov chais started at a ivariat probability measure. The ivariat σ-algebra I T is I T = {A X : A = T 1 (A)}. If X is complete, we ca equivaletly take I T = {A X : µ(a T 1 (A)) = 0}, where deotes the symmetric set differece. We say T is ergodic of I T is µ-trivial Fiishig the proof of Birkhoff s ergodic theorem (1.1.4) We begi by fiishig the proof of S (f) = 1 S (f) states: For f L 1 (X, X, µ), let S (f) = 1 j=0 f T j ad (a) There exists ϕ(f) L 1 (X, I T, µ) such that lim S (f) = ϕ(f) µ-a.e. (b) If µ(x) = 1, the ϕ(f) = E(f I T ) ad S (f) Lp ϕ(f) wheever f L p, p 1. We early fiished the proof i the last lecture. What remais to be proved are the maximal ergodic lemma ad the fact that whe µ(x) = 1, ϕ(f) = E(f I T ). Lemma (Maximal ergodic lemma). If g L 1 (X, X, µ), S + (g) = max 0 j S j (g), A = {x : S + (g) > 0}, the A gdµ 0. Proof. We first show that for all x A, g S + (g) S + (g) T. Ideed, for all j 1, S j (g) = g + S j 1 (g) T g + S + (g), so g max 1 j S j(g) S + (g) T = S + (g) = S + (g) S + (g) T where the equality is by defiitio of A. Itegratig both sides over A, gdµ A S + (g) A (S + (g) T )dµ. A Sice T is measure-preservig, the right-had side is 0. Now we show that whe µ(x) = 1, ϕ(f) = E(f I T ); that is, for all A I T, E(f1 A ) = E(ϕ1 A ). Sice µ is a probability measure, S (f) L1 ϕ(f) (proved last time), so 1 E(S (f1 A )) E(ϕ(f1 A )). 5

6 6 2. STATISTICS 310B, LECTURE NOTES, MONDAY, MARCH 2, 2015 Thus it suffices to show (1) E(ϕ(f1 A )) = E(ϕ(f)1 A ) ad (2) 1 E(S (f1 A )) = E(f1 A ) for all. For (1), ote that A I T implies that traformatio by T does ot alter membership i A, so 1 S (f1 A ) = 1 1 (f1 A ) T j = 1 j=0 1 (f T j )1 A. Takig o both sides, we obtai ϕ(f)1 A = ϕ(f1 A ), which implies (1). For (2), sice T is measure-preservig, 1 E(S (f1 A )) = 1 1 E((f1 A ) T j ) = E(f1 A ). j=0 j= Wieer s maximal iequality Theorem (Wieer s maximal iequality). If µ(x) = 1, the for all x > 0, ( ) µ max S j(f) > x 1 f dµ 1 j x X Proof. Let B = {max 1 j S j (f) > x}. By Lemma 2.1.1, E((f x)1 B ) 0. The xµ(b) = x dµ fdµ f dµ. B B X 2.3. Examples of Birkhoff Example (Positive recurret Markov chai). Suppose P µ is the law of a statioary homogeeous Markov chai, startig at a ivariat probability measure π. Assume we are i a coutable state space ad the chai is irreducible; the the chai is positive recurret ad π is uique. Without proof, the shift θ is ergodic. By 1.1.4, takig f(x) = g(x 1 ), we have 1 1 g(x j ) g(s)π(s) j=0 s S as, i.e., the Cesaro average of the states coverges to the expectatio uder the ivariat measure. Example (Beroulli shift). Let (X, X, µ) = ([0, 1), B [0,1), λ). Let Y 0 (ω) = ω ad Y = T Y 0, where T (y) = (2y) mod 1. The T is measure-preservig ad ergodic. We ca see this by represetig Y 0 as a ifiite sequece of biary digits: Y 0 = 2 (i+1) X i i=0 where X 0, X 1,... are i.i.d. Ber(1/2). The the trasformatio of Y 1 to Y correspods to shiftig the biary sequece by oe digit.

7 2.4. TWO OBSERVATIONS 7 Example (Rotatio of circle). Let α (0, 1) be a oradom parameter. For ω [0, 1), let T α (ω) = (ω + α) mod 1, so T α (ω) = (ω + α) mod 1. Idetifyig [0, 1) with {z : z = 1} ad ω with e i2πω = z, we have T α (ω) = e i2πα z, a rotatio of z by agle α. This is clearly measurepreservig. If α Q, the T α is clearly o-ergodic: writig α = K/L i lowest terms, we have T L α (ω) = ω. O the other had, if α / Q, the T α is ergodic. The by 1.1.4, ((ω+kα) mod 1) A λ(a). k= Two observatios Our first observatio allows us to build complicated statioary sequeces out of simpler oes. Theorem If (X 0, X 1,... ) is a statioary sequece ad g : S S is S c -measurable, the is also a statioary sequece. Y k = g(x k, X k+1,... ) Proof. Let g k (x) = g(x k, x k+1,... ). For B S c, let A = {x : (g 0 (x), g 1 (x),... ) B}. The P (ω : (Y 0, Y 1,... ) B) = P (ω : (X 0, X 1,... ) A) = P (ω : (X k, X k+1,... ) A) = P (ω : (Y k, Y k+1,... ) B). Our secod observatio allows us to pass from two-sided statioary sequeces to oe-sided statioary sequeces via embeddig. Theorem Ay statioary sequece {X, 0} ca be embedded i a two-sided statioary sequece {Y, Z}. Proof. For each m, let (X 0,..., X m+ ) = D (Y m,..., Y ). By statioarity, this defies a cosistet family of fiite-dimesioal distributios, ragig over m ad. Hece, by the Kolmogorov extesio theorem, there exists a law of {Y, Z}.

8

9 CHAPTER 3 Statistics 310B, Lecture Notes, Wedesday, March 4, Recurrece of statioary sequeces Cosider vector space valued {X k (ω)} ad the correspodig partial sums S (ω) = k=1 X k(ω). Let R = {S 1, S 2,..., S } deote the size of the rage of {S k } which is the umber of poits visited at time ad A = {S k 0, 1 k } deote the evet of o retur to 0. Theorem Durrett Theorem 7.3.1: Proof. The proof is based o the iequalities: (3.1.1) I A (θ m ω) R k + m=1 R a.s. E[I A I θ ]. k m=1 I Ak (θ m ω) k,. Deote the left had side iequality as (b) ad the right had side iequality as (a). Divide by throughout ad take. By Birkhoff s Ergodic thm (lettig f(.) = I A (.) for (b) ad f(.) = I Ak (.) for (a)), we get from (3.1.1) that a.s. E[I A I θ ] lim if R lim sup R E[I A k I θ ]. Now takig k, we have A k A, ad by Mootoe Covergece for Coditioal Expectatio, E[I Ak I θ ] E[I A I θ ], which completes the proof. The proof of (3.1.1) follows with the observatio {θ m (ω) A k } = {S m+r S m, r = 1, 2,..., k}. Also ote that S m+r S m = m+r k=m+1 X k = S r (θ m ω). To get (a) i (3.1.1), ote that if m is such that {S m S m+1,..., S } ad m k, the θ (ω) I k, the add for safety k to cout for possibly S m for k + 1 m, cotributig to R. To get (b), ote that θ m ω A oly if m is the time of last visit by {S k } to the poit S m. These poits must be disjoit ad hece (b) of (3.1.1). Remark. ER = k=1 P(A k) Theorem Durrett Theorem 7.3.2: {X k } statioary Z-valued with E X 1 <. The (a) E[X 1 I θ ] = 0, the P(A ) = 0. (b) If P(A ) = 0, the P(S = 0 i.o.) = 1. Remark. If we take a geeral radom walk with E X 1 <, P(o retur) = 0; this is what is called recurrece. 9

10 10 3. STATISTICS 310B, LECTURE NOTES, WEDNESDAY, MARCH 4, 2015 Special Case: {X k } i.i.d., the {S } homogeeous Markov Chai o Z ad if EX 1 = 0, the this Markov Chai is recurret (but ot ecessarily positive recurret). Notice that P(X k = 1 k) = P(X k = 1 k) = 1/2 forms a Z-valued statioary sequece for which E(X 1 ) = 0, but S = w.p.1/2 ad w.p.1/2. Proof. (a) E[X 1 I θ ] = 0, the by Birkhoff s Ergodic Theorem with f(ω) = ω 1, S / a.s. E[X 1 I θ ] = 0. Notice that R 1 + max 1 k {S k } mi 1 k {S k }. Obviously S 0 = 1 max 1 k { S k } 0, hece from (3.1.1) also R 0. By Durrett Theorem 7.3.1, this implies that E[I A I θ ] = 0. By the tower property of expectatio, it follows that E[I A ] = P(A ) = 0. (b) Proof of part (b) follows by decomposig θ j (A c ) = k=1 G j,k, j 0. θ j (A c ) = {ω : θ j (ω) / A } = {S j = S j+k for some k 1} where G j,k = {S j S j+r, 1 r k 1, S j = S j+k }. By statioarity, 1 = 1 P(A ) = P(A c ) = P(θ j (A c )). Let us defie F k = G 0,k ad similar argumet shows that 1 = j=1 P(F j), further more j=1 k=1 P(F j G j,k ) = 1. This is the evet of returig to 0 atleast twice. Repeat the argumet k times ad sed r. Theorem Durrett Theorem 7.3.3: Fix A S. Let T 0 = 0, T k = if{ > T k 1 : X A} deote the time of kth retur to A. If P(T 1 ) = 1,the (a) uder P(. X 0 A), the sequece τ k = T k T k 1 is statioary 1 (b) E[T 1 X 0 A] = P(X A). Proof. To show that the sequece τ k = T k T k 1 is statioary uder P(. X 0 A), observe P(τ 1 = m, τ 2 = X 0 A) = P(τ 2 = m, τ 3 = X 0 A). We exted the oe sided statioary sequece {X, 0} to a two-sided statioary sequece {X, Z} usig Durrett Theorem Let C k = {X 1 / A,..., X k+1 / A, X k / A} The observe, ( K k=1 C k ) c = {Xk / A for K k 1}.

11 3.1. RECURRENCE OF STATIONARY SEQUENCES 11 Lettig K, we get P ( K k=1 C k) = 1. To prove statioarity, let I j,k = {i [j, k] : X i A} ad observe that: P(τ 2 = m, τ 3 =, X 0 A) = P(X 0 A, τ 1 = l, τ 2 = m, τ 3 = ) = = = l=1 P(I 0,l+m+ = {0, l, l + m, l + m + }) l=1 P(I l,m+ = { l, 0, m, m + }) l=1 P(C l, X 0 A, τ 1 = m, τ 2 = ) l=1 The secod part of the theorem follows by computig E[T 1 X 0 A] = E[τ 1 X 0 A] = P(τ 1 k X 0 A) k=1 = P(X 0 A) 1 P(C k ) = 1/P(X 0 A) k=1 sice C k are disjoit ad their uio has probability 1. Remark. If {X } is a positively recurret Markov Chai with S coutable (i.e. has a ivariat probabaility measure π) ad A = {x}, the (b) of Thm says E[T x ] = 1 π(x). Theorem Durrett Theorem 7.3.4: Suppose φ is R-measure preservig for some probability measure P ad A X. Let T A = if{ 1 : φ (ω) A}. The, (a) P(ω A, T A = ) = 0. (b) P(φ (ω) A f.o., ω A) = 0. (c) If φ is ergodic ad P(A) > 0, the P(φ (ω) A i.o.) = 1. Proof. Let B = {ω A, T A = }. Observe that ω φ m B = φ m A, but φ (ω) / A for > m, so that φ m B are pairwise disjoit. φ measure-preservig = P(φ m B) = P(B), so we must have P(B) = 0 (or P would have ifiite mass). To prove (b), ote that for ay k, φ k is measure preservig, so (a) = 0 = P(ω A, φ k (ω) / Afor all 1) P(ω A, φ m (ω) / Afor all m k). Sice the last probability is 0 for all k, (b) follows. Fially, (c) follows by observig B = {ω : φ (ω) A i.o} is ivariat ad A by (b) ad hece P(B) > 0 ad it follows by ergodicity that P(B) = 1.

12

13 CHAPTER 4 Statistics 310B, Lecture Notes, Moday, March 9, Subadditive Ergodic Theorem Birkhoff s ergodic theorem is restricted to averages. Kigma s ergodic theorem, which we ow discuss, provides the aalogous result for subadditive fuctios. Theorem (Liggett s improvemet of Kigma s ergodic theorem). Let {X m,, 0 m < } be radom variables o the same probability space. Assume (a) X 0, X 0,m + X m,, (b) {X k,(+1)k, 1} is a statioary sequece for each k, (c) the distributio of {X m,m+k, k 1} does ot deped o m, (d) E(X 0,1 ) + < ad if 1 1 E(X 0,) >. The lim E(X 0, )/ = γ where γ = if E(X 0, )/. X lim X 0, / exists almost surely ad i L 1, ad EX = γ. If all statioary sequeces i (b) are ergodic, the X = γ. Remark. Kigma s origial versio of the theorem assumes, i lieu of (b) ad (c), that the law of {X m+k,+k, 0 m < } does ot deped o k; this is a stroger assumptio. Remark. Birkhoff s ergodic theorem (Theorem 1.1.4) is a special case of Kigma s. I that case, X m, (ω) = i=m+1 f(t i ω). Assumptio (a) holds with equality, assumptios (b) ad (c) follow from Kigma s stroger statioarity assumptio, ad (d) follows from additivity ad E X 0,1 <. Remark. Ulike Birkhoff, we do ot have a ice expressio for the limitig r.v. X outside of the ergodic case Examples Example (Number of distict poits). Let S = i=1 ξ i, where {ξ k } is a statioary sequece. let X m, = {S m+1,..., S }. The clearly X 0, X 0,m + X m,. The statioarity of the {ξ k } implies Kigma s stroger assumptio, hece implies (b) ad (c). Fially, (d) holds sice 0 X 0,. The Theorem implies R / γ i the ergodic case. 13

14 14 4. STATISTICS 310B, LECTURE NOTES, MONDAY, MARCH 9, 2015 Example (Logest commo subsequece). Let {X k } ad {Y k } be two statioary ergodic sequeces. Let L m, be the logest commo subsequece betwee positios m ad, that is, L m, = max{k : X ik = Y jk, 1 k K, m < i 1 < i 2 < < i K i, m < j 1 < j 2 < < j K } Searchig for the logest commo subsequece betwee 0 ad allows more flexibility tha searchig separately betwee 0 ad m ad betwee m ad ad the cocateatig the results. Thus we have superadditivity: L 0,m + L m, L 0, so X m, = L m, satisfies assumptio (a) of Statioarity is obvious: for each k, {L k,(+1)k, 1} is a statioary sequece because each elemet of the sequece refers to a differet block of legth k, ad the distributio of L m,m+k is the same as that of L 0,k. Fially, (d) holds because 0 L 0,. Therefore L 0, / coverges almost surely ad i L 1 to sup m 1 E(L 0,m )/m. Example (Product of radom matrices). Let A 1,..., A be a statioary sequece of k k matrices with positive etries. Defie α m, (i, j) = (A m+1... A )(i, j). Because the etries of A m+1... A are all positive, we have α 0,m (i, i)α m, (i, i) α 0, (i, i). Take X m, = log α m, (1, 1); the (a) through (c) clearly hold, ad (d) will hold if E log A m (i, j) < for all i ad j; take logs i A m (1, 1) α 0, (1, 1) k 1 sup A m (i, j). m=1 The 1 log α 0,(1, 1) coverges a.s. to X. m=1 1 i,j k Example (Logest icreasig subsequece of radom permutatio). Let π be a permutatio of {1,..., }, ad defie l(π) = max{k : 1 i 1 < i 2 < < i K, 1 π(i 1 ) < π(i 2 ) < < π(i K ) }. We call l(π) the logest icreasig subsequece of π. Choosig π uiformly at radom from all permutatios, we are iterested i the asymptotics of l(π). We proceed by cosiderig a easier Poissoized versio of the problem. If we choose i.i.d. uiform poits (X i, Y i ) i a two-dimesioal box, the all orderigs are equally likely. We may thus look for a maximal mootoe sequece (X il, Y il ) where X il ad Y il are icreasig. Istead of takig to be fixed, we cosider poits geerated by a Poisso poit process of rate 1 i the plae. The the umber of poits i [0, ] 2 is N() Pois( 2 ), which cocetrates at 2. The maximal mootoe sequece R m, i the box [m, ] 2 satisfies R 0, R 0,m + R m,. By 4.1.1, l (π)/ a.s. γ. It turs out γ = 2 i this case.