1 Convergence in Probability and the Weak Law of Large Numbers

Transcription

1 Advaced Probability Overview Sprig Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec 1.5 ad 2.2 of Durrett. 1 Covergece i Probability ad the Weak Law of Large Numbers The Weak Law of Large Numbers is a statemet about sums of idepedet radom variables. Before we state the WLLN, it is ecessary to defie covergece i probability. Defiitio 1 (Covergece i Probability). We say Y coverges i probability to Y ad write Y P Y if, > 0, P (ω : Y (ω) Y (ω) > ) 0,. Theorem 2 (Weak Law of Large Numbers). Let X, X 1, X 2,... be a sequece of idepedet, idetically distributed (i.i.d.) radom variables with E X < ad defie S = X 1 + X X. The S P EX. The proof of WLLN makes use of the idepedet coditio through the followig basic lemma. Lemma 3. Let X 1 ad X 2 be idepedet radom variables. Let f i (i = 1, 2) be measurable fuctios such that E f i (X i ) < for i = 1, 2, the Ef 1 (X 1 )f 2 (X 2 ) = Ef 1 (X 1 )Ef 2 (X 2 ). The proof of Lemma 3 follows from Lemma 34 of lecture otes set 4 ad Fubii s Theorem. The followig corollary will be used i our proof of WLLN. Corollary 4. If X 1 ad X 2 are idepedet radom variables, ad Var(X i ) <, the Var(X 1 + X 2 ) = Var(X 1 ) + Var(X 2 ). Proof: [Proof of WLLN] I this proof, we employ the commo strategy of first provig the result uder a L 2 coditio (i.e. assumig that the secod momet is fiite), ad the usig trucatio to get rid of the extraeous momet coditio. 1

2 First, we assume EX 2 <. Because the X i are iid, Var S By Chebychev s iequality, > 0, S Pr EX > = 1 2 i=1 Var(X i ) = Var(X). 1 Var S = Var(X) Thus, S P EX uder the fiite secod momet coditio. To trasitio from L 2 to L 1, we use trucatio. For 0 < t < let The, we have X k = X tk + Y tk ad X tk = X k 1 ( Xk t) Y tk = X k 1 ( Xk >t) S = 1 X tk + 1 Y tk = U t + V t Because k Y tk k Y tk, we have 1 E ad by DCT, Y tk 1 E Y tk = E( X 1 ( X >t) ) E( X 1 ( X >t) ) 0, t. Fix 1 > > 0, for ay 0 δ 1 we ca choose t such that E X 1 ( X >t) = E Yt1 < δ/6. Let µ t = E(X t1 ) ad µ = E(X). Because 0 δ 1, the we also have µ t µ E(Y t1 ) < δ/6 < /3. Let B = { U t µ t > /3} ad C = { V t > /3}. Notig that E(X 2 tk ) t2 <, we ca apply the Weak Law of Large Numbers to U t. Thus, we choose N > 0 such that > N, 2

3 Now, by Markov s iequality, we also have Pr(B ) = Pr( U t µ t > /3) < δ/2. Pr(C ) = Pr( V t > /3) 3E V t 3E Y t1 δ/2. But o B c C c = (B C ) c, we have U t µ t /3 ad V t /3, ad therefore Thus, > N, S µ U t µ t + V t + µ t µ /3 + /3 + /3. S Pr EX > Pr(B C ) δ. 2 Covergece of Radom Variables: almost sure, i probability ad i L p Let (Ω, F, P ) be a probability space. Recall that a sequece {X } of radom variables coverges to the radom variable X (all defied o that same probability space) whe P ω : lim X (ω) = X(ω) = 1, or, equivaletly, whe for each > 0, P ({ω : X X > i.o.}) = 0. Recall that the previous statemet ca be expressed as P lim sup A, = P A k, = 0, where A, = {ω : X (ω) X(ω) > }. For p 1, we say that X coverges to X i L p whe k lim E [ X X p ] = 0. Sice the L p orms are icreasig i p, covergece i L p implies covergece i L r for r < p. I the previous sectio we itroduced covergece i probability. We ow discuss the relatioship amog differet otios of covergece. Fact: Covergece i L p is differet from covergece a.s. 3

4 Example 5. Let Ω = (0, 1) with P beig Lebesgue measure. Cosider the sequece of fuctios 1, I (0,1/2], I (1/2,1), I (0,1/3], I (1/3,2/3],.... These fuctios coverge to 0 i L p for all fiite p sice the itegrals of their absolute values go to 0. But they clearly do t coverge to 0 a.s. sice every ω has f (ω) = 1 ifiitely ofte. These fuctios are i L, but they do t coverge to 0 i L. because their L orms are all 1. Example 6. Let Ω = (0, 1) with P beig Lebesgue measure. fuctios 0 if 0 < ω < 1/, f (ω) = 1/ω if 1/ ω < 1. Cosider the sequece of Each f is i L p for all p, ad lim f (ω) = 1/ω a.s. But the limit fuctio is ot i L p for eve a sigle p. Clearly, {f } =1 does ot coverge i L p. Example 7. Let Ω = (0, 1) with P beig Lebesgue measure. fuctios if 0 < ω < 1/, f (ω) = 0 otherwise. Cosider the sequece of The f coverges to 0 a.s. but ot i L p sice f p dp = p 1 for all ad fiite p. I this case, the a.e. limit is i L p, but it is ot a L p limit. Oddly eough covergece i L does imply covergece a.e., the reaso beig that L covergece is almost uiform covergece. Propositio 8. Let (Ω, F, µ) be a measure space. The f coverges to f i L if ad oly if there exists a measurable set A such that µ(a c ) = 0 ad lim f = f, uiformly o A. We ca exted covergece i probability to covergece i measure. Defiitio 9 (Covergece i Measure). Let (Ω, F, µ) be a measure space ad let f ad {f } =1 be measurable fuctios that take values i a metric space with metric d. We say that f coverges to f i measure if, for every > 0, lim µ({ω : d(f (ω), f(ω)) > }) = 0. Whe µ is a probability, covergece i measure is called covergece i probability, deoted f P f. Covergece i measure is differet from a.e. covergece. Example 5 is a classic example of a sequece that coverges i measure (i probability i that example) but ot a.e. Here is a example of a.e. covergece without covergece i measure (oly possible i ifiite measure spaces). 4

5 Example 10. Let Ω = IR with µ beig Lebesgue measure. Let f (x) = I [, ) (x) for all. The f coverges to 0 a.e. [µ]. However, f does ot coverge i measure to 0, because µ({ f > }) = for every. Example 7 is a example of covergece i probability but ot i L p. Ideed covergece i probability is weaker tha L p covergece. Propositio 11. If X X p 0 i L p for some p > 0, the X P X. Covergece i probability is also weaker tha coverges a.s. Lemma 12. If X X a.s., the X P X. Proof: Let > 0. Let C = {ω : lim X (ω) = X(ω)}, ad defie C = {ω : d(x k (ω), X(ω)) <, for all k }. Clearly, C =1 C,. Because Pr(C) = 1 ad {C } =1 is a icreasig sequece of evets, Pr(C ) 1. Because {ω : d(x (ω), X(ω)) > } C C, Pr(d(X, X) > ) 0. A partial coverse of this lemma is true ad will be proved later. Lemma 13. If X P X, the there is a subsequece {X k } such that X k a.s. X. There is a eve weaker form of covergece that we will discuss i detail later i the course. Defiitio 14 (Covergece i Distributio). A otio of covergece of a probability distributio o R (or more geeral space). We say X D X if Pr(X x) Pr(X x) for all x at which the RHS is cotiuous. Note that this is ot really a otio of covergece of radom variables, but the covergece of their distributio fuctios. This weak covergece appears i the cetral limit theorem. Fact 15. X D X Ef(X ) Ef(X) for all bouded ad cotiuous fuctio f. The relatioship betwee modes of covergece ca be summarized as follows. 5

6 X a.s. X X L X P X P X X D X 6