Itroductio to robability Ariel Yadi Lecture 2 *** Ja. 7 ***. Covergece of Radom Variables As i the case of sequeces of umbers, we would like to talk about covergece of radom variables. There are may ways to say that a sequece of radom variables approximates some limitig radom variable. We will talk about 4 such possibilities. 2. ad L p Covergece We have already see some type of covergece; amely, X X if X (ω) X(ω) for all ω Ω. This is a strog type of covergece; we are usually ot iterested i chages that have probability 0, so we will defie the followig type of covergece: Let (X ) be a sequece of radom variables o some probability space (Ω, F, ). We have already see that lim if X, lim sup X are both radom variables. Thus, the set {ω : lim if X (ω) = lim sup X (ω)} is a evet i F. Hece we ca defie: Defiitio 2.. Let (X ) be a sequece of radom variables o (Ω, F, ). Let X be a radom variable. We say that (X ) coverges to X almost surely, or, deoted covergece X if [lim X = X] = ; that is, if { [ ω : } lim X (ω) = X(ω) ] =. Aother type of covergece is a type of average covergece: Defiitio 2.2. Let p > 0. Let (X ) be a sequece of radom variables o (Ω, F, ) such that E[ X p ] < for all. Let X be a radom variable such that E[ X p ] <. We say that (X ) coverges to X i L p, deoted L p covergece X L p
2 if lim E[ X X p ] = 0. the limit is uique I the exercises we will show that for 0 < p < q, L q covergece implies L p covergece. We will also show that the limit is uique; that is L Exercise 2.3. Suppose that X ad X p Y. Show that [X = Y ] =. Suppose that X L p L X ad X q Y. Show that [X = Y ] =. L p Example 2.4. discrete radom variable So X has desity ote that Fix p > 0. Let (Ω, F, ) be uiform measure o [0, ]. For all let X be the /p ω [0, /] X (ω) = 0 otherwise. s = /p f X (s) = s = 0 0 otherwise E[ X q ] = /p q = q/p. Thus, if 0 < q < p, the E[ X 0 q ] 0, so X L q However, for q p we have that E[ X 0 q ] for all, so X 0 (ad thus X for ay X, sice the limit must be uique). We claim that X X (ω) = 0, ad thus X (ω) 0. Thus, Ideed, let ω [0, ]. If ω > 0 the for all > ω, we get that L q L q so X This example has show that [{ω : X (ω) 0}] [{0}] = 0, We ca have L p covergece without L q covergece, for q > p. We ca have covergece without L p covergece. L p Example 2.5. Thus, for all ad all p > 0, For all let X be mutually idepedet Ber(/) radom variables. E[ X p ] = 0 so X L p
3 O the other had, let A = { X > /2}. So (A ) is a sequece of mutually idepedet evets, ad [A ] = /. Thus, sice [A ] =, usig the secod Borel-Catelli Lemma, [lim sup A ] =. That is, with probability, there exist ifiitely may such that X > /2. Thus, [lim sup X /2] =. So we caot have that X Sice the limit must be uique, we caot have X for ay X. We coclude that: It is possible for (X ) to coverge i L p for all p, but ot to coverge 3. Covergece i robability Defiitio 2.6. Let (X ) be a sequece of radom variables o (Ω, F, ). We say that (X ) coverges i probability to X, deoted covergece i prob. X if for all ε > 0, Example 2.7. lim [ X X > ε] = 0. Suppose that X. That is, [ X X 0] =. Fix ε > 0. For ay let A = { X X > ε}. ote that if A occurs for ifiitely may, the lim sup X X ε > 0. Thus, by Fatou s Lemma, Thus, ad so X 0 lim sup. [A ] = [lim sup lim A ] = [A i.o.] [lim sup X X > 0] = 0. [ X X > ε] = lim sup [A ] = 0, That is: covergece implies covergece i probability. Example 2.8. Assume that X L p. The, for ay ε > 0, by Markov s iequality, [ X X > ε] E[ X X p ] ε p 0. L p
4 Thus, X. That is, covergece i L p implies covergece i probability. limit is uique The followig is i the exercises: Exercise 2.9. Let X I this sectio we will prove ad X Y. Show that [X = Y ] =. 4. The Law of Large umbers ** Ja. 9 *** Theorem 2.0 (Strog Law of Large umbers). Let X, X 2,..., X,..., be a sequece of idetically distributed ad mutually idepedet radom variables, such that E[X ] = 0 ad E[X 2 ] <. For each let The, 4.. Kolmogorov s iequality. S = S X. Lemma 2. (Kolmogorov s iequality). Let X, X 2,..., X,..., be a sequece of mutually idepedet radom variables, such that E[X ] = 0 ad E[X 2 ] <. For each let The, for ay λ > 0, roof. Fix λ > 0 ad let S = We will make a few observatios. X ad M = max S. [M λ] E[X2 ] λ 2. A = {S < λ, S 2 < λ,..., S < λ, S λ}. First ote that sice (X ) are mutually idepedet, they are ucorrelated, so E[S ] = 0 ad Var[S ] = E[S 2 ] = E[X]. 2 ext, ote that M λ if ad oly if there exists such that S λ ad S k < λ for all k <. That is, M λ implies that there exists such that S 2 A λ 2.
5 Sice (X,..., X ) are idepedet of (X +,..., X ), we have that the radom variable S A is idepedet of X + + X +2 + + X = S S. Thus, for ay we have that E[S 2 A ] = E[(S S + S ) 2 A ] = E[(S S ) 2 A ] + E[S 2 A ] + 2 E[(S S ) S A ] E[S 2 A ], where we have used that E[(S S ) S A ] = E[S S ] E[S A ] = 0 by idepedece. We ow have that E[S] 2 E[S 2 {M λ}] = E[ A S 2 ] E[S 2 A ] ote that by Boole s iequality, sice M λ implies that there exists such that S 2 A λ 2, [M λ] [S 2 A λ 2 ] λ 2 E[S 2 A ] λ 2 E[S2 ]. 4.2. Kroecker s Criterio. Lemma 2.2. Suppose that x,..., x,..., is a sequece of umbers such that x coverges. The, lim x = 0. roof. Let s = x. So s s := x. Thus, also s s. ote that for all, x = (s s ) (where we set s 0 = 0). Thus, x = (s + (( ) )s + (( 2) ( ))s 2 + + ( 2)s ) = S s s s = 0.
6 4.3. roof of Law of Large umbers. Lemma 2.3. Let X, X 2,..., X,..., be a sequece of mutually idepedet radom variables such that E[X ] = 0 for all. If k= Var[X k] < the [ ] coverges =. roof. Let S = k= X k. k= X k For every ω Ω, k= X k(ω) coverges if ad oly if S (ω) coverges, which is if ad oly if (S (ω)) form a Cauchy sequece. Thus, it suffices to show that Let > 0, ad cosider Kolmogorov s iequality gives that [(S ) is a Cauchy sequece ] =. S +m S = X + + X +2 + + X +m. [ max m S +m S λ] λ 2 E[X+k] 2 λ 2 k= k E[X 2 +k]. Takig o the left-had side, usig cotiuity of probability for the icreasig sequece of evets, [sup S +m S λ] λ 2 k E[X 2 +k]. This last term is the tail of the coverget series k E[X2 k ]. Thus, for ay r > 0 there exists (r) so that k E[X2 k ] < r 3, which implies that [if sup S +m S > r ] [sup S (r)+m S (r) > r ] < r. ow, the evets { if sup S +m S > r } are icreasig i r, so takig r with cotiuity of probability, That is, [if [if sup S +m S > 0] 0. sup S +m S = 0] =. ow let ω Ω be such that if sup S +m (ω) S (ω) = 0. This implies that for all ε > 0 there exists = (ε, ω) such that sup S +m (ω) S (ω) < ε/2.
7 Thus, for ay > (ε, ω) ad ay m, S +m (ω) S (ω) S ++m (ω) S (ω) + S + (ω) S (ω) < ε. That is, for ay ε > 0 there exists = (ε, ω) such that for all > (ε, ω) ad all m, S +m (ω) S (ω) < ε; that is, for this ω, (S (ω)) is a Cauchy sequece. We have show that {ω : if sup S +m (ω) S (ω) = 0} {ω : (S (ω)) is a Cauchy sequece }. Sice the first evet has probability, so does the secod evet, ad we are doe. The proof of the law of large umbers is ow immediate: roof of Theorem 2.0. By Kroecker s Criterio is suffices to show that [ X coverges] =. By the above Lemma it suffices to show that Var[X /] <. Sice Var[X /] = 2, this is immediate. Example 2.4. The Cetral Bureau of Statistics wats to asses how may citizes use the trai system. They poll radomly ad idepedetly chose citizes, ad set X j = if the j-th citize uses the trai, ad X j = 0 otherwise. Ay citize is equally likely to be chose, ad all are idepedet. As the sample mea j= coverges to E[X j ] = E[X ] which is the probability that a radomly chose citize uses the trai system. Sice we choose each citize uiformly, this probability is exactly the umber of citizes that use the trai, divided by the total umber of citizes. Thus, for large, C is a good approximatio for the umber of citizes that use the trai system, where C is the umber of citizes. X j j= X j