Introduction to Probability. Ariel Yadin. Lecture 7

Transcription

1 Itroductio to Probability Ariel Yadi Lecture 7 1. Idepedece Revisited 1.1. Some remiders. Let (Ω, F, P) be a probability space. Give a collectio of subsets K F, recall that the σ-algebra geerated by K, is *** Dec. 6 *** σ(k) = {G : G is a σ-algebra, K G}, ad this σ-algebra is the smallest σ-algebra cotaiig K. σ(k) ca be thought of as all possible iformatio that ca be geerated by the sets i K. Recall that a collectio of evets (A ) are mutually idepedet if for ay fiite umber of these evets A 1,..., A we have that P(A 1 A ) = P(A 1 ) P(A ). We ca also defie the idepedece of families of evets: Defiitio 7.1. Let (Ω, F, P) be a probability space. Let (K ) be a collectio of families of evets i F. The, (K ) are mutually idepedet if for ay fiite umber of families from this collectio, K 1,..., K ad ay evets A 1 K 1, A 2 K 2,..., A K, we have that P(A 1 A ) = P(A 1 ) P(A ). For example, we would like to thik that (A ) are mutually idepedet, if all the iformatio from each evet i the sequece is idepedet from the iformatio from the other evets i the sequece. This is the cotet of the ext propositio. Propositio 7.2. Let (A ) be a sequece of evets o a probability space (Ω, F, P). The, (A ) are mutually idepedet if ad oly if the σ-algebras (σ(a )) are mutually idepedet. It is ot difficult to prove this by iductio: Proof. By iductio o we show that (σ(a 1 ),..., σ(a ), {A +1 },...) are mutually idepedet. The base = 0 is just the assumptio. 1

2 2 So assume that (σ(a 1 ),..., σ(a 1 ), {A }, {A +1 },...) are mutually idepedet. Let 1 < 2 <... < k < k+1 <... < m be a fiite umber of idices such that k < < k+1. Let B j σ(a j ) for j = 1,..., k, ad B j = A j for j = k + 1,..., m. Let B σ(a ). If B = the P(B 1 B B) = 0 = P(B 1 ) P(B ) P(B). If B = Ω the P(B 1 B B) = P(B 1 B ) = P(B 1 ) P(B ), by the iductio hypotheses. If B = A the this also holds by the iductio hypotheses. So we oly have to deal with the case B = A c. I this case, for X = B 1 B, P(X B) = P(X \ (X A )) = P(X) P(X A ) = P(X)(1 P(A )) = P(X) P(B), where we have used the iductio hypotheses to say that X ad A are idepedet. Sice this holds for ay choice a a fiite umber of evets, we have the iductio step. However, we will take a more widy road to get stroger results... (Corollary 7.8) 2. π-systems ad Idepedece Defiitio 7.3. Let K be a family of subsets of Ω. We say that K is a π-system if K ad K is closed uder itersectios; that is, for all A, B K, A B K. We say that K is a Dyki system (or λ-system) if K is closed uder set complemets ad coutable disjoit uios; that is, for ay A K ad ay sequece (A ) i K of pairwise disjoit subsets, we have that A c K ad A K. The mai goal ow is to show that probability measures are uiquely defied oce they are defied o a π-system. This is the cotet of Theorem 7.7. Propositio 7.4. If F is Dyki system o Ω ad F is also a π-system o Ω, the F is a σ algebra. Proof. Sice F is a Dyki system, it is closed uder complemets, ad Ω = c F. Let (A ) be a sequece of subsets i F. Set B 1 = A 1, C 1 =, ad for > 1, C = 1 j=1 A j ad B = A \ C.

3 Sice F is a π-system ad closed uder complemets, C = ( 1 j=1 Ac j )c F, ad so B = A C c F. Sice (B ) is a sequece of pairwise disjoit sets, ad sice F is a Dyki system, A = B F. 3 Propositio 7.5. If (D α ) α is a collectio of Dyki systems (ot ecessarily coutable). The D = α D α is a Dyki system. *** leave as exercise *** Proof. Sice D α for all α, we have that D. If A D, the A D α for all α. So A c D α for all α, ad thus A c D. If (A ) is a coutable sequece of pairwise disjoit sets i D, the A D α for all α ad all. Thus, for ay α, A D α. So A D. Lemma 7.6 (Dyki s Lemma). If a Dyki system D cotais a π-system K, the σ(k) D. Proof. Let F = {D : D is a Dyki system cotaiig K}. So F is a Dyki system ad K F D. We will show that F is a σ algebra, so σ(k) F D. Suppose we kow that F is closed uder itersectios (which is Claim 3 below). Sice K F, we will the have that F is a π-system. Beig both a Dyki system ad a π system, F is a σ algebra. Thus, to show that F is a σ algebra, it suffices to show that F is closed uder itersectios. Note that F is closed uder complemets (because all Dyki systems are). Claim 1. If A B are subsets i F, the B \ A F. Proof. If A, B F, the sice F is a Dyki system, also B c F. Sice A B, we have that A, B c are disjoit, so A B c F ad so B \ A = (A c B) = (A B c ) c F. Claim 2. For ay K K, if A F the A K F. Proof. Let E = {A : A K F}. Let A E ad (A ) be a sequece of pairwise disjoit subsets i E. Sice K F ad A K F, by Claim 1 we have that A c K = K \ (A K) F. So A c E.

4 4 Sice (A K) is a sequece of pairwise disjoit subsets i F, we get that A K = (A K) F. So we coclude that E is a Dyki system. Sice K is closed uder itersectios, E cotais K. Thus, by defiitio F E. So for ay A F we have that A E, ad A K F. Claim 3. For ay B F, if A F the A B F. Proof. Let E = {A : A B F}. Let A E ad (A ) be a sequece of pairwise disjoit subsets i E. Sice B F ad A B F, by Claim 1 we have that A c B = B \ (A B) F. So A c E. Sice (A B) is a sequece of pairwise disjoit subsets i F, we get that A B = (A B) F. So we coclude that E is a Dyki system. By Claim 2, K is cotaied i E. So by defiitio, F E. Sice F is closed uder itersectios, this completes the proof. The ext theorem tells us that a probability measure o (Ω, F) is determied by it s values o a π-system geeratig F. Theorem 7.7 (Uiqueess of Extesio). Let K be a π system o Ω, ad let F = σ(k) be the σ algebra geerated by K. Let P, Q be two probability measures o (Ω, F), such that for all A K, P (A) = Q(A). The, P (B) = Q(B) for ay B F. Proof. Let D = {A F : P (A) = Q(A)}. So K D. We will show that D is a Dyki system, ad sice it cotais K, the by Dyki s Lemma it must cotai F = σ(k). Of course P (Ω) = 1 = Q(Ω), so Ω D. If A D, the P (A c ) = 1 P (A) = 1 Q(A) = Q(A c ), so A c D. Let (A ) be a sequece of pairwise disjoit sets i D. The, P ( A ) = P (A ) = Q(A ) = Q( A ). So A D.

5 Corollary 7.8. Let (Ω, F, P) be a probability space. Let (Π ) be a sequece of π-systems, ad let F = σ(π ). The, (Π ) are mutually idepedet if ad oly if (F ) are mutually idepedet. 5 Proof. We will prove by iductio o that for ay 0, the collectio (F 1, F 2,..., F, Π +1, Π +2,..., ) are mutually idepedet. For = 0 this is the assumptio. For > 1, let 1 < 2 <... < k < k+1 <... < m be a fiite umber of idices such that k < < k+1. Let A j F j, j = 1,..., k ad A j Π j, j = k + 1,..., m. For ay A F, if P(A 1 A m ) = 0 the A is idepedet of A 1 A m. So assume that P(A 1 A m ) > 0. For ay A F defie the probability measure P (A) := P(A A 1 A m ). By iductio, the collectio (F 1,..., F 1, Π, Π +1,...) are mutually idepedet, so P (A) = P(A) for ay A Π. Sice Π is a π-system geeratig F, we have by Theorem 7.7 that P (A) = P(A) for ay A F. Sice this holds for ay choice of a fiite umber of evets A 1,..., A m, we get that the collectio (F 1,..., F 1, F, Π +1,...) are mutually idepedet. *** Dec. 8 *** Corollary 7.9. Let (Ω, F, P) be a probability space, ad let (A ) be a sequece of mutually idepedet evets. The, the σ-algebras (σ(a )) are mutually idepedet. 3. Idepedet Radom Variables Let X : (Ω, F) (R, B) be a radom variable. Recall that i the defiitio of a radom variable we require that X is a measurable fuctio; i.e. for ay Borel set B B we have X 1 (B) F. We wat to defie a σ-algebra that is all the possible iformatio that ca be ifered from X. Defiitio Let (Ω, F, P) be a probability space ad let (X α ) α I : Ω R be a collectio of radom variables. The σ-algebra geerated by (X α ) α I is defied as σ(x α : α I) := σ( { X 1 α (B) : α I, B B } ).

6 6 Note that for a radom variable X : Ω R, the collectio { X 1 (B) : B B } is a σ-algebra, so σ(x) = { X 1 (B) : B B }. We ow defie idepedet radom variables. Defiitio Let (X ), (Y ) be a collectio of radom variables o some probability space (Ω, F, P). We say that (X ) are mutually idepedet if the σ-algebras (σ(x )) are mutually idepedet. We say that (X ) are idepedet of (Y ) if the σ-algebras σ((x ) ) ad σ((y ) ) are idepedet. ** keep this for ater *** A importat property (that is a cosequece of the π-system argumet above): Propositio Let (X ) be a collectio of radom variables o (Ω, F, P). (X ) are mutually idepedet, if for ay fiite umber of radom variables from the collectio, X 1,..., X, ad ay real umbers a 1, a 2,..., a, we have that P[X 1 a 1, X 2 a 2,..., X a ] = P[X 1 a 1 ] P[X 2 a 2 ] P[X a ]. Proof. Let X 1,..., X be a fiite umber of radom variables. Defie two probability measure o the Borel sets of R : For ay B B let P 1 (B) = P((X 1,..., X ) B) ad P 2 (B) = P(X 1 π 1 B) P(X π B), where π j is the projectio oto the j-th coordiate. Sice these two measure are idetical o the π-system {(, a 1 ] (, a ] : a 1, a 2,..., a R}, ad sice this π-system geerates all Borel sets o R, we get that P 1 = P 2 for all Borel sets o R. Thus, if A j σ(x j ), j = 1,...,, the for all j, A j = X 1 j (B j ) for some Borel set o R, so P(A 1 A ) = P((X 1,..., X ) B 1 B ) = P(X 1 B 1 ) P(X 1 B ) = P(A 1 ) P(A ). Example We toss two dice. Y is the umber o the first die ad X is the sum of both dice. Here the probability space is the uiform measure o all pairs of dice results, so Ω = {1, 2,..., 6} 2.

7 7 What is σ(x)? σ(x) = {{(x, y) Ω : x + y A, A {2,..., 12}}}. (We have to kow that all subsets of {2,..., 12} are Borel sets. This follows from the fact that {r} = (r 1/, r].) O the other had, Now ote σ(y ) = {{(x, y) Ω : x A, A {1,..., 6}}}. P[X = 7, Y = 3] = 1 = P[X = 7] P[Y = 3]. 36 This is maily due to P[X = 7] = 1/6. Similarly, for ay y {1,..., 6}, we have that the evets {X = 7} ad {Y = y} are idepedet. Y! Are X ad Y idepedet radom variables? NO! For example, P[X = 6, Y = 3] = = P[X = 6] P[Y = 3]. 6 For idepedece we eed all iformatio from X to be idepedet of the iformatio from We coclude this lecture with 4. Secod Borel Catelli Lemma Lemma 7.14 (Secod Borel-Catelli Lemma). Let (A ) be a sequece of mutually idepedet evets. If P(A ) = the P(lim sup A ) = P(A i.o.) = 1. Proof. It suffices to show that P(lim if A c ) = 0. For fixed m > ote that by idepedece m P( A c k) = k= m Sice lim m k= Ac k = k Ac k, we have that m (1 P(A k )). k= P( m A c k) = lim P(A k )) = m k k=(1 k (1 P(A k )) exp ( P(A k ) ), k where we have used 1 x e x.

8 8 Sice P(A ) =, we have that k P(A k) = for all fixed. Thus, P( k Ac k ) = 0 for all fixed, ad Example P(lim if A c ) = P( A c k) = P(lim k k A c k) = lim P( A c k) = 0. k A biased coi is tossed ifiitely may times, all tosses mutually idepedet. What is the probability that the sequece appears ifiitely may times? Let p be the probability the coi s outcome is 1. Let a be the outcome of the -th toss. let A be the evet that a = 0, a +1 = 1, a +2 = 0, a +3 = 1, a +4 = 0. Sice the tosses are mutually idepedet, we have that the sequece of evets (A 5+1 ) 0 are mutually idepedet. Sice P(A 5+1 ) = p 2 (1 p) 3 > 0, we have that =0 P(A 5+1) =. So the Borel-Catelli Lemma tells us that P(A 5+1 i.o.) = 1. Sice {A 5+1 i.o.} implies {A i.o.} we are doe.