Itroductio to Probability Ariel Yadi Lecture 2 1. Discrete Probability Spaces Discrete probability spaces are those for which the sample space is coutable. We have already see that i this case we ca take all subsets to be evets, so F = 2 Ω. We have also implicitly see that due to additivity o disjoit sets, the probability measure P is completely determied by its value o sigletos. That is, let (Ω, 2 Ω, P) be a probability space where Ω is coutable. If we deote p(ω) = P({ω}) for all ω Ω, the for ay evet A Ω we have that A = ω A {ω}, ad this is a coutable disjoit uio. Thus, P(A) = ω A P({ω}) = ω A p(ω). Exercise 2.1. Let Ω = {1, 2,..., } ad defie a probability measure o (Ω, 2 Ω ) by (1) P({ω}) = 1 (e 1)ω!. (2) P ({ω}) = 1 3 (3/4)ω. (Exted usig additivity o disjoit uios.) Show that both uiquely defie probability measures. Solutio. We eed to show that P(Ω) = 1, ad that P is additive o disjoit uios. Ideed, 1 P(Ω) = P({ω}) = (e 1)ω! = 1. ω=1 P (Ω) = 1 3 ω=1 (3/4) ω = 1. ω=1 Now let (A ) be a sequece of mutually disjoit evets ad let A = A. Usig the fact that ay subset is the disjoit uio of the siglto composig it, we have that ω A if ad oly if there exists a uique (ω) such that ω A. Thus, P(A) = ω A P({ω}) = P({ω}) = P(A ). ω A The same for P. 1
2 The ext propositio geeralizes the above examples, ad characterizes all discrete probability spaces. Propositio 2.2. Let Ω be a coutable set. Let p : Ω [0, 1], such that ω Ω p(ω) = 1. The, there exists a uique probability measure P o (Ω, 2 Ω ) such that P({ω}) = p(ω) for all ω Ω. (p as above is somtimes called the desity of P.) Proof. Let A Ω. Defie P(A) = ω A p(ω). We have by the assumptio o p that P(Ω) = ω Ω p(ω) = 1. Also, if (A ) is a sequece of evets that are mutually disjoit, ad A = A is their uio, the, for ay ω A there exists such that ω A. Moreover, this is uique, sice A A m = for all m, so ω A m for ll m. So P(A) = ω A p(ω) = This shows that P is a probability measure o (Ω, 2 Ω ). p(ω) = P(A ). ω A For uiquess, let P : 2 Ω [0, 1] be a probability measure such that P ({ω}) = p(ω) for all ω Ω. The, for ay A Ω, P (A) = ω A P ({ω}) = ω A p(ω) = P(A). ** Nov. 8 *** Example 2.3. (1) A simple but importat example is for fiite sample spaces. Let Ω be a fiite set. Suppose we assume that all outcomes are equally likely. That is P({ω}) = 1/ Ω for all ω Ω, ad so, for ay evet A Ω we have that P(A) = A / Ω. It is simple to verify that this is a probability measure. This is kow as the uiform measure o Ω. (2) We throw two dice. What is the probability the sum is 7? What is the probability the sum is 6? Solutio. The sample space here is Ω = {1, 2,..., 6} 2 = {(i, j) : 1 i, j 6}. Sice we assume the dice are fair, all outcomes are equally likely, ad so the probability measure
3 is the uiform measure o Ω. Now, the evet that the sum is 7 is A = {(i, j) : 1 i, j 6, i + j = 7} = {(1, 6), (2, 5), (3, 4), (4, 3), (5, 2), (6, 1)}. So P(A) = A / Ω = 6/36 = 1/6. The evet that the sum is 6 is B = {(i, j) : 1 i, j 6, i + j = 6} = {(1, 5), (2, 4), (3, 3), (4, 2), (5, 1)}. So P(B) = 5/36. (3) There are 15 balls i a jar, 7 black balls ad 8 white balls. Whe removig a ball from the jar, ay ball is equally likely to be removed. Shir puts her had i the jar ad takes out two balls, oe after the other. What is the probablity Shir removes oe black ball ad oe white ball. Solutio. First, we ca thik of the differet balls beig represeted by the umbers 1, 2,..., 15. So the sample space is Ω = {(i, j) : 1 i j 15}. Note that the size of Ω is Ω = 15 14. Sice it is equally likely to remove ay ball, we have that the probability measure is the uiform measure o Ω. Let A be the evet that oe ball is black ad oe is white. P(A) = A / Ω? How ca we compute We ca split A ito two disjoit evets, ad use additivity of probability measures: Let B be the evet that the first ball is white ad the secod ball is black. Let B be the evet that the first ball is black ad the secod ball is white. It is immediate that B ad B are disjoit. Also, A = B B. Thus, P(A) = P(B) + P(B ), ad we oly eed to compute the sizes of B, B. Note that the umber of pairs (i, j) such that ball i is black ad ball j is white is 7 8. So B = 7 8. Similarly, B = 8 7. All i all, P(A) = P(B) + P(B ) = 7 8 15 14 + 8 7 15 14 = 8 15. (4) A deck of 52 cards is shuffelled, so that ay order is equally likely. The top 10 cards are distributed amoug 2 players, 5 to each oe. What is the probability that at least oe player has a royal flush (10-J-Q-K-A of the same suit)?
4 Solutio. Each player receives a subset of the cards of size 5, so the sample space is Ω = {(S 1, S 2 ) : S 1, S 2 C, S 1 = S 2 = 5, S 1 S 2 = }, where C is the set of all cards (i.e. C = {A, 2,..., A, 2,..., }). There are ( ) 52 5 ways to choose S 1 ad ( ) 47 5 ways to choose S2, so Ω = ( ) ( 52 5 42 ) 5. Also, every choice is equally likely, so P is the uiform measure. Let A i be the evet that player i has a royal flush (i = 1, 2). For s {,,, }, let B(i, s) be the evet that player i has a royal flush of the suit s. So A i = s B(i, s). B(i, s) is the evet that S i is a specific set of 5 cards, so B(i, s) = ( 47 5 ) for ay choice of i, s. Thus, P(A i ) = B(i, s) = ( 4 Ω 47 ). s 5 Now we use the iclusio-exclusio priciple: P(A) = P(A 1 A 2 ) = P(A 1 ) + P(A 2 ) P(A 1 A 2 ). The evet A 1 A 2 is the evet that both players have a royal flush, so A 1 A 2 = 4 3 sice there are 4 optios for the first player s suit, ad the 3 for the secod. Altogether, ( ) P(A) = ( 4 47 ) + ( 4 47 ) ( 4 3 52 ) ( 5 5 5 47 ) = ( 8 47 ) 1 ( 12 52 ). 5 5 5 Exercise 2.4. Prove that there is o uiform probability measure o N; that is, there is o probability measure o N such that P({i}) = P({j}) for all i, j. Solutio. Assume such a probability measure exists. By the defiig properties of probability measures, A cotradictio. 1 = P(N) = P({j}) = p 1 =. j=0 Exercise 2.5. A deck of 52 card is ordered radomly, all orders equally likely. j=1 what is the probability that the 14th card is a ace? What is the probability that the first ace is at the 14th card? Solutio. The sample space is all the possible orderigs of the set of cards C = {A, 2,..., A, 2,..., }. So Ω is the set of all 1-1 fuctios f : {1,..., 52} C, where f(1) is the first card, f(2) the secod, ad so o. So Ω = 52!. The measure is the uiform oe.
5 Let A be the evet that the 14th card is a ace. Let B be the evet that the first ace is at the 14th card. A is the evet that f(14) is a ace, ad there are 4 choices for this ace, so if A is the set of aces, A = {f Ω : f(14) A}, so A = 4 51!. Thus, P(A) = 4 52 = 1 13. B is the evet that f(14) is a ace, but also f(j) is ot a ace for all j < 14. Thus, B = {f Ω : f(14) A, j < 14 f(j) A}. So B = 4 48 47 36 38! = 4 48! 38 37 36, ad P(B) = 4 38 37 36 52 51 50 49 = 0.031160772. Recall the iclusio-exclusio priciple: 2. Some set theory P(A B) = P(A) + P(B) P(A B). This ca be demostrated by the Ve diagram i Figure 1. A B A B A B Figure 1. The iclusio-exclusio priciple. This diagram also illustrates the ituitive meaig of A B; amely, A B is the evet that oe of A or B occurs. Similarly, A B is the evet that both A ad B occur. Let (A ) =1 be a sequece of evets i some probability space. We have A = {ω Ω : ω A for at least oe k} = {ω Ω : there exists k such that ω A }. k So k=1 i.e. k A is the set of ω Ω such that for ay k, there exists k such that ω A ; k=1 k Defiitio 2.6. Defie the set lim sup A as A = {ω Ω : ω A for ifiitely may }. lim sup A := k=1 k A.
6 The ituitive meaig of lim sup A is that ifiitely may of the A s occur. Similarly, we have that A = {ω Ω : there exists 0 such that for all > 0, ω A } k=1 k = {ω Ω : ω A for all large eough }. Defiitio 2.7. Defie lim if A := k=1 k A. lim sup A meas that A will occur from some large eough oward; that is, evetually all A occur. It is easy to see that lim if A lim sup A. This also fits the ituitio, as if all A occur evetually, the they occur ifiitely may times. Defiitio 2.8. If (A ) =1 is a sequece of evets such that lim if A = lim sup A (as sets) the we defie lim A := lim if A = lim sup A. Example 2.9. Cosider Ω = N, ad the sequece A = { j : j = 0, 1, 2,..., }. If m < ad m A, the it must be that m = 0 = 1. Thus, k A = {1} ad lim sup A = k=1 k A = {1}. Also, if m < k, ad m A, the agai m = 1, so k A does ot cotai 1 < m < k. Hece, lim if A = Thus, the limit exists ad lim A = {1}. k=1 k A = {1}. Defiitio 2.10. A sequece of evets is called icreasig if A A +1 for all, ad decreasig if A +1 A for all.
7 Propositio 2.11. Let (A ) be a sequece of evets. The, (lim if A ) c = lim sup A c. Moreover, if (A ) is icreasig (resp. decreasig) the lim A = A (resp. lim A = A ). Proof. The first assertio is de-morga. So For the secod assertio, ote that if (A ) is icreasig, the k A = 1 A lim sup A = k lim if A = k ad k A = k If (A ) is decreasig the (A c ) is icreasig. So lim sup A = (lim if ad lim A = ( ) c Ac = A. A c ) c = ( A = A k. k A = A = k k A k. A c ) c = (lim sup A A c ) c = lim if A, Exercise 2.12. Show that of F is a σ-algebra, ad (A ) is a sequece of evets i F, the lim if A F ad lim sup A F. The goal of this sectio is to prove 3. Cotiuity of Probability Measures *** Nov. 10 *** Theorem 2.13 (Cotiuity of Probability). Let (Ω, F, P) be a probability space. Let (A ) be a sequece of evets such that lim A exists. The, P(lim A ) = lim P(A ). We start with a restricted versio: Propositio 2.14. Let (Ω, F, P) be a probability space. Let (A ) be a sequece of icreasig (resp. decreasig) evets. The, P(lim A ) = lim P(A ).
8 Proof. We start with the icreasig case. Let A = A = lim A. Defie A 0 =, ad B k = A k \ A k 1 for all k 1. So (B k ) are mutually disjoit ad B k = A k = A k=1 k=1 so B = A. Thus, P(lim A ) = P( B ) = P(B ) = lim = lim P( B k ) = lim P(A ). k=1 k=1 P(B k ) The decreasig case follows from otig that if (A ) is decreasig, the (A c ) is icreasig, so P(lim A ) = P( A ) = 1 P( A c ) = 1 lim P(A c ) = lim P(A ). Lemma 2.15 (Fatou s Lemma). Let (Ω, F, P) be a probability space. Let (A ) be a sequece of evets (that may ot have a limit). The, P(lim if A ) lim if P(A ) ad lim sup P(A ) P(lim sup A ). Proof. For all k let B k = k A. So lim if A = k B k. Note that (B k ) is a icreasig sequece of evets (B k = B k+1 A k ). Also, for ay k, B k A, so P(B k ) if k P(A ). Thus, For the lim sup, P(lim if A ) = P( B ) = lim P(B ) lim if P(A k) = lim if P(A ). k lim sup P(A ) = lim sup 1 P(lim if (1 P(A c )) = 1 lim if P(A c ) A c ) = 1 P((lim sup A ) c ) = P(lim sup A ). Fatou s Lemma immediately proves Theorem 2.13. Proof of Theorem 2.13. Just ote that lim sup P(A ) P(lim sup A ) = P(lim so equality holds, ad the limit exists. A ) = P(lim if A ) lim if P(A ),
9 As a cosequece we get Lemma 2.16 (First Borel-Catelli Lemma). Let (Ω, F, P) be a probability space. Let (A ) be a sequece of evets. If P(A ) <, the P( A occurs for ifiitely may ) = P(lim sup A ) = 0. Proof. Let B k = k A. So (B k ) is decreasig, ad so the decreasig seqece P(B k ) coverges to P(lim sup A ). Thus, for all k, P(lim sup A ) P(B k ) P(A ). k Sice the right had side coverges to 0 as k by the assumptio that the series is coverget, we get P(lim sup A ) 0. Example 2.17. We have a buch of bacteria i a patry dish. Every secod, the bacteria give off offsprig radomly, ad the the parets die out. Suppose that for ay, the probability that there are o bacteria left by time is 1 exp( f()). What is the probability that the bacteria evetually die out if: f() = log. f() = 2 7 2 2 +3+5. Let A be the evet that the bacteria dies out by time. So P(A ) = 1 e f(), for 1. Note that the evet that the bacteria evetually die out, is the evet that there exists such that A ; i.e. the evet A. Sice (A ) is a icreasig sequece, we have that P( A ) = lim P(A ). I the first case this is 1. I the secod case this is ( ) lim 1 exp 2 7 2 2 + 3 + 5 = 1 e 1/2. Example 2.18. What if we take the previous example, ad the iformatio is that the probability that the bacteria at geeratio do ot die out without offsprig is at most exp( 2 log ). The, if A is the evet that the -th geeratio has offsprig, we have that P(A ) 2. Sice P(A ) <, Borel-Catelli tells us that P(A occurs for ifiately may ) = P(lim sup A ) = 0.
10 That is, P( k : k : A c ) = P(lim if A c ) = 1. So a.s. there exists k such that all geeratios k do ot have offsprig implyig that the bacteria die out with probability 1.