Chapter 1. Probability Spaces. 1.1 The sample space. Examples:

Transcription

1 Chapter 1 Probability Spaces 1.1 The sample space The ituitive meaig of probability is related to some experimet, whether real or coceptual (e.g., playig the lottery, testig whether a ewbor is a boy, measurig a perso s height). We assig probabilities to possible outcomes of the experimet. Thus, we first eed to develop a abstract model for a experimet. A experimet has two stages: (1) a actio, for example, tossig a coi, selectig a passerby or turig a roulette, ad (2) the recordig of a outcome, for example, the side show by the coi, the age of the passer-by or the umber idicated by the roulette. I probability theory a experimet (real or coceptual) is modeled by all its possible outcomes, i.e., by a set, which we call the sample space ("(9/.#$/). It should be emphasized that a set is a mathematical etity, idepedet of ay ituitive backgroud. We will usually deote the sample space by ad its elemets by!. Examples: (a) The experimet is tossig a coi ad recordig the upper side of the coi: the sample space comprises two possible outcomes, = {H, T}. (b) The experimet is tossig a coi three times ad recordig, keepig track of the the order, each of the three results: The sample space comprises all possible triples of Heads ad Tails, = {(a 1, a 2, a 3 ) a i {H, T}} = {H, T} 3.

2 20 Chapter 1 I more detail, = {(H, H, H), (H, H, T), (H, T, H), (H, T, T), (T, H, H), (T, H, T), (T, T, H), (T, T, T)}. Note that for the same actio, tossig a coi three times, we could record a di eret datum, for example, the umber of Heads. I that case the probability space would be = {0, 1, 2, 3}. The mai message from this example is that the sample space is ot uiquely determied by the actio of the experimet. (c) The experimet is throwig two distiguishable dice ad recordig the umber show by each die: The sample space comprises ordered pairs of umbers betwee 1 ad 6, = {1,...,6} 2 = {(1, 1), (1, 2),...,(5, 6), (6, 6)}. (d) The experimet is throwig two idistiguishable dice ad recordig the pair of umbers show by the dice: here we have o way distiguishig betwee (2, 3) ad (3, 2). The sample space is = {(i, j) 1 i j 6}. (e) The experimet is throwig a dart ito a uit circle ad recordig the distace from the ceter. The, = [0, 1]. If we rather record the positio of the dart, the we ca take = {(r, ) 0 r 1, 0 < 2 } = [0, 1] [0, 2 ), but also = (x, y) x 2 + y 2 1. (f) The experimet is a ifiite sequece of coi tosses recordig the outcome of each toss: the sample space is = {H, T} N. This set is o-coutable, ad it ca be idetified with the segmet (0, 1) via the biary represetatio of ratioal umbers.

3 Probability Spaces 21 (g) The experimet is watchig a grai of polle move i a fluid at rest durig a uit time iterval (a experimet performed by botaist Robert Brow i 1827): it is commo to take for sample space the set of all cotiuous fuctios from [0, 1] to R 3, = C([0, 1]; R 3 ). (h) The experimet is the followig: a perso throws a coi; if the result is Head he takes a exam i probability, which he either passes or fails; if the result is Tail he goes to sleep ad we measure the duratio of his sleep (i hours). This sample space is which we ca write also as = {(H, P), (H, F)} {(T, x) x > 0}, = {H} {P, F} {T} R +. The sample space is the primitive otio of probability theory. It provides a model of a experimet i the sese that every thikable outcome (eve if extremely ulikely) is idetified with oe, ad oly oe, sample poit. 1.2 Evets Suppose that we throw a die ad record the umber o the upper face. The set of all possible outcomes (the sample space) is = {1,...,6}. What about the result the outcome is eve? Eve outcome is ot a elemet of. It is a property shared by several elemets i the sample space. It correspods to the subset {2, 4, 6} of. The outcome is eve is therefore ot a elemetary outcome of the experimet. It is a aggregate of elemetary outcomes, which we call a evet (39&!/). Defiitio 1.1 Let be a sample space. A evet is a subset of the sample space. Let A be a evet. If the experimet yielded a outcome!, we say that A has occurred (:(9;% 39&!/%) if! A; otherwise we say that A has ot occurred. The ituitive terms of outcome ad evet have bee icorporated withi a abstract framework of a set ad its subsets. As such, we ca perform o evets

4 22 Chapter 1 set-theoretic operatios of uios, itersectios ad complemetatios. All settheoretic relatios apply as they are to evets. Let be a sample space correspodig to a certai experimet. What is the collectio of all possible evets? The obvious aswer is the power set 2 of. It turs out that i certai cases (ad this is where measure-theoretical mosters are hidig), it is ecessary to restrict the collectio of evets to a sub-collectio of 2. While we leave the reasos to a more advaced course, there are certai requiremets that we wish the collectio of evets to fulfill: 1. Closure uder complemetatio: If A is a evet so is A c. We wat this property to hold, because if we are allowed to ask whether A has occurred, we should also be allowed to ask whether A has ot occurred. 2. Closure uder coutable uios: If A are evets, so is their uio =1 A. We wat this property to hold, because if we are allowed to ask whether each A has occurred, we should be allowed to ask whether at least oe of the A has occurred. 3. is a evet. This evet stads for some outcome amog all possibilities. I other words, if we restrict the collectio of evets to a sub-collectio of the power set of, we wat this collectio to be a -algebra. (2 hrs) (2 hrs) Commet: You may woder why ot require the collectio of evet to be closed with respect to ay (ot ecessarily coutable) uio. The reaso is oce agai i the realm of measure theory. Oe may ed up with a icosistet theory. Defiitio 1.2 A pair (, F ), where is a set ad F is a is called a measurable space ($*$/ "(9/). -algebra of evets Examples: (a) Cosider the experimet of tossig three cois with = {H, T} 3. The evet secod toss was Head is {(H, H, H), (H, H, T), (T, H, H), (T, H, T)}.

5 Probability Spaces 23 (b) Cosider the experimet waitig for the fish to bite (i hours). The sample space is = {x x > 0} = (0, ). The evet waited more tha a hour ad less tha two is {x 1 < x < 2} = (1, 2).. Problem 1.1 Costruct a sample space correspodig to fillig a sigle colum i the Toto. Defie two evets that are disjoit. Defie three evets that are mutually disjoit, ad whose uio is. 1.3 Probability Roughly speakig, a probability is a assigmet of umbers to possible outcomes of experimets. As discussed, these umbers may represet a likelihood, or a measure of belief. Mathematically, however, a probability is a assigmet rule, i.e., a fuctio. The immediate issues, the are to defie this fuctio s domai, rage ad other properties. The most rudimetary costructio of a probability is the followig: to each poit! i the sample space, i.e., to each elemetary outcome, assig a umber, p(!). We require these umbers to be o-egative there is o probability less tha zero ad as a ormalizatio covetio, we require them to add up to 1. Give a evet A, we defie its probability as the sum of the probabilities of the sample poits it comprises, P(A) = p(!).! A Thus, probability is a fuctio P F R +. The fuctio P satisfies a umber of properties that are readily derived. First, by costructio, for every evet A,0 P(A) 1. Moreover, Secod, if A ad B are disjoit, the P( ) = p(!) = 1.! P(A B) = p(!) = p(!) + p(!) = P(A) + P(B),! A B! A! B

6 24 Chapter 1 where we eeded A ad B to be disjoit i order ot to double cout. But there are also some subtleties. If the sample space is fiite, the everythig is well-defied. What about a ifiite sample space? If is ifiite, yet coutable, the the sum p(!)! is well defied provided that o matter how we arrage the poits! ito a sequece, the resultig series coverges absolutely. Thus, we may require that p(!) = 1.! The sum of p(!) over every subset of, whether fiite or ifiite, it well-defied. Problems arise whe is ot coutable. A sum comprisig a ucoutable umber of addeds is ot eve defied; as it turs out, this is ot simply a lack of kowledge a omissio of the first year curriculum. We defie a otio of probability that applies to all cases, whether the probability space is coutable or ot. The idea is to attribute a probability to evets, rather tha to elemets of the sample space. I the coutable case, we ca always resort to the rudimetary costructio described above. Defiitio 1.3 Let (, F ) be a measurable space. A probability (;&9";2%) o (, F ) is a fuctio P assigig a umber to every evet i F (iterpreted as the likelihood that this evet will occur or has occurred). The fuctio P has to satisfy the followig properties: 1. For every evet A F, 0 P(A) (there is o egative probability). 2. P( ) = 1 (the probability that some outcome has occurred is oe). 3. Let (A ) be a sequece of mutually disjoit evets. The, P A = =1 =1 P(A ). This property is called coutable additivity (%**1/ ;" ;&*"*)*$!). The triple (, F, P) is called a probability space (;&9";2% "(9/). (3 hrs) (3 hrs) The followig results are immediate:

7 Probability Spaces 25 Propositio 1.1 Let (, F, P) be a probability space. The: 1. P() = For every fiite sequece of N disjoit evets (A ) P N A = =1 N =1 P(A ). 3. For every evet A F, P(A c ) = 1 P(A). 4. For every A F,P(A) 1. Proof : 1. Take a sequece (A ) with A = for every. The, By coutable additivity, which implies that P() = 0. =1 A =. =1 P() = P(), 2. Take A k = for k > ad apply the first assertio alog with coutable additivity. 3. Sice A A c =, it follows from the secod assertio that 4. The last assertio is immediate. Examples: P(A) + P(A c ) = P( ) = 1.

8 26 Chapter 1 (a) The experimet is tossig a (fair) coi. The sample space is = {H, T}. The -algebra of evets cosists of all subsets of, F = 2 = {, {H}, {T}, }. We defie a probability fuctio by takig P({H}) = P({T}) = 12. Note that this defies ideed a uique probability fuctio of F. Choosig, for example P({H}) = 12 ad P({T}) = 14 would have bee icosistet, as 1 = P( ) = P({H} {T}) = P({H}) + P({T}) = 3 4. (b) The experimet is throwig a fair die ad recordig the outcome, which is a iteger betwee 1 ad 6. The sample space is = {1,...,6}. We take the -algebra of all subsets of. This -algebra cotais 2 6 evets. We defie a probability fuctio by settig the probability of all sigletos: P({i}) = 1, i {1,...,6}. 6 This defies uiquely a probability fuctio sice every evet is a fiite disjoit uio of sigletos; its probability is the sum of the probabilities of the sigletos its cotais. Commet: You may justly ask yourself how did I kow which probability space to assig to each experimet. What I did here is to model the likelihoods of outcomes of a experimet that has t yet bee performed. A model is a model. It may relate to reality or ot. I the preset case, I applied symmetry cosideratios. Sice we believe that all outcomes of the experimet are equally likely, i.e., all sigletos should have the same probability, this probability could oly assume oe cosistet value. Havig defied what a probability is, we ca further derive properties satisfied by all probability fuctio. Propositio 1.2 (Mootoicity of probability) Let (, F, P) be a probability space. If A, B are evets such that A B, the P(A) P(B).

9 Probability Spaces 27 Proof : If A B, the B = A (B A). By additivity ad the positivity of probability, P(B) = P(A) + P(B A) P(A). Propositio 1.3 (Probability of a uio) Let (, F, P) be a probability space. For every two evets A, B F, P(A B) = P(A) + P(B) P(A B). Proof : We have A B = (A B) (B A) (A B) A = (A B) (A B) B = (B A) (A B), A B A B B A By additivity, P(A B) = P(A B) + P(B A) + P(A B) P(A) = P(A B) + P(A B) P(B) = P(B A) + P(A B). It remais to subtract the last two equatios from the first.

10 28 Chapter 1 Propositio 1.4 Let (, F, P) be a probability space. For every three evets A, B,C F, P(A B C) = P(A) + P(B) + P(C) P(A B) P(A C) P(B C) + P(A B C). Proof : Usig the biary relatio, P(A B C) = P(A B) + P(C) P((A B) C) = P(A) + P(B) P(A B) + P(C) P((A C) (B C)), ad it remais to apply the biary relatio for the last expressio. Propositio 1.5 (Iclusio-exclusio priciple (%($% %-,% 0&983)) For ay evets (A i ) i=1, P(A 1 A ) = i=1 P(A i ) P(A i A j ) + P(A i A j A k ) i< j i< j<k + +( 1) +1 P(A 1 A ).. Problem 1.2 Prove the iclusio-exclusio priciple.. Problem 1.3 Let A, B be two evets i a probability space. Show that the probability that either A or B has occurred, but ot both, is P(A) + P(B) 2 P(A B). Lemma 1.1 (Boole s iequality) Let (, F, P) be a probability space. Probability is sub-additive (;*"*)*$! ;;): for every sequece (A ) of evets, P A k P(A k ), where the right-had side may be ifiite, i which case the iequality holds trivially.

11 Probability Spaces 29 Proof : Defie the followig sequece of evets, B 1 = A 1 B 2 = A 2 A 1, B 3 = A 3 (A 1 A 2 ),,..., B = A 1 A k. The B are disjoit ad their uio equals the uio of the A. Ideed, if! A k, the the set {k N! A k } is o-empty. This set has a miimum k ad! B k. It follows that! B k, hece A k B k. The opposite directio is trivial as B A for all. It follows that P A k = P B k = P(B k ) P(A k ). 1.4 Coutable, or discrete probability spaces The simplest probability spaces are those whose sample spaces iclude coutably may poits. I such case, we may always take the collectio of evets to be the power set, F = 2. Note that a coutable set ca be arraged ito a sequece, however, geerally, a coutable set is ot ordered. For coutable probability spaces, a probability fuctio P o F is fully determied by its value for every sigleto {!}, i.e., by the probability assiged to every elemetary evet. Ideed, let P({!}) p(!) be give. Sice every evet A ca be expressed as a fiite, or coutable uio of disjoit sigletos, A = {!},! A it follows from the additivity property that P(A) = p(!).! A

12 30 Chapter 1 A particular case ofte arisig i applicatios is where the sample space is fiite ad all elemetary evets have the same probability; we deote by the size of the sample space ad by p the probability of a sigleto, i.e., p(!) = p,!. By the properties of the probability fuctio, 1 = P( ) = p(!) = p,! i.e., p = 1. The probability of every evet A is the P(A) = p(!) = pa = A! A. Commet: Recall that a probability space is a model for a experimet. There is o a priori reaso why all outcomes should be equally probable. It is a assumptio that should be made oly whe believed to be applicable. Examples: (a) Two dice are rolled. What is the probability that the sum is 7? A sample space that is coveiet for this problem is costructed by viewig the two dice as distiguishable (e.g. oe blue ad oe red) ad takig = {(i, j) 1 i, j 6}. Due to symmetry cosideratios, it is atural to assume that each of the = 36 outcomes is equally likely. The evet sum equals 7 is A = {(1, 6), (2, 5), (3, 4), (4, 3), (5, 2), (6, 1)}, so that P(A) = A = 636. (b) There are 11 balls i a jar, 6 white ad 5 black. Two balls are draw at radom. What is the probability of drawig oe white ball ad oe black ball? To solve this problem, it is helpful to imagie that the balls are umbered from 1 to 11, the six white balls beig umbered from 1 to 6 (ote the use of a thought experimet). The sample space cosists of all possible pairs, = {(i, j) 1 i < j 11},

13 Probability Spaces 31 ad its size is = Ivokig oce agai symmetry cosideratios, we assume that all pairs are equally likely. The evet oe black ball ad oe white ball is A = {(i, j) 1 i 6, 7 j 11}. The size of A is the umber of possibility to choose oe white ball out of six, ad oe black ball out of five, i.e., P(A) = = = (c) A deck of 52 cards is dealt amog four players. What is the probability that oe of the players received all 13 spades? A atural sample space is the set of all possible partitios of 52 cards amog 4 players; its size is = 52! 13! 13! 13! 13! (the umber of possibilities to order 52 cards divided by the umber of iteral orders). Oce agai, symmetry cosideratios suggest that all partitios should have the same probability. Let A i be the evet that the i-th player has all spades, ad A be the evet that some player has all spades; clearly, A = A 1 A 2 A 3 A 4. For each i, hece, A i = P(A) = 4 P(A 1 ) = 4 39! 13! 13! 13!, 39! 13! 13! 13! 13! 52! 13! 13! 13! Problem 1.4 Prove that it is ot possible to costruct a probability fuctio o the sample space of itegers, such that P({i}) = P({ j}) for all i, j.. Problem 1.5 A fair coi is tossed five times. What is the probability that there was at least oe istace of two Heads i a row? Start by buildig the probability space.

14 32 Chapter 1 Next, we study a umber of examples, all cocerig fiite probability spaces with equally likely outcomes. The importace of these examples stems from the fact that they are represetatives of classes of problems recurrig i may applicatios. Example: (The birthday paradox) I a radom assembly of people, what is the probability that oe of them share the same date of birth? We assume that < 365 ad igore leap years. We take for sample space the set of all possible date-of-birth assigmets to people (order matters). For example, if = 2, the (Jul-1, Feb-15). The size of the sample space of size = 365. By symmetry cosideratios, all birthday assigmets to people are equally probably. Let A be the evet that all dates-of-birth are di eret. The, hece Values for various are A = ( ), P(A ) = A = P(A 23 ) < 0.5 P(A 50 ) < 0.03 P(A 100 ) < Commet: Some may have guessed, erroeously, that P(A 100 ) > 23. Geeralizatio Suppose that there were N days i a year ad N people; deote by A N the evet that all dates-of-birth are di eret. The same aalysis yields that the probability of A N is P(A N ) = AN = 1 N (N 1)...(N ( 1)) = N i=0 1 i N. For ay give N ad we ca calculate this probability. However, it is more istructive to obtai a estimate that will allow us to feel whe to expect this

15 Probability Spaces 33 probability to be large (close to 1) ad whe to expect it to be small (close to 0). Sice we have a product, it is useful to evaluate its logarithm log P(A N ) = 1 i=0 log 1 i N. To boud P(A N ) from above, we recall that log(1 + x) x for all real x (exercise: prove it), hece P(A N ( 1) ) exp 2N. which is valid for ay. To get a lower boud we use the iequality x x 2 log(1 + x) which is valid for ay 1 4 x (exercise: prove it). Thus, as log as N4, log P(A N ) 1 i=0 i N Hece, for N4, = ( 1) 2N 1 i=0 i 2 N 2 = N. ( 1) 2N ( 1)(2 1) 6N 2, ( 1) exp N 3N ( 1) P(AN ) exp 2N. Whe is small compared to N the upper ad lower bouds are quite close to each other. Whe is small compared to N, the probability of all dates-of-births beig di eret is close to 1, whereas whe is large compared to N, this probability is close to 0. Commet: You may ask yourself what is so geeric about the birthday paradox. Here is a applicatio: suppose that a computer assigs IDs to users by radomly choosig a 16-bit words. As log as the umber of users is less tha the squareroot of 2 16 = 256, there is a low probability that the same ID will be assiged to two users. Example: (The iattetive secretary, or the matchig problem) A secretary places radomly letters ito evelopes. What is the probability that o letter reaches

16 34 Chapter 1 its destiatio? What is the probability that exactly k letters reach their destiatio? We start by costructig the sample space. Assume that the letters ad the evelopes are umbered from oe to. The sample space cosists of all possible assigmets of letters ito evelopes, i.e., the set of all permutatios of the umbers 1-to-. If for example = 5, the (1, 3, 5, 4, 2) (Letter 1 i Evelope 1, Letter 3 i Evelope 2, etc.). The first questio ca be reformulated as follows: take a radom oe-to-oe fuctio from {1,...,} to itself; what is the probability that it has o fixed poits? If A is the evet that o letter has reached its destiatio, the its complemet, A c is the evet that at least oe letter has reached its destiatio (at least oe fixed poit). Let B i be the evet that the i-th letter reached its destiatio, the A c = B i. i=1 We apply the iclusio-exclusio priciple: P(A c ) = i=1 P(B i ) P(B i B j ) + P(B i B j B k )... i< j i< j<k = 1 P(B 1) 2 P(B 1 B 2 ) + 3 P(B 1 B 2 B 3 )... + ( 1) +1 P(B 1 B ), where we used the symmetry of the problem to determie, for example, that P(B i ) is idepedet of i. Now, P(B 1 ) = B 1 P(B 1 B 2 ) = B 1 B 2 ( 1)! =! = ( 2)!,!

17 Probability Spaces 35 etc. It follows that P(A c ) = 1 2)! ( + 3)! ( +( 1) +1 2! 3! 0!! = 1 1 2! + 1 3! +( 1)+1 1! = ( 1) k+1. k! (5 hrs) (5 hrs) For large, P(A) = 1 P(A c ( 1) ) = k e 1, k=0 k! from which we deduce that for large, the probability that o letter has reached its destiatio is about The fact that the limit is either 0 or 1 may be surprisig (oe could have argued equally well that the limit should be both 0 ad 1...). Note that as a side result, the umber of permutatios that have o fixed poits is A = P(A) =! `=0 ( 1)`. `! Now to the secod part of this questio. Before we aswer what is the umber of permutatios that have exactly k fixed poits, let s compute the umber of permutatios that have exactly k specific fixed poits; for example 1 1,...,k k k + 1 k + 1,...,. This umber equals the umber of permutatios of k elemets ot havig a fixed poit, ( 1)` ( k)!. `! k `=0 The choice of k fixed poits is exclusive, e.g., the evets B 1,3 = {1 ad 3 are the oly fixed poits} ad B 1,4 = {1 ad 4 are the oly fixed poits}

18 36 Chapter 1 are disjoit. To fid the total umber of permutatios that have exactly k fixed poits, we eed to multiply the umber of permutatios havig k specific fixed poits by the umber of ways to choose k elemets out of. Thus, if C deotes the evet that there are exactly k fixed poits, the ad For large ad fixed k we have C = k k ( k)! ( 1)` `=0 `! P(C) = C! = 1 k k! `=0 P(C) e 1 k!. =! k! k `=0 ( 1)`. `! ( 1)`, `! We will retur to such expressios later o, i the cotext of the Poisso distributio.. Problem 1.6 Sevetee people atted a party. At the ed of it, the druk people collect at radom a hat from the hat hager. What is the probability that 1. At least someoe got their ow hat. 2. Joh Doe got his ow hat. 3. Exactly 3 people got their ow hats. 4. Exactly 3 people got their ow hats, oe of which is Joh Doe. As usual, start by specifyig the probability space.. Problem 1.7 A deck of cards is dealt out. What is the probability that the fourteeth card dealt is a ace? What is the probability that the first ace occurs o the fourteeth card? 1.5 Cotiuity of probability A importat property of the probability fuctio is its cotiuity, i the sese that if a sequece of evets (A ) has a limit, the P(lim A ) = lim P(A ). We start with a soft versio:

19 Probability Spaces 37 Theorem 1.1 (Cotiuity for icreasig sequeces) Let (A ) be a icreasig sequece of evets, the P( lim A ) = lim P(A ). Commet: We have already see that icreasig sequece of evet coverge, with Note that for every, hece lim A = A k. A k = A, P( A k ) = P(A ), but we ca t just replace by. Proof : For a icreasig sequece of evets, lim A = =1 A. Costruct the followig sequece of disjoit evets, etc. Clearly, i=1 B 1 = A 1 B 2 = A 2 A 1 B 3 = A 3 A 2, B i = A i = A ad A i = B i i=1 i=1 i=1 (every! cotaied i at least oe A is also cotaied i at least oe B ). Now, P( lim A ) = P( A i ) = P( B i ) = i=1 i=1 P(B i ) = lim = lim P( i=1 i=1 i=1 P(B i ) B i ) = lim P(A ).

20 38 Chapter 1 Commet: Sice P(A ) is a icreasig fuctio, P(A ) P( lim A ).. Problem 1.8 (Cotiuity for decreasig sequeces) Prove that if (A ) is a decreasig sequece of evets, the More precisely, P(A ) P(lim A ). P( lim A ) = lim P(A ). The ext two lemmas hold for arbitrary sequeces of evets, without assumig the existece of a limit. Lemma 1.2 (Fatou) Let (A ) be a sequece of evets, the P(lim if A ) lim if P(A ). Proof : First, ote that the lim ifs o both side represet di eret otios. The lim if o the left-had side refers to sets, lim if A = =1 k= A k G. =1 The lim if of the right-had side refers to umbers. Recall that if (a ) is a sequece of umbers, the lim if a = lim if a k, k which is also the smallest partial limit (it exists because the sequece P(A ) is bouded). The sequece of evets (G ) is icreasig, ad therefore by the cotiuity theorem for icreasig sequeces, lim P(G ) = P( lim G ) = P =1 G = P(lim if A ).

21 Probability Spaces 39 O the other had, fix. Sice G = k= A k, it follows that which i tur implies that ad hece G A k for all k, P(G ) P(A k ) for all k, P(G ) if k P(A k). The left had side coverges to P(lim if A ) whereas the right had side coverges to lim if P(A ), which cocludes the proof. Lemma 1.3 (Reverse Fatou) Let (A ) be a sequece of evets, the lim sup P(A ) P(lim sup A ). Proof : Recall that lim sup A = =1 k= A k H =1 is the set of outcomes that occur ifiitely ofte. The sequece (H ) is decreasig, ad therefore lim P(H ) = P( lim H ) = P =1 O the other had, sice H = k= A k, it follows that P(H ) sup P(A k ). k H = P(lim sup A ). The left had side coverges to P(lim sup A ) whereas the right had side coverges to lim sup P(A ), which cocludes the proof.

22 40 Chapter 1 Theorem 1.2 (Cotiuity of probability) If a sequece of evets (A ) has a limit, the P( lim A ) = lim P(A ). Proof : This is a immediate cosequece of the two Fatou lemmas, for lim sup P(A ) P(lim sup A ) = P(lim if A ) lim if P(A ). Lemma 1.4 (First Borel-Catelli) Let (A ) be a sequece of evets such that P(A ) <. The, P(lim sup A ) = P({x x A ifiitely ofte}) = 0. Proof : Let as above H = k= A k. Sice P(H ) P(lim sup A ), the for all m P(lim sup A ) P(H m ) P(A k ). k m Lettig m ad usig the fact that the right had side is the tail of a covergig series we get the desired result.. Problem 1.9 Does the reverse Borel-Catelli hold? Namely, is it true that if P(A ) = the No. Costruct a couter example. P(lim sup A ) > 0? Example: Cosider the followig sceario. At a miute to oo we isert ito a ur balls umbered 1-to-10 ad remove the ball umbered 10. At half a miute to oo we isert balls umbered 11-to-20 ad remove the ball umbered 20, ad so o. Which balls are iside the ur at oo? Clearly all itegers except for the 10.

23 Probability Spaces 41 Now we vary the situatio: the first time we remove the ball umbered 1, ext time the ball umbered 2, etc. Which balls are iside the ur at oo? oe. I the third variatio we remove each time a ball at radom (from those iside the ur). Are there ay balls left at oo? If this questio is too bizarre, here is a more sesible picture. Our sample space cosists of radom sequeces of umbers, whose elemets are distict, ad whose first elemet is i the rage 1- to-10, its secod elemet is i the rage 1-to-20, ad so o. We are askig what is the probability that such a sequece cotais all itegers? Let s focus o ball umber 1 ad deote by E the evet that it is still iside the ur after steps. We have P(E ) = = 9k 9k + 1. The evets (E ) form a decreasig sequece, whose coutable itersectio correspods to the evet that the first ball was ot ever removed. Now, P( lim E ) = lim P(E ) = lim = lim 9k 9k k + 1 9k = lim = lim lim = k 1 Thus, there is zero probability that the ball umbered 1 is iside the ur after ifiitely may steps. The same holds ball umber 2, 3, etc. If F deotes the evet that the -th ball has remaied iside the box at oo, the P( F ) =1 =1 P(F ) = 0.