Lectures 1 5 Probability Models

Transcription

1 Lectures 1 5 Probability Models Aalogy with Geometry: abstract model for chace pheomea Laguage ad Symbols of Chace Experimets: Sample space S, cosistig of all possible outcomes (elemets e, f,..., evets (subsets of S E, F,.... E occurs whe oe of its outcomes occurs. Cocepts of set theory with ames that reflect the chace experimets. Examples of differet types of sample spaces: roll two dice, gamblig was the cradle of probability theory 1 radom poit i a circle of radius oe aroud (0, 0, all sequeces of dice throws that ed with a first 6. Derived Evets (Sets: uio: E F = all outcomes i E or F itersectio: E F = EF = all outcomes i E ad F empty set: = impossible evet E ad F are mutually exclusive evets whe EF = subset: E F, E implies F or occurrece of E implies occurrece of F complemet: E c = all outcomes i S which are ot i E fiite (ifiite uios ad itersectios Ve diagrams 2 illustrate with two dice example Algebra of Evets: Note aalogy betwee ad ad multiplicatio ad additio. Illustrate with Ve diagrams. Commutative law: E F = F E ad EF = F E Associative law: (E F G = E (F G = E F G (EF G = E(F G = EF G Distributive law: (E F G = EG F G (E F (G H = EG EH F G F H (special case (E F (G F = EG F Notice that the itersectio taes precedece over the uio. De Morga s Laws: ( Ei c = E c i ad ( Ei c = E c i for fiite or ifiite uios/itersectios. Axioms of Probability 3 : Log ru frequecy otio (ituitive guidace, qualitative regularity i chaos. Postulate probabilities P (E for each evet E S, satisfyig the followig (plausible axioms: 1 Games, Gods ad Gamblig by F.N. David, Dover See wiipedia uder Ve Diagram 3 Kolmogorov (1933, Grudbegriffe der Wahrscheilicheitstheorie 1

2 Axiom 1: Axiom 2: 0 P (E 1 for all E S P (S = 1, the sure evet S Axiom 3: P ( E i = P (E i for ay sequece of evets E i, i = 1, 2,..., which are mutually exclusive (E i E j = for i j. Coutable additivity. It seems that all subsets could be evets, but we have a measurability caveat. Cosequeces: P ( = 0 sice P ( = P ( + P ( + P ( +... for ay evets E i, i = 1, 2,..., which are mutually exclusive we have ( P E i = P (E i Fiite additivity, follows simply from Axiom 3, usig E i = for i >. P (E c = 1 P (E sice E E c = S ed of Lect. 1 E F P (E P (F (= P (E + P (F E c P (E F = P (E + P (F P (EF Boole s iequality: P (E F P (E + P (F, useful 1 whe P (E + P (F 0. Boferroi s iequality: P (E F P (E + P (F 1, useful whe P (E + P (F 2. Follows from P (EF = P (E + P (F P (E F P (E + P (F 1 By iductio: ( ( P E i P (E i ( 1 ad P E i P (E i P (E F G = P (E + P (F + P (G P (EF P (EG P (F G + P (EF G The followig further geeralizes P (E F = P (E + P (F P (EF : ( P E i = P (E i P (E i1 E i ( 1 r+1 P (E i1 E i2... E ir i 1 <i 2 i 1 <i 2 <...<i r ( 1 +1 P (E 1 E 2... E (1 ( where is over all combiatios of r idices from {1, 2, 3,..., } i 1 <i 2 <...<i r The proof cosists of represetig the uio E 1 E 2... E as a disjoit uio of patches of the type E j1 E j2... E j Ej c Ej c N with probability deoted by p = P (E j1 E j2... E j E c j E c j N Such a patch maes oe cotributio p to the left side of (1 ad ( ( ( ( +... ± = 1 ( target ris per system i aircraft idustry. 100 critical systems ris = This upper boud holds o matter how such system failures might iteract. 2

3 cotributio of p o the right side of (1. Here the coutig of cotributios will be explaied just for the secod term ( 2. The mius comes from the sig of the sum of probabilities P (E i1 E i2, ad the cout ( 2 from the fact that we ca form that may itersectio pair probabilities P (E i1 E i2 with evets E i1, E i2 tae from E j1, E j2,..., E j (as log as 2, to which the patch E j1 E j2... E j Ej c Ej c N would cotribute probability p, sice the E j1 E j2... E j Ej c Ej c N E i1 E i2 for ay such choice. Ay other choice E i1 E i2 does ot itersect with the patch. The idetity (2 follows from the biomial expasio of (1 1 = 0. Sice these cotributios of patches holds for ay such patch, the idetity (1 follows. The alteratig partial sums o the right side of (1 form upper ad lower bouds for the left side of (1. Sip proof. Equally Liely Outcomes I fiite sample spaces there are ofte symmetry cosideratios that suggest that all outcomes be treated equally, i.e. be assiged the same probability. If S = {1, 2,..., N} the P ({1} = = P ({N} P ({i} = 1/N, i = 1, 2,..., N P (E = #(E/#(S = #(E/N Example 1 (Two Fair Dice: improvise Example 2 (Bowl Draw: Two balls draw radomly without replacemet from a bowl with six white ad five blac balls. If E = oe white ad oe blac ball is draw what is P (E? Two solutio paths, ordered ad uordered drawigs. Aswer: 6/11 Example 3 (Poer Hads: A had cosists of 5 cards tae from a dec of 52, cosistig of 4 suits, each havig face values: ace, 2,..., 10, jac, quee, ig. Straight: five i sequece ot of same suit = E 1 Straight flush: five i sequece from same suit = E 2 Full house: three of oe face value ad two of aother = E 3 Compute the probabilities P (E i Aswers: ( 52 5 = possible hads Lect. 2 eds at P (E 1 = ( ( 52 =.0039, P (E 2 = 4 ( = , P (E 3 = 5 5 (4 ( P (E ( 52 = Example 4 (Treatmet Radomizatio: I a radomized trial of a ew drug, 5 patiets out of 10 are radomly assiged to receive the ew drug while the other 5 get a placebo (loo alie drug. If the ew drug acts just lie the placebo, we could treat the measurable outcome for the 10 patiets as beig preordaied, i.e., have othig to do with the treatmet assigmet. What the is the chace that the treatmet group will cotai at least four of the five highest scores 1? Assume all scores are distict ad ca be raed 1, 2,..., 10, from lowest to highest. 1 differet from cotaiig at least the 4 highest scores, chace 6/252 =

4 There are ( 10 5 = 252 choices of 5 treatmet scores from the 10 available scores. Either the top 5 are chose (oe way, or exactly 4 of the top 5 are chose. Thus the total umber of ways to get at least 4 of the top 5 scores is 1 + ( ( = 26 with probability = However, if we judge the drug performace by looig at the sum of the 5 score ras for the treatmet group, what is the chace of gettig at least the third highest possible sum? The highest sum is obtaied with = 40. The secod highest sum is = 39 ad the third highest sum is obtaied i two possible ways as = = 38. Thus the desired chace is ( /252 = , which might iduce you to recosider your positio that the drug has othig to do with high scores. This is a example of the Wilcoxo test procedure. Example 5 (Lottery: Amog N ticets i a lottery there is a uique special prize. ticets are sold i radom order oe by oe. What is the chace that the prize is amog them? All N(N 1 (N + 1 draws are equally liely. The prize could be part of these ticets if it was the first or secod,..., or th ticet draw, each oe of these could be combied with (N 1(N 2 (N + 1 choices for the remaiig 1 ticets to be draw. Thus the desired probability is (N 1(N 2 (N + 1 (N 1(N 2 (N N(N 1 (N + 1 N(N 1 (N + 1 }{{} The same effect could be had if the umber of the prize ticet is radomly chose after the ticets are sold, as i a ormal lottery. The the aswer /N is more directly evidet. Example 6 (Birthday Problem: E = all persos i a room have distict birthdays. e = (x 1,..., x, x i {1, 2,..., 365} There are 365 such possible outcomes e, igorig leap years. Assume all outcomes are equally liely. P (E = (365 ( ( = = ( 1 2 ( which becomes , , , for = 22, = 23, = 50 ad = 100 respectively. It ca be be show that equally liely birthdays is least favorable for havig a birthday match. Birthday Problem Extesio: The probability of the evet A c of gettig at least oe matchig or eighborig pair of birthdays is: = N ( P = P (A c 365( 1! = 1 P (A = ( ( ( =

5 Hece P 13 =.4829, P 14 =.5375, P 23 =.8879, ad P 40 = The expressio for P (A, the probability of o equal or eighborig birthdays derives from: choices of perso 1 s birthday. 2 There are 365 o-birthdays (NB, two of which fla perso 1. Ito the remaiig places betwee NB s we eed to place the remaiig 1 birthdays (B. 3 This gives us a NB-B patter with o two B s ext to each other. 4 There are ( 1! ways of assigig the 1 B s to specific persos. Lect. 3 eded here probability P(at least oe matchig B day P(at least oe matchig or adjacet B day umber of persos Example 7 (Matchig Problem, Preset Exchage: N distict items, 1, 2,..., N, are arraged i radom order. A match occurs if the positio umber coicides with the item umber. F N =evet that o match occurs. P (F N =? Solutio: Let E i = evet that item i is i the i th positio, i.e. gives us a match. For distict i 1, i 2,..., i we have (N! P (E i1 E i2 E i = N! so that P (FN c = P (E 1 E 2... E N = P (E 1 + P (E P (E N... + ( 1 +1 P (E i1 E i2 E i ( 1 N+1 P (E 1 E 2 E N i 1 <i 2 <...<i = N 1 ( N (N! N + + ( ( 1 N+1 1 N! N! = 1 1 2! + 1 3! + ( 1+1 1! + + ( 1N+1 1 N! so that P (F N = ! 1 3! + + ( 1N 1 N! e 1 = e.g. P (F 4 =.375, P (F 5 =.367, P (F 6 =.368, 5

6 Now cosider the evet F ( = exactly matches occur. Fid P (F (. Solutio: Fixig a specific set of items the chace that remaiig items show o match is Hece there are (N! ! 1 3! + + ( 1N 1 (N!. [ ! 1 ] 3! ( 1N (N! give the aswer without derivatio ways for these N items to arrage themselves so that there is o match amog them. Thus there are ( [ N (N! ! 1 ] 3! ( 1N (N! ways of arragig all N items so that exactly of them give us a match. This gives P (F ( = 1 N! ( N = 1! [ (N! [ ! 1 ] 3! ( 1N (N! ! 1 ] 3! ( 1N (N! which for large N becomes e 1 /!, = 0, 1, 2,... (Poisso Distributio. Example 8 (Mote Hall Cups: Mote Hall 1 lets you choose oe of three doors, oe of which hides a facy car, the others just a toy goat. You get to eep what is behid your chose door. Just as you reach for your door hadle, he says: wait, I will ope oe of the other doors, with a goat behid it. He swears that he would always give you this itervetio iformatio, regardless of what you chose. What should you do, ad what are the best chaces for your choice? Origially we had chace 1/3 of gettig the car, assumig that we piced our door radomly. With the give iformatio we could improve o that to 1/2 by picig oe of the two remaiig doors at radom. A eve better choice is to switch to the other remaiig door, relyig o the 2/3 chace of havig missed the door with the car o our origial radom choice. By switchig we would get the car with probability 2/3. The power of P (E c = 1 P (E. Example 9 (Axiom 3: Illustrate Axiom 3 through the evet E of gettig a 6 before a 5 whe you roll a die util you get a 5 or 6. How to assig probabilities to all possible outcomes? Ay sequece (x 1,..., x with x i {1, 2,..., 6} should get probability 1/6. E = { (6, {(x 1, 6, x 1 = 1,..., 4}, {(x 1, x 2, 6, x 1, x 2 = 1,..., 4},... } P (E = = 1 ( ( 4 i 1 = 1 ( i= /3 1 Example 10 (Craps: I the game of craps (Ch. 2, Problem 26 you wi outright if you roll a sum of 7 or 11 with two dice, you lose, if you roll a sum of 2, 3, or 12, ad if you rolled 6 = 1 2

7 a sum i = 4, 5, 6, 8, 9, 10, the you wi whe you roll the same sum agai before a 7 shows up. Thus oe of the ways to wi is to get a 4 ad the aother 4 before a 7 shows up. What is the probability of wiig the game that way? The outcomes of this type cosist of Lect. 4 eds E = {(4,,...,, 4, 4 or 7, = 0, 1, 2,...} Istead of I should really write more precisely (4,,...,, 4, 4 or 7 ((x 1, y 1 : x 1 +y 1 = 4, (x 2, y 2 : x 2 + y 2 4 or 7,..., (x +1, y +1 : x +1 + y +1 4 or 7, (x +2, y +2 : x +2 + y +2 = 4 but as you ca see, that otatio is a bit cumbersome. Thus I will cotiue with the origial telegraphic otatio, with the uderstadig that i coutig outcomes I cout the umber of pairs (x i, y i with the give sum properties. I hope this maes it all clearer. There are 36 9 = 27 sums with 4 or 7. For fixed, the probability of all the (+2-tuples i E is P ({(4,,...,, 4, 4 or 7} = ad P (E = =0 ( 27 = /36 = = 1 36 Similarly you ca wor out the probabilities of all other ways of wiig at craps this way, where you have to get a i ad the aother i before a 7 shows up, for i = 4, 5, 6, 8, 9, 10. Of course, you eed to combie this with gettig a 7 or 11 o the first roll ad thus wiig with the first roll outright i order to get the probability of wiig at craps. Probability as a Cotiuous Set Fuctio For a icreasig sequece of evets E 1 E 2... E i E i+1... defie the limit evet as lim E = E i For a decreasig sequece of evets E 1 E 2... E i E i+1... defie the limit evet as lim E = E i The followig limit properties follow from the coutable additivity axiom 3. For either icreasig or decreasig sequeces of evets we have ( P lim E = lim P (E 1 Jaso Rosehouse (2009, The Mote Hall Problem, The Remarable Story of Math s Most Cotetious Brai Teaser, Oxford Uiversity Press. 7

8 Proof: for a icreasig sequece rewrite the uios i terms of disjoit oio layers E i = F i = E i Ei 1 c with E 0 =, i.e., E0 c = S ad E i = E = F i The F i = E i E c i 1 are the disjoit oio layers of the fiite or ifiite uio. as we have ( ( P (E = P E i = P F i = P (F i ( ( ( P (F i = P F i = P E i = P lim E ( = lim P (E = P lim E 8

9 For a decreasig sequece E use complemetatio to get E c ad use the previous result ( ( ( 1 lim P (E = lim P (E c = P lim Ec = P E c = 1 P E =1 =1 = lim P (E = P ( E = P =1 ( lim E 9