Chapter 3 Conditional Probability and Independence. Wen-Guey Tzeng Computer Science Department National Chiao Tung University

Transcription

1 Chapter 3 Conditional Probability and Independence Wen-Guey Tzeng Computer Science Department National Chiao Tung University

2 Conditional probability P(A B) = the probability of event A given the occurrence of event B Knowing that B has occurred changes the probability of A s occurrence A S Definition. If P(B)>0, the conditional probability of A given B, P( AB) P( A B) P( B) 2014 Fall 2

3 A class of 80 students. 50 of them take Linear Algebra, 60 of them take Probability, and 30 of them take both. Randomly choose a student and find that the student takes Probability What is the probability that the student takes Linear Algebra also? Sol: Let B be event that a selected student takes Probability. Let A be the event that a selected student takes Linear Algebra. P(A B )= P(AB)/P(B) = (30/80) / (60/80) 2014 Fall 3

4 Draw 8 cards from a deck of 52 cards. Given 3 of them are spades, what is P(the remaining 5 are also spades 3 are spades)=? Sol: B: the event that at least 3 of them are spades A: the event that the remaining 5 are spades P(A B) = P(AB)/P(B) = 5.44 x 10-6 P(AB)=C(13,8)/C(52,8) P(B)=[C(13,3)C(39,5)+C(13,4)C(39,4)+ +C(13,8)] / C(52,8) 2014 Fall 4

5 Box 1: 2 red balls and 3 green balls Box 2: 4 red balls and 1 green balls Choose a box with P(Box 1)=0.6, P(Box 2)=0.4, and then randomly choose two balls from the box. What is the sample space? What is the probability of the event that Box 1 is chosen and two balls are both red? the event that two red balls are chosen under the condition that Box 2 is chosen? the event that one red and one green are chosen? the event that Box 1 is chosen under the condition that two balls are red and green? 2014 Fall 5

6 P(BiDj) D1: balls are GG D2: balls are RG D3: balls are GG B1: Box 1 is chosen B2: Box 2 is chosen 2014 Fall 6

7 From the set of all families with 2 children, a family is selected at random and is found to have a girl. Assume that in a 2-child family all sex distributions are equally probable. What is the probability that the other child of the family is a girl? Sol: All possible outcomes (equally likely): bb, bg, gb, gg Let B be the event that the family has a girl. Let A be the event that the other child of the family is a girl. P(A B)=P(AB)/P(B)=P({gg} / P({bg,gb,gg})=(1/4)/(3/4)=1/ Fall 7

8 From the set of all families with 2 children, a child is randomly selected and found to be a girl. Assume that in a 2-child family all sex distributions are equally probable. What is the probability that the other child of the family is a girl? Sol: Let B be the event that a randomly selected child is a girl. Let A be the event that the second child of the family is a girl. P(A B)= P(AB)/P(B) = (1/4)/(2/4)=1/2. How do you define a sample space for this problem? 2014 Fall 8

9 Conditional probability = reduction of sample space A S C S = B 2014 Fall 9

10 All probability theorems hold under conditional probability, for P(B) 0, P(A c B) = 1-P(A B) P(E F B) = P(E B)+P(F B)-P(EF B) IF C A, P(C B) P(A B) Inclusion-exclusion principle 2014 Fall 10

11 A child mixes 10 good and 3 dead batteries. To find the dead battery, the father tests batteries one by one and without replacement. If the first 4 batteries are good, what is the probability that the fifth is dead? Sol: Old sample space S = {all sequences of 10 good and 3 dead batteries} New sample space S = {all sequences of 6 good and 3 dead batteries} P(the fifth is dead the first 4 are good} -- in the old sample space S = P(the first is dead) -- in the new sample space S = 3/ Fall 11

12 There are three boxes. One contains a 100-dollar bill and the other two are empty. You select one of them randomly. What is the probability that you win 100 dollars? I open an empty box that is not selected by you. Will you switch to the other un-opened box? Yes, always switch. Why? No, never switch. Why? Fall 12

13 Sol: Assume that the bill is in box 1. reduce the sample space Let outcome=(x, y, z) x: your first choice y: the opened box z: the final choice (switched to) Yes: always switch The reduced sample space S={ (1, 2, 3), (1, 3, 2), (3, 2, 1), (2, 3, 1)} P({3,2,1}) = P({2,3,1}) = P({1,2,3), (1,3,2)}) = 1/3 P(you win)=p({(3,2,1), (2,3,1)})=2/3 No: never switch The reduced sample space S={(1,2,1), (1,3,1), (2,3,2), (3,2,3)} P(you win) = P({(1,2,1), (1,3,1)} = 1/ Fall 13

14 Law of multiplication P( A B) P( AB) P( B) P( AB) P( B) P( A B) P(AB) P(BA) P(B)P(A B) P(A)P(B A) Extension P(A 1 A2 A3 ) P(A 1 )P(A 2 A1 )P( A3 A1 A2) 2014 Fall 14

15 Suppose 5 good fuses and 2 defective ones have been mixed up. To find the defective fuses, we test them 1-by-1, at random and without replacement. What is the probability that we are lucky and find both of the defective fuses in the first two tests? Sol: Let D 1 and D 2 be the events of finding a defective fuse in the 1 st and 2 nd tests, respectively. P(D 1 D 2 ) = P(D 1 )P(D 2 D 1 ) = 2/7x1/6 = 1/21 P(find both defective fuses in exactly 3 tests) =? 2014 Fall 15

16 Law of total probability Theorem (Law of total probability) Let P(B)>0, and P(B c )>0, then P(A) = P(A B)P(B) + P(A B c )P(B c ) Proof: P(A) = P(AB) + P(AB c ) and use the law of multiplication Fall 16

17 An insurance company rents 35% of the cars for its customers from agency I and 65% from agency II. If 8% of the cars of agency I and 5% of the cars of agency II break down during the rental periods, what is the probability that a car rented by this insurance company breaks down? Sol: P(A) = P(A I)P(I)+ P(A II)P(II) = (0.08)(0.35)+(0.05)(0.65) = Table method? Tree diagram method? 2014 Fall 17

18 Gambler s ruin problem 2 gamblers play the game of heads or tails, in which each time a fair coin lands heads up, player A wins $1 from B, and each time it lands tails up, player B wins $1 from A. Suppose that the player A initially has a dollars and player B has b dollars. If they continue to play this game successively, what is the probability that A will be ruined? the game goes forever with nobody winning? 2014 Fall 18

19 Sol: Sample space? Let E be the event that A will be ruined if he starts with i dollars, and let p i =P(E). Our aim is to calculate p a. To do so we define F to be the event that A wins the 1 st game. Then P(E) = P(E F)P(F) + P(E F c )P(F c ) => p i = p i+1 (1/2)+p i-1 (1/2) => p i+1 p i = p i p i Fall 19

20 p 0 =1, p a+b =0, and let p 1 - p 0 = x p i+1 p i = p i p i-1 = = p 2 - p 1 = p 1 - p 0 = x Thus p 1 = p 0 + x p 2 = p 0 + 2x p i = p 0 + ix and p a+b = p 0 + (a+b)x => x = -1/(a+b) So p i = 1 i/(a+b). In particular p a = b/(a+b) 2014 Fall 20

21 The same method can be used with obvious modifications to calculate q i, the probability that B is ruined if he starts with i dollars. q i = 1 i/(a+b) Since B starts with b dollars, he will be ruined with probability q b = a/(a+b). Thus the probability that the game goes on forever with nobody winning is 1-(q b +p a )= Fall 21

22 Extension If {B 1, B 2,, B n } is a partition of the sample space S of an experiment and P(B i )>0 for all i. Then, for any event A of S, P(A) = P(A B 1 )P(B 1 ) + P(A B 2 )P(B 2 ) + + P(A B n )P(B n ) 2014 Fall 22

23 Suppose that 80% of the seniors, 70% of the juniors, 50% of the sophomores, and 30% of the freshmen of a college use the library of their campus frequently. If 30% of all students are freshmen, 25% are sophomores, 25% are juniors, and 20% are seniors, what percent of all students use the library frequently? Sol: P(A)= P(A F)P(F) + P(A O)P(O) + P(A J)P(J) + P(A E)P(E) = (0.30)(0.30) + (0.50)(0.25) + (0.70)(0.25) + (0.80)(0.20) = Fall 23

24 Bayes formula P B A = P(A B)P B P(A) = P(A B)P B P(A B)P B +P(A B c )P(B c ) A conditional probability formula with very important applications on Statistics. Prior probability: P(B), P(B c ) the probability of event (hypothesis) B is believed originally Likelihoods (observation probability) : P(A B), P(A B c ) Posterior probability: P(B A) the probability of B under that A is observed 2014 Fall 24

25 In a bolt factory, 30% and 70% of production is manufactured by machines I and II, respectively. 4% and 6% of the output of machines I and II are defective. What is the probability that a randomly selected bolt that is found to be defective is manufactured by machine I? probability A: defective A c : non-defective B: manufactured by I B c : manufactured by II We want to compute P(B A)? Prior probability P(B)=0.3 Likelihoods: P(A B)=0.04, P(A B c )=0.06 Post probability: P(B A)=0.04*0.3/(0.04* *0.7)= Fall 25

26 Tree diagram method? 2014 Fall 26

27 In a double homicide, a suspect, John, is caught. The jury believes that John is guilty with 15% Later, a DNA sample is found and match John s DNA By forensic estimation, the probability that the DNA does not come from John is How certain should the jury be about that John is guilty? Sol: G: the event that John is guilty, P(G) = 0.15 I: the event that John is innocent, P(I)=0.85 D: the event that the DNA matches John s DNA P(D G)=1, P(D I)=10-9 Compute P(G D)= 2014 Fall 27

28 A car insurance company needs to decide whether to accept a car insurance from a driver. By estimation, the percentage of good drivers is 80% The probability that a good driver has a car accident within a year is 10%, and the probability that a bad driver has a car accident within a year is 40% 2014 Fall 28

29 What is the probability that a driver who has a car accident in the first year is a good driver? 2014 Fall 29

30 What is the probability that a driver who has a car accident in the first year and no car accident in the second year is a good driver? 2014 Fall 30

31 What is the probability that a driver who has no car accidents in two consecutive years is a bad driver? 2014 Fall 31

32 Extension: B 1, B 2,, B n partition S. P(B k A) = P A B k P B k P A B 1 P B P A B n P(B n ) Prior probability: P(B k ), 1 k n Likelihoods: P(A B i ), 1 i n Post probability: P(B k A ) 2014 Fall 32

33 A box contains 7 red and 13 blue balls. 2 balls are selected at random and are discarded without their colors being seen. A 3 rd ball is drawn randomly and observed to be red, What is the probability that both of the discarded balls were blue? Sol: B 1 : discarded balls are RR, P(B 1 ) = B 2 : discarded balls are RB, P(B 2 ) = B 3 : discarded balls are BB, P(B 3 ) = A: the third ball is R P(A B 1 )= P(A B 2 )= P(A B 3 )= Want to compute P(B 3 A) 2014 Fall 33

34 Independence Two events A and B are independent if P(AB)=P(A)P(B) A B A B A B 2014 Fall 34

35 Theorem:A and B are independent if either one of the following holds P(AB) = P(A)P(B) P(A B) = P(A) P(B A) = P(B) Theorem: If A and B are independent, so are A and B c A c and B A c and B c What is good for independence? easy to compute 2014 Fall 35

36 A card is drawn from a deck of 52 cards. A: the event that an ace is drawn. B: the event that a heart is drawn. Are A and B independent? P(A) = P(B)= P(AB) = 2014 Fall 36

37 An urn contains 5 red and 7 blue balls. Suppose that 2 balls are selected at random with/without replacement. Let A and B be the events that the first and the second ball are red, respectively. Are A and B independent? With replacement P(A) = P(B) = P(AB) = P(B A)P(A) = Independent? Without replacement P(B A) = P(B) = P(B A)P(A)+P(B A c )P(A c ) = Dependent? 2014 Fall 37

38 Jailer s paradox Three prisoners Alex, Bill, and Tim. One of them is condemned to death. The other two will be freed. The jailor and the judge know who is condemned to death. Alex has written a letter to his fiancée and wants to give it to either Bill or Tim, whoever goes free, to deliver. Alex asks the jailer to tell him which of the two will be freed. The jailer refuses to give that information to Alex, explaining that, if he does, the probability of Alex dying increases from 1/3 to 1/2. Really? 2014 Fall 38

39 Sol-1: Let A, B, and T be the events that Alex dies, Bill dies, and Tim dies. Let w 1 =(T, the jailer tells Alex that Bill goes free) w 2 =(B, the jailer tells Alex that Tim goes free) w 3 =(A, the jailer tells Alex that Bill goes free) w 4 =(A, the jailer tells Alex that Tim goes free) The sample space S = {w 1, w 2, w 3, w 4 }. P(w 1 )=P(w 2 )=1/3, P(w 3 )=P(w 4 )=1/6 Let J be the event that the jailer tells Alex that Tim goes free Then, P( A J ) P( AJ ) P( J ) 1 P( w4 ) 1 6 P( w2 ) P( w4 ) Fall

40 Sol-2: Zweifel, in June 1986 issue of Mathematics Magazine, page 156 He analyzes this paradox by using Bayes formula: ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( C P C J P B P B J P A P A J P A P A J P J A P 2014 Fall 40

41 Remark: If A and B are disjoint (mutually exclusive), then A and B are dependent. If A B, then A and B are dependent 2014 Fall 41

42 Extension Events A, B, and C are independent if the following all hold: P(AB) = P(A)P(B) P(AC) = P(A)P(C) P(BC) = P(B)P(C) P(ABC) = P(A)P(B)P(C) 2014 Fall 42

43 We draw cards, one at a time, at random and successively from an ordinary deck of 52 cards with replacement. What is the probability that an ace appears before a face card? Sol: Let E be the event of an ace appearing before a face card. Let A, F, and B be the events of ace, face card, and neither in the first experiment, respectively. Then P(E) = P(E A)P(A) + P(E F)P(F) + P(E B)P(B) = 1(4/52) + 0(12/52)+ P(E B)(36/52) P(E B) = P(E) P(E)=4/52+P(E)(36/52) So, P(E)=1/ Fall 43