Rules of Probability

Transcription

1 Rules of Probability

2 The additive rule P[A B] = P[A] + P[B] P[A B] and P[A B] = P[A] + P[B] if P[A B] = f

3 The additive rule for more than two events n n P A i P Ai P Ai A j i1 i1 i j P A A A i j k i j k 1 1 n P A1 A2 An and if A i A j = f for all i j. n n then P A i P Ai i1 i1

4 The Rule for complements for any event E P E 1 P E

5 Conditional Probability, Independence and The Multiplicative Rue

6 Then the conditional probability of A given B is defined to be: P A B P A B PB if PB 0

7 The multiplicative rule of probability P A and B P A P B A if P A 0 PBP A B if PB 0 PAB PAPB if A and B are independent. This is the definition of independent

8 The multiplicative rule for more than two events P A1 A2 An P A1 P A2 A1 P A3 A2 A1 P A n An An A 1 2 1

9 Proof P A A A P A A A A 1 2 n 1 2 n1 n and continuing we obtain P A A A P A A A A 1 2 n1 n 1 2 n1 P A1 A2 An 2 P A n1 A1 A2 An 2 P A n A1 A2 An 1 P A1 P A2 A1 P A3 A2 A1 P A n An An A 1 2 1

10 Example What is the probability that a poker hand is a royal flush i.e , J, Q, K,A 2. 10, J, Q, K,A 3. 10, J, Q, K,A 4. 10, J, Q, K,A

11 1. 10, J, Q, K,A 2. 10, J, Q, K,A 3. 10, J, Q, K,A 4. 10, J, Q, K,A Solution Let A 1 = the event that the first card is a royal flush card. Let A i = the event that the i th card is a royal flush card. i = 2, 3, 4, P A1 52, P A2 A1 51, P A3 A2 A1 50, P A4 A3 A2 A1, P A5 A4 A3 A2 A

12 P Royal Flush P A A A A A Another solution is by counting 4 4 PRoyal Flush The same result

13 Independence for more than 2 events

14 Definition: The set of k events A 1, A 2,, A k are called mutually independent if: P[A i1 A i2 A im ] = P[A i1 ] P[A i2 ] P[A im ] For every subset {i 1, i 2,, i m } of {1, 2,, k } i.e. for k = 3 A 1, A 2,, A k are mutually independent if: P[A 1 A 2 ] = P[A 1 ] P[A 2 ], P[A 1 A 3 ] = P[A 1 ] P[A 3 ], P[A 2 A 3 ] = P[A 2 ] P[A 3 ], P[A 1 A 2 A 3 ] = P[A 1 ] P[A 2 ] P[A 3 ]

15 P[A 1 ] =.4, P[A 2 ] =.5, P[A 3 ] =.6 A 1 A 2 P[A 1 A 2 ] = (0.4)(0.5) = P[A 1 A 3 ] = (0.4)(0.6) = 0.24 P[A 2 A 3 ] = (0.5)(0.6) = 0.30 P[A 1 A 2 A 3 ] = (0.4)(0.5)(0.6) = 0.12 A 3

16 Definition: The set of k events A 1, A 2,, A k are called pairwise independent if: P[A i A j ] = P[A i ] P[A j ] for all i and j. i.e. for k = 3 A 1, A 2,, A k are pairwise independent if: P[A 1 A 2 ] = P[A 1 ] P[A 2 ], P[A 1 A 3 ] = P[A 1 ] P[A 3 ], P[A 2 A 3 ] = P[A 2 ] P[A 3 ], It is not necessarily true that P[A 1 A 2 A 3 ] = P[A 1 ] P[A 2 ] P[A 3 ]

17 P[A 1 ] =.4, P[A 2 ] =.5, P[A 3 ] =.6 A 1 A 2 P[A 1 A 2 ] = (0.4)(0.5) = P[A 1 A 3 ] = (0.4)(0.6) = 0.24 P[A 2 A 3 ] = (0.5)(0.6) = 0.30 P[A 1 A 2 A 3 ] = 0.14 (0.4)(0.5)(0.6) = 0.12 A 3

18 Bayes Rule Due to the reverend T. Bayes Picture found on website: Portraits of Statisticians maths/histstat/people/welco me.htm#h P A B P A P B A PAP B A P AP B A

19 Proof: P A B P A B PB P A B PA B P A B P A P B A PAP B A P AP B A

20 Example: We have two urns. Urn 1 contains 14 red balls and 12 black balls. Urn 2 contains 6 red balls and 20 black balls. An Urn is selected at random and a ball is selected from that urn. Urn 1 Urn 2 If the ball turns out to be red what is the probability that it came from the first urn?

21 Solution: Let A = the event that we select urn 1 A P A = the event that we select urn 2 1 P A 2 Let B = the event that we select a red ball 14 6 P B A, P B A We want P A B. Note: the desired conditional probability is in the reverse direction of the given conditional probabilities. This is the case when Bayes rule should be used

22 Bayes rule states P A B P A P B A PAP B A P AP B A

23 Example: Testing for a disease Suppose that 0.1% of the population have a certain genetic disease. A test is available the detect the disease. If a person has the disease, the test concludes that he has the disease 96% of the time. It the person doesn t have the disease the test states that he has the disease 2% of the time. Two properties of a medical test Sensitivity = P[ test is positive disease] = 0.96 Specificity = P[ test is negative disease] = = 0.98 A person takes the test and the test is positive, what is the probability that he (or she) has the disease?

24 Solution: Let A = the event that the person has the disease A = the event that the person doesn t have the disease 0.001, P A P A Let B = the event that the test is positive. P B A 0.96, P B A 0.02 We want P A B. Note: Again the desired conditional probability is in the reverse direction of the given conditional probabilities.

25 Bayes rule states PAP B A P A B PAP B A P AP B A Thus if the test turns out to be positive the chance of having the disease is still small (4.58%). Compare this to (.1%), the chance of having the disease without the positive test result.

26 An generlization of Bayes Rule Let A 1, A 2,, A k denote a set of events such that S A1 A2 Ak and Ai Aj f for all i and j. Then P A i B P Ai P B Ai P A P B A P A P B A 1 1 k k

27 If A 1, A 2,, A k denote a set of events such that S A1 A2 Ak and Ai Aj f for all i and j. Then A 1, A 2,, A k is called a partition of S. S A 1 A 2 A k

28 Proof B B A B A 1 k and B Ai B Aj f for all i and j. A 1 A 2 A k B Then P B P B A P B A 1 k PA 1 P B A P A 1 k P B A k

29 and P A i B P A i PB B P Ai P B Ai P A P B A P A P B A 1 1 k k

30 Example: We have three urns. Urn 1 contains 14 red balls and 12 black balls. Urn 2 contains 6 red balls and 20 black balls. Urn 3 contains 3 red balls and 23 black balls. An Urn is selected at random and a ball is selected from that urn. Urn 1 Urn 2 Urn 3 If the ball turns out to be red what is the probability that it came from the first urn? second urn? third Urn?

31 Solution: Let A i = the event that we select urn i S A1 A2 A3 1 PA 1 P A2 P A3 3 Let B = the event that we select a red ball P B A1, P B A2, P B A We want P A i B for i 1, 2,3. Note: the desired conditional probability is in the reverse direction of the given conditional probabilities. This is the case when Bayes rule should be used

32 Bayes rule states P A 1 B P A1 P B A1 PA 1 P B A1 P A2 P B A2 P A3 P B A P A B P A B

33 Example: Suppose that an electronic device is manufactured by a company. During a period of a week 15% of this product is manufactured on Monday, 23% on Tuesday, 26% on Wednesday, 24% on Thursday and 12% on Friday.

34 Also during a period of a week 5% of the product is manufactured on Monday is defective 3 % of the product is manufactured on Tuesday is defective, 1 % of the product is manufactured on Wednesday is defective, 2 % of the product is manufactured on Thursday is defective and 6 % of the product is manufactured on Friday is defective. If the electronic device manufactured by this plant turns out to be defective, what is the probability that is as manufactured on Monday, Tuesday, Wednesday, Thursday or Friday?

35 Solution: Let A 1 = the event that the product is manufactured on Monday A 2 = the event that the product is manufactured on Tuesday A 3 = the event that the product is manufactured on Wednesday A 4 = the event that the product is manufactured on Thursday A 5 = the event that the product is manufactured on Friday Let B = the event that the product is defective

36 Now P[A 1 ] = 0.15, P[A 2 ] = 0.23, P[A 3 ] = 0.26, P[A 4 ] = 0.24 and P[A 5 ] = 0.12 Also P[B A 1 ] = 0.05, P[B A 2 ] = 0.03, P[B A 3 ] = 0.01, P[B A 4 ] = 0.02 and P[B A 5 ] = 0.06 We want to find P[A 1 B], P[A 2 B], P[A 3 B], P[A 4 B] and P[A 5 B]. We will apply Bayes Rule P A i B PAi P B Ai P A1 P B A1 P A5 P B A5

37 i P[A i ] P[B A i ] P[A i ]P[B A i ] P[A i B] Total

38 The sure thing principle and Simpson s paradox

39 The sure thing principle Suppose P A C P B C P A C P B C P A P B and then Example to illustrate Let A = the event that horse A wins the race. B = the event that horse B wins the race. C = the event that the track is dry C = the event that the track is muddy

40 Proof: P A C P B C or PAC PB C PA PAC P AC implies P A C P B C P C P C implies P A C P A C P B C P B C P C P C or P AC P B C P B C P B C P B

41 Does Simpson s Paradox P D S C P D S C and P D S C P D S C imply P D S P D S? Example to illustrate D = death due to lung cancer S = smoker C = lives in city, C = lives in country

42 Solution P D S P D S C P D S C P D S P S P S similarly P D S C P S C P D S C P S C PS C PS P S C PS P D S C P C S P D S CP C S P D S P D S C P C S P D S C P C S

43 if P D S C P D S C P D S C P D S C and whether P D S P D S C P C S P D S CP C S is greater than P D S P D S C P C S P D S C P C S depends also on the values of P C S, P C S 1 P C S, P C S and P C S 1P C S

44 Suppose P D S C 0.90 P D S C =0.60 and whether than and P D S C 0.40 P D S C 0.10 P C S.10, P C S 1 P C S.90, P C S =.80 and P C S 1 P C S 0.20 P D S P D S C P C S P D S CP C S P D S P D S C P C S P D S C P C S

45 Bayes Rule