Math 42, Discrete Mathematics

Transcription

1 c Fall 2018 last updated 11/14/2018 at 18:13:19 For use by students in this class only; all rights reserved. Note: some prose & some tables are taken directly from Kenneth R. Rosen, and Its Applications, 8th Ed., the ocial text adopted for this course.

2 Equal-Likelihood Denitions An experiment is a procedure that yields one of a given set of possible outcomes. The sample space of the experiment is the set of possible outcomes. An event is a subset of the sample space. Intro to If we run the experiment once, a single outcome x from the sample space is generated. For a given event E, we say that E occurred" if x E. If x / E, we say that E did not occur. For the moment, we consider experiments where we have nitely many equally-likely outcomes. In a given application, we must be very careful to verify that the outcomes are, in fact, equally likely before using any of the results in this section. c R. P.

3 Equal-Likelihood Examples Example The experiment consists of tossing two dice and then counting the number of spots showing. Since a standard die may show from 1 to 6 spots, the sample space for this experiment is S = {2, 3,..., 12}. (1) Intro to But are the outcomes in S equally likely? No! If we look at Figure 1, it appears that we're much more likely to see 7 spots showing than either 2 or 12 spots. However, if we modify our sample space, changing it to S = {(1, 1), (1, 2),..., (1, 6), (2, 1),..., (6, 6)}, (2) then it looks like for the basic toss of two dice we have equally likely outcomes. c R. P.

4 Equal-Likelihood Examples Figure 1: Tossing Two Dice Intro to c R. P.

5 Equal-Likelihood Denition If S is a nite nonempty sample space of equally likely outcomes, and E is an event, i.e., E S, then the probability of E is p(e) = E S. (3) Intro to Remarks Since E S, we have 0 E S, and thus 0 p(e) 1. (4) Note that, the empty event, can never occur, and p( ) = 0. On the other hand, the event S is certain to occur, since it contains every possible outcome of the experiment; p(s) = 1. c R. P.

6 Equal-Likelihood Warning In more advanced probability discussions, where the sample space is an uncountable set, it's possible to have an event E with p(e) = 0 without the event's being impossible. Similarly, it's possible to have an event F with p(f) = 1 without the event F being certain. Intro to c R. P.

7 Equal-Likelihood Examples Example The experiment consists of ipping a fair coin, i.e., a coin in which heads and tails are equally likely to occur, ve times. Record the results. The sample space for this experiment consists of ordered quintuples of H's and T's like (H, H, H, H, H) or (T, H, T, H, T). The Product Rule tells us that the sample space S will contain 2 5 = 32 distinct outcomes, i.e., S = 32. Our intuition tells us that these outcomes will be equally likely. Since there are so many dierent outcomes in the sample space, we try to avoid writing S out explicitly, if we can help it. Intro to c R. P.

8 Equal-Likelihood Examples Example (cont'd) 1. Let E 1 be the event all ve ips come up heads." Then E 1 = {(H, H, H, H, H)} and so p(e 1 ) = 1/ Let E 2 be the event exactly two of the ve ips result in heads." Then E 2 = ( 5 2) = 10. Why? So p(e 2 ) = 10/32 = 5/ Let E 3 be the event at most two of the ips result in heads." We will have at most two heads out of the ve ips if we have exactly no heads or exactly one heads or exactly two heads. The Sum Rule from counting yields E 3 = ( ) ( ) ( ) 5 2 = = 16. Intro to Thus p(e 3 ) = 16/32 = 1/2. c R. P.

9 of a Union Theorem If E 1 and E 2 are two events in the sample space S, then p(e 1 E 2 ) = p(e 1 ) + p(e 2 ) p(e 1 E 2 ). (5) Intro to Proof. We know from Section 2.2 that E 1 E 2 = E 1 + E 2 E 1 E 2. (6) If we divide both sides of (6) by S and apply (3), we get (5). c R. P.

10 of a Complement Corollary Let E be an event in a sample space S. The probability of the event Ē = S E, the complementary event of E, is given by Intro to p(ē) = 1 p(e). (7) Proof. Apply (5) to E 1 = E and E 2 = Ē. c R. P.

11 Equal-Likelihood Examples Example Suppose that a certain dinner is attended by 50 women and 50 men. At the end of the dinner, door prizes are awarded to ve of the attendees. What is the probability that exactly three of the door prizes will be awarded to women? Our experiment will consist of selecting ve names, at random," from the list of attendees. So our sample space S has S = ( ) = 75, 287, 520. Now we count the number of ways in which the selection will result in exactly three women being chosen. This will be the number of ways of choosing three women from the 50 women present times the number of ways of choosing 2 men from the 50 men present. That is, ( )( ) = = 24, 010, 000. (8) 3 2 c R. P. Intro to

12 Equal-Likelihood Examples Example (cont,d) So now we get that the probability that exactly three women win prizes is p = 24, 010, 000/75, 287, Remark If we had 500 women and 500 men and we gave out 5 prizes as before and then asked for the probability that exactly three women won prizes, the probability would be ) p = ( 500 )( ( ). (9) Intro to But maybe your calculator would rebel. What to do? c R. P.

13 Sampling With or Without Replacement It turns out that the probability that exactly three women will win door prizes is the same whether we select ve names all at once or one at a timeprovided we do not replace" a name once it is chosen. But what if we did replace a name after it was chosen? Then the problem would be like the problem of ipping a coin ve times and counting the number of heads. We found earlier that probability of getting exactly two heads if we ip a coin ve times is ( 5 2) /2 5 = 5/16. But since ( 5 ) ( 2 = 5 ) 3, the probability of getting exactly three heads is Intro to also 5/16 = So the probability that three women will win door prizes if we draw ve names with replacement" is exactly , not so far away from the we found with 50 women and 50 men. And the approximation only gets better as we increase the number of diners. (See Table 1.) c R. P.

14 Sampling With or Without Replacement Table 1: Sampling With or Without Replacement Sampling Without Replacement # Women # Men p(#women = 3) ,000 1, ,000 5, ,000 10, Intro to Sampling With Replacement n n c R. P.

15 Distributions The assumption we used in the previous section that all outcomes should be equally likely is unnecessarily restrictive. As we saw in the tossing two dice and counting the number of spots" example, the most natural sample space was S = {2,..., 12}, but its outcomes were not equally likely. It also turns out that the assumption that S be nite can exclude certain experiments of interest to us. For example, suppose our experiment is to ip a fair coin until we get heads for the rst time. Then record the number of ips up to and including that rst heads. The natural sample space for this experiment would be S = {1, 2, 3,...}. (10) That is, there's no largest number of ips requiredeven though it's very unlikely that we'll ever need even 100 ips, for example. c R. P. Intro to

16 Distributions Denition Assume that we are given a sample space S that is either nite or countably innite. (That is either S = {x 1, x 2, x 3,..., x n } or S = {x 1, x 2, x 3,...}.) Then a probability distribution p on S is a function p : S R Intro to that satises two properties 1. 0 p(x) 1 for all x S, and 2. n i=1 p(x i) = 1, if S has n elements, or i=1 p(x i) = 1, if S is countably innite. Remark Sometimes we'll write x S p(x) = 1 to cover both cases in 2 above. c R. P.

17 Distributions Remarks Any real-valued function on S that satises the two conditions in the denition above is a probability distribution. But not all probability distributions are useful. We need to choose those functions p that somehow model phenomena we're interested in studying. Intro to c R. P.

18 Distributions Examples Example If S = n and we let p be the constant function p : S R given by p(x) = 1/n for all x S, then p is a probability distribution. This is just the equal-likelihood probability we studied in Section 7.1. We call it the uniform distribution. The experiment of selecting an element from a sample space with a uniform distribution is called selecting an element of S at random. Intro to c R. P.

19 Distributions Examples Example Recall the experiment of tossing two dice and recording the total number of spots showing. We saw earlier that the natural sample space for this experiment would be S = {2, 3,..., 12}. But, as we noted before, these outcomes are not equally likely. But now, by consulting Figure 1 above, we can give a probability distribution for this sample space. Intro to Table 2: Distribution for Tossing Two Dice x p(x) c R. P.

20 Distributions Examples Example Toss a fair coin until the rst heads appears. Record the total number of tosses needed. As noted earlier, the sample space will be S = {1, 2, 3,...}. Now let us build the probability distribution one value at a time. Since the coin is fair, the probability of getting heads on the rst toss is 1/2, and thus p(1) = 1 2. Now suppose we tossed a coin twice. The equally likely outcomes would be HH, HT, TH, and TT. So the probability of getting heads for the rst time only on the second toss would be 1/4, and thus we see that p(2) = 1 4 = ( 1 2 )2. Intro to c R. P.

21 Distributions Examples Example (cont'd) Continuing in this way by considering three total tosses, four total tosses, etc., we get that p(n) = ( 1 2 )n. Note that this p is a legitimate probability distribution, since 0 p(n) = ( 1 2 )n 1 for all n and Intro to p(n) = n=1 n=1 ( ) 1 n = 1/ /2 = 1, by the formula for the sum of a convergent geometric series. c R. P.

22 Distributions Denition The probability of an event E is the sum of the probabilities of the outcomes in E. That is, p(e) = x E p(x). (11) Intro to Note that when E is an innite set, x E p(x) is a convergent innite series. Note also that if E F, p(e) = x E p(x) x F p(x) = p(f). (12) Since E S, (12) yields p(e) p(s) = 1 for all events E. c R. P.

23 Some Facts About Distributions Some of the properties of the uniform probability distribution that we discussed in Section 7.1 continue to hold for probability distributions in general. Facts 1. If E 1 and E 2 are two events in the sample space S, then p(e 1 E 2 ) = p(e 1 ) + p(e 2 ) p(e 1 E 2 ). (13) Intro to 2. If E 1 E 2 = we say E 1 and E 2 are mutually exclusivethen p(e 1 E 2 ) = p(e 1 ) + p(e 2 ). (14) 3. p(ē) = 1 p(e). c R. P.

24 Distributions Examples Suppose we test 160 people and examine them with regard to their smoking status and blood cholesterol count. Table 3 shows the results. Table 3: Smoking Status vs. Blood Cholesterol Count Blood Cholesterol Count Smoking Status Low Moderate High Heavy Light Nonsmoker Intro to Now suppose we put the names of all the participants in a bin and then choose one of the names at random, i.e., with equal likelihood. c R. P.

25 Distributions Examples 1. Find the probability that we've chosen a person with low blood cholesterol. p(lbc) = 41/ Find the probability that the name chosen is that of a heavy smoker. p(hs) = 44/ Find the probability that we've chosen a heavy smoker with low blood cholesterol. p(hs LBC) = 6/ Suppose we want to know the probability that a heavy smoker has low blood cholesterol. Notice the dierence between this question and the question above about the probability of being a heavy smoker and having low blood cholesterol. The dierence is that we're only considering heavy smokers here. p(lbc HS) = 6/44. Intro to c R. P.

26 Smoking vs. Blood Cholesterol Table 4: Smoking Status vs. Blood Cholesterol Count (with marginal totals) Blood Cholesterol Count Smoking Status Low Moderate High Heavy Light Nonsmoker Intro to c R. P.

27 Conditional Remark We read the notation p(lbc HS) as the probability of LBC, given HS." This is a conditional probability because it is conditioned or based on an assumption. Intro to Warning If we calculate p(hs LBC), the probability that someone with low blood cholesterol is a heavy smoker, we nd p(hs LBC) = 6/41 6/44 = p(lbc HS). Thus already in this example we learn that we should never expect p(e F) to be the same as p(f E). Also note that p(e F) p(e F). c R. P.

28 Conditional Note that in the example above, we have p(lbc HS) = 6 44 = 6/160 44/160 p(hs LBC) = = p(hs) p(lbc HS). p(hs) The last equation in (15) holds because since HS LBC = LBC HS, it must be true that p(hs LBC) = p(lbc HS). (15) Intro to c R. P.

29 Conditional The observation (15) suggests the following Denition Let E and F be events with p(f) > 0. The conditional probability of E given F, denoted by p(e F), is dened as p(e F) = p(e F). (16) p(f) Intro to Remarks Since E F F, we have 0 p(e F) p(f), and hence 0 p(e F) 1. Sometimes we say, the probability of E, if F." Here the if" signals that we're assuming that F occurred. c R. P.

30 Conditional Notice that in our smoking and blood cholesterol example above, p(lbc) = 41/ > /44 = p(lbc HS). This suggests that being a heavy smoker might somehow lower one's likelihood of having low blood cholesterol. This leads to the following Intro to Denition The events E and F are independent if p(e F) = p(e), provided p(f) 0. (17) Remark This denition may be rephrased informally as E and F are independent if the fact that F has occurred doesn't aect the likelihood that E has occurred." c R. P.

31 Conditional Remarks If p(e) and p(f) are both nonzero, then (16) and (17) together imply that E and F are independent if and only if p(e F) = p(e)p(f). (18) Intro to Furthermore, if p(e) and p(f) are both nonzero, then E and F are independent if and only if p(f E) = p(f), (19) since p(e F) = p(f E). It can be shown that E and F are independent if and only if Ē and F are independent if and only if E and F are independent if and only if Ē and F are independent. c R. P.

32 Don't Confuse Mutually Exclusive" with Independent" Warning Do not confuse mutually exclusive with independent! Mutually exclusive events with nonzero probability are never independent: p(e F) = p(e F) p(f) = p( ) p(f) = 0 = 0 p(e). (20) p(f) Intro to If E and F are mutually exclusive, then p(e F) = p(e) + p(f). E and F are independent if and only if p(e F) = p(e)p(f). If you use the equation p(e F) = p(e)p(f), then you are assuming that E and F are independentwhether you want to or not and whether it's true or not! c R. P.

33 Independent" vs. Mutually Exclusive" Example My great niece will give birth to a single baby in February. The two events G, the child is a girl," and B, the child is a boy," are mutually exclusive. But they are denitely not independent, since Intro to p(b G) = p(b G) p(g) = p( ) p(g) = 0 = = p(b). 0.5 c R. P.

34 Independence Remark Independence is a two-edged sword. Sometimes we infer independence from reasoning and, as a result, may use (18), p(e F) = p(e)p(f). Example If we toss a single die, we know that the probability of two spots showing is p(d 1 ) = 1/6. If we toss the die a second time, the probability of two spots showing is p(d 2 ) = 1/6. Intro to But we believe that these events are independentthe die doesn't remember how it landed the rst time!so we conclude that the probability of two spots showing on both tosses is p(d 1 D 2 ) = p(d 1 )p(d 2 ) = (1/6)(1/6) = 1/36. c R. P.

35 Independence On the other hand, if p(e F) = p(e)p(f) is true, the events are independenteven if we don't have a specic a priori reason to believe so. Example Toss two dice, a red one and a white one. Let E be the event the number of spots showing on the red die exceeds 3" and let F be the event the total number of spots showing on both dice is 7." Then from Figure 1 we get p(e) = 1/2, p(f) = 1/6, and p(e F) = 3/36 = 1/12 = p(e)p(f). So E and F are independent. Sometimes we hedge our bets and say statistically independent." Intro to c R. P.

36 Bayes' Theorem Example Amyotrophic Lateral SclerosisALS, or Lou Gehrig's Disease"is a devastating, invariably fatal degenerative disease of the muscles. The median survival time after diagnosis is 3 years. Fortunately, its incidence is quite rare: about 2 people per 100,000 in the US population are diagnosed annually. So suppose you are tested for ALS and the test comes out positive, i.e., it says you have the disease. Should you immediately give away all your wordly possessions and retire to a monastery? Or go on a three-year spree of unlimited indulgence? Not so fast! As we will see using conditional probabilities, in tests for rare diseases there are many false positives," i.e., results where you do not actually have the disease even though the test says you do. Intro to c R. P.

37 Bayes' Theorem Example Example Suppose there is a test for ALS with the following characteristics: if you have the disease, it says so 98% of the time. On the other hand, if you don't have the disease, 99% of the time it correctly says that you don't. Question: if you test positive for ALS via this test, how likely is it that you actually have the disease? Let D be the event you have the disease," and let T be the event you test positive for the disease." We are given Intro to p(d) = p(t D) = 0.98 p( T D) = (21) And what we want is p(d T). (22) c R. P.

38 Bayes' Theorem Example One of the rst things to note is that we don't already know the answer; the likelihood we seek is not one of the numbers already given. Somehow we need to reverse the order of D and T in our conditional probabilities. We start by expanding and rewriting (22): Intro to p(d T) = p(d T) p(t) = p(t D) p(t) = p(t D)p(D). (23) p(t) We've made some progress; we can now calculate the numerator of the fraction in (23). But what about p(t), the denominator? c R. P.

39 Law of Total We compute p(t) by putting together several of the results we've seen before. We know that S = D D, and this is a disjoint union, i.e., a union of disjoint sets. As a result T = T S = T (D D) = (T D) (T D), (24) and this union is still disjoint. But since the union in (24) is disjoint, we have Intro to p(t) = p(t D) + p(t D) = p(t D)p(D) + p(t D)p( D). (25) The equation p(t) = p(t D)p(D) + p(t D)p( D) (26) is sometimes called the Law of Total. c R. P.

40 Bayes' Theorem We now combine (23) and (26) to get Bayes' Theorem: Example p(d T) = p(t D)p(D) p(t D)p(D) + p(t D)p( D). (27) (cont'd) Plugging the information given in (21) into (27), we get p(d T), the desired likelihood: Intro to p(d T) = (28) This means that if you test positive for the disease, you're likelihood of having it is less than 0.2%! c R. P.

41 Bayes' Theorem Example Example (cont'd) Let's bring this down to earth. Suppose we look at 10,000,000 Americans. Of these, 200 will be diagnosed with ALS. But how many will test positive? The set of those people among the 10,000,000 under consideration who test positive for ALS consists of the (disjoint) union of those who have ALS and test positive and those who don't have ALS but nonetheless (falsely) test positive. Since 98% of those who have ALS will test positive, 196 people will have the disease and test positive. There are 9,999,800 people in our study population who don't have ALS. And 1% of those99,998falsely test positive. So we have 100,194 altogether who test positive. c R. P. Intro to

42 Bayes' Theorem Example Example (cont'd) The empirical conditional probability of having the disease given a positive test is then # who have ALS and test positive # who test positive = , (29) Intro to Remarks The accuracy numbers of the test sounded pretty impressive at rst. What went wrong? Which of the accuracy numbers should be made better to reduce the number of false positives? The number p(t D) = 1 p( T D) should be made smaller so that the denominator of (29) won't be so large. c R. P.

43 Bayes' Theorem Example Suppose a court has decided that a fatal auto accident was caused by a defective part in a car. The manufacturer bought this part from one of three sourcesa, B, and Cand it is impossible to tell from which source the particular part came. Source A's parts are 99% reliable, and A supplied 70% of the carmaker's needs. Source B's parts are 95% reliable, and B supplied 25% of the carmaker's needs. Source C's parts are only 80% reliable, but C supplied only 5% of the parts the carmaker used. In the award of damages, the judge needs to determine a fair way in which to decide who should pay. Source A believes it shouldn't pay because its parts are much more reliable than B's and C's. Source C says that while its parts are less reliable than A's and B's, it's much less likely that the defective part came from C since C provided parts for only 5% of the carmakers cars. c R. P. Intro to

44 Bayes' Theorem Example The judge used Bayes' Theorem to decide what each supplier's share of the damage award should be. Specically, p(a D) = p(b D) = p(c D) = p(a D) p(d) p(b D) p(d) p(c D) p(d) = = = p(d A) p(d) p(d B) p(d) p(d C) p(d) Here we computed the denominators by = p(d A)p(A) p(d) = p(d B)p(B) p(d) = p(d C)p(C) p(d) p(d) = p(d A)p(A) + p(d B)p(B) + p(d C)p(C) = = = 0.24 = 0.42 (30) (31) = 0.34 (32) (33) c R. P. Intro to

45 Law of Total Example Example Suppose a certain club has 8 women and 6 men as members. If two members are chosen at random to represent the club at a conference, what is the probability that one woman and one man is chosen? Solution (1) Using counting methods from Chapter 6, we know that there are ( ) ways of picking two members from the total membership of 14, and there are ( 8 1)( 6 1) ways of choosing a woman and a man from the club. So, since at random" implies equal likelihood, the probability of choosing a man and a woman to represent the club is ( 8 )( 6 1 1) ( 8+6 ) = (34) 2 c R. P. Intro to

46 Law of Total Example Solution (2) When we discussed such a problem earlier, I claimed that choosing the two members in sequence, i.e., one after another, gives the same result as choosing the two members at the same time. Let's see if we can prove that. Let W i be the event a woman is chosen on the i-th draw. Then the event we seeka woman and a man are chosenis the event (W 1 W 2 ) (W 1 W 2 ), (35) and this union is disjoint. But now (35) implies that Intro to p(1 & 1 ) = p(w 1 W 2 ) + p(w 1 W 2 ) = p(w 2 W 1 )p(w 1 ) + p(w 2 W 1 )p(w 1 ). (36) c R. P.

47 Law of Total Example Solution (2) (cont'd) But this gives as in Solution 1. p(1 & 1 ) = = 48 91, (37) Intro to Just for fun, let us compute p(w 2 ). We know that p(w 1 ) = 8/14, but what about p(w 2 )? p(w 2 ) = p((w 2 W 1 ) (W 2 W 1 )) = p(w 2 W 1 ) + p(w 2 W 1 ) = p(w 2 W 1 )p(w 1 ) + p(w 2 W 1 )p(w 1 ) = (7/13)(8/14) + (8/13)(6/14) = 8/14. (38) c R. P.

48 Bayes' Theorem Example Example Three chips are placed in a box. One is red on both sides, a second is blue on both sides, and the third is red on one side and blue on the other. One chip is selected at random and placed on a table. Suppose that the color showing on that chip is red. What is the probability that the color underneath is also red? Intro to Any guesses? Solution Let R be the event a red chip is chosen"; let B be the event a blue chip is chosen"; let T be the event the two-color chip is chosen." Let F be the event the color showing on the chosen chip is red." c R. P.

49 Bayes' Theorem Example Solution (cont'd) Observe that if the color showing on the chosen chip is red, the underside of that chip will also be red if and only if we have chosen the red chip. So we need p(r F). We have p(r F) = p(r F) p(f) = p(f R) p(f) = p(f R)p(R) p(f) (39) Intro to and p(f) = p((f R) (F B) (F T)) = p(f R) + p(f B) + p(f T) = p(f R)p(R) + p(f B)p(B) + p(f T)p(T), (40) where addition is justied by the mutual exclusivity of the uniands. c R. P.

50 Bayes' Theorem Example Solution (cont'd) But we have p(r) = p(b) = p(t) = 1/3; p(f R) = 1; p(f B) = 0; and p(f T) = 1/2. Intro to Putting these numbers into (39) and (40), we get p(r F) = 1/3 1/2 = 2 3. c R. P.