C Examples of Bayes Calculations

Transcription

1 There are two buckets containing poker chips. Bucket A has Bucket B has 700 red chips 300 blue chips 300 red chips 700 blue chips Flip a coin to choose a bucket at random. From the chosen bucket, select 12 chips at random. (The chips should be chosen without replacement.) Suppose that this process results in 8 red chips and 4 blue chips. Given this information, what is the conditional probability P(Bucket A 8 red, 4 blue)? Said another way, given that you got 8 red and 4 blue chips, what s the probability that it came from bucket A? Clearly this probability is larger than Clearly too, you can work out this probability using Bayes theorem. For this exercise, simply guess the conditional probability. Then... you can see the solution on the next page. 1 gs2003

2 SOLUTION: The formal use of Bayes theorem gives this result: P(Bucket A 8 red, 4 blue) = ( A) P( 8 red, 4 blue Bucket A) P( 8 red, 4 blue) P Bucket Certainly P(Bucket A) = 1 2. The conditional probability in the numerator is the hypergeometric calculation , The denominator can be developed as follows: P = P( {Bucket 1 8 red, 4 blue} {Bucket 2 8 red, 4 blue} ) = P(Bucket 1 8 red, 4 blue) + P(Bucket 2 8 red, 4 blue) = P(Bucket 1) P(8 red, 4 blue Bucket 1) + P(Bucket 2) P Bucket 2) We ve know all the pieces to this except the last conditional probability. This can also be done as a hypergeometric calculation. It is , All this work leads to P = = Finally, we have P(Bucket A 8 red, 4 blue) = ( A) P( 8 red, 4 blue Bucket A) P( 8 red, 4 blue) P Bucket = = gs2003

3 Thus, the selection leads to extremely strong evidence in favor of Bucket A. Most people realize that this conditional probability should be larger than 0.50, but very few will appraise it even as high as So what s the message? Kahneman and Tversky point out that people simply do not understand the strength of the evidence that is presented to them! You can do problems like this with a method called the hypothetical hundred thousand. You start with a table like this: Bucket A Bucket B 100,000 You can do this with any large round-number total, but certainly 100,000 seems nice. You imagine that you re going to play the game 100,000 times. Certainly, we would expect to end up with 50,000 plays from each of the two buckets, so we move to this step: Bucket A 50,000 Bucket B 50, ,000 If this were really done, we would have some statistical fluctuation around this 50,000 point. However, the 50,000 values do represent the expected numbers of times for the two buckets. If you re working from Bucket A, the probability of getting would be ; we found this above. The expected number of times you d get would then be ,000 = 11,625. We enter this number in the table: Bucket A 11,625 50,000 Bucket B 50, ,000 3 gs2003

4 This value was an exact integer, for the precision at which we re working. If it were non-integer, we would keep the decimal part. Certainly the number in the upper right must be 50,000-11,625 = 38,375. Now we re at this point: Bucket A 11,625 38,375 50,000 Bucket B 50, ,000 We ll repeat this for the Bucket B values, getting to here: Bucket A 11,625 38,375 50,000 Bucket B ,625 50, ,000 The value 375 was obtained as ,000. Next give the column totals: Bucket A 11,625 38,375 50,000 Bucket B ,625 50,000 12,000 88, ,000 This says that if you play this game 100,000 times, you d expect to see the result 12,000 times. Of these, fully 11,625 came from Bucket A, so the conditional probability is 11,625 = This is almost 97%. 12,000 This example can be found in Kahneman, Slovic, and Tversky, Judgment Under Uncertainty: Heuristics and Biases, page 361. The technique called hypothetical hundred thousand is of uncertain origin. The name appears in Mind on Statistics, 2 nd edition, by Jessica Utts and R. F. Heckard, 2003, page 228. It s also mentioned in an article by Jessica Utts, What Educated Citizens Should Know About Statistics and Probability in The American Statistician, May gs2003

5 This example has a story line related to the power of legal evidence. In a certain city, taxis are easily identified by rooftop lights. There are two taxi companies in this city, Green and Blue, with cabs colored accordingly. One night, there is a hit-and-run accident involving a taxi. The vehicle was clearly a taxi, as multiple witnesses saw the rooftop light. However, only one witness (Ralph) claimed to be able to identify the cab by color. Ralph claimed that this was a Green cab. We would like to know the probability that the accident was really committed by a Green cab. Here are some facts: Ralph was given an identification test under lighting conditions similar to those the night of the accident. When the test cab was blue, he identified it correctly 80% of the time. When the test cab was green, he identified it correctly 80% of the time. A number of other people were given the same identification test, and they also produced results close to the 80% that Ralph got. In this town, 85% of the taxis are Blue. The single fallible witness saw the accident and reported that the taxi was green. As a result, the green taxi company became the defendant. Now P(E I ) represents the probability of the witness s report given that the green company is innocent; this is not directly relevant, as it merely says something about the quality of the witness. The very relevant quantity is P(I E), which is the probability that the green company is innocent, given the evidence. OK. here s the solution. Let s simulate 100,000 versions of the accident. R a l p h s a y s Accident committed by Blue Green Blue 85,000 Green 15, ,000 Thus, if we re going to set up 100,000 simulated versions, 85% of these ought to be caused by Blue taxis. With no witnesses, after all, we d blame Blue with probability gs2003

6 Now let s fill in Ralph s judgments. R a l p h s a y s Accident committed by Blue Green Blue 68,000 17,000 85,000 Green 3,000 12,000 15, ,000 We ve done this so that 80% of the time (in each row) Ralph makes the correct call. Let s now get the totals. R a l p h s a y s Accident committed by Blue Green Blue 68,000 17,000 85,000 Green 3,000 12,000 15,000 71,000 29, ,000 Thus, among the 29,000 times that Ralph would identify the taxi as Green, only 12,000 of those corresponded to actual Green taxis. Thus, we ve found P(taxi was Green Ralph said Green ) = 12,000 29, This is far from convincing. It is quite far from 80%, and it s even below 50%! (This is of course well above 15%, which is the overall proportion of Green taxis.) 6 gs2003

7 EXAMPLE: The probability that a medical test will correctly detect the presence of a certain disease is 98%. The probability that this test will correctly detect the absence of the disease is 95%. The disease is fairly rare, found in only 0.5% of the population. If you have a positive test (meaning that the test says yes, you got it ) what is the probability that you really have the disease? The best way to deal with such problems is to apply the proportions exactly to a large population. Say that you have 100,000 people. With the given facts, 0.5% of these, or ,000 = 500 actually have the disease. Our state of information is this: Test POSITIVE Test NEGATIVE Have disease? YES 500 Have disease? NO 99, ,000 We re told that the test will correctly detect the presence in 98% of the people who actually have the disease. Thus, we expand our information to this (using 98% 500 = 490 and getting the 10 by subtraction): Test POSITIVE Test NEGATIVE Have disease? YES Have disease? NO 99, ,000 We are also told that the test will correctly note the absence of the disease in 95% of the people who don t have the disease. Since 95% 99,500 = 94,525, we complete the table as follows: Test POSITIVE Test NEGATIVE Have disease? YES Have disease? NO 4,975 94,525 99,500 5,465 94, ,000 Of course, in this final step we can note the column totals also. Conditional on a positive test, what s the probability of actually having the disease? We see that out of 5,465 persons showing positive on the test, only 490 have the disease. Our 490 probability is then %. This is PV+ = P(D T+) = predictive value 5,465 7 gs2003

8 positive. You should compare this to 0.5%, the probability of having the disease if the test is not done at all. Conditional on a negative test, what s the probability of NOT having the disease? It s 94, This is PV- = P(D T-) = predictive value negative. This should 94,535 be compared to 0.995, the probability of not having the disease if the test is not done at all.) Also, P(T+ D ) is called sensitivity, while P(T- D ) is called specificity. This is highly entertaining in the medical context, but it also applies directly to screening for uncommon industrial defects. 8 gs2003