Priors, Total Probability, Expectation, Multiple Trials

Transcription

1 Priors, Total Probability, Expectation, Multiple Trials CS 2800: Discrete Structures, Fall 2014 Sid Chaudhuri

2 Bayes' Theorem Given: prior probabilities of hypotheses, and the probability that each hypothesis produces the observed evidence Produces: probability of a particular hypothesis, given the observed evidence. P( A B)= P(B A) P( A) P (B)

3 Estimating P(B) Total Probability Theorem: If A 1, A 2, A 3, is a (finite or countably infinite) partition of S (they are pairwise disjoint subsets and their union is S), and B is any event in S, then P(B) = P( i (B A i )) (Set theory) = i P(B A i ) (Axiom 3) = i P(B A i ) P(A i ) (Definition)

4 S A 2 A 1 A 3

5 S B A 2 A 1 B A 1 B A 2 B A 3 A 3

6 Recall: Medical Diagnosis Suppose: 1 in 1000 people carry a disease, for which there is a pretty reliable test Probability of a false negative (carrier tests negative) is 1% (so probability of carrier testing positive is 99%) Probability of a false positive (non-carrier tests positive) is 5% A person just tested positive. What are the chances (s)he is a carrier of the disease?

7 A counter-intuitive result The reliable test gives a positive result, but the chance the patient has the disease is only about 2%. Informally: the rarity of the disease outweighs the small chance of error.

8 Should we stop worrying? Scrubs

9 No. Should we stop worrying?

10 Should we stop worrying? No. The consequences of being wrong are pretty severe.

11 Should we stop worrying? No. The consequences of being wrong are pretty severe. How can we better interpret this result?

12 You live Possibilities

13 Possibilities You live You die (let's assume the disease is always fatal)

14 Possibilities You live You die (let's assume the disease is always fatal) Improvement to your current state: +1 (if we agree living longer can enrich our life experience)

15 Possibilities You live Improvement to your current state: +1 (if we agree living longer can enrich our life experience) You die (let's assume the disease is always fatal) Improvement to your current state:

16 Possibilities You live Improvement to your current state: +1 (if we agree living longer can enrich our life experience) You die (let's assume the disease is always fatal) Improvement to your current state: (Yes, I'm pulling these numbers out of a (nonprobabilistic) posterior, but they're illustrative)

17 Expectation P(Living Positive) Utility of living + P(Dying Positive) Utility of dying

18 Expectation P(Living Positive) Utility of living + P(Dying Positive) Utility of dying =

19 Expectation P(Living Positive) Utility of living + P(Dying Positive) Utility of dying =

20 Expectation P(Living Positive) Utility of living + P(Dying Positive) Utility of dying = The exact value isn't meaningful in this example. Just note that despite a small probability of a bad outcome, the highly negative weighted average suggests we might have reason to be worried.

21 Expectation The expectation (or expected outcome, or expected value...) of an experiment with possible outcomes {x 1, x 2, x 3,...} is E(x) = i P(x i ) x i

22 Expectation The expectation (or expected outcome, or expected value...) of an experiment with possible outcomes {x 1, x 2, x 3,...} is E(x) = i P(x i ) x i If the outcomes are not numbers (or more generally, not elements of a vector space) which can be meaningfully added/scaled, we can assign them numeric utility values u(x), and inspect the expected utility i P(x i ) u(x i )

23 Expected Weather in Ithaca P(sunny) P(rainy) Expected weather P(cloudy) P(snow)

24 Expected Weather in SoCal P(sunny) Expected weather P(cloudy) P(rainy) P(snow)

25 Expectation Think of it as a center of mass, with the probabilities being the masses and the outcomes/utilities their (point) positions A useful statistic, but only a guide, not a complete picture of human decision-making Gold Medal Bodies

26

27

28

29 Multiple Tests First test: T 1 = { Positive, Negative } Second test: T 2 = { Positive, Negative } Ignoring patient status for the moment

30 Multiple Tests First test: T 1 = { Positive, Negative } Second test: T 2 = { Positive, Negative } Ignoring patient status for the moment T = T 1 T 2 = { (Positive, Positive), (Positive, Negative), (Negative, Positive), (Negative, Negative) }

31 Multiple Tests First test: T 1 = { Positive, Negative } Second test: T 2 = { Positive, Negative } Ignoring patient status for the moment T = T 1 T 2 = { (Positive, Positive), (Positive, Negative), (Negative, Positive), (Negative, Negative) } Positive in first test = A 1 =?

32 Multiple Tests First test: T 1 = { Positive, Negative } Second test: T 2 = { Positive, Negative } Ignoring patient status for the moment T = T 1 T 2 = { (Positive, Positive), (Positive, Negative), (Negative, Positive), (Negative, Negative) } Positive in first test = A 1 = { (Positive, Positive), (Positive, Negative) }

33 Multiple Tests First test: T 1 = { Positive, Negative } Second test: T 2 = { Positive, Negative } Ignoring patient status for the moment T = T 1 T 2 = { (Positive, Positive), (Positive, Negative), (Negative, Positive), (Negative, Negative) } Positive in first test = A 1 = { (Positive, Positive), (Positive, Negative) } Positive in second test = A 2 =?

34 Multiple Tests First test: T 1 = { Positive, Negative } Second test: T 2 = { Positive, Negative } Ignoring patient status for the moment T = T 1 T 2 = { (Positive, Positive), (Positive, Negative), (Negative, Positive), (Negative, Negative) } Positive in first test = A 1 = { (Positive, Positive), (Positive, Negative) } Positive in second test = A 2 = { (Positive, Positive), (Negative, Positive) }

35 Multiple Tests D = { Carrier, NotCarrier } Sample space of first test S 1 = T 1 D Sample space of second test S 2 = T 2 D Full sample space S = T 1 T 2 D E.g. event that first test is positive, second is unknown, and person is a carrier: { (Positive, Positive, Carrier), (Positive, Negative, Carrier) }

36 Multiple Tests E.g. event that person is a carrier: { (Positive, Positive, Carrier), (Positive, Negative, Carrier), (Negative, Positive, Carrier), (Negative, Negative, Carrier) } E.g. event that first test is positive: { (Positive, Positive, Carrier), (Positive, Negative, Carrier), (Positive, Positive, NotCarrier), (Positive, Negative, NotCarrier) }

37 Multiple Independent Tests We often make the assumption that the test results are independent of each other (but not, of course, of the patient's carrier status). In other words: P(Test 1 positive and Test 2 positive Carrier) = P(Test 1 positive Carrier) P(Test 2 positive Carrier) and similarly for all other combinations.

38 Thought for the Day #1 When might this assumption be unjustified?

39 P (C Y 1 Y 2 ) Multiple Independent Tests

40 Multiple Independent Tests P (C Y 1 Y 2 ) Is Carrier Second test positive (said Yes ) First test positive (said Yes )

41 Multiple Independent Tests P (Y 1 Y 2 C) P (C ) P (C Y 1 Y 2 ) = (Bayes' Theorem) P (Y 1 Y 2 )

42 Multiple Independent Tests P (C Y 1 Y 2 ) = P (Y 1 Y 2 C) P (C ) P (Y 1 Y 2 ) (Bayes' Theorem) = P (Y 1 Y 2 C ) P (C) P (Y 1 Y 2 C ) P (C )+ P (Y 1 Y 2 C ') P (C ') (Total Probability Theorem)

43 Multiple Independent Tests P (C Y 1 Y 2 ) = P (Y 1 Y 2 C) P (C ) P (Y 1 Y 2 ) (Bayes' Theorem) = = P (Y 1 Y 2 C ) P (C) P (Y 1 Y 2 C ) P (C )+ P (Y 1 Y 2 C ') P (C ') (Total Probability Theorem) P (Y 1 C) P (Y 2 C) P (C) P (Y 1 C) P (Y 2 C) P (C)+ P (Y 1 C ') P (Y 2 C ') P (C ') (Independence assumption)

44 Multiple Independent Tests P (C Y 1 Y 2 ) = P (Y 1 Y 2 C) P (C ) P (Y 1 Y 2 ) (Bayes' Theorem) = P (Y 1 Y 2 C ) P (C) P (Y 1 Y 2 C ) P (C )+ P (Y 1 Y 2 C ') P (C ') (Total Probability Theorem) = = P (Y 1 C) P (Y 2 C) P (C) P (Y 1 C) P (Y 2 C) P (C)+ P (Y 1 C ') P (Y 2 C ') P (C ') (Independence assumption) (From problem statement. The modeling assumption is that these values also apply to the product space)

45 Multiple Independent Tests P (C Y 1 Y 2 ) = P (Y 1 Y 2 C) P (C ) P (Y 1 Y 2 ) (Bayes' Theorem) = P (Y 1 Y 2 C ) P (C) P (Y 1 Y 2 C ) P (C )+ P (Y 1 Y 2 C ') P (C ') (Total Probability Theorem) = = P (Y 1 C) P (Y 2 C) P (C) P (Y 1 C) P (Y 2 C) P (C)+ P (Y 1 C ') P (Y 2 C ') P (C ') (Independence assumption) = (From problem statement. The modeling assumption is that these values also apply to the product space)

46 Thought for the Day #2 Can you write down, in full, all the outcomes in each event involved in the previous slide? Note that these events are all subsets of S = T 1 T 2 D

47 Thought for the Day #3 Can you work out the probability of the patient being a carrier, for every other combination of test results?