Discrete Probability

Size: px

Start display at page:

Download "Discrete Probability"

Leo Floyd
5 years ago
Views:

1 Discrete Probability Mark Muldoon School of Mathematics, University of Manchester M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 1/38

2 Overview Mutually exclusive Independent More about combining Today we ll discuss ways to learn how to think about that are influenced by chance. Basic probability: famous Frenchmen, coins and dice Definitions and rules: mutually exclusive and Expectation: given probabilities, what can we compute? An application: going to the dogs Conditional probability: for example, the probability that a child smokes, given that her parents do. More applications: why it s very hard to detect rare disorders. M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 2/38

Famous Frenchmen Mutually exclusive Independent More about combining Blaise Pascal (1623-1662) If God does not exist, one will

Pierre de Fermat (1601-1665)... a man of great erudition, has contact with men of learning everywhere.

3 Famous Frenchmen Mutually exclusive Independent More about combining Blaise Pascal ( ) If God does not exist, one will lose nothing by believing in him, while if he does exist, one will lose everything by not believing... we are compelled to gamble. Pierre de Fermat ( )... a man of great erudition, has contact with men of learning everywhere. But he is rather preoccupied, he does not report cases well and is confused. M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 3/38

4 What does probability mean? Mutually exclusive Independent More about combining To say that an event has probability p means that the long-term average of is p. Number of times event occurs Number of times it could have occured Example 3.1 (A fair coin) Two possible outcomes: Heads and Tails Each assumed equally likely, so P(Heads) = P(Tails) = 1/2 M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 4/38

5 Properties of probabilities Mutually exclusive Independent More about combining Probabilities are numbers 0 p 1. Given an exhaustive list of possible outcomes, their probabilities add up to one. The pair A happens and A doesn t happen are exhaustive, so P(not A) = 1 P(A) M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 5/38

6 Mutually exclusive Mutually exclusive Independent More about combining Two are mutually exclusive if one precludes the other, for example: Toss a coin and get Heads and Get Tails on the same toss. Example 3.2 (Drawing cards) Consider drawing a card from an ordinary deck: what it the probability of getting an ace? M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 6/38

7 Mutually exclusive Mutually exclusive Independent More about combining Two are mutually exclusive if one precludes the other, for example: Toss a coin and get Heads and Get Tails on the same toss. Example 3.3 (Drawing cards) Consider drawing a card from an ordinary deck: what it the probability of getting an ace? Answer Number of aces Number of cards in deck = 4 52 = M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 6/38

8 Which are mutually exclusive? Mutually exclusive Independent More about combining Example 3.4 (Mutually exclusive ) Consider the following, which are possible outcomes when drawing a card from a standard, 52-card deck Event A: drawing a Heart. Event B: drawing a face card. Event C: drawing a three. Which of the pairs of A&B, A&C, B&C, are mutually exclusive? M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 7/38

9 Which are mutually exclusive? Mutually exclusive Independent More about combining Example 3.5 (Mutually exclusive ) Consider the following, which are possible outcomes when drawing a card from a standard, 52-card deck Event A: drawing a Heart. Event B: drawing a face card. Event C: drawing a three. Which of the pairs of A&B, A&C, B&C, are mutually exclusive? Answer: Only B & C are mutually exclusive. M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 7/38

10 Addition rule for mutually exclusive Mutually exclusive Independent More about combining The previous example suggests a rule for working out the probability of either of two mutually exclusive happening: If A & B are mutually exclusive, P(A or B) = P(A) + P(B). Example 3.6 (Rolling a die) A single roll of a die may show a 1 or a 2, but not both. The probability that it shows either a 1 or a 2 is 1/6 + 1/6 = 1/3. M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 8/38

11 Independent Mutually exclusive Independent More about combining Two are independent if knowing that one has happened tells us nothing about whether the other will happen. Example 3.7 (Tossing two coins) Consider tossing a penny and a pound coin. Use h & t to show the result for the penny, and H & T, for the pound. Possible outcomes are {hh, ht, th, tt}. Each is equally likely. M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 9/38

12 Multiplication rule for Mutually exclusive Independent More about combining By counting it is clear that P(Heads on penny) = (2/4) = 0.5 P(Heads on pound) = (2/4) = 0.5 P(Heads on both) = (1/4) = 0.25 Example suggests a rule for probability of two independent happening together: If A & B are, P(A and B) = P(A) P(B). M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 10/38

13 More about combining Mutually exclusive Independent More about combining Finally, there is a rule for combining the probabilities of that are not mutually exclusive (i.e. those for which P(A & B) 0). Generally, P(A or B) = P(A) + P(B) - P(A & B). To see how this works consider rolling two dice, one six-sided and one four-sided and consider A The four-sided die comes up an even number; B The sum of the two rolls is an even number. M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 11/38

14 Outcomes for the two dice Mutually exclusive Independent More about combining Outcomes contributing to event A appear in dashed boxes. Those contributing to event B are circled. Six outcomes contribute to both. M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 12/38

15 Using the rule Mutually exclusive Independent More about combining Counting up from the diagram, one can see the rule in action P(A or B) = P(A) + P(B) P(A and B) = (12/24) + (12/24) (6/24) = (1/2) + (1/2) (1/4) = (3/4) M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 13/38

16 Review: mutually exclusive Mutually exclusive Independent More about combining If A & B are mutually exclusive, which of the following statements are true? a) P(A or B) = P(A) + P(B) b) P(A and B) = 0 c) P(A and B) = P(A) P(B) d) P(A) = P(B) e) P(A) + P(B) = 1 M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 14/38

17 Review: mutually exclusive Mutually exclusive Independent More about combining If A & B are mutually exclusive, which of the following statements are true? a) P(A or B) = P(A) + P(B) b) P(A and B) = 0 c) P(A and B) = P(A) P(B) d) P(A) = P(B) e) P(A) + P(B) = 1 Answer: only a) and b) are true. M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 14/38

18 Review: independence Mutually exclusive Independent More about combining The probability of a woman having condition X is 0.20 while the probability of her having Y is If these probs are independent, which of the following is true? a) prob. she has both conditions is 0.01; b) prob. she has both conditions is 0.25; c) prob. she has either condition, or both, is 0.24; d) if she has X, prob. she has Y also is 0.01; e) if she has Y, prob. she has X also is M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 15/38

19 Review: independence Mutually exclusive Independent More about combining The probability of a woman having condition X is 0.20 while the probability of her having Y is If these probs are independent, which of the following is true? a) prob. she has both conditions is 0.01; b) prob. she has both conditions is 0.25; c) prob. she has either condition, or both, is 0.24; d) if she has X, prob. she has Y also is 0.01; e) if she has Y, prob. she has X also is Answer: a), c) and e) are true. M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 15/38

Probability: going to the dogs Mutually exclusive Independent More about combining Everyone knows that the best way to make money at the track is to be the

20 Probability: going to the dogs Mutually exclusive Independent More about combining Everyone knows that the best way to make money at the track is to be the bookie but what does the bookie do? The entry above is the sort of data he has to work with. M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 16/38

21 The bookie s job Mutually exclusive Independent More about combining And his job is to choose entries in the following table Punter bets Bookie pays 1 on Farloe Melody 3 if he wins 3 on Chini Chin Chin 1 if she wins So that he makes a decent living and doesn t have to worry about a run of bad luck that would force into bankruptcy. a a This problem was inspired by one in M. Baxter & A. Rennie, Financial Calculus, CUP, (1998). M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 17/38

22 A mathematician s dog track Mutually exclusive Independent More about combining Simplify the problem drastically Only two dogs in the race, A and B. We, the bookies, know true probability of dogs winning: P(A) = 0.25 and P(B) = Punters bet 10K on dog A, 20K on dog B. How should we set the odds? M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 18/38

23 First effort: fair odds Mutually exclusive Independent More about combining Set the odds using the probabilities we know Punter bets Bookie pays 1 on dog A 3 if he wins 3 on dog B 1 if she wins How much money should we expect to make? M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 19/38

24 The two possible outcomes Mutually exclusive Independent More about combining If dog A wins we get to keep all the money wagered on dog B, but have to pay out 3 for every pound wagered on dog A. Our winnings are thus [ 20, , 000] = 10K while if dog B wins we get to keep the money wagered on A, but have to pay out 1 for every 3 wagered on dog B: [ 10, 000 (1/3) 20, 000] = (1/3) 10, 000 These are the two contributions two our earnings... M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 20/38

25 Expected earnings Mutually exclusive Independent More about combining Bearing in mind the definition of probability as long-term frequency, if we ran this race many times we would expect to earn P(A wins)(profit if A wins) + P(B wins)(profit if B wins) = (0.25) ( 10K) + (0.75) ((1/3) 10K) = 2.5K + ( 2.5K) = 0 Two major problems: a) Average long-term earnings are zero b) Sometimes lose money, so may go broke M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 21/38

26 Summary Mutually exclusive Independent More about combining Assembling all the results into one display Punter bets We pay 1 on dog A 3 if he wins 3 on dog B 1 if she wins Outcome Prob. Financial result Contrib. A wins 0.25 ( 20K K) - 2.5K B wins 0.75 ( 10K - (1/3) 20K) 2.5K Totals M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 22/38

27 Alter odds (payments) slightly Mutually exclusive Independent More about combining Punter bets We pay 5 on dog A 14 if he wins 15 on dog B 4 if she wins Outcome Prob. Financial result Contrib. A wins 0.25 ( 20K - (14/5) 10K) - 2K B wins 0.75 ( 10K - (4/15) 20K) 3.5K Totals K So now we can make money in the long run, but go broke first. M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 23/38

28 Odds based on the betting Mutually exclusive Independent More about combining Punter bets We pay 1 on dog A 2 if he wins 2 on dog B 1 if she wins Outcome Prob. Financial result Contrib. A wins 0.25 ( 20K K) 0 B wins 0.75 ( 10K - (1/2) 20K) 0 Totals 1.0 0K Neither outcome causes us to lose money... M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 24/38

29 Shade odds in our favour Mutually exclusive Independent More about combining Punter bets We pay 5 on dog A 9 if he wins 10 on dog B 4 if she wins Outcome Prob. Financial result Contrib. A wins 0.25 ( 20K - (9/5) 10K) B wins 0.75 ( 10K - (4/10) 20K) Totals 1.0 M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 25/38

30 Shade odds in our favour Mutually exclusive Independent More about combining Punter bets We pay 5 on dog A 9 if he wins 10 on dog B 4 if she wins Outcome Prob. Financial result Contrib. A wins 0.25 ( 20K - (9/5) 10K) 0.5K B wins 0.75 ( 10K - (4/10) 20K) 1.5K Totals 1.0 2K We make money no matter what happens!?! M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 25/38

31 Reality check Mutually exclusive Independent More about combining Our problem too abstract Essentially the same phenomenon as the stock market: gambling on underlying random system can make money, no matter which way the gamble goes Bookies and brokerage houses do sometimes go broke. Real bookies must offer odds without knowing how much money will ultimately be wagered: adapt as bets come in. M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 26/38

32 Conditional probability Mutually exclusive Independent More about combining Want a concise notion/notation for the probability that one event occurs, given that another has. Example 3.8 Roll a six-sided die: what is the probability of getting a two, given that the result is an even number? There are three possible even numbers, { 2, 4, 6 } and only one of them is a 2, so by direct counting the probability is (1/3). M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 27/38

33 The notation P(A B) Mutually exclusive Independent More about combining Write conditional probabilities as P(A B) and read them as the probability of A given B". Examples include: P( It will rain tomorrow it is raining now ) P( It will rain tomorrow one is in Manchester ) P( Woman gets breast cancer mother and sister did ) M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 28/38

34 A sum rule Mutually exclusive Independent More about combining The simplest rule about conditional probabilities underlies reasonable statements such as: P( rain Manchester ) + P( no rain Manchester ) = 1. More formally, the rule is If one has an exhaustive list of mutually exclusive then their conditional probabilities add up to one. M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 29/38

35 Recovering ordinary probabilities Mutually exclusive Independent More about combining Sometimes one needs to pass from conditional probabilities back to non-conditional ones. The main tool one needs is the formula: P(A&B) = P(A B) P(B) ( ) M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 30/38

36 Using conditional probability Mutually exclusive Independent More about combining Epidemiology of lung cancer Divide subjects into three groups Heavy smokers: more that 40 a day Smokers: up to 39 per day Non-smokers: none Find risk of cancer for each group, e.g. continued... P(lung cancer heavy smoker). M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 31/38

37 Using... Mutually exclusive Independent More about combining Use conditional probabilities to find risk for general population: P( subject develops lung cancer ) = P( [cancer & heavy smoker] or [cancer & smoker] or [cancer & non-smoker]) = P( cancer & heavy smoker ) + P( cancer & smoker ) + P( cancer & non-smoker ) M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 32/38

38 Using... Mutually exclusive Independent More about combining Then use ( ) to say P( subject develops lung cancer ) = P( cancer heavy smoker ) P( heavy smoker ) + P( cancer smoker ) P( smoker ) + P( cancer non-smoker ) P( non-smoker ) M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 33/38

39 Bayes Theorem Mutually exclusive Independent More about combining On the left side of ( ) the A and B play the same role: A&B means the same thing as B&A. On the right things seem to be different: P(A B) is not generally the same as P(B A), but P(A B) P(B) = P(A&B) = P(B&A) = P(B A) P(A) which means P(A B) P(B) = P(B A) P(A) This final expression is sometimes known as Bayes Theorem. M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 34/38

40 Application: screening for Mutually exclusive Independent More about combining Consider a screening program where target condition is rare (prevalence 0.5%); test correctly flags 99% of ill patients; test also flags 5% of healthy subjects (false positive). What is the probability that a patient with a positive test result is actually ill? M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 35/38

41 Formulate the problem Mutually exclusive Independent More about combining Using the language of conditional probability we want P( ill positive test ); we have P( pos ill ) = 0.99 P( pos well ) = 0.05 P( ill ) = P( well ) = (1 P( ill )) = Start with Bayes Theorem P( ill pos ) P( pos ) = P( pos ill ) P( ill ) M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 36/38

42 Solve for necessary probabilities Mutually exclusive Independent More about combining P( ill pos ) = P( pos ill ) P( ill ) P( pos ) We need P( pos ), but we can find it with a calculation similar to the one about probability of lung cancer sketched earlier: P( pos ) = P( pos ill ) P( ill ) + P( pos well ) P( well ) M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 37/38

43 Assemble results Mutually exclusive Independent More about combining P( ill pos ) = P( pos ill ) P( ill ) P( pos ill ) P( ill ) + P( pos well ) P( well ) = ( )/( ) 0.09 Discouraging: a positive result from a powerful-seeming test gives only a lukewarm indication that the patient is actually ill. Rare true positive results are overwhelmed by false alarms. M05: Mathematical Methods, January 30, 2007 Discrete Probability - p. 38/38

Today we ll discuss ways to learn how to think about events that are influenced by chance.

Overview Today we ll discuss ways to learn how to think about events that are influenced by chance. Basic probability: cards, coins and dice Definitions and rules: mutually exclusive events and independent