Lecture 04: Conditional Probability. Lisa Yan July 2, 2018

Transcription

1 Lecture 04: Conditional Probability Lisa Yan July 2, 2018

2 Announcements Problem Set #1 due on Friday Gradescope submission portal up Use Piazza No class or OH on Wednesday July 4 th 2

3 Summary from last time Sample space, S: The set of all possible outcomes of an experiment. Event space, E: A subset of S. If we have equally likely outcomes, then P(E) = E / S. Two key tactics to counting probabilities: DeMorgan s law and calculating P(E c ) helps us avoid tricky counting situations. We can treat indistinct objects as distinct if it helps create equally likely outcomes. 3

4 Counts of balls and urns Suppose we have n=2 balls, labeled A and B, are placed in m=2 urns, Urns 1 and 2, where each ball is equally likely to be placed in any urn. Possibilities Urn 1 Urn 2 A, B - A B B A - A, B Equally likely Counts Urn 1 Urn 2 Prob 2 0 1/ / /4 Not equally likely 4

5 Goals for today Conditional probability Definition of conditional probability Chain Rule Law of Total Probability Bayes Theorem The Monty Hall problem 5

6 Dice, our misunderstood friends Problem: Roll two 6-sided dice, yielding values D 1 and D 2. Let E be event: D 1 + D 2 = 4. What is P(E)? Solution: S = 36 E = {(1,3), (2,2), (3,1)} P(E) = 3/36 = 1/12 6

7 Dice, our misunderstood friends Roll two 6-sided dice, yielding values D 1 and D 2. Let E be event: D 1 + D 2 = 4. Let F be event: D 1 = 2. What is P(E, given F already observed)? Solution: S = {(2,1), (2,2), (2,3), (2,4), (2,5), (2,6)} E = {(2,2)} P(E, given F already observed) = 1/6 7

8 Conditional Probability Conditional probability is the probability that E occurs given that F has already occurred. This is known as Conditioning on F. Written as: P(E F) Means: P(E, given F already observed) Sample space, S à all possible outcomes consistent with F (i.e. S Ç F) Event space, E à all outcomes in E consistent with F (i.e. E Ç F) 8

9 Conditional Probability Conditional probability is the probability that E occurs given that F has already occurred. This is known as Conditioning on F. With equally likely outcomes: P(E F) = # of outcomes in E consistent with F # of outcomes in S consistent with F EF = = SF EF F P (E) = P (E F )=

10 Conditional Probability, in general General definition: P(E F) = P(EF) P(F) Implies: (even when outcomes are not equally likely) The Chain Rule P(EF) = P(E F) P(F) By commutativity of set intersection, P(EF) = P(FE) = P(F E) P(E) What if P(F) = 0? à P(E F) undefined, of course! You have observed the impossible! 10

11 Slicing up the spam P(E F) = P(EF) P(F) For equally likely outcomes: = EF F 24 s are sent, 6 each to 4 users. 10 of the 24 s are spam. All possible outcomes are equally likely. E = user 1 receives 3 spam s. What is P(E)? F = user 2 receives 6 spam s. What is P(E F)? G = user 3 receives 5 spam s. What is P(G F)? Solution: 10 P(E) = P(E F) = No way to choose 5 spam from 4 remaining spam s! P(G F) = 6 = 0 11

12 Bit strings A bit string with m 0 s and n 1 s sent on a network. All distinct arrangements of bits are equally likely. E = first bit received is a 1 F = k of first r bits received are 1 s What is P(E F)? Solution: n 1 P(E F) = P(EF) P(F) P(F E) = k 1 r k, P(E) = m + n 1 r 1 m = P(F E)P(E) P(F) n m + n, P(F) = By definition of conditional probability n k m r k m + n r 12

13 Bit strings A bit string with m 0 s and n 1 s sent on a network. All distinct arrangements of bits are equally likely. E = first bit received is a 1 F = k of first r bits received are 1 s What is P(E F)? Solution: P(E F) = P(EF) = P(F E)P(E) By definition of P(F) P(F) conditional probability n 1 m n m k 1 r k n P(E F) = m + n 1 k r k k m + n m + n = r r 1 r 13

14 Bit strings A bit string with m 0 s and n 1 s sent on a network. All distinct arrangements of bits are equally likely. E = first bit received is a 1 F = k of first r bits received are 1 s What is P(E F)? Solution 2: P(E F) = P(picking one of k 1 s out of r bits) = k r Rock on!!! By insight 14

15 Generalized Chain Rule Chain Rule: P(EF) = P(E F) P(F) P(E 1 E 2 E 3 E n ) = P(E 1 )P(E 2 E 1 )P(E 3 E 1 E 2 ) P(E n E 1 E 2 E n-1 ) Proof: P(E 1 E 2 E 3 E n ) = P(E 1 ) x P(E 2 E 3 E n E 1 ) = P(E 1 ) x P(E 2 E 1 ) x P(E 3 E n E 1 E 2 ) = P(E 1 ) x P(E 2 E 1 ) x P(E 3 E 1 E 2 ) x P(E 4 E n E 1 E 2 E 3 ) = P(E 1 ) x P(E 1 E 2 ) x P(E 3 E 1 E 2 ) x x P(E n E 1 E 2 E n-1 ) 15

16 Card piles A deck of 52 cards is randomly divided into 4 piles (13 cards per pile). What is P(each pile contains exactly one ace)? Solution: E 1 = {Ace Spades (AS) in any one pile} E 2 = {AS and Ace Hearts (AH) in different piles} E 3 = {AS, AH, Ace Diamonds (AD) in different piles} E 4 = {All 4 aces in different piles} Compute P(E 1 E 2 E 3 E 4 ) = P(E 1 ) P(E 2 E 1 ) P(E 3 E 1 E 2 ) P(E 4 E 1 E 2 E 3 ) 16

17 Card piles E 1 = {Ace Spades (AS) in any one pile} E 2 = {AS and Ace Hearts (AH) in different piles} E 3 = {AS, AH, Ace Diamonds (AD) in different piles} E 4 = {All 4 aces in different piles} P(E 1 ) = 1 P(E 2 E 1 ) = 39/51 (39 cards not in AS pile) P(E 3 E 1 E 2 ) = 26/50 (26 not in AS or AH piles) P(E 1 E 2 E 3 E 4 ) 39 x 26 x 13 = 51 x 50 x 49 P(E 4 E 1 E 2 E 3 ) = 13/49 (13 not in AS,AH,AD piles)

18 Card piles A deck of 52 cards is randomly divided into 4 piles (13 cards per pile). What is P(each pile contains exactly one ace)? Solution 2: E: multipart experiment: Deal 4 aces into different piles. 4! 12,12,12,12 2. Distribute rest of cards such that each pile has 13 cards. S: Distribute all cards such that each pile has 13 cards. P(E) = E S ,13,13,13 18

19 Law of Total Probability E = EF È EF c Note: EF Ç EF c = Æ S F E E = S EF + S EF c P(E) = P(EF) + P(EF c ) = P(F)P(E F) + P(F c ) P(E F c ) If you cannot calculate P(E) directly, it helps to introduce a new event F and remove it. 19

20 General Law of Total Probability If F 1, F 2,, F n are a set of mutually exclusive and exhaustive events, then > P E = = P? F? P E F? 20

21 Break Attendance: tinyurl.com/cs109summer

22 Majoring in CS What is the probability that a randomly chosen student on campus is a CS major if: 25% of students on campus are juniors If a student is a junior, then they have 30% likelihood of being a CS major If a student is not a junior, then they have 20% likelihood of being a CS major Solution: M = a student is a CS major, J = a student is a junior We know: P(J) = 0.25, P(M J) = 0.30, P(M J c ) = 0.20 P(M) = P(J) P(M J) + P(J c ) P(M J c ) 0.25 x x 0.2 = General Law of Total Probability > P E = = P Law of Total probability? F? P E F? 22

23 Thomas Bayes Rev. Thomas Bayes (~ ): British mathematician and Presbyterian minister 23

24 A moment of silence 24

25 Bayes Theorem Most common form: P F E = P E F P F P E Expanded form: P F E = P E F P F P E F P F + P E F B P(F B ) 25

26 Bayesian Interpretation #1 P F E = P E F P F P E Want to find (WTF): P(F E) Know: P(E F) Bayes is a way of flipping the condition. 26

27 HIV testing A test is 98% effective at detecting HIV. However, the test has a false positive rate of 1%. 0.5% of the US population has HIV. E = you test positive for HIV with this test F = you actually have HIV What is P(F E)? Solution: P F E = P(E F) = 0.98 P(E F c ) = 0.01 P(F) = P E F P F P E F P F + P E F B P(F B ) true positive false positive prior = P F E = (0.98) (0.005) (0.98)(0.005) + (0.01)( ) P E F P F P E F P F + P E F B P(F B ) Bayes Theorem 27

28 Bayes Theorem Intuition People who test positive, E People with HIV, F All People, S 28

29 Say we have 1000 people: 5 have HIV and test positive 985 do not have HIV and test negative. 10 do not have HIV and test positive.» 0.333

30 Why it s still good to get tested Ec = you test negative for HIV F = you actually have HIV P(F E c )? HIV + HIV Test = P(E F) 0.01 = P(E F c ) Test 0.02 = P(E c F) 0.99 = P(E c F c ) P(F E c ) = P(F E c ) = P(E c F) P(F) P(E c F) P(F) + P(E c F c ) P(F c ) (0.02)(0.005)» (0.02)(0.005) + (0.99)( ) Your confidence of not having HIV goes up! 30

31 Bayesian Interpretation #2 Before the test: prior: The test: likelihood: After the test: posterior: you may have HIV, but you are 95.5% confident. (0.5% population has HIV) the test returns useful information based on whether you have HIV (98% yes if HIV, 1% yes if not HIV) you now know have more information about whether you have HIV, based on test results. posterior likelihood prior P E F P F P F E = P E Normalizing constant 31

32 Bayesian Interpretations posterior likelihood prior P E F P F P F E = P E Normalizing constant You probably think that you are better now, better now, You only say that 'cause I'm not around, not around Post Malone, 2018 Posterior 32

33 Wayne Brady + Monty Hall Problem 33

34 Monty Hall Problem aka Let s Make a Deal Behind one door is a prize (equally likely to be any door). Behind the other two doors is nothing 1. We choose a door 2. Host opens 1 of other 2 doors, revealing nothing 3. We are given an option to change to the other door. Doors A,B,C Should we switch? Note: If we don t switch, P(win) = 1/3 (random) We are comparing P(win) and P(win switch). 34

35 If we switch Solution: Without loss of generality, say we pick A (out of Doors A,B,C). P(A prize) = 1/3 P(B prize) = 1/3 P(C prize) = 1/3 1.Host opens B or C 2.We switch 3.We always lose 1.Host must open C 2.We switch to B 3.We always win 1.Host must open B 2.We switch to C 3.We always win P(win A prize, picked A, switched) = 0 P(win B prize, picked A, switched) = 1 P(win C prize, picked A, switched) = 1 P(win picked A, switched) = 1/3 * 0 + 1/3 * 1 + 1/3 * 1 = 2/3 35

36 Monty Hall, 1000 envelope version Start with 1000 envelopes (of which 1 is the prize). 1. You choose 1 envelope. P(envelope is prize) = 1/1000 P(other 999 envelopes have prize) = 999/ I open 998 of remaining 999 (showing they are empty). 999/1000 = P(998 empty envelopes had prize) + P(last other envelope has prize) = P(remaining other envelope has prize) 3. Should you switch? P(you win without switching) = P(you win with switching) = 1 original # envelopes original # envelopes - 1 original # envelopes 36

37 Summary P(F E) P(EF) P(E) Definition of Conditional Probability P(E F)P(F) P(E) Chain Rule Law of Total Probability P(E F)P(F) = P(E F)P(F) + P(E F c )P(F c ) P(EF) = P(E F) P(F) P(E) = P(F)P(E F) + P(F c ) P(E F c ) Bayes Theorem Monty Hall example: conditional probability hijinks. 37