1.7: Bayes Theorem. Jiakun Pan. Feb 4, 2019

Transcription

1 1.7: Bayes Theorem Jiakun Pan Feb 4, 2019

2 Bayesian Filtering For experiments of two parts, Bayesian filtering is an idea of predicting the second part based on the first part:

3 Bayesian Filtering For experiments of two parts, Bayesian filtering is an idea of predicting the second part based on the first part: programs regard incoming s with too many typos as spams, and quarantine automatically.

4 Bayesian Filtering For experiments of two parts, Bayesian filtering is an idea of predicting the second part based on the first part: programs regard incoming s with too many typos as spams, and quarantine automatically. When your FICO score goes up, your credit limit gets raised.

5 Bayesian Filtering For experiments of two parts, Bayesian filtering is an idea of predicting the second part based on the first part: programs regard incoming s with too many typos as spams, and quarantine automatically. When your FICO score goes up, your credit limit gets raised. If you drive safely for a whole year, your auto insurance policy will be cheaper.

6 Bayesian Filtering For experiments of two parts, Bayesian filtering is an idea of predicting the second part based on the first part: programs regard incoming s with too many typos as spams, and quarantine automatically. When your FICO score goes up, your credit limit gets raised. If you drive safely for a whole year, your auto insurance policy will be cheaper. To diagnose cold symptoms, physicians may ask patients if they have traveled outside the U.S. within 30 days.

7 Bayesian Filtering For experiments of two parts, Bayesian filtering is an idea of predicting the second part based on the first part: programs regard incoming s with too many typos as spams, and quarantine automatically. When your FICO score goes up, your credit limit gets raised. If you drive safely for a whole year, your auto insurance policy will be cheaper. To diagnose cold symptoms, physicians may ask patients if they have traveled outside the U.S. within 30 days. Recommendation letter matters heavily in application of jobs in modern society. Etc..

8 Bayesian Reasoning However, sometimes we only know the outcome of the second part of an experiment, and want to attribute this to some reason, which is in the unobserved first part.

9 Bayesian Reasoning However, sometimes we only know the outcome of the second part of an experiment, and want to attribute this to some reason, which is in the unobserved first part. Then Bayesian reasoning comes to help.

10 Bayesian Reasoning However, sometimes we only know the outcome of the second part of an experiment, and want to attribute this to some reason, which is in the unobserved first part. Then Bayesian reasoning comes to help. Recall the GRE sample problem...

11 GRE Sample There are two boxes each of which contains four balls. Box A has one red ball and three black balls, while box B has two red balls and two black balls. You randomly pick a ball from them, and it turns out to be red. Question:What is the probability of the ball is from box A? Answer:1/3.

12 Solving the Sample Problem Let S be the ball is picked from the two boxes, E be event the ball is picked from box A, and F be the ball is red, then P(E F ) stands for the probability of

13 Solving the Sample Problem Let S be the ball is picked from the two boxes, E be event the ball is picked from box A, and F be the ball is red, then P(E F ) stands for the probability of the ball is picked from box A, provided that it is red.

14 Solving the Sample Problem Let S be the ball is picked from the two boxes, E be event the ball is picked from box A, and F be the ball is red, then P(E F ) stands for the probability of the ball is picked from box A, provided that it is red. By definition, we know P(E F ) = P(E F ), P(F )

15 Solving the Sample Problem Let S be the ball is picked from the two boxes, E be event the ball is picked from box A, and F be the ball is red, then P(E F ) stands for the probability of the ball is picked from box A, provided that it is red. By definition, we know P(E F ) = P(E F ), P(F ) and the probability of the picked ball is the red ball of box A is P(E F ) = n(e F ) n(s) = 1 8, so it boils down to determine P(F ).

16 Solving the Sample Problem (Continued) By definition, P(F ) = n(f ) n(s) = 3 8.

17 Solving the Sample Problem (Continued) By definition, So the answer is P(F ) = n(f ) n(s) = 3 8. P(E F ) = 1/8 3/8 = 1 3.

18 Solving the Sample Problem (Continued) By definition, So the answer is P(F ) = n(f ) n(s) = 3 8. P(E F ) = 1/8 3/8 = 1 3. Bayes reasoning will be crucial when P(F ) can t be found easily, for example, when n(s) and n(f ) are unknown, or when probability is provided after the first step, so that you only have partial information on P(F ).

19 Sketch of a Novel A millionaire was murdered.

20 Sketch of a Novel A millionaire was murdered. By the investigation of a detective, there are three suspects.

21 Sketch of a Novel A millionaire was murdered. By the investigation of a detective, there are three suspects. A, maid of the victim; B, manager of the millionaire s family foundation; and C, professional assassin who killed for money.

22 Sketch of a Novel A millionaire was murdered. By the investigation of a detective, there are three suspects. A, maid of the victim; B, manager of the millionaire s family foundation; and C, professional assassin who killed for money. The detective also found the crime was committed in one blow by one person. In other words, only one of A, B, and C did try to murder, and the other two were sleeping alone at home.

23 Sketch of a Novel (Continued) Combining all information about likelihood (incentive, alibi, twitters, etc), the detective thinks the probability for A, B, and C to murder is 60%, 35%, and 5% respectively.

24 Sketch of a Novel (Continued) Combining all information about likelihood (incentive, alibi, twitters, etc), the detective thinks the probability for A, B, and C to murder is 60%, 35%, and 5% respectively. Estimating successful rate, the detective guesses A has 5% probability to complete this move, if A made the attempt.

25 Sketch of a Novel (Continued) Combining all information about likelihood (incentive, alibi, twitters, etc), the detective thinks the probability for A, B, and C to murder is 60%, 35%, and 5% respectively. Estimating successful rate, the detective guesses A has 5% probability to complete this move, if A made the attempt. Similarly, B has 15% successful rate, and C has 95%.

26 Sketch of a Novel (Continued) Combining all information about likelihood (incentive, alibi, twitters, etc), the detective thinks the probability for A, B, and C to murder is 60%, 35%, and 5% respectively. Estimating successful rate, the detective guesses A has 5% probability to complete this move, if A made the attempt. Similarly, B has 15% successful rate, and C has 95%. Question: How suspicious is each of these suspects?

27 Sketch of a Novel (Continued) Let E 1, E 2, E 3 be event A, B, C tried to murder respectively, F be event the rich man died of murder. How to translate the probability of A tried to murder, given that the try turned out to be successful?

28 Sketch of a Novel (Continued) Let E 1, E 2, E 3 be event A, B, C tried to murder respectively, F be event the rich man died of murder. How to translate the probability of A tried to murder, given that the try turned out to be successful? P(E 1 F ).

29 Sketch of a Novel (Continued) Let E 1, E 2, E 3 be event A, B, C tried to murder respectively, F be event the rich man died of murder. How to translate the probability of A tried to murder, given that the try turned out to be successful? P(E 1 F ). By definition, and by product rule we have that P(E 1 F ) = P(E 1 F ), (1) P(F )

30 Sketch of a Novel (Continued) Let E 1, E 2, E 3 be event A, B, C tried to murder respectively, F be event the rich man died of murder. How to translate the probability of A tried to murder, given that the try turned out to be successful? P(E 1 F ). By definition, P(E 1 F ) = P(E 1 F ), (1) P(F ) and by product rule we have that P(E 1 F ) = P(E 1 )P(F E 1 ) = 60% 5% = 0.03.

31 Sketch of a Novel (Continued) Let E 1, E 2, E 3 be event A, B, C tried to murder respectively, F be event the rich man died of murder. How to translate the probability of A tried to murder, given that the try turned out to be successful? P(E 1 F ). By definition, P(E 1 F ) = P(E 1 F ), (1) P(F ) and by product rule we have that P(E 1 F ) = P(E 1 )P(F E 1 ) = 60% 5% = Similarly, P(E 2 F ) = 35% 15% = , and P(E 3 F ) = 5% 95% = It remains to find P(F ).

32 Sketch of a Novel (Continued) Recall that we assumed only one in the three tried to kill, and nobody else, so

33 Sketch of a Novel (Continued) Recall that we assumed only one in the three tried to kill, and nobody else, so events E 1 F, E 2 F, and E 3 F make a partition of event F.

34 Sketch of a Novel (Continued) Recall that we assumed only one in the three tried to kill, and nobody else, so events E 1 F, E 2 F, and E 3 F make a partition of event F. Here by partition we mean a separation with no omission or overlap.

35 Sketch of a Novel (Continued) Recall that we assumed only one in the three tried to kill, and nobody else, so events E 1 F, E 2 F, and E 3 F make a partition of event F. Here by partition we mean a separation with no omission or overlap. The formula for mutually exclusive events tells us

36 Sketch of a Novel (Continued) Recall that we assumed only one in the three tried to kill, and nobody else, so events E 1 F, E 2 F, and E 3 F make a partition of event F. Here by partition we mean a separation with no omission or overlap. The formula for mutually exclusive events tells us P(F ) = P(E 1 F ) + P(E 2 F ) + P(E 3 F ) = = 0.13, and after applying formula (1), the answer comes out.

37 Sketch of a Novel (Continued) Recall that we assumed only one in the three tried to kill, and nobody else, so events E 1 F, E 2 F, and E 3 F make a partition of event F. Here by partition we mean a separation with no omission or overlap. The formula for mutually exclusive events tells us P(F ) = P(E 1 F ) + P(E 2 F ) + P(E 3 F ) = = 0.13, and after applying formula (1), the answer comes out. Answer: The probability of A, B, and C to commit the murder is 23.08%, 40.38%, and 36.54%, respectively.

38 The Bayes Theorem Now we can formulate the theorem.

39 The Bayes Theorem Now we can formulate the theorem. With sample space S and positive integer m, we assume E 1,E 2,...,E m make a partition of S, and another event F, where P(E 1 ),..., P(E m ) > 0, and P(F ) > 0.

40 The Bayes Theorem Now we can formulate the theorem. With sample space S and positive integer m, we assume E 1,E 2,...,E m make a partition of S, and another event F, where P(E 1 ),..., P(E m ) > 0, and P(F ) > 0. Then for any i = 1, 2,..., m, we have

41 The Bayes Theorem Now we can formulate the theorem. With sample space S and positive integer m, we assume E 1,E 2,...,E m make a partition of S, and another event F, where P(E 1 ),..., P(E m ) > 0, and P(F ) > 0. Then for any i = 1, 2,..., m, we have P(E i F ) = P(E i )P(F E i ) P(E 1 )P(F E 1 ) + P(E 2 )P(F E 2 ) P(E m )P(F E m ), in the name of Presbyterian minister Thomas Bayes ( ).

42 Summary Bayes theorem is not new to us, in the sense that we can derive it from the definition of conditional probability in the previous section.

43 Summary Bayes theorem is not new to us, in the sense that we can derive it from the definition of conditional probability in the previous section. With this new tool, you will need to make a decision on which method to use, when being asked to find out some conditional probability.

44 Summary Bayes theorem is not new to us, in the sense that we can derive it from the definition of conditional probability in the previous section. With this new tool, you will need to make a decision on which method to use, when being asked to find out some conditional probability. Do NOT apply this theorem regardlessly. Sometimes the straightforward method by definition works better.

45 Summary Bayes theorem is not new to us, in the sense that we can derive it from the definition of conditional probability in the previous section. With this new tool, you will need to make a decision on which method to use, when being asked to find out some conditional probability. Do NOT apply this theorem regardlessly. Sometimes the straightforward method by definition works better. Probability trees will be useful for these problems.

46 Summary Bayes theorem is not new to us, in the sense that we can derive it from the definition of conditional probability in the previous section. With this new tool, you will need to make a decision on which method to use, when being asked to find out some conditional probability. Do NOT apply this theorem regardlessly. Sometimes the straightforward method by definition works better. Probability trees will be useful for these problems. It needs more practice to build the skills of problem solving.