Conditional Probability, Total Probability Theorem and Bayes Rule

Transcription

1 Conditional robability, Total robability Theorem and Bayes Rule Berlin Chen Department of Computer Science & Information Engineering National Taiwan Normal University Reference: - D.. Bertsekas, J. N. Tsitsiklis, Introduction to robability, Sections.-.4

2 Conditional robability (/) Conditional probability provides us with a way to reason about the outcome of an experiment, based on partial information Suppose that the outcome is within some given event B, we wish to quantify the likelihood that the outcome also belongs some other given event A Using a new probability law, we have the conditional probability of given, denoted by, which is defined as: A B ( A B) ( ) ( ) A I B A B ( B) ( ) ( ) If B has zero probability, A B is undefined We can think of ( A B) as out of the total probability of the elements of B, the fraction that is assigned to possible outcomes that also belong to A A B robability-berlin Chen

3 Conditional robability (/) When all outcomes of the experiment are equally likely, the conditional probability also can be defined as ( A B) number of elements of A I number of elements of B B Some examples having to do with conditional probability. In an experiment involving two successive rolls of a die, you are told that the sum of the two rolls is 9. How likely is it that the first roll was a 6?. In a word guessing game, the first letter of the word is a t. What is the likelihood that the second letter is an h?. How likely is it that a person has a disease given that a medical test was negative? 4. A spot shows up on a radar screen. How likely is it that it corresponds to an aircraft? robability-berlin Chen

4 Conditional robabilities Satisfy the Three Axioms Nonnegative: ( A B) 0 Normalization: ( ) ( Ω I B) Ω B ( B) ( B) ( B) Additivity: If and A are two disjoint events A A B A ( ) ( A U A ) A U A B ( I B ) ( B ) (( A I B ) U ( A I B )) ( B ) ( A I B ) + ( A I B ) ( B ) ( A B ) + ( A B ) distributive disjoint sets robability-berlin Chen 4

5 Conditional robabilities Satisfy General robability Laws roperties probability laws ( A U A B ) ( A B ) ( A B ) ( U A B ) ( A B ) + ( A B ) ( A A B ) + A I Conditional probabilities can also be viewed as a probability law on a new universe, because all of the conditional probability is concentrated on. B B robability-berlin Chen 5

6 Simple Examples using Conditional robabilities (/) robability-berlin Chen 6

7 Simple Examples using Conditional robabilities (/) robability-berlin Chen 7

8 Simple Examples using Conditional robabilities (/) C SS FS N SF FF robability-berlin Chen 8

9 Using Conditional robability for Modeling (/) It is often natural and convenient to first specify conditional probabilities and then use them to determine unconditional probabilities An alternative way to represent the definition of conditional probability ( A I B) ( B) ( A B) robability-berlin Chen 9

10 Using Conditional robability for Modeling (/) ( A I B) c ( A I B ) ( A ) c I B c c ( A I B ) robability-berlin Chen 0

11 Multiplication (Chain) Rule Assuming that all of the conditioning events have positive probability, we have ( ) ( ) ( ) ( ) ( ) n n I A A A A A A I A L A I i i n i Ai The above formula can be verified by writing ( n A ) ( A ) ( A I A ) ( A ) ( A I A I A ) ( A I A ) ( n I ) i Ai ( A ) i L n I i i Ii For the case of just two events, the multiplication rule is simply the definition of conditional probability ( A I A ) ( A ) ( A ) A robability-berlin Chen

12 Multiplication (Chain) Rule: Examples (/) Example.0. Three cards are drawn from an ordinary 5-card deck without replacement (drawn cards are not placed back in the deck). We wish to find the probability that none of the three cards is a heart. ( ) { the ith card is not a heart}, i,, A i ( A I A I A ) ( A ) ( A A ) ( A A I A ) C C 9 5? robability-berlin Chen

13 Multiplication (Chain) Rule: Examples (/) Example.. A class consisting of 4 graduate and undergraduate students is randomly divided into 4 groups of 4. What is the probability that each group includes a graduate student? A A A { graduate students and are at different groups} { graduate students,, and are at different groups} { graduate students,,, and 4 are at different groups} ( A ) ( A I A I A ) ( A ) ( A A ) ( A A I A ) ( A ) A ( A A ) ( A A I A ) ( ) 8 4 robability-berlin Chen

14 Total robability Theorem (/) Let A, L, A n be disjoint events that form a partition of the sample space (each possible outcome is included in one and only one of the events A, L, A n ) and assume that ( A i ) > 0, for all i. Then, for any event B, we have ( B) ( A I B) + L + ( A I B) ( A ) ( B A ) + L + ( A ) ( B A ) n n n robability-berlin Chen 4

15 Total robability Theorem (/) Figure.: robability-berlin Chen 5

16 Some Examples Using Total robability Theorem (/) Example.. robability-berlin Chen 6

17 Some Examples Using Total robability Theorem (/) Example.4. (,),(,4) (,),(,),(,4) (4) robability-berlin Chen 7

18 U B i i Some Examples Using Total robability Theorem (/) Example.5. Alice is taking a probability class and at the end of each week she can be either up-to-date or she may have fallen behind. If she is up-to-date in a given week, the probability that she will be up-to-date (or behind) in the next week is 0.8 (or 0., respectively). If she is behind in a given week, the probability that she will be up-to-date (or behind) in the next week is 0.4 (or 0.6, respectively). Alice is (by default) up-to-date when she starts the class. What is the probability that she is up-to-date after three weeks? : up - to - date : behind ( U ) ( U ) ( U U ) + ( B ) ( U B ) ( U ) ( B ) ( U ) ( U) ( U U) + ( B ) ( U B ) ( U) ( B ) ( B ) ( B ) ( B U ) + ( B ) ( B B ) ( U ) 0. + ( B ) ( U) 0.8, ( B ) ( U ) ( B ) ( U ) As we know that > Recursion formulea ( Ui+ ) ( Ui ) ( Bi ) ( Bi + ) ( Ui ) 0. + ( Bi ) ( U ) 0.8, ( B ) robability-berlin Chen 8

19 Bayes Rule Let A, A, K, A n be disjoint events that form a partition of the sample space, and assume that ( A i ) 0, for all i. Then, for any event such that we have ( A B) i ( Ai I B) ( B) ( A ) ( B A ) i ( B) B ( B) > 0 i ( A ) ( ) i B Ai n ( ) ( ) k Ak B Ak ( A ) ( ) i B Ai ( A ) ( B A ) + L+ ( A ) ( B A ) Multiplication rule Total probability theorem n n robability-berlin Chen 9

20 Inference Using Bayes Rule (/) 惡性腫瘤 Figure.4: 良性腫瘤 robability-berlin Chen 0

21 Inference Using Bayes Rule (/) Example.8. The False-ositive uzzle. A test for a certain disease is assumed to be correct 95% of the time: if a person has the disease, the test with are positive with probability 0.95 ( ( B A) ), and if the person does not have the disease, the test results are negative with probability 0.95 ( c B c A 0.95 ). A random person drawn from a certain population has probability 0.00 ( ( A ) ) of having the disease. Given that the person just tested positive, what is the probability of having the disease ( A B )? ( ) A : the event that the person has a disease B : the event that the test results are positive ( A B ) ( A ) ( B A ) ( B ) ( A ) ( B A ) c c ( A ) ( B A ) + ( A ) ( B A ) B A c c c B A robability-berlin Chen

22 Recitation SECTION. Conditional robability roblems, 4, 5 SECTION.4 robability Theorem, Bayes Rule roblems 7,, 4, 5 robability-berlin Chen

23 Homework - Chapter : Additional roblems ( roblems 4, 6,, 7 robability-berlin Chen