Tutorial 3: Random Processes

Transcription

1 August 18 Semester 2018-II Tutorial 3: Random Processes Lecturer: Utpal Mukherji/ Parimal Parag Prepared by: Prathamesh Mayekar Note: LaTeX template courtesy of UC Berkeley EECS dept. 3.1 Conditional Probability Let A be any event in the event space. Let f n (A) denote the frequency of an event A; f n (A) = # of times A occurrs n number of trials. It s seems reasonable that P (A) f n (A) for a large enough n. Now suppose there is another event B in the event space with P (B) 0. We would like to come up with definition for P (A B) - the probability that a event A occurs given that an event B has occurred. First, let s see if we can have a heuristic understanding of this probability using the concept of frequencies of events. If we consider f n (A), it is silly to count f n (A B c ), since we know B has occurred. Thus to come with f n (A B) (the frequency of event A given B) we only count those occurrences of A where B also occurs, this is nf n (A B). The number of trials is the number of occurrences of B, nf n (B). Therefore, P (A B) nf n(a B), nf n (B) for a sufficiently large n. This heuristic motivates the following definition. Definition Let A, B events, P (B) > 0. The conditional probability of A given B is P (A B) P (A B)/P (B). Example Suppose an urn contains seven black balls and five white balls. We draw two balls from the urn without replacement. Assuming that each ball in the urn is equally likely to be drawn, what is the probability that both drawn balls are black. Solution: Let B 1, B 2 denote, respectively, the events that the first and second balls drawn are black. Now, given that the first ball selected is black, there are six remaining black balls and five white balls, and so P (B 2 B 1 ) = 6/11. Also P (B 1 ) = 7/12, therefore P (B 1 B 2 ) = P (B 1 )P (B 2 B 1 ) = Partition Equation Let (E 1, E 2,...) be a finite or countable partition of Ω. Then if A A, P (A) = n P (A E n )P (E n ). Proof. A = n A E n. 3-1

2 3-2 Tutorial 3: Random Processes Since E n are disjoint, A E n are disjoint. Therefore, P (A) = P ( n A E n ) = n P (A E n ) = n P (A E n )P (E n ). Example Suppose Bob and Alice have three six sided fair dices with the sides numbered as follows: die A -1,1,6,6,8,8 die B -2,2,4,4,9,9: die C -3,3,5,5,7,7. Alice and Bob decide to play a game using this dices. The game proceeds as follows: both Alice and Bob pick a dice from the three available and roll them, and the one who ends up with the lowest number loses. a) Supposes that Bob chooses dice A and Alice chooses dice B. Show that probability that Alice wins is greater than or equal 1 2. b) Supposes that Bob chooses dice B and Alice chooses dice C. Show that probability that Alice wins is greater than or equal 1 2. c) Supposes that Bob chooses dice C and Alice chooses dice A. Show that probability that Alice wins is still greater than or equal Bayes Formula Let (E n ) be a finite or countable partition of Ω, and suppose P (A) 0. Then P (E n A) = P (A E n )P (E n ) m P (A E m)p (E m ). Example In answering a question on a multiple-choice exam a student either knows the answer or guesses. Let p be the probability that she knows the answer and 1 p the probability that she guesses. Assume that when student guesses an answer she will be correct with probability 1/m, where m is the number of multiple-choice alternatives. What is the conditional probability that a student knew the answer to a question given that she answered it correctly? Solution: Let K denote the event that the student knew the answer to the question and C denote the event that she answered it correctly. To compute P (K C). By Baye s formula, P (K)P (C/K) P (K C) = P (K)P (C/K) + P (K c )P (C/K c ) p = p + (1 p)(1/m) mp = (m 1)p + 1.

3 Lecture 3: Random Processes Conditional Independence Definition Events A and B are conditionally independent given C if P (A B C) = P (A C)P (B C). 1. An example where two events are (unconditionally) independent but not conditionally independent (when conditioned on a third event) This example is to demonstrate that if there exist 3 events A, B and C such that then it need not necessarily be true that P (A B) = P (A) P (B), P (A B C) = P (A C) P (B C). Consider two independent throws of a fair die. 1 Verify that Ω = {(1, 1), (1, 2),..., (1, 6), (2, 1),..., (2, 6),..., (6, 1),..., (6, 6)} Let F be the set of all subsets of Ω. With this, we have specified what Ω and F are. Notice that we have also specified P : F [0, 1] by saying that we are throwing a fair die. Let A be the event that the first throw results in a 5. So, A = {(5, 1), (5, 2),..., (5, 6)} Let B be the event that the second throw results in a 3. Write out what B looks like. What is A B? Now, let us calculate P (A B). Recall that the two throws are independent of each other. P (A B) = 1 36 = = P (A) P (B). Thus, A and B are independent. Now, let C be the event that the sum of the results of the two throws is equal to 8. i.e., C = {(2, 6), (3, 5), (4, 4), (5, 3), (6, 2)}. Clearly, A B C = {(5, 3)} A C = {(5, 3)} and, B C = {(5, 3)}. 1 A die has six faces, numbered 1, 2, 3, 4, 5, 6. A throw of a die results in one of the six faces. For a fair die, all faces are equally likely to result.

4 3-4 Tutorial 3: Random Processes Now, conditioned on event C, are events A and B independent? Similarly, P (B C) = 1 5. Thus, P (A B C) P (A B C) = P (C) = 1/36 5/36 = 1 5 P (A C) P (A C) = P (C) = 1/36 5/36 = 1 5 P (A B C) P (A C) P (B C). Therefore, conditioned on the event C, events A and B are not conditionally independent. 2. An example where two events are conditionally independent (conditioned on a third event), but not independent by themselves unconditionally This example is to demonstrate that if there exist 3 events A, B and C such that P (A B C) = P (A C) P (B C), then it need not necessarily be true that P (A B) = P (A) P (B). Consider two coins C 1 and C 2. Let the probability of C 1 resulting in H be 1/3 and that in T be 2/3. Let the probability of C 2 resulting in H be 1/4 and that in T be 3/4. Now, let C 0 be a fair coin. i.e., the probability of C 0 resulting in H is 1/2 and that in T is 1/2. We go through the following procedure: Toss C 0. If it results in H, pick C 1 and toss it twice. If the toss of C 0 results in T, pick C 2 and toss it twice. Note that there are three coin tosses in all. Assume that all the tosses are independent of each other. Let C be the event that the first toss (i.e., toss of C 0 ) results in H. Let A be the event that the second toss results in H. Let B be the event that the third toss results in H. Now, P (A B C) = P (second and third tosses result in H, given that coin C 1 is picked) = Similarly we can see that P (A C) = 1 3 and, P (B C) = 1 3.

5 Lecture 3: Random Processes 3-5 Therefore, P (A B C) = P (A C) P (B C). So, conditioned on the event C, events A and B are conditionally independent. Now, P (A B) = P (A B C) P (C) + P (A B C C ) P (C C ) = = P (A) = P (A C) P (C) + P (A C C ) P (C C ) = = Similarly, P (B) = Therefore, P (A) P (B) = 288. Thus, A and B are not independent. P (A B) P (A) P (B). References 1. Ross, Sheldon M. Introduction to probability models. Academic press, Jacod, Jean, and Philip Protter. Probability essentials. Springer Science and Business Media, Mitzenmacher, Michael, and Eli Upfal. Probability and computing: Randomized algorithms and probabilistic analysis. Cambridge university press, Karthik P N, and Sahasrananand R. Tutorial on conditional probability and indpendence. E2 202 : Random Processes, Fall 2017, ECE, IISc.