Week 4 Classical Probability, Part II
Week 4 Objectives This week we continue covering topics from classical probability. The notion of conditional probability is presented first. Important results/tools related to conditional probabilities are the multiplication rule and tree diagrams; the Law of Total Probability and Bayes Theorem. These are presented and their usefulness in computing probabilities is illustrated. Finally the notion of independent events is introduced and its application in system reliability is discussed.
1 2 3
Updating Probabilities When the response variable is multivariate, knowledge of the value of one variable may help predict another. For now, the word prediction will mean update the probabilities of events regarding the other variable. A man s height helps update the probability that he weighs over 170lb. A person s education level helps update the probability of that person being in a certain income category. In this unit we will learn how to update probabilities when given other relevant information. The updated probabilities are called conditional probabilities.
Given partial information regarding the outcome of simple random selection amounts to simple random selection from a restricted population. If the outcome of rolling a die is known to be even, what is the probability it is a 2? Answer: The restricted population is {2, 4, 6}, so the probability of a 2 is 1/3. If the selected card from a deck is known to be a figure card, what is the probability it is a king? Hint: The restricted population consists of 12 cards, including the 4 kings.
A household is selected at random. Given the household has a cat what is the probability that it also has a dog? Use the probabilities shown in the figure in next page, where A = {household has a cat}, B = {household has a dog}. Hint: The challenge presented by this example is that it provides only the relative sizes of the subpopulations of interest, i.e., only probabilities instead of the actual sizes. To overcome this, think of the entire population of households as consisting of 100 households.
A-B A B B-A 0.1 0.5 0.2 0.2 Figure: Probabilities for having a cat or a dog.
The Multiplication Rule The conditional probability of the event A given that event B has occurred is denoted by P(A B) and equals P(A B) = P(A B), provided P(B) > 0 P(B) The definition of P(A B) yields the formulas: P(A B) = P(A B)P(B) or P(A B) = P(B A)P(A) which which known as the multiplication rule. The rule extends to more than two events. A version with three events is: P(A B C) = P(A)P(B A)P(C A B)
Examples using the Multiplication Rule 40% of bean seeds come from supplier A and 60% come from supplier B. Seeds from supplier A have 50% germination rate while those from supplier B have a 75% rate. What is the probability that a randomly selected seed came from supplier A and will germinate? Solution: Let A = {seed comes from supplier A} and G = {seed will germinate}. By the multiplication rule, P(A G) = P(G A)P(A) = 0.5 0.4 = 0.2. Three players are dealt a card in succession. What is the probability that the 1st gets an ace, the 2nd gets a king, and the 3rd gets a queen? Answer: P(A B C) = P(A)P(B A)P(C A B) = 4 4 4 52 51 50.
Of the customers entering a department store 30% are men and 70% are women. The probability a male shopper will spend more than $50 is 0.4, and the corresponding probability for a female shopper is 0.6. The probability that at least one of the items purchased is returned is 0.1 for male shoppers and 0.15 for female shoppers. Find the probability that the next customer to enter the department store is a woman who will spend more than $50 on items that will not be returned. Solution: With A = {next customer is female}, B = {customer spends >$50} and C = {no item is returned}, P(A B C) = P(A)P(B A)P(C A B) = 0.7 0.6 0.85 = 0.357.
Tree Diagrams The multiplication rule typically applies in situations where the events whose intersection we wish to compute are associated with different stages of an experiment. In the experiment in the department store example there are three stages: a) record the customer s gender, b) record how much the customer spends, and c) record whether any of the items purchased is returned. A tree diagram displays all outcomes of the experiments as sequences of branches leading from the start on the left to one of the end points on the right. The tree diagram for the department store experiment is shown next.
0.3 M 0.4 0.6 > 50 < 50 0.1 0.9 0.1 0.9 R R c R R c 0.7 W 0.4 0.6 < 50 > 50 0.15 0.85 0.15 0.85 R R c R R c Figure: Tree diagram for the department store experiment
More on tree diagrams By the general product rule, the department store experiment has 2 2 2 = 8 different outcomes, which correspond to the 8 sequences of branches leading to the 8 end points on the right. The first group of branches list the probabilities of each outcome in the first stage. For example, 0.3 and 0.7 are the probabilities for the entering customer to be male and female, respectively. The remaining branches list conditional probabilities. The probability of each outcome is the product of the probabilities listed on the corresponding sequence of branches. Identify the sequences of branches for the outcome customer is female who spends >$50 on items that are not returned, and compute the probability of this outcome.
The Law of Total Probability Let the events A 1, A 2..., A k form a partition of the sample space S (they are disjoint and their union is S), and let B denote an event whose probability we want to calculate. B A A A A 1 2 3 4 If we know P(B A j ) and P(A j ) for all j = 1, 2,..., k, the Law of Total Probability states that P(B) = P(A 1 )P(B A 1 ) + + P(A k )P(B A k )
Examples using the Law of Total Probability 40% of bean seeds come from supplier A and 60% come from supplier B. Seeds from supplier A have 50% germination rate while those from supplier B have a 75% rate. What is the probability that a randomly selected seed will germinate? Solution: Let A = {seed comes from supplier A}, B = {seed comes from supplier B} and G = {seed will germinate}. The events A and B form a partition of the sample space. By the Law of Total Probability P(G) = P(A)P(G A) + P(B)P(G B) = 0.4 0.5 + 0.6 0.75 = 0.65.
Two players are dealt a card in succession. What is the probability that the 2nd gets a king? If the player with a king wins a prize, does it matter which player is the first? Answer: Set A 1 = {1st player gets a king}, A 2 = A c 1 and B = {2nd player gets a king}. The answer is P(B) = 4/52. It does not matter who goes first because the order does not change the probability of getting a king.
Consider rolling two dice repeatedly and recording their sum. Find the probability of B = {5 happens before 7}. Hint: Set A 1 ={sum of 1st roll is 5}, A 2 ={sum of 1st roll is 7}, A 3 = {sum of 1st roll is neither 5 nor 7}. Then P(B) = P(B A 1 )P(A 1 ) + P(B A 2 )P(A 2 ) + P(B A 3 )P(A 3 ) = P(A 1 ) + 0 + P(B)P(A 3 ). (Why?) Use the pmf for the sum of two die rolls (derived before and reshown bellow) to find P(A 1 ) and P(A 3 ), and solve for P(B). x 2 3 4 5 6 7 8 9 10 11 12 p(x) 0.028 0.056 0.083 0.111 0.139 0.167 0.139 0.111 0.083 0.056 0.028
Two consecutive traffic lights have been synchronized to make a run of green lights more likely. In particular, if a driver finds the first light to be red, the second light will be green with probability 0.9, and if the first light is green the second will be green with probability 0.7. The probability of finding the first light green is 0.6. Find the probability of B = {a driver finds the 2nd traffic light green}. Solution. Let A = {a driver finds the 1st traffic light green}. Because A and A c constitute a partition of the sample space, by the Law of Total Probability P(B) = P(A)P(B A) + P(A c )P(B A c ) = 0.6 0.7 + 0.4 0.9 = 0.42 + 0.36 = 0.78.
In the context of the previous example, construct a tree diagram for the experiment which records whether or not a driver finds each of the two traffic lights green, and recalculate the probability of the event B. Solution. The tree diagram is shown in the next page. It has 2 2 = 4 outcomes. The event B is composite and consists of the two outcomes corresponding to the sequences of branches that end with a G. Thus, P(B) is the sum of the probabilities of these two outcomes, i.e., P(B) = 0.6 0.7 + 0.4 0.9 = 0.78.
0.4 R 0.1 0.9 R G 0.6 G 0.3 0.7 R G Figure: Tree diagram for the traffic light experiment
Bayes Theorem Consider events B and A 1,..., A k as in the Law of Total Probability. Now, however, we ask a different question: Given that B has occurred, what is the probability that a particular A j has occurred? The answer is provided by the Bayes theorem: ( P(A j B) = P(A ) j B) P(A j )P(B A j ) = P(B) k j=1 P(A i)p(b A i ) where the parenthetic expression serves as the intermediate step needed for the proof of the theorem.
Examples using Bayes Theorem 40% of bean seeds come from supplier A and 60% come from supplier B. Seeds from supplier A have 50% germination rate while those from supplier B have a 75% rate. Given that a randomly selected seed germinated, what is the probability that it came from supplier A? Answer: P(A G) = P(A)P(G A) P(A)P(G A) + P(B)P(G B) = 0.2 0.65 = 0.31.
Two players are dealt a card in succession. Given that the 2nd player got an ace, what is the probability that the 1st got an ace? Solution: Set A 1 = {1st player gets an ace}, A 2 = A c 1 and B = {2nd player gets an ace}. By Bayes Theorem, P(A 1 B) = = P(A 1 )P(B A 1 ) P(A 1 )P(B A 1 ) + P(A 2 )P(B A 2 ) (4/52)(3/51) (4/52)(3/51) + (48/52)(4/51) = 3 51.
Seventy percent of the light aircraft that disappear while in flight in a certain country are subsequently discovered. Of the aircraft that are discovered, 60% have an emergency locator, whereas 10% of the aircraft not discovered have such a locator. Suppose a light aircraft has disappeared. (A tree diagram would help verify the below answers.) 1 What is the probability that it has an emergency locator and it will not be discovered? Answer: 0.1 0.3 = 0.03 (use the multiplication rule). 2 What is the probability that it has an emergency locator? Answer: 0.1 0.3 + 0.6 0.7 = 0.45 (use the Law of Total Probability). 3 If it has an emergency locator, what is the probability that it will not be discovered? Answer: 0.03/0.45 (use Bayes Theorem).
Probability of an Intersection The formula for the probability of A B yields P(A B) = P(A) + P(B) P(A B). A simpler formula is possible if A and B are independent: For independent events, P(A B) = P(A)P(B). The above formula also serves as the definition of independent events. The empty set is independent from any other event.
Independent events arise in connection with independent experiments or independent repetitions of the same experiment. Two experiments are independent if there is no mechanism through which the outcome of one experiment will influence the outcome of the other. A die is rolled twice. Are the two rolls independent? Two cards are drawn without replacement from a deck of cards. Are the two draws independent? In two independent repetitions of an experiment, any event associated with the first repetition will be independent of any event associated with the second repetition.
Toss a coin twice and set A 1 = {H in toss 1}, A 2 = {H in toss 2}. Find the probability of two heads. Solution. Since the two tosses are independent, P(A 1 A 2 ) = P(A 1 )P(A 2 ) = 1 1 2 2 = 1 4. Alternatively, the independence of the events A 1 and A 2 can be shown from the fact that the 4 outcomes of two coin tosses are equally likely. Indeed, P(A 1 A 2 ) = 1 4 = 1 1 2 2 = P(A 1)P(A 2 ), which implies that the two events are independent.
(Fair coin toss with an unfair coin) A coin results in H with probability p (e.g., p = 0.3). Flip this coin twice. If the outcome is (H,H) or (T,T) ignore the flips and flip again twice. Repeat until the outcome of the two flips is either (H,T) or (T,H). In the first case you say you got tails, and in the second case you say you got heads. Prove that these events of heads and tails are equally likely. Solution: Ignoring the outcomes (H,H) and (T,T), is equivalent to conditioning on the the event B = {(H, T ), (T, H)}. Thus, P((T, H) B) = = P((T, H) B) P(B) = P((T, H)) P(B) (1 p)p p(1 p) + (1 p)p = 0.5.
(Efron s Dice) Consider again Efron s dice, i.e., Die A: four 4s and two 0s; Die B: six 3s; Die C: four 2s and two 6s; Die D: three 5 s and three 1 s. Find the probabilities that A > B, B > C, C > D, and D > A using the properties of probability and the concept of independence. Solution (partial): Note that P(C > D) equals P(C = 2 and D = 1) + P(C = 6 and D = 5 or 1) = P(C = 2)P(D = 1) + P(C = 6)P(D = 5 or 1) = 2 1 3 2 + 1 3 = 2 3.
Events can be independent even if they are do not arise in connection with independent experiments. A card is drawn at random from a deck of 52 cards. Show that the events A = {the card is a five} and B = {the card is a spade} are independent. If A and B are independent, then so are A c and B. If a card is drawn at random, the events A c = {the card is not a five} and B = {the card is a spade} are independent. The sample space S is independent from any other event.
If A and B are independent, we have P(A B) = P(A) and P(B A) = P(B). Conversely, any of the above relations implies that A and B are independent. Read Example 2.6-4, p. 92. Being independent should not be confused with being disjoint. In fact, if A and B are disjoint then they are not independent unless one, or both, of them equals the empty set.
Independence of Multiple Events Definition The events A 1,..., A n are mutually independent if P(A i1 A i2... A ik ) = P(A i1 )P(A i2 )... P(A ik ) for any sub-collection A i1,..., A ik of k events chosen from A 1,..., A n All conditions in the definition are needed because, for example, the equality P(A B C) = P(A)P(B)P(C) does not imply that A, B, C are independent. This is demonstrated in the following example.
Consider rolling a die and define the events A = {1, 2, 3}, B = {3, 4, 5}, C = {1, 2, 3, 4}. Verify that P(A B C) = P(A)P(B)P(C), but that A, B are not independent (and thus A, B, C are not mutually independent). Solution: First, since A B C = {3}, it follows that P(A B C) = 1 6 = P(A)P(B)P(C) = 1 1 4 2 2 6. Next, A B = {3}, so P(A B) = 1 6 P(A)P(B) = 1 1 2 2.
Independence is often assumed as it facilitates the computation of probabilities. At 25 o C, 20% of a type of diodes have efficiency below 0.3mW/mA. Five such diodes are selected at random. Find: (a) Only the 2nd diode has efficiency below 0.3 at 25 o C. (b) Exactly one has efficiency below 0.3 at 25 o C. (c) Exactly two has efficiency below 0.3 at 25 o C. Solution: (a) (0.2)(0.8 4 ) = 0.082. (Why?) (b) 5 0.082 = 0.41. (Why?) (c) ( 5 2) 0.2 2 0.8 3 = 0.205. (Why?) What is the probability of two heads in 5 flips of a coin?
The three components of the series system shown in the figure fail with probabilities p 1 = 0.1, p 2 = 0.15 and p 3 = 0.2, respectively, independently of each other. What is the probability the system will fail? 1 2 3 Figure: Components connected in series Answer: Assume the components fail independently. Then, P(system fails) = 1 P(system does not fail) = 1 0.9 0.85 0.8 = 0.388.
The three components of the parallel system shown in figure function with probabilities p 1 = 0.9, p 2 = 0.85 and p 3 = 0.8, respectively, independently of each other. What is the probability the system functions? Answer: Assume the components function independently. Then, P(system functions) = 1 P(system does not function) = 1 0.1 0.15 0.82 = 0.997.
1 2 3 Figure: Components connected in parallel
Read Example 2.6-9, p. 95