ECE 450 Lecture 2 Recall: Pr(A B) = Pr(A) + Pr(B) Pr(A B) in general = Pr(A) + Pr(B) if A and B are m.e. Lecture Overview Conditional Probability, Pr(A B) Total Probability Bayes Theorem Independent Events Compound Experiments Binomial Distribution ECE 450 D. van Alphen 1
Conditional Probability Defn: The conditional probability of event A, given that event B has occurred is Pr( A B) P(B) Note (by symmetry of the definition): Pr( B A) Pr(A B), Pr(B) Pr(A B) Pr(A) P(A) From (1) & (2), we have two new ways of writing Pr(A B): Pr(A B) = Pr(A) Pr(B A) = Pr(B) Pr(A B) 0 0 (1) (2) ECE 450 D. van Alphen 2
Verification Example (Is the definition reasonable?) Experiment: Toss a single die, and find Pr(2 even) Pr( 2 even) Pr(2 even) Pr(even) Pr(2) Pr(even) 1 6 3 6 1/ 3 Another way to look at it - Let B be the event of getting an even number: B is called the restricted S B B 1 sample space; 2 is now 5 2 4 one of 3 equally likely outcomes. 3 6 ECE 450 D. van Alphen 3
Experiment: Toss 2 coins Another Example Sample Space: S = {HH, HT, TH, TT} Find the conditional probability of obtaining two heads when flipping two coins, given that at least one head was obtained; i.e., find Pr(2 heads at least 1 head) Def: A: event of obtaining 2 heads = {HH} Then B: event of obtaining at least one head = {HH, HT, TH} Pr( A Pr(A B) B) Pr(B) Pr(A) Pr(B) 1 4 3 4 1/ 3 ECE 450 D. van Alphen 4
Side Note & Definition Note: Conditional probabilities are themselves probabilities; thus, they satisfy all the axioms for probabilities : 1. Pr(A B) 0 2. Pr(B B) = 1 3. Pr(A C B) = Pr(A B) + Pr(C B) if A and C are m.e. Another definition: consider a collection of subsets, {A i }, (i = 1,, n ), of S. The collection is S A 1 A 2 a partition of S if:... A n A i A j = f, i j U A i = S, i = 1,, n ECE 450 D. van Alphen 5
Total Probability Theorem Let {A i } be a partition of S, and let B be a subset of S: S A 1 A 2 A 3 B... A n Strategy: To find Pr(B), break apart B, into mutually exclusive pieces Then Pr(B) = Pr[ (A 1 B) (A 2 B)... (A n B) ] m.e = Pr(A 1 B) + Pr(A 2 B) + + Pr(A n B) (see box, bottom of p.2) Pr(B) = Pr(B A 1 )Pr(A 1 ) + Pr(B A 2 )Pr(A 2 ) + + Pr(B A n )Pr(A n ) ECE 450 D. van Alphen 6
Example Using Total Probability Transistor types X, Y and Z make up 45%, 25%, and 30% of the total number of transistors in a box, respectively. Let B be the event that a transistor fails before 1000 hrs. Given the reliability information: Pr(B X) =.15 Pr(B Y) =.4 Pr(B Z) =.25 Find the probability that a randomly chosen transistor from the box fails before 1000 hours. Pr(B) = + + = + + =.243 ECE 450 D. van Alphen 7
Bayes Rule Recall from p. 2 (again, the box near the bottom) Pr(A B) = Pr(A B) Pr(B) = Pr(B A) Pr(A) Focus here; solve for Pr(A B) Pr(B A)Pr(A) Pr( A B), Pr(B) 0 Pr(B) (Bayes Rule) Pr( A j B) n Pr( B j1 A Pr( B j A )Pr( A j j ) )Pr( A j ) By ECE 450 D. van Alphen 8
Bayes Rule: Note & Example Note: Use Bayes Rule when asked to find some Pr(A B), but it would be easier to find Pr(B A). ( Backwards conditional probability ) Example: Say an observed transistor (from the previous example) fails before 1000 hrs. Find the probability that it was a Type Z transistor. Let B: event that transistor fails before 1000 hrs. We want: Pr(Z B), non-trivial We know Pr(B Z) =.25 (the easier problem; given on p. 7) Using Bayes Rule, next page: ECE 450 D. van Alphen 9
Bayes Rule Example, continued Pr( Z B) Pr(B Z)Pr(Z) Pr(B).30864 Answer from p. 7 example In-Class Practice Two cards are drawn without replacement from a 52-card deck. Find the probability that the 2 nd is a queen, given that the 1 st is a queen. Find the probability that both the 1 st and 2 nd are queens. Unordered answers: 1/221, 3/51 ECE 450 D. van Alphen 10
Bayes Rule In-Class Example A medical test for a particular type of cancer has the following properties: It correctly detects the cancer (when present) with probability 95%; It incorrectly detects the cancer (when there is no cancer present) with probability 20%. Suppose that this particular type of cancer is present in only 1% of people of your age/sex/ethnicity. Find the probability that you actually have this cancer, given that your test is positive (i.e., cancer was detected). Let c: event that you have cancer. Let d: event that cancer is detected. ECE 450 D. van Alphen 11
Pr(d c) =.95; Pr( ) =.05 (complements) Pr(d c ) =.2; Pr(d c ) = ; Pr(c) =.01 (a priori) Find Pr(c d) ECE 450 D. van Alphen 12
(Statistically) Independent Events Defn: 2 events A and B are statistically independent ( ) if and only if Pr(A B) = Pr(A) (knowing B has occurred tells me nothing about whether or not A has occurred) Equivalently: and Pr(B A) = P(B) Pr(A B) = Pr(A) Pr(B) (caution: only true for independent events) ECE 450 D. van Alphen 13
Example Using Independence Definition Experiment: Toss 2 dice. Define the events A: sum = 7 Question: Are A, B, B: 1 face = 6 Pr(A B) Pr Pr{(6,1) (1,6)} 2 / 36 2 Pr( B) Pr{(6,1) (6,2) (6,6)} 11/ 36 11 Pr(A) = 6/36 = 1/6 A, B not A, B dependent ECE 450 D. van Alphen 14
Facts/Thoughts About Independence Fact 1: Complementary events are dependent events. Fact 2: Unions, intersections & complements of independent events are independent. Consider the 2012 election (Obama/Biden vs. Romney/Ryan): Event A: Obama wins as Pres. Event B: Romney wins as Pres. Event C: Biden wins as VP. Event D: Ryan wins as VP. Is there any pair of events from this set that is an independent pair? Let A: it rains today; B: it rains tomorrow. Are A and B independent? ECE 450 D. van Alphen 15
Example Using Independence Say the switches in the circuit below are closed ( -ly of each other) at any time with probability 0.1. Find the probability of a closed path from point A to point B. A B Labeling for the Solution: A bottom top right ECE 450 D. van Alphen 16 B Note: for a closed path to exist, the right switch must be closed; and, either the bottom switch must be closed or both of the top switches must be closed.
Example, continued Pr(closed path) = Pr[ right and ( top or bottom )] (both switches) = Pr(right) Pr(top or bottom), by Fact 2, p. 14 = (0.1) [Pr(top) + Pr(bottom) Pr(both top & bottom)] by Corollary 4 (both switches: (.1)(.1) ) A =.1 [(.01) +.1 (.01)(.1)] =.0109 top right B Note: Don t round off! bottom ECE 450 D. van Alphen 17
Compound Experiments Consider experiments: E a, E b with sample spaces: S a, S b Say S a = {a 1,, a n } and S b = {b 1,, b m }. Define S a x S b (the Cartesian Cross Product) as the set of ordered pairs with the 1 st element from S a and the second element from S b Do experiments E a & E b (jointly), and get pairs of outcomes from S a x S b The joint performance of E a & E b is said to be a compound experiment. ECE 450 D. van Alphen 18
Cross Product Examples & Bernoulli Trials Example 1: Toss a coin 2 times, with S 1 = S 2 = {H, T}. S = S 1 x S 2 = {,,, } Example 2: Toss a coin 3 times, with S 1 = S 2 = S 3 = {H, T}. S = S 1 x S 2 x S 3 = { (HHH), (HHT), (HTH), (HTT), (THH), (THT), (TTH), (TTT) } Definition: A Bernoulli Trial is an experiment with only 2 possible outcomes, sometimes called success and failure. Success 1 yes, Failure 0 no ECE 450 D. van Alphen 19
Binomial Experiments Definition: A Binomial Experiment is an experiment consisting of n independent Bernoulli Trials. Let A k denote the event of getting k successes (and thus n-k failures) in n trials. Let p be the probability of success on each trial. Let q = 1 p be the probability of failure on each trial. Example: Consider a 5-trial binomial experiment, and say we are interested in the probability of having 2 successes (first), followed by 3 failures. Pr{1, 1, 0, 0, 0} = = (O.K. to multiply since the events are ) ECE 450 D. van Alphen 20
Binomial Experiments, continued Refined example: Say we want to know the probability of getting 2 successes, in 5 independent trials (order doesn t matter): 5 2 Pr{A 2 } = p 2 q 3 (since there are 5 C 2 ways to decide where to put the 2 successes in the string of 5 outcomes) In general, the probability of getting k successes in n (independent) Bernoulli tials is n k p n (k) = Pr{A k } = p k q n k (Binomial Experiments) ECE 450 D. van Alphen 21
Binomial Experiment Examples 1. 10 missiles are fired at a tank; each missile has a (0.2) probability of hitting the tank, independently of the other missiles. Find the probability that exactly 3 missiles hit the tank. p 10 (3) = Pr{ A 3 } = ( ) 3 ( ) 7.2013 2. A certain football player can catch 2/3 of the passes thrown to him. He needs to catch at least 3 more passes for his team to win the game. Find the probability that his team wins if the quarterback throws to him 5 more times. Pr{win} = Pr{at least 3 catches} = Pr{exactly 3 catches or exactly 4 catches or exactly 5 catches} ECE 450 D. van Alphen 22
Binomial Experiment Examples, continued = Pr{A 3 A 4 A 5 } = Pr{A 3 } + Pr{A 4 } + Pr{A 5 } = + + =.790123 ** Note: we can add these probabilities, because the events catch exactly 1 pass, catch exactly 2 passes, and catch exactly 3 passes are: events. ECE 450 D. van Alphen 23
Binomial Experiment Examples 3. Using a normal deck of cards, say we cut the deck 5 times; find the probability of getting an ace on at least 3 cuts. Pr{at least 3 aces} = Pr{exactly 3 aces or exactly 4 aces or exactly 5 aces) = Pr{A 3 } + Pr{A 4 } + Pr{A 5 } = 5 3 1501 1 13 3 2 12 13 =.00404 13 5 5 4 (Bernoulli Trials) 1 13 4 1 12 5 13 5 1 13 5 0 12 13 m.e. ECE 450 D. van Alphen 24
Danger!! 1. Pr(A B) = Pr(A) + Pr(B) Pr(A B) = Pr(A) + Pr(B) if A, B m.e. 2. Pr(A B) = Pr(A) Pr(B A) = Pr(A) Pr(B) if A, B independent ECE 450 D. van Alphen 25
Review Pr(A B) = (defn) = (Bayes Rule) Total Probability: If {A i } is a partition of sample space S, then Pr(B) = If A and B are independent, then Pr(A B) = and Pr(A B) = (Binomial Experiments): The probability of getting k successes in n independent Bernoulli trials is: ECE 450 D. van Alphen 26