Lecture 3 Probability Basics Thais Paiva STA 111 - Summer 2013 Term II July 3, 2013
Lecture Plan 1 Definitions of probability 2 Rules of probability 3 Conditional probability
What is Probability? Probability is a measure or estimation of how likely it is that something will happen or that a statement is true (Wikipedia). Here are some motivating examples: 1 American roulette is a gambling game characterized by 38 pockets. What is the probability that the ball falls in one of the pockets? 2 If a basketball player s free throw percentage is 80%, what is the probability that he makes his next two free throws? 3 In May 2008, the S&P index was quoted 1400. Today it is quoted 1281. What is the probability that the S&P index will get back to 1400 in the next 8 months? We will see that the definition of probability is not unique.
The Classical Definition (P.S. Laplace) The probability of an event is the number of cases favorable to it the number of all cases possible when nothing leads us to expect that any one of these cases should occur more than any other. In American roulette, P(double-zero) = 1 38 0.02632 The classical definition is based on the assumption that each pocket is equally possible.
The Frequentist Definition The probability of an event is the limit as the number of repetitions of the number of times the event is observed number of repetitions of the experiment P(double-zero) = lim #spins number of double-zeros number of spins Requires one to conceptualize an infinite number of repetitions
Flipping a Coin We are interested in the probability of a coin coming up heads. Classical I will theorize that the coin has two sides and it is unbiased such that both sides are equally likely. So the answer is 0.5. Frequentist I will carry out an empirical experiment by flipping a coin thousand of times and record how many times it comes up heads. We will describe a third definition (Bayesian) later in the course.
Probability Model: Sample Space To describe probability mathematically, we first need to define what outcomes we can assign probabilities to. Call this the Sample Space, S, the set of all possible outcomes (Wonnacott calls this the outcome set ). For example, the sample space for 1 Tossing a coin: S = {H, T} 2 Tossing a coin twice: S = {HH, HT, TH, TT} 3 Rolling a die: S = {1, 2, 3, 4, 5, 6} 4 A person s height in inches: S = the interval 0 to 100 We have a lot of freedom in defining a sample space.
Probability Model: Event Now that we have a list of possible outcomes, we need to define what random phenomenon we are interested in. An event is an outcome or a set of outcomes from the sample space. Events for tossing a coin twice: S = {HH, HT, TH, TT } Heads then tails: E = {HT } At least one head: E = {HH, HT, TH} Not both heads: E = {HT, TH, TT } Events for someone s height: S = [0, 100] Greater than 5 feet: E = (60, 100] Between 5 and 7 feet: E = [60, 84]
Probability Model - Kolmogorov s axioms A probability model is a function that maps any event from the sample space to a real number. It must obey the following axioms: 1 A probability is a real number between 0 and 1 0 P(E) 1 2 Probability of the entire sample space is 1 P(S) = 1 3 The complement of an event E (that E does not occur), written as E c, is P(E c ) = 1 P(E)
Probability Model - Kolmogorov s axioms 4 If two events A and B are disjoint (mutually exclusive), then P(A or B) = P(A) + P(B) Two events are disjoint: They have no outcomes in common They cannot occur together Example: tossing two coins: A = {HT, TH}; B = {HH} With these rules, we can derive many useful properties of probability. These rules apply regardless of the definition of probability we adopt.
Selective Service Example For the Selective Service lottery, put all birth-dates for 2005 in a box. Mix thoroughly and draw one out. What is the probability that the date is in December? What is the probability that the date is not in December? What is the probability that the date is in January or February? What is the sample space and the corresponding events for the above? Why is the assumption that the numbers be thoroughly mixed in the box important?
Selective Service Example Suppose the officer draws two dates from the box (without replacement). What is the probability that both are in December? There are 31 365 ways to draw December on the first draw, and after removing that date there are 30 364 chances for December on the second draw. Then P(1 st in Dec & 2 nd in Dec) = 31 365 30 364 = 0.0069998 In defining the probability, we are counting the number of different ways in which the event can happen: there are 365 364 ways to draw two different dates. Of these, 31 30 are both in December.
Tree Diagram Consider flipping a coin three times, P(HHH) = 1 8
Tree Diagram Consider flipping a coin three times, P( two heads) = 4 8
Addition (Union) Rule The addition rule for OR, where or means that either event A or B or both occur: Example P(A or B) = P(A) + P(B) P(A and B) Roll a die: S = {1, 2, 3, 4, 5, 6} Define the events A = (2, 3, 4) and B = (4, 5). A and B = (4) with A and B = A Intersection B = A B P(A or B) = 3 6 + 2 6 1 6 = 4 6
Union Rule: Mutually Exclusive Events If events A and B are mutually exclusive, It is impossible for both to happen P(A and B) = 0 P(A or B) = P(A) + P(B) For example, the chance of rolling a 1 or 2 on a six-sided die is A = (1) and B = (2), therefore P(1 or 2) = 1 6 + 1 6 = 2 6 = 1 3 Note this is the same as Axiom 4.
Multiplicative Rule for Independent Events Events A and B are independent if the occurrence of one does not affect the occurrence of the other. Then P(A and B) = P(A) P(B). For example, two coins are flipped and the chance of both being heads is Also, P(H and H) = P(H) P(H) = 1 2 1 2 = 1 4. P(H or H) = P(H) + P(H) P(H)P(H) = 1 2 + 1 2 1 4 = 3 4
Conditional Probability Sometimes events can overlap (share similar outcomes in the sample space). When P(A) > 0, the conditional probability of B given A is P(B A) = Can you intuitively tell the difference? P(height > 6 4 basketball player) P(height > 6 4 ) P(B and A) P(A) Does knowing (given) the information that someone is a basketball player increase the likelihood that he/she is taller than 6 4?
Conditional Probability: Special Cases 1 A and B are mutually exclusive: If A occurs, B cannot occur. P(B A) = 2 A and B are independent: P(B and A) P(A) = 0 P(A) = 0 Knowing A offers no information for B and vice versa. P(B A) = P(A B) = P(B and A) P(A) P(A and B) P(B) = AND = P(A) P(B) P(A) P(A) P(B) P(B) = P(B) = P(A)
Tulip Example A bag contains 30 tulip bulbs: 12 Red 10 Yellow 8 Purple We first draw one bulb and then draw a second one without replacement. Evaluate the following probabilities: P(2Red) = P(1 st R & 2 nd R) = P(2 nd R 1 st R)P(1 st R) = 11 11 + 10 + 8 12 12 + 10 + 8 = 11 29 12 30 15%
P(1 st R & 2 nd Y ) = P(2 nd Y 1 st R)P(1 st R) = 10 11 + 10 + 8 12 12 + 10 + 8 = 10 29 12 30 14% P(1 st Y & 2 nd R) = P(2 nd R 1 st Y )P(1 st Y ) = 12 12 + 9 + 8 10 12 + 10 + 8 = 12 29 10 30 14%
Microchips Example A box contains 100 microchips. After testing, it turns out that Factory 1 (F1) Factory 2 (F2) Defective (D) 15 5 Good (G) 45 35 We can calculate the marginal probabilities: P(D) = 15 + 5 15 + 5 + 45 + 35 = 20 100 = 0.2 P(F 1) = 15 + 45 15 + 5 + 45 + 35 = 60 100 = 0.6 P(G) = 1 0.2 = 0.8 P(F 2) = 1 0.6 = 0.4
Microchips Example Question 1: Are factory and defective chips independent? P(D & F 1) = 0.15 P(D)P(F 1) = 0.2 0.6 = 0.12 0.15 Answer: not independent Question 2: Are factory and defective chips mutually exclusive? P(D & F 1) = 0.15 > 0 Answer: not mutually exclusive
The Monty Hall Problem Suppose you are on a TV show where you need to pick one of three doors. Behind one door, there is a prize. The remaining two doors are empty. Suppose you pick door A. The TV show presenter opens door B, which he knows to be empty. So what just happened? 1 Your chance of winning is still 1/3, because at least one of the remaining door B and C would be empty anyhow (no new relevant information has been provided) 2 Your chance of winning is now 1/2 Which of the two statements is correct?
The Monty Hall Problem Define O xy to be the event such that the TV show presenter opens door x if you choose y W x to be the event such that x is the winning door Therefore you are interested in knowing Conditional probabilities tell us that P(W A O BA ). P(W A O BA ) = P(W A & O BA ) P(O BA )
The Monty Hall Problem
The Monty Hall Problem The previous graph tells us that Therefore P(W A & O BA ) = 1 6 P(W B & O BA ) = 0 P(W C & O BA ) = 1 3 Remember that P(W A O BA ) = P(W A & O BA ) P(O BA ) = 1/6 1/6 + 0 + 1/3 = 1 3 P(O BA ) = P(W A & O BA ) + P(W B & O BA ) + P(W C & O BA ) implies marginalization with respect to W (.) Having an opened door does not give you any extra information!
The Monty Hall Problem with Switching Suppose now to play the same three door game again. However, after the TV show presenter opens an empty door, you can decide to switch to the remaining closed door. Is switching: A good strategy? A bad strategy? A neutral strategy (it does not matter)?!
The Monty Hall Problem with Switching
The Monty Hall Problem with Switching The previous graph shows you that: The non-switching strategy gives you a probability of winning equal to 1 6 + 1 6 = 1 3 The switching strategy gives you a probability of winning equal to 1 3 + 1 3 = 2 3 A study showed that only about 13% of the people will switch! To see many more variants see the Wikipedia page.
Summary Key words: Sample space and events Mutually exclusive Independence Addition and multiplicative rules Conditional probability Tree diagrams