Probability Probability theory is the branch of math that deals with unpredictable or random events Probability is used to describe how likely a particular outcome is in a random event the probability of obtaining heads in a single coin flip the probability of getting into a car accident today Probabilities are always between 0 and 1 A probability near 1 indicates that the event is very likely A probability near 0 indicates that the event is very unlikely
Interpretations of Probability Classical or Analytic view Definition of probability in terms of an analysis of possible (and equally likely) outcomes for a set of events Examples: Proportion of ways in which two 6-sided dice can yield 10 total dots Proportion of 5-card combinations that can form a royal flush Frequentist view Definition of probability in terms of past performance p=(f/n) Example: If we roll a die 1000 times and it comes up 6 250 times, then we estimate the probability of rolling a 6 as 0.25 (Note: such a die is probably loaded) Subjective or Bayesian view Probability as degree of belief Examples: Probability that a particular candidate will win the next election Probability that the die in the example above is fair
Basic Terminology Trial: one of a number of repetitions of an experiment E.g, a coin flip, die roll, or measurement procedure Event (Variable): the outcome (value) of a trial E.g., heads or tails, the showing face on a die, the value of the measurement Mutually exclusive events: events related such that the occurrence of one precludes the occurrence of the other E.g., heads vs. tails for a single coin flip, color of M&M for a single draw from a bag, the suit of a card on a single draw Exhaustive set: a set of events encompassing all possibilities E.g., heads & tails, all possible M&M colors, all possible card suits
Standard Deck of Cards 52 cards: 4 suits x 13 ranks
Mutually Exclusive Events All Events Intersecting Events All Events Set A Set B Set A A B Set B All Events Set C Mutually Exclusive & Exhaustive Set Set A Set B Note: Here, Set C is meant to indicate items that are neither in A nor in B
Basic Terminology Marginal (unconditional) probability: the probability of one event, ignoring the occurrence or nonoccurrence of other (simultaneous) events Denoted: P(event1) Examples: P(height>68 ), P(suit=clubs) All Events P(A)
Marginal Probability: P(diamond) Basic Concepts of Probability P(diamond) = 13/52 = 0.25
Basic Terminology Joint probability: the probability of (simultaneous) occurrence of two or more events Denoted: P(event1,event2) or P(event1 event 2) Examples: P(height>68, gender=female), P(suit = clubs, rank = king) Mutually Exclusive Events All Events Intersecting Events All Events Set A Set B Set A P(A,B) Set B P(A,B) P(A,B) = 0 P(A,B) > 0
Joint Probability: P(diamond, face card) P(diamond) = 13/52 = 0.25 P(face card) = 12/52 = 0.231 P(diamond,face card) = 3/52 = 0.058
Basic Terminology Conditional probability: the probability that one event will occur given the occurrence of some other event Denoted: P(event1 event2) Examples: P(height>68 gender=female), P(suit=clubs color=black) All Events P(A B) = P(A,B)/P(B) P(A,B) Set A P(A,B) Set B P(B)
Conditional Probability: P(diamond face card) P(diamond face card) = 3/12 = 0.25
Conditional Probability: P(face card diamond) P(face card diamond) = 3/13 = 0.231
Basic Terminology Independent events: events related such that the occurrence of one has no effect on the probability of occurrence of the other E.g., successive coin flips, successive M&M draws from bag sampled with replacement A special category of independent events is IID or independent and identically distributed events Many of the techniques in this course depend on IID assumptions Dependent events: events related such that the occurrence of one affects the probability of occurrence of the other E.g., single die outcome in pair & sum of die outcomes, successive M&M draws from bag sampled without replacement
Independent Events: (diamond, face card) P(diamond) = 13/52 = 0.25 P(diamond face card) = 3/12 = 0.25
Dependent Events: (diamond,red card) P(diamond) = 13/52 = 0.25 P(diamond red card) = 13/26 = 0.5
Basic Laws of Probability Additive law: Given a set of mutually exclusive events (e.g., {A,B,C}), the combined probability of occurrence of any of the events in the set (e.g., P(A or B or C)) is equal to the sum of their separate probabilities. Examples: For a fair flipped coin, the probability of heads is 0.5 and the probability of tails is 0.5. Since these outcomes are mutually exclusive, the probability of obtaining either heads or tails is 0.5+0.5 = 1 For a fair 6-sided die, the probability of occurrence of each face is 1/6. Since the die can only land on one face at a time (the face outcomes are mutually exclusive), the probability of obtaining a 4 or a 5 or a 6 is (3*1/6) = ½. Note: the combined probability of occurrence of any of the events in a set is always greater than (or equal to) the probability of occurrence of any individual event within the set
Additive Law: P(diamond or spade) Basic Concepts of Probability P(diamond) = 13/52 = 0.25 P(spade) = 13/52 = 0.25 P(diamond or spade) = P(diamond) + P(spade) = 0.5
Basic Laws of Probability Multiplicative law: Given a set of independent events (e.g., {D,E,F}), the combined probability of occurrence of all the events in the set (P(D,E,F))is the product of their individual probabilities. Example: Successive coin flips: the probability of a particular sequence (e.g., H,T) is equal to the product of the individual probabilities (e.g., P(H,T) = P(H)P(T) = 0.5 x 0.5 = 0.25) Note: because probabilities are [0,1], the combined probability of occurrence all of the events in a set of independent events is always smaller than (or equal to) the probability of any individual event in the set
Multiplicative Law: P(diamond, face card) P(diamond) = 13/52 = 0.25 P(face card) = 12/52 = 0.231 P(diamond,face card) = P(diamond) P(face card) = 0.058
Examples Using the Additive and Multiplicative Laws 1. What is the probability of obtaining two heads in two successive coin flips? a) 0.5 b) 1.0 c) 0.25 d) 2.0 Fair Coin Distribution ( ) = P( Tails) = 0.5 P Heads Official M&M's Color Distribution Color p brown 0.3 red 0.2 blue 0.1 orange 0.1 green 0.1 yellow 0.2
Examples Using the Additive and Multiplicative Laws 2. What is the probability of obtaining either two heads or two tails in two successive coin flips? Fair Coin Distribution ( ) = P( Tails) = 0.5 P Heads a) 0.5 b) 1.0 c) 0.25 d) 0.125 Official M&M's Color Distribution Color p brown 0.3 red 0.2 blue 0.1 orange 0.1 green 0.1 yellow 0.2
Examples Using the Additive and Multiplicative Laws 3. What is the probability that one M&M drawn from a bag will be either green or blue or red? Fair Coin Distribution ( ) = P( Tails) = 0.5 P Heads a) 0.002 b) 0.4 c) 0.2 d) 0.125 Official M&M's Color Distribution Color p brown 0.3 red 0.2 blue 0.1 orange 0.1 green 0.1 yellow 0.2
Examples Using the Additive and Multiplicative Laws 4. When drawing two M&Ms from a bag with replacement, what is the probability of getting one red and one brown one (ignoring the order of the draws)? Fair Coin Distribution ( ) = P( Tails) = 0.5 P Heads a) 0.12 b) 0.06 c) 0.5 d) 0.05 Official M&M's Color Distribution Color p brown 0.3 red 0.2 blue 0.1 orange 0.1 green 0.1 yellow 0.2
Generalizing the Additive & Multiplicative Laws The book simplifies things by giving you rules only for mutually exclusive or independent events. However, both laws can be written more generally using conditional and joint probabilities Generalized Additive Law: P(A or B) = P(A)+P(B) P(A,B) Note: if A and B are not mutually exclusive, then we have to subtract off their intersection (A B) to keep from counting it twice Generalized Multiplicative Law: P(A,B) = P(A B) P(B) Note: if A and B are independent, then P(A B) = P(A), which is how we get the law for the independent case.
Generalized Additive Law (number or red card) P(red card) = 26/52 P(number card) = 40/52 P(number or red card) = 20/52
Relating Joint and Conditional Probabilities Given a set of two simultaneous outcomes (e.g., gender and height) the joint probability of both outcomes is equal to the product of the conditional probability of one event (conditioned on the other) multiplied by the marginal probability of the other. Example: (the score tall means height > 68 inches) or ( tall, female) = ( tall female) ( female) P P P ( tall,female) = ( female tall) ( tall) P P P Equivalently, P ( female tall) = P ( tall, female) P ( tall)
P(female) = 0.51 female P(male) = 0.49 male ( tall, female) = ( tall female) ( female) P P P = 0.07 0.51 = 0.0357 ( tall, male) = ( tall male) ( male) P P P = 0.71 0.49 = 0.347 Sample Questions: What is the marginal probability of being tall? ( ) ( ) ( ) P tall = P tall, female + P tall, male = 0.3470 + 0.0357 = 0.38 Are height and gender independent? P P( tall, male) = 0.347 ( ) P( ) = 0. 38 = P( tall, male) P( tall) P( male) tall male 0.49 0.18
Contingency Tables (crosstabs) Basic Concepts of Probability Event Frequencies Death Sentence Race Yes No Total Black 95 425 520 Nonblack 19 128 147 Total 114 553 667 Event Probabilities (f/n) Death Sentence Race Yes No Total Black 0.142 0.637 0.780 Nonblack 0.028 0.192 0.220 Total 0.171 0.829 1.000 Sample Questions: Are race of the defendant and application of the death sentence independent? P P ( black, death) = 0.142 ( ) P( ) = 0.780 = P( black, death) P( black ) P( death) black death 0.171 0.133 How likely is a defendant to receive the death penalty given that he is black versus nonblack? P P ( death black ) ( death nonblack ) ( death, black) P ( black) P = = 0.142 / 0.780 = 0.182 ( death, nonblack ) P( nonblack ) P = = 0.028 / 0.220 = 0.127