Probability Probability theory is the branch of math that deals with random events Probability is used to describe how likely a particular outcome is in a random event the probability of obtaining heads in a single coin flip the probability of getting into a car accident today Probabilities are always between 0 and 1 A probability near 1 indicates that the event is very likely A probability near 0 indicates that the event is very unlikely
Interpretations of Probability Analytic view Definition of probability in terms of an analysis of the sample space, or set of possible outcomes Examples: Proportion of ways in which two 6-sided dice can yield 10 total dots Proportion of 5-card combinations that can form a royal flush Frequentist view Definition of probability in terms of past performance p=(f/n) Example: If we roll a die 1000 times and it comes up 6 250 times, then we estimate the probability of rolling a 6 as 0.25 (Note: such a die is probably loaded) Subjective view
Basic Terminology Trial: one of a number of repetitions of an experiment E.g, a coin flip, die roll, or measurement procedure Event: the outcome of a trial E.g., heads or tails, the showing face on a die, the value of the measurement Mutually exclusive events: events related such that the occurrence of one precludes the occurrence of the other E.g., heads vs. tails for a single coin flip, color of M&M for a single draw from a bag Exhaustive set: a set of events encompassing all possibilities E.g., heads & tails, all possible M&M colors
Mutually Exclusive Events All Events Intersecting Events All Events Set A Set B Set A A B Set B All Events Set C Mutually Exclusive & Exhaustive Set Set A Set B Note: Here, Set C is meant to indicate items that are neither in A nor in B
Basic Terminology Marginal (unconditional) probability: the probability of one event, ignoring the occurrence or nonoccurrence of other (simultaneous) events Denoted: P(event1) Example: P(height>68 ) All Events P(A)
Basic Terminology Joint probability: the probability of (simultaneous) occurrence of two or more events Denoted: P(event1,event2) or P(event1 event 2) Example: P(height>68, gender=female) Mutually Exclusive Events All Events Intersecting Events All Events Set A Set B Set A P(A,B) Set B P(A,B) P(A,B) = 0 P(A,B) > 0
Basic Terminology Conditional probability: the probability that one event will occur given the occurrence of some other event Denoted: P(event1 event2) Example: P(height>68 gender=female) All Events P(A B) = P(A,B)/P(B) P(A,B) Set A P(A,B) Set B P(B)
Basic Terminology Independent events: events related such that the occurrence of one has no effect on the probability of occurrence of the other E.g., successive coin flips, successive M&M draws from bag sampled with replacement A special category of independent events is IID or independent and identically distributed events Many of the techniques in this course depend on IID assumptions Dependent events: events related such that the occurrence of one affects the probability of occurrence of the other E.g., single die outcome in pair & sum of die outcomes, successive M&M draws from bag sampled without replacement
Basic Laws of Probability Additive law: Given a set of mutually exclusive events (e.g., {A,B,C}), the combined probability of occurrence of any of the events in the set (e.g., P(A or B or C)) is equal to the sum of their separate probabilities. Examples: For a fair flipped coin, the probability of heads is 0.5 and the probability of tails is 0.5. Since these outcomes are mutually exclusive, the probability of obtaining either heads or tails is 0.5+0.5 = 1 For a fair 6-sided die, the probability of occurrence of each face is 1/6. Since the die can only land on one face at a time (the face outcomes are mutually exclusive), the probability of obtaining a 4 or a 5 or a 6 is (3*1/6) = ½. Note: the combined probability of occurrence of any of the events in a set is always greater than (or equal to) the probability of occurrence of any individual event within the set
Basic Laws of Probability Multiplicative law: Given a set of independent events (e.g., {D,E,F}), the combined probability of occurrence of all the events in the set (P(D,E,F))is the product of their individual probabilities. Example: Successive coin flips: the probability of a particular sequence (e.g., H,T) is equal to the product of the individual probabilities (e.g., P(H,T) = P(H)P(T) = 0.5 x 0.5 = 0.25) Note: because probabilities are [0,1], the combined probability of occurrence all of the events in a set of independent events is always smaller than (or equal to) the probability of any individual event in the set
Examples Using the Additive and Multiplicative Laws 1. What is the probability of obtaining two heads on two successive coin flips? 2. What is the probability of obtaining either two heads or two tails on successive coin flips? 3. What is the probability that one M&M drawn from a bag will be either green or blue or red? 4. When drawing two M&Ms from a bag, what is the probability of getting one red and one brown one (ignoring the order of the draws)? 5. When drawing three M&Ms from a bag, what is the probability that at least two of them are brown? Fair Coin Distribution PTails 0.5 P Heads Official M&M's Color Distribution Color p brown 0.3 red 0.2 blue 0.1 orange 0.1 green 0.1 yellow 0.2
Generalizing the Additive & Multiplicative Laws The book simplifies things by giving you rules only for mutually exclusive or independent events. However, both laws can be written more generally using conditional and joint probabilities Generalized Additive Law: P(A or B) = P(A)+P(B) P(A,B) Note: if A and B are not mutually exclusive, then we have to subtract off their intersection (A B) to keep from counting it twice Generalized Multiplicative Law: P(A,B) = P(A B) P(B) Note: if A and B are independent, then P(A B) = P(A), which is how we get the law for the independent case.
Relating Joint and Conditional Probabilities Given a set of two simultaneous outcomes (e.g., gender and height) the joint probability of both outcomes is equal to the product of the conditional probability of one event (conditioned on the other) multiplied by the marginal probability of the other. Example: (the score tall means height > 68 inches) or tall, female tall female female P P P tall, female female tall tall P P P Equivalently, P female tall P tall, female P tall
P(female) = 0.51 female P(male) = 0.49 male tall, female tall female female P P P 0.070.51 0.0357 tall, male tall male male P P P 0.710.49 0.347 Sample Questions: What is the marginal probability of being tall? Are height and gender independent?
Crosstabulations (Contingency Tables) Event Frequencies Death Sentence Race Yes No Total Black 95 425 520 Nonblack 19 128 147 Total 114 553 667 Sample Questions: Are race of the defendant and application of the death sentence independent? Event Probabilities (f/n) Death Sentence Race Yes No Total Black 0.142 0.637 0.780 Nonblack 0.028 0.192 0.220 Total 0.171 0.829 1.000 How likely is a defendant to receive the death penalty given that he is black versus nonblack?