Probability & Random Variables

Transcription

1 & Random Variables

2 Probability Probability theory is the branch of math that deals with random events, processes, and variables What does randomness mean to you? How would you define probability in your own words? 2

3 Probability Probability theory is the branch of math that deals with random events, processes, and variables Probability is used to describe how likely a particular outcome is in a random event the probability of obtaining heads in a single coin flip the probability of getting into a car accident today Probabilities are always between 0 and 1 A probability near 1 indicates that the event is very likely A probability near 0 indicates that the event is very unlikely 3

4 Interpretations of Probability Classical or Analytic view Definition of probability in terms of an analysis of possible (and equally likely) outcomes for a set of events. I.e., p=(favorable cases/all possible cases) Examples: Proportion of ways in which two 6-sided dice can yield 10 total dots Proportion of 5-card combinations that can form a royal flush Probability Frequentist view Definition of probability in terms of past performance p=(f/n) Example: If we roll a die 1000 times and it comes up times, then we estimate the probability of rolling a 6 as 0.25 (Note: such a die is probably loaded) Subjective or Bayesian view Probability as degree of belief Examples: Probability that a particular candidate will win the next election Probability that the die in the example above is fair Probability that a sound I hear in the middle of the night is caused by a burglar 4

5 Basic Terminology Trial: one of a number of repetitions of an experiment E.g, a coin flip, die roll, or measurement procedure Event (Variable): the outcome (value) of a trial E.g., heads or tails, the showing face on a die, the value of the measurement Disjoint (mutually exclusive) events: events related such that the occurrence of one precludes the occurrence of the other E.g., heads vs. tails for a single coin flip, color of M&M for a single draw from a bag, the suit of a card on a single draw Sample space (exhaustive set): the set of all possible outcomes E.g., heads & tails, all possible M&M colors, all possible card suits Complementary events: a set of two disjoint events that form an exhaustive set E.g., the outcomes 1 and not 1 in a single dice roll 5

6 Axioms of Probability (Kolmogorov) A function P() on a sample space Ω is a probability measure if: 1. The measure of the function on the entire sample space is 1 P 1 2. The measure of any subset of the sample space is greater than or equal to 0 0, P A A 3. The measure of the union of any set of mutually exclusive events in the sample space is equal to the sum of the measures of the individual events P B C P B P( C) B, C B C ( ), s.t. 6

7 Mutually Exclusive Events All Events Intersecting Events All Events Set A Set B Set A A B Set B All Events Set C Mutually Exclusive & Exhaustive Set Set A Set B Note: Here, Set C is meant to indicate items that are neither in A nor in B 7

8 Standard Deck of Cards 52 cards: 4 suits x 13 ranks variables: suit, color, rank, rank type (face vs. number) 8

9 Basic Terminology Marginal probability: the probability of one event, ignoring the occurrence or nonoccurrence of other (simultaneous) events Denoted: P(event1) Examples: P(height>68 ), P(suit=clubs) All Events P(A) 9

10 Marginal Probability: P(diamond) P(diamond) = 13/52 =

11 Basic Terminology Joint probability: the probability of (simultaneous) occurrence of two or more events Denoted: P(event1,event2) or P(event1 event 2) Examples: P(height>68, gender=female), P(suit = clubs, rank = king) Mutually Exclusive Events All Events Intersecting Events All Events Set A Set B Set A P(A,B) Set B P(A,B) P(A,B) = 0 P(A,B) > 0 11

12 Joint Probability: P(diamond, face card) P(diamond) = 13/52 = 0.25 P(face card) = 12/52 = P(diamond,face card) = 3/52 =

13 Basic Terminology Conditional probability: the probability that one event will occur given the occurrence of some other event Denoted: P(event1 event2) Examples: P(height>68 gender=female), P(suit=clubs color=black) All Events P(A B) = P(A,B)/P(B) P(A,B) Set A P(A,B) Set B P(B) 13

14 Conditional Probability: P(diamond face card) P(diamond face card) = 3/12 =

15 Conditional Probability: P(face card diamond) P(face card diamond) = 3/13 =

16 Basic Terminology Independent events (variables): events (variables) related such that the occurrence of one has no effect on the probability of occurrence of the other E.g., successive coin flips, successive M&M draws from bag sampled with replacement A special category of independent events is IID or independent and identically distributed events Many of the techniques in this course depend on IID assumptions Dependent events (variables): events (variables) related such that the occurrence of one affects the probability of occurrence of the other E.g., single die outcome in pair & sum of die outcomes, successive M&M draws from bag sampled without replacement 16

17 A Note on Replacement Most of the tools that we use in statistics assume very large populations We normally sample from populations without replacement However, as population size (N) grows, the effects of withdrawing individual scores disappears, allowing us to treat individual scores as IID Statistics for IID samples are much easier to compute 17

18 Independent Events: (diamond, face card) Probability P(diamond) = 13/52 = 0.25 P(diamond face card) = 3/12 =

19 Dependent Events: (diamond,red card) P(diamond) = 13/52 = 0.25 P(diamond red card) = 13/26 =

20 Basic Laws of Probability Additive law: Given a set of mutually exclusive events (e.g., {A,B,C}), the combined probability of occurrence of any of the events in the set (e.g., P(A or B or C)) is equal to the sum of their separate probabilities. Examples: For a fair flipped coin, the probability of heads is 0.5 and the probability of tails is 0.5. Since these outcomes are mutually exclusive, the probability of obtaining either heads or tails is = 1 For a fair 6-sided die, the probability of occurrence of each face is 1/6. Since the die can only land on one face at a time (the face outcomes are mutually exclusive), the probability of obtaining a 4 or a 5 or a 6 is (3*1/6) = ½. Note: the combined probability of occurrence of any of the events in a set is always greater than (or equal to) the probability of occurrence of any individual event within the set 20

21 Additive Law: P(diamond or spade) P(diamond) = 13/52 = 0.25 P(spade) = 13/52 = 0.25 P(diamond or spade) = P(diamond) + P(spade) =

22 Basic Laws of Probability Multiplicative law: Given a set of independent events (e.g., {D,E,F}), the combined probability of occurrence of all the events in the set (P(D,E,F))is the product of their individual probabilities. Example: Successive coin flips: the probability of a particular sequence (e.g., H,T) is equal to the product of the individual probabilities (e.g., P(H,T) = P(H)P(T) = 0.5 x 0.5 = 0.25) Note: because probabilities are [0,1], the combined probability of occurrence all of the events in a set of independent events is always smaller than (or equal to) the probability of any individual event in the set 22

23 Multiplicative Law: P(diamond, face card) Probability P(diamond) = 13/52 = 0.25 P(face card) = 12/52 = P(diamond,face card) = P(diamond) P(face card) =

24 Generalizing the Additive & Multiplicative Laws The book simplifies things by giving you rules only for mutually exclusive or independent events. However, both laws can be written more generally using conditional and joint probabilities Generalized Additive Law: P(A or B) = P(A)+P(B) P(A,B) Note: if A and B are not mutually exclusive, then we have to subtract off their intersection (A B) to keep from counting it twice Generalized Multiplicative Law: P(A,B) = P(A B) P(B) Note: if A and B are independent, then P(A B) = P(A), which is how we get the law for the independent case. 24

25 Generalized Additive Law (number or red card) P(red card) = 26/52 P(number card) = 40/52 P(number,red card) = 20/52 25

26 Relating Joint and Conditional Probabilities Probability Given a set of two simultaneous outcomes (e.g., gender and smoking) the joint probability of both outcomes is equal to the product of the conditional probability of one event (conditioned on the other) multiplied by the marginal probability of the other. Example: or smoke, female smoke female female P P P P female smoke smoke,female female smoke smoke P P P Equivalently, P smoke, female P Bayes Rule smoke 26

27 Bayes Rule (A Practical Example) Lupus is an (overdiagnosed) autoimmune disorder whose prevalence in the population is estimated about 2% The most common test for lupus correctly identifies the disease in 98% of people who have it and correctly rejects the disease in 74% of people who do not have it. A patient tests positive for lupus. What is the probability that the patient has the disease? Remind me to say something here about the distinction between parameters and variables 27

28 Bayes Rule (A Practical Example) P P P P lupus test lupus test lupus 0.74 Plupus Plupus test P lupus 1 test lupus P lupus test P P lupus, test P test Plupus Ptest lupus lupus, test Plupus, test Plupus Ptest lupus lupus test lupus lupus test lupus P P P P 0.02(0.98) Plupus test (0.98) 0.98( 0.26) It s never lupus. House, MD 28