Conditional Probability Idea have performed a chance experiment but don t know the outcome (ω), but have some partial information (event A) about ω. Question: given this partial information what s the probability that the outcome is in some event B? Example: Toss a coin 3 times. We are interested in event B that there are 2 or more heads. The sample space has 8 equally likely outcomes. Ω = { HHH, HHT, HTH, THH, HTT, THT, TTH, TTT} The probability of the event B is Suppose we know that the first coin came up H. Let A be the event the first outcome is H. Then A { HHH, HHT, HTH, HTT} and A B = HHH, HHT, The conditional probability of B given A is 3 3 8 P( A B) = = 4 4 8 P A = { HTH} ( ) week 2 1
Given a probability space (Ω, F, P) and events A, B F with P(A) > 0 The conditional probability of B given the information that A has occurred is P ( ) ( A B) P B A = P A Example: ( ) We toss a die. What is the probability of observing the number 6 given that the outcome is even? Does this give rise to a valid probability measure? Theorem If A F and P(A) > 0 then (Ω, F, Q) is a probability space where Q : is defined by Q(B) = P(B A). Proof: F R week 2 2
The fact that conditional probability is a valid probability measure allows the following: ( ) ( ) P B A = 1 P B A, A, B F, P(A) >0 P ( B B A) = P( B A) + P( B A) P( B B A) 1 2 1 2 1 2 for any A, B 1, B 2 F, P(A) >0. week 2 3
Multiplication rule For any two events A and B, For any 3 events A, B and C, P ( A B) = P P ( B A) P( A) ( A B C) = P( A) P( B A) P( C A B) In general, P n I i= 1 A i = P n 1 ( A ) P( A A ) P( A A A ) P A A 1 2 1 3 1 2 n I i= 1 i Example: An urn initially contains 10 balls, 3 blue and 7 white. We draw a ball and note its colure; then we replace it and add one more of the same colure. We repeat this process 3 times. What is the probability that the first 2 balls drawn are blue and the third one is white? Solution: week 2 4
Law of total probability Definition: For a probability space (Ω, F, P), a partition of Ω is a countable collection { B i } of events such that B i F, B B = Φ and B = Ω. Theorem: If { B, B,... 1 2 } is a partition of Ω such that P B i > 0 then Proof: ( A) P( A B ) P( ) i j U i for any A. P = i B i F i i ( ) i week 2 5
Examples 1. Calculation of for the Urn example. P ( ) B 2 2. In a certain population 5% of the females and 8% of the males are lefthanded; 48% of the population are males. What proportion of the population is left-handed? Suppose 1 person from the population is chosen at random; what is the probability that this person is left-handed? week 2 6
{ } Bayes Rule Let B, 2,... be a partition of Ω such that P( ) > 0 for all i then 1 B for any A F. P ( B A) j = P( A B j ) P( B j ) P A Bi P Bi i ( ) ( ) Example: A test for a disease correctly diagnoses a diseased person as having the disease with probability 0.85. The test incorrectly diagnoses someone without the disease as having the disease with probability 0.1 If 1% of the people in a population have the disease, what is the probability that a person from this population who tests positive for the disease actually has it? (a) 0.0085 (b) 0.0791 (c) 0.1075 (d) 0.1500 (e) 0.9000 B i week 2 7
Independence Example: Roll a 6-sided die twice. Define the following events A : 3 or less on first roll B : Sum is odd. If occurrence of one event does not affect the probability that the other occurs than A, B are independent. week 2 8
Definition Events A and B are independent if Note: Independence disjoint. Two disjoint events are independent if and only if the probability of one of them is zero. Generalized to more than 2 events: A collection of events 1, 2 is (mutually) independent if for any subcollection P ( A B) P( A) P( B) P = { A A,... A } i, 1 i2 { A A,... } i m A n ( A A A ) = P( A ) P( A ) P( A ) i 1 i2 im i1 i 2 im Note: pairwize independence does not guarantee mutual independence. week 2 9
Example Roll a die twice. Define the following events; A: 1st die odd B: 2nd die odd C: sum is odd. week 2 10
Example Let R, S and T be independent, equally likely events with common ( ) probability 1/3. What is P R S T? Solution: week 2 11
Claim If A, B are independent so are A, B and A, B and A, B. Proof: week 2 12
Random Variables Example: We roll a fair die 6 times. Suppose we are interested in the number of 5 s in the 6 rolls. Let X = number of 5 s. Then X could be 0, 1, 2, 3, 4, 5, 6. X = 0 corresponds to the 56 elements of our 66 elements of Ω. X = 1 corresponds to the elements etc. X is an example of a random variable. Probability models often stated terms of random variables. E.g. - model for the # of H s in 10 flips of a coin. - model for the height of a randomly chosen person. - model for size of a queue. week 2 13
Discrete Probability Spaces (Ω, F, P) week 2 14
Discrete Random Variable Definition: A random variable X is said to be discrete if it can take only a finite or countably infinite number of distinct values. A discrete random variable X maps the sample space Ω onto a countable set. Define a probability mass function (pmf) or frequency function on X such that Where the sum is taken over all possible values of X. Note that there is a theorem that states that there exists a probability triple and random variable whenever we have a function p such that Definition: The probability distribution of a discrete random variable X is represented by a formula, a table or a graph which provides the list of all possible values that X can take and the pmf for each value week 2 15
Examples of Discrete Random Variables Discrete Uniform Distribution We roll a fair die. Let X = the # that comes up. We have that This is an example of equiprobable outcomes, that is X ( ω ) = ω To state the probability distribution of X we need to give its possible values and its pmf X is a discrete Uniform random variable. X has a uniform distribution. week 2 16
Bernoulli Distribution week 2 17
Binomial Distribution Roll a die n time and count the number of times 6 came up. Let X be the number of 6 s in n rolls. X has image {1, 2,, n} The probability distribution of X is given by the following formula In general, if identical Bernoulli trail is repeated n times independently and X is a random variable that count the number of success in the n trails then the probability distribution of X is given by Where p is the probability of success on any one experiment. X is a Binomial random variable. X has a Binomial Distribution. Question: is this a valid pmf? Prove! week 2 18
Geometric Distribution We roll a fair die until the first 6 comes up. Let X = the number of rolls until we get the first 6. Possible values of X: {1, 2, 3,..} The probability distribution of X is given by the following formula In general, if identical Bernoulli trail is repeated independently until the first success is obtained and X is a random variable that count the number of trials until the first success then the probability distribution of X is given by X is a Geometric random variable. X has a Geometric Distribution. Question: is this a valid pmf? Prove! week 2 19
In general for a Geometric distribution: Memory-less property of geometric random variable: for i > j week 2 20
Negative Binomial Distribution We roll a fair die until the second 6 comes up. This is the waiting time for the second 6. Let X = the number of rolls until we get two 6 s. Possible values of X: {2, 3, 4,..} The probability distribution of X is given by the following formula Is this a valid pmf? Prove! In general, X is the total number of experiments when waiting for rth success in a sequence of independent Bernoulli trails. The probability distribution of X is given by X has a Negative Binomial random Distribution. week 2 21
Hypergeometric Distribution A hat contains 12 tickets, 7 black and 5 white. Three tickets are drawn at random. Let X = the # of black tickets drawn. X could be 0, 1, 2, 3. The probability mass for each value can be calculated using combinatorics. For example, week 2 22
Poisson Distribution Model for the number of events occurring in a time (or space) interval where λ (a parameter of the distribution) is the rate of the occurrence of the events per one unit of time (or space). A Poisson random variable X = number of events per one unit of time (space). Possible values for X: {0, 1, 2, } The probability distribution of X is given by Is this a valid pmf? Prove! week 2 23