Homework 3 due Monday, November 19 at 5 PM. Hint added to Problem 2.2.

Transcription

1 advprobnotes111607b Page 1 Covariance and conditional probability Friday, November 16, :59 PM Homework 3 due Monday, November 19 at 5 PM. Hint added to Problem 2.2. If we have two random variables and want to discuss how dependent they are, one of the simplest ways to quantify such a relationship is through the covariance of the random variables The sign of the covariance describes the qualitative positive or negative correlation between the random variables, but to quantify it, it is cleaner to use the correlation coefficient (nondimensionalized covariance): Consider now sums of possibly dependent random variables:

2 advprobnotes111607b Page 2 We see that it is not difficult to calculate the mean and variance of sums of arbitrary random variables. However, computing the full probability distribution is much more complicated in general than for the case where the random variables are independent. (No simple generalization of convolution formulas.) sampling of some property from a finite population: Each member of the population has some characteristic quantity and there are m members of the population. Suppose we want to estimate the average value of this quantity over the population by looking at finite sample of size n. Define random variables as the values sampled: If we think of the sample as being done one after the other without replacement, then

3 advprobnotes111607b Page 3 What are the statistics of the sample mean? Note that the random variables are not independent. (because sampling without replacement) So the full probability distribution is hard to calculate but let's look at mean and standard deviation. Note that the sample mean is a random variable! But how accurate is the estimator? Calculate its standard deviation (through the variance) to get a quantitative assessment.

4 To calculate c, one can proceed directly or through a clever trick. advprobnotes111607b Page 4

5 advprobnotes111607b Page 5 Slicker way to calculate Covariance just gives partial information about the relationship between random variables, just as variance just gives partial information about the probability distribution of the random variable. In particular, two random variables with zero covariance can be highly

6 advprobnotes111607b Page 6 zero covariance can be highly dependent. To characterize more completely the relationship between random variables, one uses either joint probability distribution (already discussed) and the notion of conditional probabilities and distributions, which we'll discuss next. This conditional probability formula suggests the following perspective: "Given B", then B becomes like a new probability space, and the conditional probability is the probability measure inherited on this probability space. The denominator is a normalizing factor.

7 advprobnotes111607b Page 7 Basic laws of conditional probability: All rules involving standard probabilities also hold true for conditional probabilities, provided all conditions are the same. In other words, one can take any probability rule and add the same condition to every term (provided the condition has positive probability).

8 advprobnotes111607b Page 8 condition has positive probability). And so on because conditional probability distributions are just normal probability distributions which are associated to a probability space consistent with the given information. One can also add common conditions to conditional probability formulas: Prisoner's dilemma (probability style) Prisoners A, B and C: 2 of them are to be executed (all choices are equally likely). A asks the guard for the name of one prisoner, other than himself, to be executed, since there must exist such a name. Guard says B will be executed. Given this informatin what is the probability that A is executed?

9 So the guard's information does not affect prisoner A's probability of being executed. We assumed the guard would always choose the name of the other prisoner to be executed if A is also to be executed. But suppose that A knew the guard would have probability p to tell A he will be executed when A is scheduled to be executed. Then we change the above calculation by: advprobnotes111607b Page 9

10 advprobnotes111607b Page 10 since the guard would have told him he would be executed, if he were to. in agreement with the Result of the prisoner's reasoning -- because prisoner A had some chance of being informed of his sad fate, and since he wasn t, there is legitimate reason to have more hope. (Kind of like having Dr. House not telling you that you have some horrible illness being a good sign, whereas other doctors may want to hide the truth of a serious illness from a patient and so not hearing the bad news does not necessarily mean there isn't bad news.)