Review of probability and statistics 1 / 31

Transcription

1 Review of probability and statistics 1 / 31

2 2 / 31 Why? This chapter follows Stock and Watson (all graphs are from Stock and Watson). You may as well refer to the appendix in Wooldridge or any other introduction to statistics. When analyzing observational data, we must rely on basic concepts of probability theory and statistics. We would like to identify causal effects in a population, but usually only have a sample from this population available. Under what conditions can we make statements about the population? One key assumption: Sample has to be a random draw.

3 3 / 31 General Probability The probability of an outcome is the proportion of the time that the outcome occurs in the long run. Random variables A random variable is a numerical summary of a random outcome. Probability distribution The probability distribution of a discrete random variable is the list of all possible values of the variable and the probability that each value will occur. These probabilities sum to 1.

4 4 / 31 Expected value The expected value of a random variable Y, denoted E(Y), is the long-run average value of the random variable over many repeated trials or occurrences. It is computed as a weighted average of the possible outcomes of a random variable where the weights are the probabilities of the outcome k E(Y) = y i p i It is also called the expectation of Y or the mean of Y and is denoted by µ Y. The mean of Y is also referred to as the first moment of Y. Question: What is the expected value when throwing a dice? i=1

5 5 / 31 Variance The variance measures the dispersion or the spread of a probability distribution. [ σy 2 = var(y) = E (Y µ Y ) 2] = k (y i µ Y ) 2 p i i=1 The standard deviation σ Y is the square root of the variance.

6 6 / 31 Joint distribution The joint probability distribution of two random variables, say X and Y, is the probability that the random variables simultaneously take on certain values, say x and y. The probabilities of all possible combinations sum to 1. We can express the joint probability distribution as Pr(X = x, Y = y).

7 7 / 31 Conditional distribution The distribution of a random variable Y conditional on another variable X taking on a specific value is called the conditional distribution of Y given X. We can express the conditional distribution as PR(Y = y X = x) = Pr(X = x, Y = y). Pr(X = x)

8 Joint and conditional distribution 8 / 31

9 9 / 31 Conditional expectation The conditional expectation (or mean) of Y given X is the mean of the conditional distribution of Y given X. If Y takes on values y 1,..., y k, then the conditional mean of Y given X = x is E (Y X = x) = k y i Pr(Y = y i X = x i ) i=1 Question: What is the expected number of computer crashes given the computer is old? The conditional variance follows the same logic.

10 10 / 31 Covariance & correlation These are two measures that describe to which extent two random variables move together. The covariance between X and Y is the expected value E [(X µ X )(Y µ Y )] and denoted cov(x, Y). If X can take on l values and Y can take on k values, then the covariance is given by cov(x, Y) = σ XY = E [(X µ X ) (Y µ Y )] k l = (x j µ X ) (y i µ Y ) Pr (X = x j, Y = y i ) i=1 j=1

11 11 / 31 Covariance & correlation The units of the covariance are the units of X multiplied by the units of Y This makes the covariance difficult to interpret. The correlation overcomes this problem and is defined as the covariance between X and Y divided by their standard deviation: corr(x, Y) = cov(x, Y) var(x)var(y) = σ XY σ X σ Y As units cancel, correlation is unitless and ranges between -1 and 1.

12 12 / 31 Correlation & conditional mean If the conditional mean of Y does not depend on X, then Y and X are uncorrelated, that is if E(Y X) = µ Y then cov(x, Y) = corr(x, Y) = 0

13 13 / 31 Random sampling Almost all the statistical and econometric procedures used here involve averages of a sample of data. Characterizing the distribution of sample averages is thus essential to understand the performance of these techniques. Random sampling makes the sample average a random variable! It has a probability distribution, called sampling distribution.

14 14 / 31 Random sampling Independent distribution Suppose you record your commuting time to the university over the whole year. If knowing the commuting time at one randomly drawn day provides no information about the commuting time on another specific day: Then, the values of the commuting time are independently distributed.

15 15 / 31 Random sampling Identical distribution When Y i has the same marginal distribution for all i = 1,..., n, then Y i,..., Y n are said to be identically distributed. That means, for every draw it is equally likely to obtain a certain value of the random variable. We will mostly rely on the assumption that a random variable is i.i.d. (independently and identically distributed)

16 16 / 31 Random sampling With random sampling, each observation of the sample has the same expected value, variance and standard deviation: E(Y i ) = µ Y var(y i ) = σ 2 Y std.dev(y i ) = σ Y

17 Random sampling For the sample average Ȳ = 1 n n i=1 Y i, we get E(Ȳ) = 1 n E (Y i ) = µ Y n i=1 var(ȳ) = σ 2 Ȳ = var ( 1 n ) n Y i i=1 = 1 n var (Y n 2 i ) + 1 n 2 i=1 n n i=1 j=1,j i cov (Y i, Y j ) = σ2 Y n std.dev.(ȳ) = σ Ȳ = σ Y n Note: σ 2 Ȳ is the var. of the sampling distribution of the sample average Ȳ, while σ 2 Y is the var. of each individual Y i 17 / 31

18 18 / 31 Approximations It is crucial to characterize the sampling distribution. We discuss two tools: 1 Law of large numbers: When n is large, the sample mean will be close to the population mean. 2 Central limit theorem: When n is large, the sampling distribution of the sample average is approximately normally distributed.

19 19 / 31 Estimation Suppose you want to know the mean value of Y, that is µ Y, in a population. (Let ˆµ Y be the estimator of µ Y.) A natural way to estimate this mean is to compute the sample average Ȳ from a sample of n i.i.d. observations. Alternatively, one could use the first observation of the sample, Y 1. Both Y 1 and Ȳ are estimators of µ Y. What makes one estimator better than another?

20 20 / 31 Estimation An estimator is a function of a sample of data to be drawn randomly from a population. An estimate is the numerical value of the estimator when it is actually computed using data from a specific sample. An estimator is a random variable because of randomness in selecting the sample, while an estimate is a nonrandom number.

21 21 / 31 Estimation What are desirable characteristics of the sampling distribution of an estimator? Unbiasedness: The estimator ˆµ Y is unbiased if E(ˆµ Y ) = µ Y. Consistency: The probability that ˆµ Y is within a small interval of the true value of µ Y approaches 1 as the sample size increases. Efficiency: Among two unbiased estimators, choose the one with the lower variance.

22 22 / 31 Estimation Is Ȳ a good estimator? It is unbiased because E(Ȳ) = µ Y. It is also consistent as the law of large number ensures that Ȳ p µ Y. Y 1 is also unbiased, but has a higher variance: var(y 1 ) = σ 2 Y > var(ȳ) = σ 2 Y/n So Ȳ is more efficient than Y 1 and should therefore be preferred!

23 23 / 31 Estimation Ȳ is the least-squares estimator of µ Y. n i=1 (Y i m) 2 is a measure of the total squared distance between the estimator m and the sample point. The sample average provides the best fit to the data in the sense that n (Y i m) 2 i=1 is minimized. That is, choose m = Ȳ. (For proof see Appendix 3.2 in Stock and Watson.)

24 24 / 31 Hypothesis tests We want to find answers to questions like: Are mean earnings the same for men and women in reality? Is it true that graduate students earn 20 EUR on average? The key is that we want to find answers about the population, not about the sample (that would be trivial!) Null and alternative hypothesis (µ Y,0 = 20): H 0 : E(Y) = µ Y,0 H 1 : E(Y) µ Y,0

25 25 / 31 Hypothesis test Ȳ act : the value of the sample average actually computed in the data set at hand

26 26 / 31 Hypothesis tests Problem: The formula for the p-value depends on the variance of the population distribution which is unknown. ( ) p value = 2Φ Ȳ µ Y,0 σ Ȳ Solution: We can replace the unknown standard deviation of the population by its estimate, the standard error of Ȳ. ( ) p value = 2Φ Ȳ µ Y,0 SE(Ȳ)

27 27 / 31 Hypothesis tests How do we obtain the standard error? We need the sample variance s 2 Y = 1 n 1 n (Y i Ȳ) 2 i=1 Two modifications compared to the population variance: 1 µ Y is replaced by Ȳ because it is unknown. 2 Degrees-of-freedom-adjustment: Divide by n 1 instead of n.

28 28 / 31 Hypothesis tests How do we obtain the standard error? Because the standard deviation of the sampling distribution of Ȳ is σ Ȳ = σ Y / n, we employ s Y / n as an estimator for σ Ȳ. This estimator is called the standard error of Ȳ. So we have SE(Ȳ) = ˆσ Ȳ = s Y n

29 Hypothesis tests The t-statistic: The standardized sample average plays a key role and has a special name, the t-statistic: t = Ȳ µ Y,0 SE(Ȳ) For large n, t is approximately distributed N(0, 1). This value can be used to reject the null for given significance levels. For instance, at 5% Reject H 0 if t act > / 31

30 30 / 31 Confidence intervals Because of random sampling, it is impossible to learn the true value with certainty. However, we can construct a set of values that contains the true value with a given probability. Such a set is called confidence set or confidence interval. The prespecified probability is referred to as confidence level.

31 31 / 31 Confidence intervals A 95% two-sided confidence interval for µ Y is an interval constructed so that it contains the true value of µ Y in 95% of all possible random samples. When the sample size n is large, the 95% confidence interval for µ Y = {Ȳ ± 1.96SE(Ȳ)}. the 90% confidence interval for µ Y = {Ȳ ± 1.64SE(Ȳ)}. the 99% confidence interval for µ Y = {Ȳ ± 2.58SE(Ȳ)}.