14.30 Introduction to Statistical Methods in Economics Spring 2009

Size: px

Start display at page:

Download "14.30 Introduction to Statistical Methods in Economics Spring 2009"

Posy Simpson
5 years ago
Views:

1 MIT OpenCourseWare Introduction to Statistical Methods in Economics Spring 009 For information about citing these materials or our Terms of Use, visit:

2 .30 Introduction to Statistical Methods in Economics Lecture Notes Konrad Menzel May 5, 009 Constructing Hypothesis Tests If X i has support S X, then the sample X = (X,...,X n ) has support S n The critical region of a test is a region C X S n X. X of the support of the sample for which we reject the null. The following example illustrates most of the important issues in a standard setting, so you should loo at this carefully and mae sure that you now how to apply the same steps to similar problems. Example Suppose X,...,X n are i.i.d. with X i N(µ, ), and we are interested in testing H 0 : µ = 0 against H A : µ =. Let s first loo at the case n = : X Reject : C x μ A Don't reject μ o X We could design a test which rejects for values of X +X which are too large to be compatible with µ = 0. We can also represent this rejection region on a line: This representation is much easier to use if n is large, so it s hard to visualize the rejection region in terms of X,...,X n directly. However, by condensing the picture from n to a single dimension we may loose the ability of specifying really odd-shaped critical regions, but typically those won t be interesting for practical purposes anyway. So in this example, we will base our testing procedure on a test statistic T n (X,...,X n ) = X n and reject for large values of T n. How do we choose? - we ll have to trade off the two types of error. Suppose now that n = 5, and since X i N(µ, ), { ( ) N 0, Tn := X n ( 5 ) under H 0 N, under H 5 A

3 X + X 0 Don't reject C x : reject X n 0 Don't reject C x : reject Now we can calculate the probabilities of type I and type II error ( ) ( ) 0 = P(X > µ = 0) = Φ = Φ /5 /5 ( ) β = P(X µ = ) = Φ /5 Therefore, fixing any one of, β, determines the other two, and that choice involves a specific tradeoff between the probability of type I and type II error - if we increase, the significance level goes down, 3 but so does power β. Specifically, if we choose = 5, 6.7%, and β 5.87%. For different sample sizes, we can graph the trade-off between the probability of type I and type II error through the choice of as follows: A low value of would give high power, but also a high significance level, so that increasing would β n = n = 5 n = 00 move us to the left along the frontier. How should we choose? Recall that in the usual setting, the first priority is to control the probability of false rejections, so we ll choose to eep the probability of a type I error at an acceptably low level, usually 5% or %. Of course, as n, for fixed, the power of the test, β goes to. As a convention, we usually say that a rejection at the significance level = 5% is significant, whereas a rejection at = % is highly significant.

4 Example In the previous example, the maintained hypothesis was that µ {0, }, but this is a highly artificial assumption, and usually we have no reason to believe that this is the case. Suppose that as before X,...,X 5 is an i.i.d. sample with X i N(µ, ), but now we want to test H 0 : µ = 0 against H A : µ 0 Now H A is a two-sided composite hypothesis (i.e. under the alternative µ could tae several values, some on the left, some on the right of µ 0 ). Also we ll again loo at a test that s only based on the sample mean, X 5 - what should the critical region now loo lie? Intuitively, it maes sense to reject H 0 for both large and small values of X, i.e. we are unliely to see values in either tail if the null hypothesis is true, and the alternative hypothesis states that we are interested in evidence that µ is either greater or smaller than 0. f(μ μ o ) = f(x n μ o ) μ o Therefore, we are going to choose two values and such that [ ( )] ( ) = P(X 5 > µ = 0) + P(X 5 < µ = 0) = Φ + Φ /5 /5 C x What is β? Since the alternative does not specify a single probability law, but rather a continuum of them, β is not well-defined, i.e. for fixed µ, [ ( )] ( ) µ µ β(µ) = P(X 5 > µ) + P(X 5 < µ) = Φ + Φ /5 /5 Usually for a desired significance level, we choose, symmetrically about the value postulated by the null hypothesis (note that since the normal distribution is single-peaed and symmetric, this maes the critical region as large as possible. The last example should remind you of the way we constructed confidence intervals for µ from a normal population with nown variance: the above procedure is in fact identical to the following:. construct a confidence interval [A(X), B(X)] for µ (case, see notes for last class). reject H 0 if µ 0 = 0 / [A(X), B(X)] Since we construct the interval [A(X), B(X)] in such a way that P θ (A(X) < θ < B(X)) = implicitly under the null assumptions, so that P(θ 0 / [A(X), B(X)] H 0 : θ = θ 0 ) = 3

5 f(x (μ ) β(μ ) f(x n (μ ) -β(μ) β(μ ) μ 0 μ 0 μ Evaluation and Construction of Tests As in our discussion of estimation, we first introduced the general idea and saw a few examples. Now we will see how to choose among tests and how to construct them from scratch.. Properties of Tests For any test we care about its level of significance = P(type I) and its power β = P(type II). If H 0 and H A are both simple hypotheses, and β are well-defined for given, and we can simply choose the test with the highest β, the most powerful test. If H A is composite, and H 0 is simple, we need a metric for comparing power functions β(θ) = P(type II θ) for a given size. A test is uniformly most powerful (UMP) when it is at least as powerful at every θ H A as any other test of the same size. In general, a UMP test need not exist. Example 3 Sometimes it is possible to find a uniformly most powerful test: Suppose X i N(µ, ), and we are interested in testing H 0 : µ = 0 against H A : µ > 0 Recall that the most powerful test for H 0 : µ = 0 against H A : µ = too the form of reject if X >, the general form of the test does not change no matter what µ A as long as µ A > µ 0. Therefore a most powerful test for H 0 : µ = 0 against H A : µ > 0 will also tae the form reject if X n >. For the following important result, denote f 0 (x) = f 0 (x,...,x n ) the joint p.d.f. of the sample X,...,X n under the simple null hypothesis H 0 : µ = µ 0, and f A (x) the joint p.d.f. of the sample under H A : µ = µ A. Proposition (Neyman-Pearson Lemma) In testing f 0 against f A (where both H 0 and H A are simple hypotheses), the critical region { } f 0 (x) C() = x : < f A (x) is most powerful for any choice of 0. Note that the choice of depends on the specified significance level of the test. This means that the most powerful test rejects if for the sample X,...,X n, the lielihood ratio f 0 (X,...,X n ) r(x,...,x n ) = f A (X,...,X n )

6 is low, i.e. the data is much more liely to have been generated under H A. F o (x) F A (x) = r(x) Reject μ o μ A The most powerful test given in the Neyman-Pearson Lemma explicitly solves the trade-off between size = P(reject H 0 ) = f 0 (x)dx C() and power C x () β = P(reject H A ) = f A (x)dx C() at every point x in the sample space (where the integrals are over many dimensions, e.g. typically f 0 (x) x R n ). From the expressions for and β we can see that the lielihood ratio f A (x) gives the price of including x with the critical region in terms of how much we pay in terms of size relative to the gain in power from including the point in the critical region C X. Therefore, we should start constructing the critical region by including the cheapest points x - i.e. those with a small lielihood ratio. Then we can go down the list of x ordered according to the lielihood ratio and continue including more points until the size of the test is down to the desired level. Example A criminal defendant (D) is on trial for a purse snatching. In order to convict, the jury must believe that there is a 95% chance that the charge is true. There are three potential pieces of evidence the prosecutor may or may not have been able to produce, and in a given case the jury taes a decision to convict based only on which out of the three clues it is presented with. Below are the potential pieces of evidence, assumed to be mutually independent, the probability of observing each piece given the defendant is guilty, and the probability of observing each piece given the defendant is not guilty guilty not guilty lielihood ratio. D ran when he saw police coming /. D has no alibi /3 3. Empty purse found near D s home / 5

7 In the notation of the Neyman-Pearson Lemma, x can be any of the 3 possible combinations of pieces of evidence. Using the assumption of independence, we can therefore list all possible combinations of clues with their respective lielihood under each hypothesis and the lielihood ratio. I already ordered the list by the lielihood ratios in the third column. In the last column, I added () = f 0 (x) r(x) the cumulative sum over the ordered list of combinations x. guilty f A (x) not guilty f 0 (x) lielihood ratio r(x) = f A (x) (). all three clues 6/000 9/ /000. no alibi,found purse /000 / / ran,no alibi 3/000 8/ /000. no alibi 6/000 89/ / ran,found purse /000 / / found purse 6/000 9/ / ran 36/000 89/ / none of the clues /000 / The jury convicting the defendant only if there is at least 95% confidence that the charge is true corresponds to a probability of false conviction (i.e. if the defendant is in fact innocent) of less than 5%. In the terminology of hypothesis test, the sentence corresponds to a rejection of the null hypothesis that the defendant is innocent using the most powerful test of size = 5%. Looing at the values of () in the last column of the table, we can read off that including more than the first two combinations of the evidence raises the probability of a false conviction to more than 5%. Therefore, the jury should convict the defendant if he doesn t have an alibi and the empty purse was found near his home, regardless whether he ran when he saw the police. In principle, the jury could in addition randomize when the defendant ran, had no alibi, but no purse was found (that is case 3): if in that case, the jury convicted the defendant with probability 8, the probability of a false conviction would be exactly equal to 5%, but this would probably not be considered an acceptable practice in criminal justice. Example 5 We can now show that a test based on the mean is in fact most powerful in the normal case. Suppose X i N(µ, ), and we test H 0 : µ = 0 against H A : µ =, where we observe an i.i.d. sample X = (X,...,X 5 ) of 5 observations. Since the observations are i.i.d. normal, the lielihood ratio evaluated at the observed sample is given by f(x µ = 0) e π r(x) = = (X i ) f(x µ = ) 5 i= π e 5 = e 8 [(X i X i +) (X i )] i= 5 P 5 5 = e 8 i= X i = e 8 00 X 5 (X i 0) We can see that r(x) depends on the sample only through the sample mean X5 and is strictly decreasing in X5. Therefore, the critical region of a most powerful test taes the form C X () = {x : r(x) } f 0 (x) 6

Hypothesis Testing. Testing Hypotheses MIT Dr. Kempthorne. Spring MIT Testing Hypotheses

Hypothesis Testing. Testing Hypotheses MIT Dr. Kempthorne. Spring MIT Testing Hypotheses Testing Hypotheses MIT 18.443 Dr. Kempthorne Spring 2015 1 Outline Hypothesis Testing 1 Hypothesis Testing 2 Hypothesis Testing: Statistical Decision Problem Two coins: Coin 0 and Coin 1 P(Head Coin 0)