Economics 520 Lecture Note 9: Hypothesis Testing via the Neyman-Pearson Lemma CB 8., 8.3.-8.3.3 Uniformly Most Powerful Tests and the Neyman-Pearson Lemma Let s return to the hypothesis testing problem within the Neyman-Pearson framework. Recall that we have a random variable X, with PDF/PMF f X x;θ), and we have a null and alternative hypothesis: H 0 : θ Θ 0, H a : θ Θ c 0. We need to construct a test statistic T X ) and a critical region C T, such that we reject the null hypothesis if T X ) C T. The power function of a test is defined as βθ) = Pr θ T X ) C T ) Given a prespecified significance level α for example α =.05), we require our test to satisfy, for all θ Θ 0, βθ) α. Subject to this restriction, we want βθ) for θ Θ c 0 to be as large as possible. Now we define a criterion that will measure optimality of a test. It requires that the probability of a type II error is minimized for all values of the parameter consistent with the alternative hypothesis. Definition Consider all tests of level α for the null hypothesis θ Θ 0 against the alternative θ Θ c 0. A test with power function βθ) is uniformly most powerful if, for all alternative tests with level α and power function β θ), βθ) β θ) for all θ Θ c 0. There is no guarantee that uniformly most powerful tests actually exist. We first study a simple case where such tests are easy to find. We focus on the case where both the null hypothesis and the alternative hypothesis are simple, that is, where the sets Θ 0 and Θ c 0 contain a single element each: H 0 : θ = θ 0, H a : θ = θ.
If a hypothesis contains more than a single point, we say that it is a composite hypothesis.) Result Neyman Pearson lemma) Consider testing the null hypothesis H 0 : θ = θ 0 against the alternative H a : θ = θ using a critical region of the form Let C X = x : f X x;θ ) k f X x;θ 0 ). α = f X x;θ 0 )dx. C X This test is the uniformly most powerful test of level α. Proof: Let βθ) denote the power function of the test proposed. Consider any other test with a critical region C X and power function β θ). Define φx) = x C X, and Consider φ x) = x C X. φx) φ x)) f X x;θ ) k f X x;θ 0 )). If this expression differs from zero, we must either have φx) φ x) = or φx) φ x) =. If φx) φ x) =, f X x;θ ) k f X x;θ 0 )) must be nonnegative by the form of the critical region C X, so the entire expression is nonnegative. If φx) φ x) =, the second factor must be 0, and the product again is nonnegative. Hence, and therefore x φx) φ x)) f X x;θ ) k f X x;θ 0 )) 0, φx) φ x)) f X x;θ ) k f X x;θ 0 ))dx x φx) φ x)) f X x;θ )dx k φx) φ x)) f X x;θ 0 )dx = βθ ) β θ ) k βθ 0 ) β θ 0 )) 0. x If both tests are level α tests, βθ 0 ) = β θ 0 ) = α, and so it must be the case that βθ ) β θ ) 0, and the second test cannot be the most powerful test. 2
Example Let us consider some examples of applications of the Neyman-Pearson Lemma. Suppose X has an exponential distribution with arrival rate λ. We wish to test the hypothesis that λ = against the alternative that λ = 2: H 0 : λ = ; H a : λ = 2. By the Neyman-Pearson lemma, we should use a critical region of the form C X = x : f X x;2) k f X x;) = x : 2 exp 2x) k exp x) = x : exp 2x) expk )exp x) = x : 2x k x = x : x k. All that is left to determine is k. Suppose we wish to test at the 0.05 level. Then we choose k to satisfy 0.05 = Pr X k H 0 ) = k 0 exp x)d x = exp k ), or and the critical region is k = ln0.95) 0.053, C X = [0, ln0.95)]. Example 2 Suppose X,..., X n are iid normal with mean µ and unit variance. We wish to test the null hypothesis µ = µ 0 against the alternative hypothesis that µ = µ, for some µ and µ 0 with µ > µ 0 : H 0 : µ = µ 0 ; H a : µ = µ. 3
By Neyman-Pearson, we want the test to reject the null if f x,..., x n ;µ ) k f x,..., x n ;µ 0 ) or equivalently: This ratio of likelihood functions is L µ ) L µ 0 ) f x,..., x n ;µ ) f x,..., x n ;µ 0 ) k. = exp 2 i x i µ ) 2) exp 2 i x i µ 0 ) 2) = exp 2 i [x 2 i 2x i µ + µ 2 ]) exp 2 i [x 2 i 2x i µ 0 + µ 2 0 ]) = exp µ µ 0 ) i x i ) C, where C is a constant which does not depend on x. Since µ µ 0 > 0, this ratio is larger than k if and only if or equivalently, x n The critical region is therefore of the form x i k, i x i k. i C X = x,..., x n ) : x k. Suppose we wish to test at the 0.05 level. Then 0.05 = Pr x k µ = µ 0 ). Under the null the distribution of x is normal with mean µ 0 and variance /n: x N µ 0, n ), so x µ 0 /n N 0,). Using a table for the standard normal distribution, we can determine that Pr ) x µ0.645 = 0.05. /n 4
So and Pr Pr x µ 0.645 n ) = 0.05, x µ 0 +.645 n ) = 0.05. Hence the critical region should be C X = x,..., x n ) : x > µ 0 +.645/ n. Example 2 also illustrates an important phenomenon. There, the critical region does not depend on the value of the parameter under the alternative hypothesis, µ. Whether the alternative is µ = µ 0 + or µ = µ 0 + 4 leads to exactly the same critical region. Thus, we can use the same test if we are testing the composite alternative hypothesis H a : µ > µ 0. Moreover, since the test is most powerful for each specific point in the alternative, the test is uniformly most powerful against the composite alternative. Uniformly most powerful tests do not always exist. They exist for some special models like the normal model, when the alternative is one-sided i.e. H a : µ > µ 0 or H a : µ < µ 0 ). What if we consider the same normal model, and test H 0 : µ = µ 0, against the two-sided alternative H : µ µ 0. If the alternative is µ = µ > µ 0 the critical region for the most powerful test is of the form C X = x,..., x n ) : x k. If the alternative is µ = µ < µ 0 the critical region of the most powerful test is of the form C X = x,..., x n ) : x k. There is therefore no test that is most powerful for all values under the alternative. In other words, there is no uniformly most powerful test. One way to get around this problem, is to impose some additional restrictions on the test, and look for uniformly most powerful tests within the restricted set of tests. A test is unbiased if the power function βθ ) βθ 0 ) for all θ Θ c 0 and all θ 0 Θ 0. That is, the probability of rejecting the null hypothesis, or of an observation in the critical region, is at least 5
as large for values of the parameters consistent with the alternative θ Θ c 0 ) as for values of the parameters consistent with the null hypothesis θ Θ 0 ). Let us consider this approach in detail for the case with a normal distribution with unknown mean and known variance. Let X,..., X n be independent and normally distributed with unknown mean µ and known variance σ 2. We are interested in testing the null hypothesis H 0 : µ = µ 0, against the alternative H : µ µ 0. Let us consider the ratio of density functions to determine the critical region: f x,..., x n µ ) f x,..., x n µ 0 ) = 2πσ2)n/2 exp n 2σ 2 i= x2 i 2µ n i= x i + nµ 2 )) 2πσ 2 ) n/2 exp n 2σ 2 i= x2 i 2µ n 0 i= x i + nµ 2 )) 0 = exp σ 2 µ µ 0 ) ) x i exp µ 2 µ2 0 )n/2σ2 )). Hence if we are looking for a uniformly most powerful test against the alternative hypothesis H : µ > µ 0, the critical region ought to be of the form C X = x,..., x n ) : x k. If we were to test against the alternative hypothesis H : µ < µ 0, the critical region ought to be of the form C X = x,..., x n ) : x k. It therefore appears sensible to base a test on the value of x, the sample average, which is a sufficient statistic for µ. It seems fairly clear that the critical region should be of the form C X = x,..., x n ) : x a or x b. Unbiasedness of the test implies that b βµ) = a 2πσ 2 /n exp ) 2σ 2 x µ)2 d x, /n is maximized at µ 0. The function is maximized at µ = a +b)/2, so that for unbiasedness we must have b µ 0 = µ 0 a. Hence the critical region is C X = x,..., x n ) : x µ 0 c or x µ 0 + c, 6
with the value of c determined by the size of the test. Under the null hypothesis the distribution of x is normal with mean µ 0 and variance σ 2 /n. Hence, if we wish to test at the 0% level, recalling that for a standard normal random variable Z Pr.645 < Z <.645) = 0.90, the critical region is C X = x,..., x n ) : x µ 0.645 σ/ n, x µ 0 +.645 σ/ n. This is the uniformly most powerful unbiased test. If we wish to test at the 5% level, the critical region is C X = x,..., x n ) : x µ 0.96 σ/ n, x µ 0 +.96 σ/ n. Equivalently we can use the critical region C X = x,..., x n ) : n x µ 0 ) 2 /σ 2 3.84, which uses the Chi squared distribution for the square of a standard normal random variable. In fact a common way of doing the test is to calculate the test statistic, here n x µ 0 ) 2 /σ 2 which under the null hypothesis has a known distribution, in this case a χ 2 ) distribution. We reject the null hypothesis if the test statistic exceeds the critical value, in this case 3.84 at the 5% level or 2.706 at the 0% level. 7