5. Likelihood Ratio Tests

Size: px

Start display at page:

Download "5. Likelihood Ratio Tests"

Edgar Scott
6 years ago
Views:

1 1 of 5 7/29/2009 3:16 PM Virtual Laboratories > 9. Hy pothesis Testig > Likelihood Ratio Tests Prelimiaries As usual, our startig poit is a radom experimet with a uderlyig sample space, ad a probability measure P. I the basic statistical model, we have a observable radom variable X takig values i a set S. I geeral, X ca have quite a complicated structure. For example, if the experimet is to sample objects from a populatio ad record various measuremets of iterest, the X = (X 1, X 2,..., X ) where X i is the vector of measuremets for the i th object. The most importat special case occurs whe (X 1, X 2,..., X ) are idepedet ad idetically distributed. I this case, we have a radom sample of size from the commo distributio. I the previous sectios, we developed tests for parameters based o atural test statistics. However, i other cases, the tests may ot be parametric, or there may ot be a obvious statistic to start with. Thus, we eed a more geeral method for costructig test statistics. M oreover, we do ot yet kow if the tests costructed so far are the best, i the sese of maximizig the power for the set of alteratives. I this ad the ext sectio, we ivestigate both of these ideas. Likelihood fuctios, similar to those used i maximum likelihood estimatio, will play a key role. Tests of Simple Hypotheses Suppose that X has oe of two possible distributios. Our simple hypotheses are H 0 : X has probability desity fuctio f 0 versus H 1 : X has probability desity fuctio f 1 We will use subscripts o the probability measure P to idicate the two hypotheses. The test that we will costruct is based o the followig simple idea: if we observe X = x, the the coditio f 1 (x) > f 0 (x) is evidece i favor of the alterative; the opposite iequality is evidece agaist the alterative. Thus, let L(x) = f 0(x) f 1 (x), x S The fuctio L is the likelihood ratio fuctio for the hypotheses ad L(X) is the likelihood ratio statistic. Restatig our earlier observatio, ote that small values of L are evidece i favor of H 1. Thus it seems reasoable that the likelihood ratio statistic may be a good test statistic, ad that we should cosider tests of the followig form, where k is a costat:

2 2 of 5 7/29/2009 3:16 PM Reject H 0 if ad oly if L k 1. Show that the sigificace level of the test is α = P 0 (L k) As usual, we ca try to costruct a test by choosig k so that α is a prescribed value. If X has a discrete distributio, this will oly be possible whe α is a value of the distributio fuctio of L(X). A importat special case of this model occurs whe the distributio of X depeds o a parameter θ that has two possible values. Thus, the parameter space is Θ = {θ 0, θ 1 }, ad f 0 deotes the probability desity fuctio of X whe θ = θ 0 ad f 1 deotes the probability desity fuctio of X whe θ = θ 1. I this case, the hypotheses are equivalet to H 0 : θ = θ 0 versus H 1 : θ = θ 1 The Neyma-Pearso Lemma The followig exercises establish the Neyma-Pearso Lemma, amed for Jerzy Neyma ad Ego Pearso. This result shows that the test give above is most powerful. Let 2. Use the defiitios of L ad R to show that R = {x S : L(x) k} P 0 (X A) k P 1 (X A) for A R P 0 (X A) k P 1 (X A) for A R c 3. Show that if A S the P 1 (X R) P 1 (X A) 1 k (P 0(X R) P 0 (X A)) Hit: Write R = (R A) (R A) ad A = (A R) (A R). Use the additivity of probability ad the results i Exercise Cosider the tests with rejectio regios R ad A. Recall that the size of a rejectio regio is the sigificace of the test with that rejectio regio. Use Exercise 3 to show that if the size of R is at least as large as the size of A the the test with rejectio regio R is more powerful tha the test with rejectio regio A: P 0 (X R) P 0 (X A) P 1 (X R) P 1 (X A) The Neyma-Pearso lemma is a beautiful result, ad is more useful tha might be first apparet. I may importat cases, the same most powerful test works for a rage of alteratives, ad thus is a uiformly most powerful test for this rage. I the followig subsectios, we will cosider some of these special cases.

3 3 of 5 7/29/2009 3:16 PM Tests for the Expoetial Model Suppose that X = (X 1, X 2,..., X ) is a radom sample from the expoetial distributio with scale parameter The sample variables might represet the lifetimes from a sample of devices of a certai type. We are iterested i testig the simple hypotheses H 0 : b = b 0 versus H 1 : b = b 1, where b 0 > 0 ad b 1 > 0 are distict specified values. Recall that the sum of the variables is a sufficiet statistic for b: Y = i =1 Recall also that Y has the gamma distributio with shape parameter ad scale parameter For α ( 0, 1), we will deote the quatile of order α for the this distributio by γ, b(α). 5. Show that the likelihood ratio statistic is L = ( b 1 b 0 ) X i exp (( 1 b 1 1 b 0 ) Y ) 6. Show that the followig tests are most powerful test at the α level Suppose that b 1 > b 0. Reject H 0 : b = b 0 versus H 1 : b = b 1 if ad oly if Y γ, b0 (1 α). Suppose that b 1 < b 0. Reject H 0 : b = b 0 versus H 1 : b = b 1 if ad oly if Y γ, b0 (α). Note that the tests i Exercise 6 do ot deped o the value of b 1. This fact, together with the mootoicity of the power fuctio ca be used to shows that the tests are uiformly most powerful for the usual oe-sided tests. 7. Show that The test i Exercise 6 (a) is uiformly most powerful for the hypotheses H 0 : b b 0 versus H 1 : b > b 0 The test i Exercise 6 (b) is uiformly most powerful for the hypotheses H 0 : b b 0 versus H 1 : b < b 0 Tests for the Beroulli Model Suppose that X = (X 1, X 2,..., X ) is a radom sample of size from the Beroulli distributio with success parameter p. The sample could represet the results of tossig a coi times, where p is the probability of heads. We wish to test the simple hypotheses H 0 : p = p 0 versus H 1 : p = p 1, where p 0 ( 0, 1) ad p 1 ( 0, 1) are distict specified values. I the coi tossig model, we kow that the probability of heads is

4 4 of 5 7/29/2009 3:16 PM either p 0 or p 1, but we do't kow which. Recall that the umber of successes is a sufficiet statistic for p: Y = i =1 Recall also that Y has the biomial distributio with parameters ad p. For α ( 0, 1), we will deote the quatile of order α for the this distributio by b, p( α); although sice the distributio is discrete, oly certai values of α are possible. 8. Show that the likelihood ratio statistic is L = ( 1 p 0 1 p 1 ) X i Y p 0 (1 p 1 ) ( p 1 (1 p 0 )) 9. Show that the followig tests are most powerful test at the α level Suppose that p 1 > p 0. Reject H 0 : p = p 0 versus H 1 : p = p 1 if ad oly if Y b, p0 (1 α). Suppose that p 1 < p 0. Reject H 0 : p = p 0 versus H 1 : p = p 1 if ad oly if Y b, p0 (α). Note that the tests i Exercise 9 do ot deped o the value of p 1. This fact, together with the mootoicity of the power fuctio ca be used to shows that the tests are uiformly most powerful for the usual oe-sided tests. 10. Show that The test i Exercise 9 (a) is uiformly most powerful for the hypotheses H 0 : p p 0 versus H 1 : p > p 0 The test i Exercise 9 (b) is uiformly most powerful for the hypotheses H 0 : p p 0 versus H 1 : p < p 0 Uiformly Most Powerful Tests The oe-sided tests that we derived i the ormal model, for μ with σ kow, for μ with σ ukow, ad for σ with μ ukow are all uiformly most powerful. O the other had, oe of the two-sided tests are uiformly most powerful. A Noparametric Example Suppose that X = (X 1, X 2,..., X ) is a radom sample, either from the Poisso distributio with parameter 1 or from the geometric distributio o N with parameter 1. Thus, we wish to test the hypotheses 2

5 5 of 5 7/29/2009 3:16 PM H 0 : X has probability desity fuctio f 0 (x) = e 1 1 x!, x N H 1 : X has probability desity fuctio f 1 (x) = ( Show that the likelihood ratio statistic is 2) x+1, x N L = 2 2Y e U where Y = i =1 X i ad U = i =1 X i! 12. Show that the most powerful tests have the followig form, where d is a costat: reject H 0 if ad oly if l(2) Y l(u) d Geeralized Likelihood Ratio The likelihood ratio statistic ca be geeralized to composite hypotheses. Suppose agai that the probability desity fuctio f θ of the data variable X depeds o a parameter θ, takig values i a parameter space Θ. Cosider the hypotheses H 0 : θ Θ 0 versus H 1 : θ Θ 0, where Θ 0 Θ. We defie L(x) = max { f θ(x) : θ Θ 0 } max { f θ (x) : θ Θ} The fuctio L is the likelihood ratio fuctio ad L(X) is the likelihood ratio statistic. By the same reasoig as before, small values of L(x) are evidece i favor of the alterative hypothesis. Virtual Laboratories > 9. Hy pothesis Testig > Cotets Applets Data Sets Biographies Exteral Resources Key words Feedback

6. Sufficient, Complete, and Ancillary Statistics

6. Sufficient, Complete, and Ancillary Statistics Sufficiet, Complete ad Acillary Statistics http://www.math.uah.edu/stat/poit/sufficiet.xhtml 1 of 7 7/16/2009 6:13 AM Virtual Laboratories > 7. Poit Estimatio > 1 2 3 4 5 6 6. Sufficiet, Complete, ad Acillary