Chapter 8 Hypothesis Testing

Transcription

1 Leture 5 for BST 63: Statistial Theory II Kui Zhang, Spring Chapter 8 Hypothesis Testing Setion 8 Introdution Definition 8 A hypothesis is a statement about a population parameter Definition 8 The two omplementary hypotheses in a hypothesis testing problem are alled the null hypothesis and the alternative hypothesis They are denoted by H and H, respetively Setting: Let be a parameter of interest with : where and H versus H : : Definition 83 A hypothesis testing proedure or hypothesis test is a rule that speifies: For whih sample values the deision is made to aept H as true For whih sample values H is rejeted and H is aepted as true Rejetion Region or Critial Region: subset of the sample spae for whih H will be rejeted

2 Leture 5 for BST 63: Statistial Theory II Kui Zhang, Spring Aeptane Region: omplement of the rejetion region Philosophial Notes: Rejet H and aepting H Aepting H and not rejeting H Our Main Conern: assertion of H or H Test Statisti W ( X ): funtion of the sample; used together with the rejetion region to make deisions about the hypotheses being tested Setion 8 Methods of Finding Tests 8 Likelihood Ratio Tests Definition 8 The likelihood ratio test statisti for testing H: versus H: is sup L( x) ( x) sup L( x)

3 Leture 5 for BST 63: Statistial Theory II Kui Zhang, Spring A likelihood ratio test (LRT) is any test that has a rejetion region of the form { x: ( x ) }, where is any number satisfying LRT and MLE: Let ˆ be the MLE of under the unrestrited parameter spae and ˆ be the MLE of under the restrited parameter spae Then ( x) L( ˆ x) L( ˆ x ) Example 8 (Normal LRT) Let X,, Xn be iid from n(,) We want to test H: versus H: where is a fixed number set by the experimenter Show that ( x ) exp[ nx ( ) /] so that the LRT rejets H for small values of ( x ) Therefore the rejetion region is whih is equivalent to { x: ( x ) }, { x : x (log )/ n} Example 83 (Exponential LRT) Let X,, Xn be iid from an exponential population with pdf 3

4 Leture 5 for BST 63: Statistial Theory II Kui Zhang, Spring where f ( x ) exp[ ( x )] I[, ) ( x), We want to test H: versus H:, where is a fixed number set by the experimenter Show that, x() ; ( x ) exp[ nx ( () )], x() so that the LRT rejets H for small values of ( x ) Therefore the rejetion region is log( ) { x: ( x ) } whih is equivalent to { x : x() } n Note: In both of the above examples, the rejetion region only depends on the suffiient statisti for Theorem 84 If T ( X) is a suffiient statisti for and *( t) and ( x ) are the LRT statistis based on T and X, respetively, then *( T ( x)) ( x ) for every x in the sample spae Example 85 (LRT and Suffiieny): In Example 8, we ould have used the likelihood assoiated with the suffiient statisti X using the fat that X ~ n(,/ n) whih rejets for large values of X 4

5 Leture 5 for BST 63: Statistial Theory II Kui Zhang, Spring Similarly, in Example 83, we an use the likelihood assoiated with the suffiient statisti X () : L( x() ) nexp[ n( x() )] I[, ] ( x() ), whih rejets for large values of X () Example 86 (Normal LRT with unknown variane) Let X,, Xn be iid from n (, ) and an experimenter is interested only in testing We want to test H: versus H: The LRT statisti where ˆ n ( xi ) / n i max L(, x) max L(, x) ( x) max (, ) (, ) {, } {, } L x L ˆ ˆ x {, }, ˆ ; L( ˆ ˆ ˆ, x)/ L(, x), 8 Bayesian Tests Bayesian Formulation of Hypothesis Testing Classial Approah: The parameter is fixed If is known, then P( x ) and P( o x ) for all x 5

6 Leture 5 for BST 63: Statistial Theory II Kui Zhang, Spring If is known, then P( x ) and P( o x ) for all x In pratie, P( x ) and P( x ) are unknown and do not depend on x Hene these probabilities are not used Bayesian Approah: The parameter is random and is assigned a prior distribution The P( x) P( H is true x ) and P( x) P( H is true x) may be omputed and make sense A way to use the posterior distribution to make deisions about H and H is to deide to aept H as true ( X) ( X ) and rejet H otherwise, ie, the rejetion region is { x: P( x ) /} if P P One may also define a rejetion region as { x: P( x ) p } where, say 99, whih is set by the researher p p Example 87 (Normal Bayesian Test) Let X,, Xn be iid from n (, ) and let the prior distribution on be n (, ), where,, are known We want to test H: versus H: Reall that the posterior distribution ( x) is normal with 6

7 Leture 5 for BST 63: Statistial Theory II Kui Zhang, Spring mean = n x and variane = n n Deision rule: Aept H if and only if P( X) P( X ) or P( X) P( X ) / and aept Hotherwise Note that ( x) is symmetri, hene, an equivalent deision rule is to aept H when and aept H otherwise X ( ) n Setion 83 Methods of Evaluating Tests 83 Error Probabilities and the Power Funtion Two types of Error: Type I error: If but the hypothesis test inorretly deides to rejet H Type II error: If but the hypothesis test deides to aept H 7

8 Leture 5 for BST 63: Statistial Theory II Kui Zhang, Spring Let R denote the rejetion region for a test Then P(Type I Error) if ; P ( X R) P(Type II Error) if Definition 83 The power funtion of a hypothesis test with rejetion region R is the funtion of ( ) P ( X R) Example 83 (Binomial power funtion) Let X ~ binomial(5, ) Consider: Test : R {all suesses are observed} Test : R { X 3,4,or 5} H : / versus H : / Example 833 (Normal power funtion) Let X,, Xn be iid from The LRT for this test has rejetion region defined by H: versus H: n(, ) where is known Consider X R / n 8

9 Leture 5 for BST 63: Statistial Theory II Kui Zhang, Spring Therefore X ( ) P( ) P( Z ) / n / n Example 834 (ontinuation of Example 833) Suppose that the experimenter would like the maximum probability of a Type I error of and a minimum probability of a Type II error of if How do we hoose and n? Definition 835 For, a test with power funtion ( ) is a size test if sup ( ) Definition 836 For, a test with power funtion ( ) is a level test if sup ( ) Notes: Some authors use the terms level and size interhangeably The set of level tests ontains the set of size tests In more omplex problems (eg, intersetion-union and union-intersetion tests), it is diffiult if not impossible to obtain a size test so one will just have to settle for a level test The ommonly used values in pratie are, 5, and 9

10 Leture 5 for BST 63: Statistial Theory II Kui Zhang, Spring Fixing the level of a test is ontrolling the Type I error but not the Type II error H and H should be set up properly so that the more important error to ontrol is the Type I error H typially is the hypothesis that you want your data to support and hene is alled the researh hypothesis Example 837 (Size of LRT) A size LRT is onstruted by hoosing the appropriate suh that sup P( ( X) ) In Example 8, H: versus H: so that z / R{ X } n In Example 83, H: versus H: so that P ( X ) exp( n( )) if ( log( )) / n () We need to show that this is a size test Note that for any P ( X ) P ( X ) () () Therefore, sup sup P( X ) P ( X ( log( )) / n ) () ()

11 Leture 5 for BST 63: Statistial Theory II Kui Zhang, Spring Definition 839 A test with power funtion ( ) is unbiased if ( ') ( '') for every ' and '' Example 83 (Conlusion of Example 833) Let X,, Xn be iid from for testing H: versus H: has power funtion given by X / n / n ( ) P( ) P( Z ) n(, ) where whih is an inreasing funtion of for a fixed value of Therefore, this test is unbiased sine ( ) ( ) sup ( ) t t for all is known The LRT 83 Most Powerful Tests Definition 83 Let be a lass of tests for testing H: versus H: A test in lass, with power funtion ( ), is uniformly most powerful (UMP) lass test if ( ) '( ) for every and every '( ) that is a power funtion of a test in lass Theorem 83 (Neyman-Pearson Lemma) Consider testing H: versus H:, where the pdf or pmf orresponding to i is f( x i), i,, then any test with rejetion region R is a UMP test if it satisfies

12 Leture 5 for BST 63: Statistial Theory II Kui Zhang, Spring and for some k, and f ( x ) x R if f( x ) kf( x ) or, equivalently, k f ( x ) f ( x ) x R if f( x ) kf( x ) or, equivalently, f ( x ) P ( R) X k Corollary 833 Consider the hypothesis problem posed in Theorem 83 Suppose T ( X ) is a suffiient statisti for and gt ( i ) is the pdf or pmf of T orresponding to,,, then any test based on T with rejetion region S (subset of the sample spae T ) is a UMP level test if it satisfies gt ( ) t S if gt ( ) kgt ( ) or, equivalently, k gt ( ) and for some k, and P ( T S) t S if i i gt ( ) gt ( ) kgt ( ) or, equivalently, k gt ( ) Example 834 (UMP Binomial Test) Let X ~ binomial(, ) We want to test H : / versus H : 3/4 Then

13 Leture 5 for BST 63: Statistial Theory II Kui Zhang, Spring f ( 3 / 4), f ( / ) 4 f ( 3 / 4) 3, and f ( / ) 4 f ( 3 / 4) 9 f ( / ) 4 Test : If we hoose 3 k 9, then by Neyman-Pearson Lemma, the UMP test rejets Hwhen X The 4 4 orresponding size of this test is PX ( ) 4 Test : If we hoose k 3, then by Neyman-Pearson Lemma, the UMP test rejets H when X = or The 4 4 orresponding size of this test is 3 PX ( or ) 4 Notes: (on Example 834) If we hoose k or 4 9 k, the UMP test has size 4 and, respetively Notes: (on Example 834 ontinued) 3

14 Leture 5 for BST 63: Statistial Theory II Kui Zhang, Spring 3 If k then the UMP test has rejetion region X and aeptane region X but does not tell us 4 what to do when X In this ase, we hoose to have X in either the rejetion or aeptane region The resulting size of the test will depend on this hoie beause this is in a disrete setting Example 835 (UMP Normal Test) Let X,, Xn be iid from n(, ) where test for H: versus H: where, we reall that X is a suffiient statisti for and X ~ n(, / n) Note that n ( / n) exp[ ( x ) ] / ( ) n exp[ { x( ) ( )}] ( ) / n ( / n) exp[ ( x ) ] gx gx Hene, by Corollary 833, the UMP test rejets when Equivalently, we rejet when Notes: The inequality was reversed beause gx ( ) n exp[ { x( ) ( )}] k gx ( ) (log( k))( / n) ( ) x ( ) is known To obtain the UMP 4

15 Leture 5 for BST 63: Statistial Theory II Kui Zhang, Spring As k inreases from to, the right hand size of the rejetion region inequality goes from - to To omplete the UMP test proedure of size, we find the onstant suh that P ( X ) P( Z ) / n In this ase, z so that / n z n Types of Hypotheses: Simple Hypothesis: H : Composite Hypothesis: more than one possible distribution a One-sided Hypothesis: H : b Two-sided Hypothesis: H : Question: How do we find the UMP test for omposite hypotheses? 5

16 Leture 5 for BST 63: Statistial Theory II Kui Zhang, Spring Definition 836 A family of pdfs and pmfs { gt ( ) : } for a univariate random variable T with real-valued parameter has a monotone likelihood ratio (MLR) if, for every, gt ( )/ gt ( ) is a monotone (noninreasing or nondereasing) funtion of t on {: t g( t ) or g( t ) } Note that / if Note: Any regular exponential family with gt ( ) ht ( ) ( )exp[ w( ) t] has an MLR if w( ) is a nondereasing funtion or noninreasing funtion Theorem 837 (Karlin-Rubin) Consider testing H: versus H: Suppose that T is a suffiient statisti for and the family of pdfs and pmfs { gt ( ) : } of T has an MLR and gt ( )/ gt ( ) for is nondereasing funtion of t Then for any t, the test that rejets H if and only if T t is a UMP level test, where P ( T t ) Proof: Let ( ) be the power funtion for the rejetion region: { T t} () Consider the simple hypothesis: H : versus H : ( ) Consider the test has the rejetion gt ( ) region: {: t k} gt ( ), sine gt ( ) gt ( ) gt ( ) gt ( ) k and when is the nondereasing funtion of t, we an find k suh that when T t, gt ( ) T t, k From Corollary 83, this test is the UMP level gt ( ) 6

17 Leture 5 for BST 63: Statistial Theory II Kui Zhang, Spring P ( T t ) ( ) test Define ( T ), whih is level test, and P ( ( T)) P ( ( T)), therefore, ( ) So we proved that ( ) ( ) for, thus sup ( ) ( ) () Consider the simple hypothesis: H : versus H : ( ) Consider the test has the rejetion gt ( ) gt ( ) region: {: t k}, sine gt ( ) gt ( ) gt ( ) gt ( ) k and when is the nondereasing funtion of t, we an find k suh that when T t, gt ( ) T t, k From Corollary 83, this test is the UMP level gt ( ) P ( T t ) ( ) test For any another level test with power funtion *( ), beause *( ), whih is also a level test for H : versus H : ( ), thus ( ) *( ), so the proposed test, R { T t } is UMP of level Note: Consider testing H: versus H: Suppose that T is a suffiient statisti for and the family of pdfs and pmfs { gt ( ) : } of T has an MLR and gt ( )/ gt ( ) for is noninreasing funtion of t Then for any t, the test that rejets H if and only if T t is a UMP level test, where P ( T t ) 7

18 Leture 5 for BST 63: Statistial Theory II Kui Zhang, Spring Consider testing H: versus H: Suppose that T is a suffiient statisti for and the family of pdfs and pmfs { gt ( ) : } of T has an MLR and gt ( )/ gt ( ) for is nondereasing funtion of t Then for any t, the test that rejets H if and only if T t is a UMP level test, where P ( T t ) Consider testing H: versus H: Suppose that T is a suffiient statisti for and the family of pdfs and pmfs { gt ( ) : } of T has an MLR and gt ( )/ gt ( ) for is noninreasing funtion of t Then for any t, the test that rejets H if and only if T t is a UMP level test, where P ( T t ) Note: For many problems there is no UMP level test This is beause the lass of level tests is so large that no one test dominates all the others in terms of power Example 838 (Continuation of Example 835) Consider testing that rejets H if H : versus ' H : using the test ' X z n Note that T X n n ~ (, / ) has a MLR To see this, let Then 8

19 Leture 5 for BST 63: Statistial Theory II Kui Zhang, Spring gt ( ) n exp [( t ) ( t) ] exp[ n( ) / ( )]exp[ nt( ) / ] gt ( ) Sine, this ratio is an inreasing funtion of t By Karlin-Rubin Theorem, the above test is a UMP level To see that this is a level test, note that the power funtion is given by whih is a dereasing funtion of Therefore, z n / n ( ) P( X ) P( Z z ) sup ( ) ( ) 834 p-values Definition 836 A p-value p( X ) is a test statisti satisfying p( x ) for every sample point x Small values of p( X ) give evidene that H is true A p -value is valid if, for every and every, P ( p( X ) ) Notes: Given a valid p -value, a level test rejets H if and only if p( X ) 9

20 Leture 5 for BST 63: Statistial Theory II Kui Zhang, Spring p -value reports the results of a test in a ontinuous sale and gives the reader a hoie of what they want to use The smaller the p -value is, the stronger the evidene for rejeting H Theorem 837 Let W ( X ) be a test statisti suh that large values of W give evidene that H is true For eah sample point x, define p( x) sup P ( W( X) W( x )) Then p( X) is a valid p-value Example 838 (Two-sided normal p-value) Let X,, Xn be iid from for H: versus H: : n(, ) The LRT test (Exerise 838) X rejets H if S / n is large X Under H: regardless of the value of, ~ t S / n n Therefore, x p( x ) P( Tn ), s / n where Tn has Student s t distribution with n degrees of freedom

21 Leture 5 for BST 63: Statistial Theory II Kui Zhang, Spring Example 839 (One-sided normal p-value) Let X,, Xn be iid from for H: versus H: : n(, ) The LRT test (Exerise 838) Note that so that the supremum ours at X rejets H if is large S / n X P W W P W S / n (, ) ( ( X) ( x)) ( ( x)),, Therefore, X P W S / n S / n P ( T, n W( ) ) x S / n PT ( W( x)) (for ) ( ( x) ), n x px ( ) PT ( nwx ( )) PT ( n ) s / n Alternative Method for Finding p-values:

22 Leture 5 for BST 63: Statistial Theory II Kui Zhang, Spring Let W ( X ) be a test statisti for whih large values give evidene that H is true and S( X ) be a suffiient statisti only for the model{ f( x ): } Note that this is a valid p -value sine p( x) PW ( ( X) W( x) SS( X )) P ( p ( x ) ) P ( p ( x ) S s ) P ( S s ) P ( S s ) s s This is also true in the ontinuous ase (where integrals are used in plae of summation) Note: This alternative method is usually helpful in dealing with disrete distribution for S