Chapter 8 Hypothesis Testing

Transcription

1 Chapter 8 for BST 695: Speial Topis i Statistial Theory Kui Zhag, Chapter 8 Hypothesis Testig Setio 8 Itrodutio Defiitio 8 A hypothesis is a statemet about a populatio parameter Defiitio 8 The two omplemetary hypotheses i a hypothesis testig problem are alled the ull hypothesis ad the alterative hypothesis They are deoted by H ad H, respetively Settig: Let be a parameter of iterest with : where ad H versus H : : Defiitio 83 A hypothesis testig proedure or hypothesis test is a rule that speifies: For whih sample values the deisio is made to aept H as true For whih sample values H is rejeted ad H is aepted as true Rejetio Regio or Critial Regio: subset of the sample spae for whih H will be rejeted

2 Chapter 8 for BST 695: Speial Topis i Statistial Theory Kui Zhag, Aeptae Regio: omplemet of the rejetio regio Philosophial Notes: Rejet H ad aeptig H Aeptig H ad ot rejetig H Our Mai Coer: assertio of H or H Test Statisti W ( X ): futio of the sample; used together with the rejetio regio to make deisios about the hypotheses beig tested Setio 8 Methods of Fidig Tests 8 Likelihood Ratio Tests Defiitio 8 The likelihood ratio test statisti for testig H: is sup L( x) ( x) sup L( x )

3 Chapter 8 for BST 695: Speial Topis i Statistial Theory Kui Zhag, A likelihood ratio test (LRT) is ay test that has a rejetio regio of the form { x: ( x ) }, where is ay umber satisfyig LRT ad MLE: Let ˆ be the MLE of uder the urestrited parameter spae ad ˆ be the MLE of uder the restrited parameter spae The ( x) L( ˆ x) L( ˆ x ) Example 8 (Normal LRT) Let X,, X be iid from (,) We wat to test H: where is a fixed umber set by the experimeter Show that ( x ) exp[ x ( ) /] so that the LRT rejets H for small values of ( x ) Therefore the rejetio regio is whih is equivalet to { x: ( x ) }, { x : x (log )/ } Example 83 (Expoetial LRT) Let X,, X be iid from a expoetial populatio with pdf 3

4 Chapter 8 for BST 695: Speial Topis i Statistial Theory Kui Zhag, where f ( x ) exp[ ( x )] I[, ) ( x), We wat to test H :, where is a fixed umber set by the experimeter Show that, x() ; ( x ) exp[ x ( () )], x() so that the LRT rejets H for small values of ( x ) Therefore the rejetio regio is log( ) { x: ( x ) } whih is equivalet to { x : x() } Note: I both of the above examples, the rejetio regio oly depeds o the suffiiet statisti for Theorem 84 If T ( X) is a suffiiet statisti for ad *( t) ad ( x ) are the LRT statistis based o T ad X, respetively, the *( T ( x)) ( x ) for every x i the sample spae Example 85 (LRT ad Suffiiey): I Example 8, we ould have used the likelihood assoiated with the suffiiet statisti X usig the fat that X ~ (,/ ) whih rejets for large values of X 4

5 Chapter 8 for BST 695: Speial Topis i Statistial Theory Kui Zhag, Similarly, i Example 83, we a use the likelihood assoiated with the suffiiet statisti X () : L( x() ) exp[ ( x() )] I[, ] ( x() ), whih rejets for large values of X () Example 86 (Normal LRT with ukow variae) Let X,, X be iid from (, ) ad a experimeter is iterested oly i testig We wat to test H : The LRT statisti where ˆ ( xi ) / i max L(, x) max L(, x) {, } {, } L x L ˆ ˆ x {, } ( x) max (, ) (, ), ˆ ; L( ˆ ˆ ˆ, x)/ L(, x), 8 Bayesia Tests Bayesia Formulatio of Hypothesis Testig Classial Approah: The parameter is fixed If is kow, the P( x ) ad P( o x ) for all x 5

6 Chapter 8 for BST 695: Speial Topis i Statistial Theory Kui Zhag, If is kow, the P( x ) ad P( o x ) for all x I pratie, P( x ) ad P( x ) are ukow ad do ot deped o x Hee these probabilities are ot used Bayesia Approah: The parameter is radom ad is assiged a prior distributio The P( x) P( H is true x ) ad P( x) P( H is true x) may be omputed ad make sese A way to use the posterior distributio to make deisios about H ad H is to deide to aept H as true P( X) P( X ) ad rejet H otherwise, ie, the rejetio regio is { x: P( x ) /} if Oe may also defie a rejetio regio as { x: P( x ) p } where, say 99, whih is set by the researher p p Example 87 (Normal Bayesia Test) Let X,, X be iid from (, ) ad let the prior distributio o be (, ), where,, are kow We wat to test H : Reall that the posterior distributio ( x) is ormal with 6

7 Chapter 8 for BST 695: Speial Topis i Statistial Theory Kui Zhag, mea = x ad variae = Deisio rule: Aept H if ad oly if P( X) P( X ) or P( X) P( X ) / ad aept Hotherwise Note that ( x) is symmetri, hee, a equivalet deisio rule is to aept H whe ad aept H otherwise X ( ), 83 Uio-Itersetio ad Itersetio-Uio Tests Uio-itersetio method Let be a arbitrary idex set whih may be fiite or ifiite Defie H : 7

8 Chapter 8 for BST 695: Speial Topis i Statistial Theory Kui Zhag, Suppose that for eah a test is available for whih rejets H whe { x: T( x ) R} H : versus : H The the rejetio regio for the uio-itersetio test is { x: T ( x) R } Note: We aept H oly if H is aepted for all If at least oe H is rejeted, we rejet H I partiular, if the rejetio regio for H does ot deped o the the rejetio regio for H is { x: T( x) } { x:sup T( x) }, ad T( x) sup T ( x) is the test statisti Appliatio: ANOVA (Chapter ) ull hypothesis beig a itersetio of hypotheses based o otrasts (used to ompare treatmets), ie where is a set of ostats ad ai s satisfy H k k a i i i : for all a, ai k i 8

9 Chapter 8 for BST 695: Speial Topis i Statistial Theory Kui Zhag, Example 88 (Normal uio-itersetio test) Let X,, X be iid from (, ) We wat to test H :{ : } { : } Test : H : evrsus H : X LRT rejets H if t S / Test : H : evrsus H : X LRT rejets H if t S / Therefore, the uio-itersetio test for H: formed by ombiig tests ad X X Rejet H is t or t S / S / If t t the the rejetio regio for the uio-itersetio test is X Rejet H if t S / 9

10 Chapter 8 for BST 695: Speial Topis i Statistial Theory Kui Zhag, It turs out that this test is also the LRT for this problem alled the two-sided t test Itersetio-uio method Let be a arbitrary idex set whih may be fiite or ifiite Defie H : Suppose that for eah, a test is available for whih rejets H whe { x: T( x ) R} H : versus : H The the rejetio regio for the itersetio-uio test is ie, rejet H if ad oly if H is rejeted for all { x: T ( x) R } I partiular, if the rejetio regio for H does ot deped o the the rejetio regio for H is { x: T ( x) } { x:if T( x) }

11 Chapter 8 for BST 695: Speial Topis i Statistial Theory Kui Zhag, ad T( x) if T( x) is the test statisti Appliatio: Aeptae samplig where all stadards must be met for a produt to be aeptable Example 89 Let X, X be iid measuremets of breakig stregth from (, ) ad Y,, Ym be iid Beroulli( ) beig the results of m flammability tests, where Yi if the uit passes the test ad Yi otherwise Stadards to be met: 5 ad 95 H :{ 5 or 95} { 5 ad 95} Test : H : 5 versus H : 5 LRT rejets H if X 5 S / t Test : H : 95 versus H : 95 LRT rejets H if m Y i i b Therefore, the itersetio-uio test for H :{ 5 or 95}

12 Chapter 8 for BST 695: Speial Topis i Statistial Theory Kui Zhag, x m rejets H if {( x, y) : t ad y } i i b S / Setio 83 Methods of Evaluatig Tests 83 Error Probabilities ad the Power Futio Two types of Error: Type I error: If but the hypothesis test iorretly deides to rejet H Type II error: If but the hypothesis test deides to aept H Let R deote the rejetio regio for a test The P(Type I Error) if ; P ( X R) P(Type II Error) if Defiitio 83 The power futio of a hypothesis test with rejetio regio R is the futio of ( ) P ( X R)

13 Chapter 8 for BST 695: Speial Topis i Statistial Theory Kui Zhag, Example 83 (Biomial power futio) Let X ~ biomial(5, ) Cosider: Test : R {all suesses are observed} Test : R { X 3,4,or 5} H : / versus H : / Example 833 (Normal power futio) Let X,, X be iid from The LRT for this test has rejetio regio defied by H : X R / (, ) where is kow Cosider Therefore X ( ) P( ) P( Z ) / / Example 834 (otiuatio of Example 833) Suppose that the experimeter would like the maximum probability of a Type I error of ad a miimum probability of a Type II error of if How do we hoose ad? 3

14 Chapter 8 for BST 695: Speial Topis i Statistial Theory Kui Zhag, Defiitio 835 For, a test with power futio ( ) is a size test if sup ( ) Defiitio 836 For, a test with power futio ( ) is a level test if sup ( ) Notes: Some authors use the terms level ad size iterhageably The set of level tests otais the set of size tests I more omplex problems (eg, itersetio-uio ad uio-itersetio tests), it is diffiult if ot impossible to obtai a size test so oe will just have to settle for a level test The ommoly used values i pratie are, 5, ad Fixig the level of a test is otrollig the Type I error but ot the Type II error H ad H should be set up properly so that the more importat error to otrol is the Type I error H typially is the hypothesis that you wat your data to support ad hee is alled the researh hypothesis Example 837 (Size of LRT) A size LRT is ostruted by hoosig the appropriate suh that sup P( ( X) ) 4

15 Chapter 8 for BST 695: Speial Topis i Statistial Theory Kui Zhag, I Example 8, H : so that z / R{ X } I Example 83, H : so that P ( X ) exp( ( )) if ( log( )) / () We eed to show that this is a size test Note that for ay Therefore, P ( X ) P ( X ) () () sup sup ( ) ( ( log( )) / ) P X() P X() Example 838 (Size of uio-itersetio test) Reall Example 88 Let X,, X be iid from (, ) The test for H: : For ay (, ) ad H :, X X rejets H if t or t S / S / X S / ~( t ) 5

16 Chapter 8 for BST 695: Speial Topis i Statistial Theory Kui Zhag, Choose t L ad t U suh that TL t, ad TU t, where I this ase P (Type I error) = for all (, ) I pratie, we usually hoose TL TU t, / Defiitio 839 A test with power futio ( ) is ubiased if ( ') ( '') for every ' ad '' Example 83 (Colusio of Example 833) Let X,, X be iid from for testig H : has power futio give by (, ) where X ( ) P( ) P( Z ), / / whih is a ireasig futio of for a fixed value of Therefore, this test is ubiased sie ( ) ( ) sup ( t) t for all is kow The LRT 83 Most Powerful Tests 6

17 Chapter 8 for BST 695: Speial Topis i Statistial Theory Kui Zhag, Defiitio 83 Let be a lass of tests for testig H: A test i lass, with power futio ( ), is uiformly most powerful (UMP) lass test if ( ) '( ) for every ad every '( ) that is a power futio of a test i lass Theorem 83 (Neyma-Pearso Lemma) Cosider testig H:, where the pdf or pmf orrespodig to i is f( x i), i,, usig a test with rejetio regio R that satisfies ad for some k, ad The f ( x ) x R if f( x ) kf( x ) or, equivaletly, k f ( x ) f ( x ) x R if f( x ) kf( x ) or, equivaletly, f ( x ) P ( R) X a (Suffiiey) Ay test that satisfies the above oditios is a UMP level test b (Neessity) If there exists a test satisfyig the above oditios with k, the every UMP level test is a size test ad every UMP level test has the rejetio regio defied i this Theorem exept perhaps o a set A satisfyig P ( XA) P ( X A) k 7

18 Chapter 8 for BST 695: Speial Topis i Statistial Theory Kui Zhag, Corollary 833 Cosider the hypothesis problem posed i Theorem 83 Suppose T ( X ) is a suffiiet statisti for ad gt ( i ) is the pdf or pmf of T orrespodig to i, i,, The ay test based o T with rejetio regio S (subset of the sample spae T ) is a UMP level test if it satisfies gt ( ) t S if gt ( ) kgt ( ) or, equivaletly, k gt ( ) ad for some k, ad P ( T S) t S gt ( ) ( ) ( ) or, equivaletly, k gt ( ) if gt kgt Example 834 (UMP Biomial Test) Let X ~ biomial(, ) We wat to test H : / versus H : 3/4 The f ( 3 / 4), f ( / ) 4 f ( 3 / 4) 3, ad f ( / ) 4 f ( 3 / 4) 9 f ( / ) 4 Test : If we hoose 3 k 9, the by Neyma-Pearso Lemma, the UMP test rejets H whe X The 4 4 orrespodig size of this test is 8

19 Chapter 8 for BST 695: Speial Topis i Statistial Theory Kui Zhag, PX ( ) 4 Test : If we hoose k 3, the by Neyma-Pearso Lemma, the UMP test rejets H whe X = or The 4 4 orrespodig size of this test is 3 PX ( or ) 4 Notes: (o Example 834) If we hoose k or 4 9 k, the UMP test has size 4 ad, respetively Notes: (o Example 834 otiued) 3 If k the the UMP test has rejetio regio X ad aeptae regio X but does ot tell us 4 what to do whe X I this ase, we hoose to have X i either the rejetio or aeptae regio The resultig size of the test will deped o this hoie beause this is i a disrete settig 9

20 Chapter 8 for BST 695: Speial Topis i Statistial Theory Kui Zhag, Example 835 (UMP Normal Test) Let X,, X be iid from (, ) where is kow To obtai the UMP test for H: versus where, we reall that X is a suffiiet statisti for ad X ~ (, / ) Note that ( / ) exp[ ( x ) ] / ( ) exp[ { x( ) ( )}] ( ) / ( / ) exp[ ( x ) ] gx gx Hee, by Corollary 833, the UMP test rejets whe Equivaletly, we rejet whe Notes: The iequality was reversed beause gx ( ) exp[ { x( ) ( )}] k gx ( ) (log( k))( / ) ( ) x ( ) As k ireases from to, the right had size of the rejetio regio iequality goes from - to To omplete the UMP test proedure of size, we fid the ostat suh that P ( X ) P( Z ) /

21 Chapter 8 for BST 695: Speial Topis i Statistial Theory Kui Zhag, I this ase, z so that / z Types of Hypotheses: Simple Hypothesis: H : Composite Hypothesis: more tha oe possible distributio a Oe-sided Hypothesis: H : b Two-sided Hypothesis: H : Questio: How do we fid the UMP test for omposite hypotheses? Defiitio 836 A family of pdfs ad pmfs { gt ( ) : } for a uivariate radom variable T with real-valued parameter has a mootoe likelihood ratio (MLR) if, for every, gt ( )/ gt ( ) is a mootoe (oireasig or odereasig) futio of t o {: t g( t ) or g( t ) } Note that / if

22 Chapter 8 for BST 695: Speial Topis i Statistial Theory Kui Zhag, Note: Ay regular expoetial family with gt ( ) ht ( ) ( )exp[ w( ) t] has a MLR if w( ) is a odereasig futio or oireasig futio Theorem 837 (Karli-Rubi) Cosider testig H : Suppose that T is a suffiiet statisti for ad the family of pdfs ad pmfs { gt ( ) : } of T has a MLR ad gt ( )/ gt ( ) for is odereasig futio of t The for ay t, the test that rejets H if ad oly if T t is a UMP level test, where P ( T t ) Proof: Let ( ) be the power futio for the rejetio regio: { T t} () Cosider the simple hypothesis: H : versus H : ( ) Cosider the test has the rejetio gt ( ) regio: {: t k} gt ( ), sie gt ( ) gt ( ) is the odereasig futio of t, we a fid k suh that whe T t, gt ( ) k gt ( ) ad whe gt ( ) T t, k From Corollary 83, this test is the UMP level gt ( ) P ( T t ) ( ) test Defie ( T ), whih is level test, ad P ( ( T)) P ( ( T)), therefore, ( ) So we proved that ( ) ( ) for, thus sup ( ) ( ) () Cosider the simple hypothesis: H : versus H : ( ) Cosider the test has the rejetio gt ( ) gt ( ) regio: {: t k}, sie gt ( ) gt ( ) is the odereasig futio of t, we a fid k suh that whe T t,

23 Chapter 8 for BST 695: Speial Topis i Statistial Theory Kui Zhag, gt ( ) gt ( ) k ad whe T t, k From Corollary 83, this test is the UMP level gt ( ) gt ( ) P ( T t ) ( ) test For ay aother level test with power futio *( ), beause *( ), whih is also a level test for H : versus H : ( ), thus ( ) *( ), so the proposed test, R{ T t } is UMP of level Note: Cosider testig H : Suppose that T is a suffiiet statisti for ad the family of pdfs ad pmfs { gt ( ) : } of T has a MLR ad gt ( )/ gt ( ) for is oireasig futio of t The for ay t, the test that rejets H if ad oly if T t is a UMP level test, where P ( T t ) Cosider testig H : Suppose that T is a suffiiet statisti for ad the family of pdfs ad pmfs { gt ( ) : } of T has a MLR ad gt ( )/ gt ( ) for is odereasig futio of t The for ay t, the test that rejets H if ad oly if T t is a UMP level test, where P ( T t ) Cosider testig H : Suppose that T is a suffiiet statisti for ad the family of pdfs ad pmfs { gt ( ) : } of T has a MLR ad gt ( )/ gt ( ) for is oireasig futio 3

24 Chapter 8 for BST 695: Speial Topis i Statistial Theory Kui Zhag, of t The for ay t, the test that rejets H if ad oly if T t is a UMP level test, where P ( T t ) Note: For may problems there is o UMP level test This is beause the lass of level tests is so large that o oe test domiates all the others i terms of power Example 838 (Cotiuatio of Example 835) Cosider testig that rejets H if H : versus ' H : usig the test ' X z Note that T X ~ (, / ) has a MLR To see this, let The gt ( ) exp [( t ) ( t) ] exp[ ( ) / ( )]exp[ t( ) / ] gt ( ) Sie, this ratio is a ireasig futio of t By Karli-Rubi Theorem, the above test is a UMP level To see that this is a level test, ote that the power futio is give by z ( ) P ( X ) P( Z z ), / whih is a dereasig futio of Therefore, 4

25 Chapter 8 for BST 695: Speial Topis i Statistial Theory Kui Zhag, sup ( ) ( ) Example 839 (Noexistee of UMP test) Let X,, X be iid (, ) where H: A level test for this problem is ay test that satisfies P (Rejet H ) is kow We wat to test Test : Cosider Usig the same argumet as i Example 838, the test that rejets H if X ( z / ) has the highest power Furthermore, by part (b) of Neyma-Pearso Lemma, ay other level test that has as a high power as Test must have the same rejetio regio as Test exept for a set A satisfyig P ( X A) i Test : Cosider the test that rejets H is X z / Let ( ) ad ( ) be the power futio for Test ad, respetively The for ay, 5

26 Chapter 8 for BST 695: Speial Topis i Statistial Theory Kui Zhag, X P X z P z / / PZ ( z) (sie ) ( ) ( / ) ( ) PZ ( z) X P z / / ( ) P ( X z / ) ( ) Therefore, Test is ot the UMP level test Sie by Neyma-Pearso the UMP level test would have to be Test, the there exists o UMP test i this problem Notes: Whe o UMP level test exists withi the lass of all tests, the ext best thig is to fid a UMP level test withi the lass of ubiased tests Tests ad are ot ubiased tests Example 83 (olusio of Example 839 Ubiased test) Test 3: Rejets H : i favor of H: if ad oly if X z / / / or X z / 6

27 Chapter 8 for BST 695: Speial Topis i Statistial Theory Kui Zhag, Although Tests ad have slightly higher power tha Test 3 for some values of, it turs out that Test 3 is the UMP test amog all ubiased tests Skip: Setio 833 ad p-values Defiitio 836 A p-value p( X ) is a test statisti satisfyig p( x ) for every sample poit x Small values of p( X ) give evidee that H is true A p -value is valid if, for every ad every, P ( p( X ) ) Notes: Give a valid p -value, a level test rejets H if ad oly if p( X ) p -value reports the results of a test i a otiuous sale ad gives the reader a hoie of what they wat to use The smaller the p -value is, the stroger the evidee for rejetig H 7

28 Chapter 8 for BST 695: Speial Topis i Statistial Theory Kui Zhag, Theorem 837 Let W ( X ) be a test statisti suh that large values of W give evidee that H is true For eah sample poit x, defie p( x) sup P ( W( X) W( x )) The p( X) is a valid p-value Example 838 (Two-sided ormal p-value) Let X,, X be iid from for H: : (, ) The LRT test (Exerise 838) X rejets H if S / is large X Uder H: regardless of the value of, ~ t S / Therefore, x p( x ) P( T ), s / where T has Studet s t distributio with degrees of freedom Example 839 (Oe-sided ormal p-value) Let X,, X be iid from for H : : (, ) The LRT test (Exerise 838) 8

29 Chapter 8 for BST 695: Speial Topis i Statistial Theory Kui Zhag, Note that so that the supremum ours at X Rejets H if is large S / X P W W P W S / (, ) ( ( X) ( x)) ( ( x)),, Therefore, X P S / S / P ( T, W( ) ) x S / PT ( W( x)) (for ) ( W( x) ), x px ( ) PT ( Wx ( )) PT ( ) s / Alterative Method for Fidig p-values: Let S( X ) be a suffiiet statisti oly for the model{ f( x ): } p( x) PW ( ( X) W( x) SS( X )) Note that this is a valid p -value sie 9

30 Chapter 8 for BST 695: Speial Topis i Statistial Theory Kui Zhag, P ( p ( x ) ) P ( p ( x ) S s ) P ( S s ) P ( S s ) s s This is also true i the otiuous ase (where itegrals are used i plae of summatio) Note: This alterative method is usually helpful i dealig with disrete distributio for S Example 833 (Fisher s Exat Test) FYI 3