Some Properties of the Exact and Score Methods for Binomial Proportion and Sample Size Calculation

Transcription

1 Some Properties of the Exact ad Score Methods for Biomial Proportio ad Sample Size Calculatio K. KRISHNAMOORTHY AND JIE PENG Departmet of Mathematics, Uiversity of Louisiaa at Lafayette Lafayette, LA , USA I this article, we poit out some iterestig relatios betwee the exact test ad the score test for a biomial proportio p. Based o the properties of the tests, we propose some approximate as well as exact methods of computig sample sizes required for the tests to attai a specified power. Sample sizes required for the tests are tabulated for various values of p to attai a power of 0.80 at level We also propose approximate ad exact methods of computig sample sizes eeded to costruct cofidece itervals with a give precisio. Usig the proposed exact methods, sample sizes required to costruct 95% cofidece itervals with various precisios are tabulated for p =.05(.05).5 ad precisios. The approximate methods for computig sample sizes for score cofidece itervals are very satisfactory ad the results coicide with those of the exact methods for may cases. Key words: Clopper Pearso iterval; Oe-sided limits; coverage probability; Expected legth; Sizes; Wilso iterval Address correspodece to K. Krishamoorthy, Departmet of Mathematics, Uiversity of Louisiaa at Lafayette, Lafayette, LA 70504, USA; Fax: ; krisha@louisiaa.edu

2 2 1. Itroductio The biomial distributio is the oldest subject i statistical scieces which has bee receivig cotiuous iterest amog researchers ad practitioers. The biomial model is commoly postulated for makig iferece about the proportio p of idividuals i a populatio with a particular attribute of iterest. Eve though there are several iferetial methods are proposed i the literature, all of them are subject to some criticisms. For example, the exact cofidece itervals due to Clopper Pearso (1934) for the biomial proportio are too coservative, yieldig cofidece itervals that are uecessarily wide. There are articles that recommed the approximate score cofidece itervals due to Wilso (1927) for the biomial proportio (e.g., Agresti ad Coull (1998)). The review article by Brow, Cai ad DasGupta (2001) evaluates the merits of several approximate as well as some exact cofidece itervals for the biomial proportio. These authors recommed the score iterval, amog others, for practical use because of its simplicity ad satisfactory accuracy. Eve though score itervals are i geeral shorter tha the exact cofidece itervals, their coverage probabilities are very ustable, ad may go well below the omial level for some parameter ad sample size combiatios (see Brow et al. (2001), Casella (2001) ad Geyer ad Meede (2005) for two-sided itervals, ad Cai (2005) for oe-sided itervals). There are other alterative approaches which are less coservative ad produce shorter itervals tha the Clopper Pearso exact itervals. Blyth ad Still (1983) ad Cai ad Krishamoorthy (2005) proposed shorter itervals for biomial proportios which cotrol the coverage probabilities very close to the omial level. These itervals are shorter tha the classical exact itervals, but they are computatioally itesive, ad are ot so simple as the Clopper Pearso exact method or the score iterval. Radomized itervals (Blyth ad Hutchiso (1960)) ad, recetly, fuzzy itervals (Geyer ad Meede (2005)) are also proposed for estimatig biomial proportio, but these itervals are ot commoly used i applicatios. Eve though other approaches produce shorter itervals with some desirable properties, the score itervals ad the exact itervals are still popular ad they appear i commo text books. Softwares such as Miitab ad S-Plus, ad olie calculators (e.g., use the exact method to compute cofidece itervals. Departmet of Health, Washigto state govermet ( recommeds the score iterval, Draft Guidace for Idustry ad FDA Staff ( suggests usig either of the itervals, ad the Natioal Istitute of Stadard ad Techology (NIST) describes the score ad the exact itervals. As see i the precedig paragraphs, there are may articles compared the score itervals, Wald itervals ad the exact itervals with respect to coverage probabilities ad expected legths. However, ot much compariso studies were made for hypothesis testig, especially for oe-sided hypothesis testig. At first glace, oe may thik

3 3 that the iterval estimatio problem is dual to hypothesis testig, ad so the properties of hypothesis tests ca be easily deduced from those of iterval estimatio procedures. However, as will be see later i the sequel, this is ot always the case. Aother importat problem that has ot bee well addressed i the literature is the sample size calculatio for computig cofidece itervals with a specified precisio (margi of error) or for hypothesis testig with a specified power. For example, as the score iterval is recommeded for applicatios, oe may wat to kow the reductio i sample size by usig the score iterval istead of the exact iterval. The primary goal of this paper is to outlie sample size calculatio methods for Wilso s approach ad the exact methods, ad provide a useful referece for practitioers of statistics. Keepig these objectives i mid, this article is orgaized as follows. I the followig sectio, we outlie the score tests, exact tests ad the exact methods of computig their powers. Exact size ad power properties of the tests are preseted. Sample size computatio for a give power ad omial level is outlied. Exact sample sizes required to attai a power of 0.80 at level 0.05 are give i Tables 1a ad 1b. Also, coveiet simple approximatios to the sample sizes are give for oe-tail ad two-tail tests. I Sectio 3, we first outlie the score ad the exact cofidece iterval procedures. Expressios for computig exact expected legths are give. Exact sample sizes required to compute 95% cofidece itervals with various precisios are evaluated ad preseted i Table 2. We also provide simple approximatios to compute the sample sizes. Compariso of the exact sample sizes with those based o the approximatios idicate that the approximatios are remarkably accurate for the score itervals. Some illustratios for usig table values are give i Sectio 4, ad some cocludig remarks are give i Sectio The Tests Let X be a biomial(, p) radom variable with probability mass fuctio (pmf) f(x, p) = p x (1 p) x, x = 0, 1,...,. (1) x Suppose we wat to test H 0 : p p 0 vs. H a : p > p 0, (2) where p 0 is a specified value of p, based o a observed value k of X. 2.1 The score test The score test is based o the z-score statistic Z(X,, p 0 ) = X p 0 p0 (1 p 0 ) = ˆp p 0 p0 (1 p 0 )/, (3)

4 4 where the sample proportio ˆp = X/. This test rejects the ull hypothesis i (2) whe Z(k,, p 0 ) z α, where z α deotes the upper αth quatile of the stadard ormal distributio. For testig two-sided alterative hypothesis, that is, whe H 0 : p = p 0 vs. H a : p p 0, (4) the score test rejects the ull hypothesis i (4) whe Z(k,, p 0 ) z α/ The exact test The exact tests are based o the exact p-value that ca be computed usig the biomial pmf. I particular, the p-value for testig (2) is give by P (X k, p 0 ) = p i i 0(1 p 0 ) i. (5) i=k This exact test that rejects H 0 i (2) whe the p-value i (5) is less tha or equal to α. The p-value for a two-sided alterative hypothesis, that is, for testig (4), is give by 2 mi{p (X k, p 0 ), P (X k, p 0 )}. These exact tests are level α tests i the sese that the Type I error rates ever exceed the omial level α for all (, p) cofiguratios. 2.3 Size ad power properties of the tests The exact power of the score test for testig oe-sided hypotheses i (2) is give by ( ) P (Z(X,, p 0 ) z α, p) = P X [p 0 + z α p0 (1 p 0 )] +, p, (6) where X biomial(, p) ad [x] + deotes the smallest iteger greater tha or equal to x. Note that, whe p = p 0, the above expressio gives the size (Type I error rate) of the score test. The power fuctio for testig (4) is give by P ( X [p 0 + z α/2 p0 (1 p 0 )] +, p ) + P ( X [p 0 z α/2 p0 (1 p 0 )], p (7) where [x] deotes the largest iteger less tha or equal to x. For a give p p 0, power β ad level α, the sample size required for a two-tail score test is the smallest value of for which the above power i (2) is at least β, ad the Type I error rate is at most α. The power of the exact test for testig (2) is give by p k (1 p) k I (P (X k, p 0 ) α), (8) k k=0 ),

5 5 where I(.) is the idicator fuctio. We ote that the expressio i (8) with p = p 0 gives the size of the exact test. A power expressio for a two-tail test ca be obtaied usig (8) with the idicator fuctio replaced by I(2 mi{p (X k, p 0 ), P (X k, p 0 )} α). The exact sizes ad powers of the score test ad the exact test are computed usig (6) ad (8) respectively. The sizes ad powers of both tests are plotted as a fuctio of i Figure 1a for testig H 0 : p.3 vs. H a : p >.3; powers are computed at p =.35. It is clear from these graphs that the sizes of the score test are always greater tha or equal to those of the exact test. I particular, we ote from Figure 1a that wheever the size of the score test is below the omial level it coicides with that of the exact test. The fluctuatio i powers reflects the size behaviors for all. Specifically, wheever the size of the score test is above the omial level.05, it offers more power tha the exact test; otherwise the powers of the tests are the same. These fidigs idicate that the sample sizes eeded for both tests to attai a give power are the same if the score test is required cotrol the Type I error rate withi the omial level. We also computed the sizes ad powers of the tests as a fuctio of for other values of p. As the plots exhibited similar patters as those i Figure 1a, they are ot preseted here. I Figure 1b, we plotted the sizes ad powers of the tests as a fuctio of for a two-tail test. We first observe that the size behaviors are differet from those for the right-tail test give i Figure 1a. We also see from Figure 1b that the sizes of the score test are always larger tha that of the exact test; however, there are may sample sizes for which the Type I error rates of the score test are below the omial level ad greater tha those of the exact test. For these sample sizes, the score test ot oly cotrols the Type I error rates, but also provides large power tha the exact test.

6 exact score Figure 1a. Sizes ad powers of the tests for testig H 0 : p.3 vs. H a : p >.3 at α =.05; powers are computed at p = exact score Figure 1b. Sizes ad powers of the tests for testig H 0 : p =.3 vs. H a : p.3 at α =.05; powers are computed at p =.35

7 7 2.4 Sample size calculatio for hypothesis tests For a give p, p 0 ad α, the sample size required for the score test (for oe-sided hypothesis) to attai a power of β is the smallest value of for which the power i (6) is at least β. That is, the smallest sample size for which ( ) P X [p 0 + z α p0 (1 p 0 )] +, p β. (9) Notice that the power fuctios of both tests are oscillatig with respect to the sample size (see Figure 1a); the power does icrease for a large icrease i sample size, but it may decrease for small chages i sample size. Therefore, to compute a accurate sample size, a forward search method, startig from a small value of, ca be used to fid the smallest value of that satisfies (9). Ay other search method may produce a sample size that is uecessarily larger tha the oe required to attai a specified power. Usig (7), the sample size for a two-tail score test ca be computed similarly. A approximatio to the sample size ca be obtaied usig the ormal approximatio to the biomial(, p) distributio. Specifically, a approximatio to the power i (9) ca be expressed as ( ) p0 + z α p0 q 0 p 1 Φ, (10) pq where q = 1 p, q 0 = 1 p 0 ad Φ is the stadard ormal distributio fuctio. Equatig the above power to β ad solvig for, we get (z β pq zα p0 q 0 ) 2. (11) (p p 0 ) 2 It should be oted that z c, for 0 < c < 1, is the upper cth quatile of the stadard ormal distributio. If the hypotheses are two-sided, the it is ot feasible to obtai a simple approximatio to the sample size usig the approach give for oe-sided hypotheses. A ituitive approximatio is (11) with z α replaced by z α/2. That is, (z β pq zα/2 p0 q 0 ) 2. (12) (p p 0 ) 2 The sample size for the oe-sided exact test is the smallest value of for which the power i (8) is at least β. That is, the least value of for which k=0 p k (1 p) k I (P (X k, p 0 ) α) β. (13) k The sample size for a two-tail test ca be expressed similarly.

8 8 For ay give p, p 0, α ad power β, we first determie the sample size s for the score test to attai the power of at least β ad the size at most α, ad the we determie the sample size for the exact test usig forward search startig from s. Usig this approach, we computed the sample sizes required for right-tail tests for various values of p ad p 0, α = 0.05 ad power As the size ad power studies i Sectio 2.3 idicated, the attaied powers of both tests are the same for all the cases cosidered i Table 1a. For example, whe = 49, the power of the score test is at (p, p 0 ) = (.85,.70) ad the attaied level is At this sample size parameter cofiguratio, the exact test also attaied the same level ad power. This is true for all the cases reported i Table 1a. Sample sizes are preseted i Table 1a for right-tail tests; the sample size required for a left-tail test ca be obtaied from Table 1a (see Sectio 4). If the hypotheses are two-sided, the there are may situatios where the score ad the exact tests require differet sample sizes, ofte the score test requires smaller samples tha the exact test (see Table 1b). Thus, for two-sided hypotheses, the score test may be preferable to the exact test. We also checked the approximatios i (11) ad (12) for their accuracies. Sample sizes based o these approximatios ad the correspodig exact oes are give i Table 3 for some values of (p, p 0 ). Compariso of the sample sizes shows that the approximatio is ot very satisfactory i some situatios. Nevertheless, the approximatio ca provide a reasoable estimate of the sample size, ad it ca be used as a iitial value for determiig the exact sample size usig (9). Table 3. Compariso of the exact ad approximate sample sizes for the score test to attai a power of at least 0.80 at level 0.05 right-tail test two-tail test (p, p 0 ) Apprx. exact Apprx. exact (.95,.90) (.85,.80) (.90,.85) (.75,.50) (.65,.60) (.85,.60) (.75,.60) (.45,.30) (.40,.35)

9 9 3. Cofidece Itervals We shall ow preset the score itervals, exact itervals ad expressios for computig their expected legths. 3.1 Score itervals ad expected legths The score cofidece iterval is obtaied by ivertig the score test statistic. Specifically, solvig the iequality ˆp p z α/2 p(1 p)/ for p, we get the score iterval as ˆp + z2 α/ z2 α/2 ± z α/2 ˆp(1 ˆp) + zα/2 2 /(4). (14) 1 + z2 α/2 Oe-sided limits ca be obtaied by replacig z α/2 by z α. Oe half of the expected legth of the score iterval i (14) is give by x=0 p x (1 p) x x (x/)(1 x/) + zα/2 2 /(4). (15) 1 + z2 α/2 z α/2 A approximatio to the above expressio ca be obtaied usig the followig lemma. Lemma 3.1 Let f(ˆp) be a real valued fuctio. The, Ef(ˆp) = f(p) + O( 1 ). (16) Proof. Usig a Taylor series expasio aroud p, we have f(ˆp) = f(p) + (ˆp p)f (ˆp) ˆp=p + Now, takig expectatio o both sides, we get (16). (ˆp p)2 f (ˆp) ˆp=p ! Applyig the above lemma, we see that (15) is approximately equal to p(1 p) + zα/2 2 /(4). (17) 1 + z2 α/2 z α/2

10 10 A approximatio to the sample size that is required to costruct a cofidece iterval with a give precisio d ca be calculated as follows. Settig (17) equal to d, ad solvig the resultig equatio for, we get [ zα/2 2 (pq 2d 2 ) + ] (pq 2d 2 ) 2 d 2 (4d 2 1), (18) 2d 2 where q = 1 p. Similarly, a approximate sample size to fid a 1 α upper limit withi a precisio of d from the true p ca be computed as the solutio of the equatio ( zα ) pq + z 2 α /(4) p = d. (19) 1 + z2 α Solvig (19) for, we get [ zα 2 pq + d 2d(p + d) + ] (pq + d 2d(p + d)) 2 d 2 ((2(p + d) 1) 2 1). (20) 2d 2 These above approximatios are very satisfactory as will be show later i Sectio Exact cofidece itervals ad expected legths The edpoits of the Clopper-Pearso (1934) cofidece iterval for a biomial proportio p ca be obtaied as solutios of the followig two equatios. I particular, for a give sample size ad a observed umber of successes k, the lower limit p L for p is the solutio of the equatio i=k p i i L(1 p L ) i = α 2, ad the upper limit p U is the solutio of the equatio k i=0 p i i U(1 p U ) i = α 2. Usig a relatio betwee the biomial ad beta distributios, the edpoits ca be expressed as p L = B 1 (α/2; k, k + 1) ad p U = B 1 (1 α/2; k + 1, k), where B 1 (c; a, b) deotes the cth quatile of a beta distributio with the shape parameters a ad b. The iterval (p L, p U ) is a exact 1 α cofidece iterval for p, i the sese that the coverage probability is always greater tha or equal the specified cofidece level

11 11 1 α. Oe-sided 1 α lower limit for p is B 1 (α; k, k + 1) ad oe-sided 1 α upper limit for p is B 1 (1 α; k + 1, k). Whe k =, the lower limit is α 1 ad the upper limit is set to be 1; whe k = 0, the upper limit is 1 α 1 ad the lower limit is set to be 0. For a give cofidece level 1 α ad p, the expected legth of (p L, p U ) is give by k=0 p k (1 p) k (p U p L ), (21) k where p L ad p U are as defied above. Suppose that oe wats to compute the sample size required to have a 1 α cofidece iterval with a specified precisio d the the sample size ca be computed as the solutio of the equatio k=0 p k (1 p) k (p U p L ) = 2d. (22) k For a give p ad d, the required sample size for estimatig the proportio withi the precisio d is the smallest value of for which the above expected legth is less tha or equal to 2d. A approximatio to the expected legth i (21) ca be obtaied usig Lemma 3.1, ad is give by B 1 (1 α/2; p + 1, p) B 1 (α/2; p, p + 1). (23) Our umerical compariso (ot reported here) of (23) ad the exact expected legths i (21) showed that the approximatio is satisfactory for 40, ad is accurate up to two decimal places for 100 regardless of the values of p. 3.3 Sample size calculatio for a give precisio We plotted the exact expected legths of the score ad exact itervals as a fuctio of p i Figure 2a ad as a fuctio of i Figure 2b. All the plots clearly idicate that the score itervals are i geeral shorter (ot withstadig coverage probabilities) tha the exact itervals. The differece betwee the expected legths appears to dimiish as the sample size icreases. Furthermore, we see i Figure 2a that the expected legths of both cofidece itervals are icreasig with icreasig p i (0,.5] ad i Figure 2b that they are decreasig with icreasig.

12 = 40 exact score p = 80 exact score p Figure 2a Figure 2b. Expected legths of exact ad score cofidece itervals as a fuctio of p p =.25 exact score p =.5 exact score Expected legths of exact ad score cofidece itervals as a fuctio of For a give precisio d, value of p ad cofidece level 1 α, the exact sample size required to compute the score iterval is the smallest value of for which (15) is less tha or equal to d ad the coverage probability is at least 1 α. We first computed the sample size s usig the approximatio i (18), ad the usig the s as a iitial value i (15) we carried out forward/backward search to fid the exact sample size required to costruct the score iterval with a specified precisio ad coverage probability at least 1 α. The sample sizes for the exact itervals are computed similarly usig s as a iitial value i (22), ad a forward search method. The sample sizes for 95% cofidece itervals are give i Table 2. We also computed the approximate sample sizes for the score itervals usig (18), ad compared them with the exact oes based o (15). Our compariso idicted that the approximate sample sizes are idetical to those based o the exact approach i may cases. I geeral, the approximatio is very satisfactory. The cases where the approximate sample sizes are ot equal to the exact oes are idicated i Table 2. For

13 13 example, whe p =.10, d =.01 ad the cofidece level is 0.95, the approximate sample size based o (18) is 3460 ad the exact oe is 3464, which is four uits greater tha the exact oe; this is idicated by placig umber 4 at the superscript (see Table 2). As show i Table 2, i may cases the approximate sample sizes coicide with those based o the exact methods. We observe from Table 2 that the sample sizes eeded for the score itervals are smaller tha those for the exact itervals for all the cases. 4. Some Illustratios for usig Tables Sample size for a give power: Suppose that oe wats to determie the sample size for testig H 0 : p 0.6 vs. H a : p > 0.6 at level 0.05 whe the true value of p is 0.7. The the required sample size for either of the tests to get the power of 0.80 ca be obtaied by referrig to the value (p, p 0 ) = (.7,.6) i Table 1a, ad is 143; the actual Type I error rate is If it is a two-tail test, that is, H 0 : p = 0.6 vs. H a : p 0.6, the the sample size ca be foud by referrig to (p, p 0 ) = (.7,.6) i Table 1b, ad is 181; the actual size is Sample size for a left-tail test ca be obtaied from Table 1a as follows. We first ote that, testig H 0 : p p 0 vs. H a : p < p 0 based o X is equivalet to testig H 0 : q q 0 vs. H a : q > q 0 based o X, because the umber of failures X also follows the biomial(, q) distributio. For example, the problem of fidig the sample size for testig H 0 : p.3 vs. H a : p <.3 whe the true value of p = 0.25 is equivalet to the oe for testig H 0 : q.7 vs. H a : q >.7 whe the true value of q = 1 p =.75. Thus, treatig (q, q 0 ) as (p, p 0 ), ad referrig to (p, p 0 ) = (.75,.7) i Table 1a, we get the sample size for the exact test (or for the score test) as 501. Sample size for cofidece itervals: Suppose that a researcher believes that the true proportio p of idividuals with a attribute of iterest i a populatio is 0.20, ad he wats to costruct a 95% cofidece iterval for p withi a margi of error ±1%. The, by referrig to p =.2 ad precisio 0.01 i Table 2, we get 6144 for the score iterval ad 6245 for the exact iterval. If p >.5, the referrig to 1 p ad the specified precisio, we ca get the sample size from Table 2. For example, if p =.6, the the required sample size to estimate p withi ±5% ca be foud by referrig to the value (1 p, d) = (.4,.05) i Table 2, ad is 365 for the score iterval ad 386 for the exact iterval. 5. Some Cocludig Remarks I this article, we have show that, if the sample size for a test has to be determied so that it should cotrol both Type I ad Type II error rates, the for oe-sided hypotheses, the score ad exact tests require the same sample size. This fidig idicates the sample size for the exact test (if oe wats to use the exact test) ca be computed usig the power

14 14 expressio i (6) because it is easier to compute tha the power of the exact test i (8). However, for two-sided hypotheses, the score test ca be recommeded for applicatios as it requires smaller samples i may situatios. The score method is also preferable to the exact method for costructig cofidece itervals as the former requires cosiderably smaller samples tha the latter to attai the same power. We hope that our fidigs are useful for researcher ad practitioers to choose betwee the score ad exact methods for applicatios. Refereces Agresti A, Coull, B. A. (1998). Approximate is better tha exact for iterval estimatio of biomial proportio. The America Statisticia 52: Blyth C. R., Hutchiso D. W. (1960). Tables of Neyma-shortest cofidece itervals for the biomial parameter. Biometrika 47: Blyth C. R., Still H. A. (1983). Biomial cofidece itervals. J. Amer. Statist. Assoc. 78: Brow L. D., Cai T., Das Gupta A. (2001). Iterval estimatio for a biomial proportio (with discussio). Statistical Sciece 16: Cai T. (2005). Oe-sided cofidece itervals i discrete distributios. J. Statist. Pla. Ifer. 131: Cai Y., Krishamoorthy, K. (2005). A simple improved iferetial method for some discrete distributios. Computatioal Statistics ad Data Aalysis 48: Casella G. (2001). Commet o Brow, Cai, ad DasGupta A. Statistical Sciece 16: Clopper C.J., Pearso E. S. (1934) The use of cofidece or fiducial limits illustrated i the case of biomial. Biometrika 26: Geyer C. J. Meede G. D. (2005). Fuzzy ad radomized cofidece itervals ad p-values. Statistical Sciece 20: Wilso E. B. (1927) Probable iferece, the law of successios ad statistical iferece. J. Amer. Statist. Assoc. 22: