On the Distribution of Type II Errors in Hypothesis Testing

Transcription

1 Applied Mathematics,,, doi:436/am Pblished Olie Febrary ( O the Distribtio of Type II Errors i Hypothesis Testig Abstract Skip Thompso Departmet of Mathematics & Statistics, Radford Uiversity Radford, USA thompso@radforded Received October 6, ; revised November 6, ; accepted November 3, Whe a statistical test of hypothesis for a poplatio mea is performed, we are faced with the possibility of committig a Type II error by ot rejectig the ll hypothesis whe i fact the poplatio mea has chaged We cosider this isse ad qatify matters i a maer that differs a bit from what is commoly doe I particlar, we defie the probability distribtio fctio for Type II errors We the explore some iterestig properties that we have ot see metioed elsewhere for this probability distribtio fctio Fially, we discss several Maple procedres that ca be sed to perform varios calclatios sig the distribtio Keywords: Complemetary Error Fctio, Hypothesis Testig, Power Crves, Power Srfaces, Type II Errors Itrodctio Both the probability of committig a Type I error ad the probability of committig a Type II error mst be cosidered whe a statistical test of hypothesis of a poplatio mea is performed There is a vast literatre dealig with the role of each type of error Both [] ad [] cotai sefl discssios ad refereces to the relevat literatre For a give sample size, it is possible to calclate ad cotrol directly; bt it is ot possible to calclate sice the ew poplatio mea is ot kow Varios techiqes have bee developed to qatify the role of Type II errors A particlarly good descriptio of these techiqes may be fod i [] For example, operatig-characteristic crves are ofte sed to estimate sample sizes eeded to keep the probability of a Type II error below a prescribed level Similarly, the power of a test is sed to assess the ability of a test to detect chages i the poplatio mea For a give sample size, it is cstomary to postlate a ew vale (or several ew vales) for the poplatio mea ad compte sig each sch mea The size of the gives a idicatio whether the sample size is adeqate I this paper we will maitai the spirit of this approach bt we will qatify Type II errors sig a differet perspective I Sectio, we will briefly review Type II errors We will se to deote the probability of a Type II error if the ew poplatio mea is eqal to I Sectio 3, we will go a bit frther ad covert ito a probability distribtio ad explore properties of this distribtio I Sectio 4, we will illstrate how the distribtio ca be sed to aswer iterestig qestios that are sally addressed sig operatig crves ad power crves ad how it may be sed to qatify covetioal wisdom regardig Type II errors By covertig ito a probability distribtio, we will fid that these qestios ca be addressed i a systematic ad coveiet maer Type II Errors I this sectio we review Type II errors briefly A detailed discssio of Type II errors (ad hypothesis testig i geeral) ca be fod i ay mathematical statistics text, for example, [] We assme that the paret poplatio of iterest is ormally distribted with stadard deviatio If is the sigificace level for a two tailed test, the ll hypothesis will ot be rejected for a sample of size if the sample mea x is sch that the stadardized statistic x z falls i the iterval Copyright SciRes

2 9 S THOMPSON, where deotes the iverse stadard ormal vale determied by the right tail of size We will se the complemetary error fctio to facilitate or discssio This fctio is defied as x t erfcx e dt () We ote the followig sefl properties of the complemetary error fctio x lim erfc () x x lim erfc (3) x x e erfcxdx xerfcx (4) The cmlative distribtio fctio for the stadard ormal distribtio ca be expressed sig the complemetary error fctio as x t e dt erfc x (5) The probability of a Type II error is eqal to M t M e dt (6) where M ad M are the -based stadardized vales defied by so that M, (7) erfc t M ( ) (8) M( ) The probability of a Type II error approaches a maximm limitig vale as Frthermore, a bit of reflectio shows that is symmetric abot We defie a probability distribtio for as follows et M ( ) T erfc t d (9) M( ) The probability distribtio is the () Of corse, i ay practical tests of hypothesis, the ew T poplatio mea is ot a radom variable We are beig cavalier ad regardig it as sch simply for the prposes of aalyzig the properties of Qatities obtaied by itegratig ca be iterpreted simply as the fractio of all possible ew poplatio meas that yield Type II errors of varios sizes 3 Distribtio Properties of Type II Errors I this sectio we will explore several importat ad iterestig properties of the distribtio Property We claim that T, iterestigly eogh, is eqal to the legth of the cofidece iterval abot, that is, T () Whe the itegral i Eqatio () is expaded, there reslts a expressio with fiftee terms (Refer to [3] for the actal expressio ad simplificatio) De to Eqatios ()-(4), all bt two eight terms approach as ad sice M ad M approach as The remaiig two ozero terms are ad erfc erfc The argmets i the erfc factors approach as ; so each factor approaches Therefore, T as claimed As a matter of iterest, we give also a more covetioal proof (based o the stadard ormal rather tha the complemetary error fctio) of the fact that Ideed, d Copyright SciRes

3 S THOMPSON 9 x d e dxd w x e dxdw w w x e dwdx w x x x e xe dx x e dx Property Give vales ad, we have M p erfc t d T M so that p M M erfc erfc d T which i tr is eqal to erfc erfc d T Breakig this itegral ito two, sig the sbstittios M i x, ad sig Eqatio (4), we see that x e p xerfcx T xerfc x x e M M M M () Eqatio () allows s to work with the probability distribtio sig the erfc fctio withot the eed to itegrate it directly Property 3 If we se Eqatio (3) ad Property, ad we let, we fid that the cotribtio of the two terms M i is We ths obtai a coveiet represetatio for the cmlative distribtio fctio for p x x e xerfc( x) T M M (3) Property 4 For a give vale of p i,, deote by ad R the vales of for which p with R (We refer to these vales as the left ad right iverses of p, respectively) I this case, Eqatio (3) ca be expressed i a simpler form that more clearly shows the depedece o p ad : M p p erfc M M M e e (4) Ideed, sig Eqatio 3 shows that is eqal to xerfc T x x e p x M M (5) Expadig this expressio sig Property 4 ad Eqatio () yields M M erfc M M erfc M M e e We ca rewrite the factor cotaiig the two vales of erfc as M M M erfc erfc MM M erfc Sice p, the first parethesized term is eqal to p The secod parethesized term is eqal to Makig these sbstittios ad simplifyig establishes Eqatio (4) Property 5 The mea of this distribtio is eqal to de to symmetry The stadard distribtio is eqal to (6) 3 Copyright SciRes

4 9 S THOMPSON For vales of i the rage to, 3 rages from approximately 5 to The size of this factor accots i part for the roded shape of To establish this property, we start with the itegral T d We obtai a complicated atiderivative with twety-six terms (Refer to [3] for the actal expressio ad simplificatio) However, gropig terms ad sig Eqatios () ad (3) show that all bt two of the terms approach as The two groped terms that do ot approach as are 3 s s lim s erfc erfc s s s ad s s lim s erfc erfc s s s I both groped terms ad s erfc s as s erfc as s Makig these sbstittios ad simplifyig leads to d s 3 (7) T as claimed Property 6 Workig with the secod derivative of shows that the iflectio poits of occr whe s t where t is the iqe positive soltio of t t te t e (Refer to [3] for details) We ote that t is i the iterval,3 for l 5 3 Property 7 Give a iterval, that we sspect cotais the ew poplatio mea, the average probability of a Type II error for this iterval is eqal to T p Cstomarily, is calclated for a particlar vale of the poplatio mea or for a few particlar vales This simple property provides a iterval-orieted versio of By droppig the factor of T, we ca obtai similar average vales for Property 8 Give a probability level p, the probability that does ot exceed p is eqal to p p p d where p is the left iverse of p p see this, first ote that we ca calclate p (8) To p sig Property 4 Sppose p is costrcted sig two sets of poplatio parameters,, ad, ad,, ad The defiitio of M ad M ad the p leads to fact that Solvig for i terms of ad sbstittig the reslts ito Property 4 for the secod set of parameters shows that the correspodig terms i Property 4 are eqal for the two sets of parameters so that p, p p, p p is ths a fctio of p ad (via ) p qatifies itrisically the well-kow difficlty of obtaiig Type II errors withi prescribed levels de to the rodedess of Property 9 A slight extesio of Property 8 is possible Give two probability levels p ad p with p p, the probability that will be betwee these vales is eqal to beta_cdf, beta_cdf, p,, (9) T where, ad, are the left iverses of p ad p, respectively 4 Usig ad The Maple Compter Algebra System [4] ca be sed to illstrate varios calclatios reqired to address qestios of iterest Relevat calclatios are implemeted i a Maple worksheet [3] ad several axiliary worksheets that are available from the athor s web site I the procedres discssed here, beta_erfc is the fctio defied by Eqatio 8 ad beta_cdf is the cmlative probability distribtio fctio defied by Eqatio (3) fsolve is the Maple oliear eqatio solver It shold be oted that the actal procedres i [3] are a bit more complicated de to the eed for error checkig ad the eed to deal with merical difficlties cased by the effects of floatig poit calclatios; bt we wo t fss abot the details here Iterested readers may wish to cosider implemetig similar procedres sig their fa- Copyright SciRes

5 S THOMPSON 93 vorite statistical comptig package The ses of are well kow [] For example, give a particlar vale for the ew poplatio mea we ca calclate the probability of a Type II error sig Eqatio (8) or we ca perform the calclatio as sal sig Eqatio (6) Frthermore, give a iterval, that we sspect cotais the ew poplatio mea, we ca calclate the average probability of a Type II error for this iterval sig Property 7 Power crves ad operatig-characteristic crves [] are ofte sed to help determie appropriate sample sizes to obtai Type II error probabilities of differet sizes Sch a crve is the graph of obtaied sig varios sample sizes Rather tha geerate a set of oe-dimesioal operatig-characteristic crves i the sal fashio we ca cosider as a fctio of ad ad plot the srface, or the srface the, as i the followig abbreviated code segmet beta_padn := x -> 5erfc(-x/sqrt); beta_punn := (,) -> beta_padn((-)/(sigma/sqrt())+) - beta_padn((-)/(sigma/sqrt())-); plot3d(-beta_punn(,),=uu,=45,axe s=boxed, grid=[5,5]); The srface ca be redered i varios ways Figre depicts a power srface for a typical set of poplatio parameters By workig with the srface cotors ad cross sectioal slices, we ca obtai the iformatio sally obtaied by sig oe-dimesioal power crves I particlar, we ca stdy the qestio of determiig the sample sizes reqired to yield Type II errors of varios sizes To see how we might proceed, cosider the followig example Sppose the poplatio parameters are 74 ad 3 Frther sppose we wish to se a sigificace level We wold like to determie the miimm sample size that yields a Type II error eqal to whe the ew poplatio mea is eqal to 85 While it is simple eogh to solve the oliear eqatio, mi, we ca se the power srface to estimate mi as accrately as desired Figre shows the portio of the srface for which, If we follow the srface arod the bottom for 85 til reachig the cotor crve for 85, we see that a sample size betwee 85 ad 9 will sffice Solvig the correspodig oliear eqatio shows that mi 87 For this example, mi 87 agrees with the sal two-tailed estimate [] ad this approach is applicable to other types of tests i which a simple estimate is ot readily available The sefless of this approach is ehaced de to the fact the srface ca be geerated qickly withot the eed to perform tedios ad time cosmig itegratios Also, oce the srface has bee geerated, it Figre The power srface, Figre Top portio of the srface, ca viewed ad maiplated i ay maer that is desired Similarly, by cosiderig as a fctio of ad, we ca plot the power srface, as i the followig abbreviated code segmet Copyright SciRes

6 94 S THOMPSON beta_pada := x -> 5erfc(-x/sqrt); beta_puna := (,alpha) -> beta_pada((-)/(sigma/sqrt())+(alpha)) - beta_pada((-)/(sigma/sqrt())-(alpha)); plot3d(-beta_puna(,alpha),=uu,alpha=, axes=boxed,grid=[5,5]); Figre 3 depicts the srface obtaied for a typical set of poplatio parameters As with Figre we ca se the srface cotors ad cross sectioal slices to cosider the effect of sig differet sigificace levels For the same reaso that iverse ormal calclatios are eeded whe workig with a ormal distribtio, we eed to be able to ivert to fid the vale of p for which pxp p The followig procedre ca be sed to perform this task It ses fsolve to fid give that p beta_cdf beta_cdf_iv := proc(p) # Calclate the iverse of beta_cdf local eq,, p: global beta_cdf, U, U, : := '': p := 'p': if (p = 5) the := : retr(); fi: if (p < 5) the eq := beta_cdf(p) = p: fsolve(eq, {p}, U): assig(%): else eq := beta_cdf(p) = p: fsolve(eq, {p}, U): assig(%): fi: := p: retr(): ed proc: Give a particlar probability p, we ca perform a iverse calclatio to fid the vales ad R for which R p i mch the same way as i the above iversio of R ca be calclated as i the followig procedre arcpr := proc(p) local eq, star, : global beta_erfc, U, : star := 'star': := '': eq := beta_erfc(star) = p: fsolve(eq, {star}, U): assig(%): := star: retr(): ed proc: Figre 3 The power srface, A similar procedre arcp ca be sed to calclate Of corse, oly oe of the procedres is actally eeded de to symmetry ad R satisfy R Note that R is precisely the amot by which the poplatio mea mst chage i oe directio or the other i order that the probability of a Type II error does ot exceed p Oce the vales p ad R p are available, we kow that will ot exceed p if or R Sppose we wish to calclate the probability p p this will happe Property 8 allows s to do so As examples, sppose p The yields p 4 while 5 yields p 6 ad yields p 9 I the latter case, we the ca say there is a 9% chace of committig a Type II error that does ot exceed %, that is to say, 9% of all possible vales for the ew poplatio mea yield a Type II error ot exceedig % Althogh p is smaller tha for most cases of iterest its size is easily explaied by the roded shape of If p is small eogh that ad R differ sigificatly from, the area of the regio der betwee ad R ca be early oe Sice this area is p, p teds the to be ear zero Iterpreted i aother way, the tails whose combied size is p ca be qite small for a roded distribtio sch as p serves as a measre of ad a remider that keepig the probability of a Type II error below a prescribed level ca be qite a challege Copyright SciRes

7 S THOMPSON 95 Figre 4 The crves p p for =,, 5, 5 # Calclate pstar, the probability that the # probability of a Type II error will ot # exceed p local eq, star,,, pstar: global beta_cdf, beta_erfc, U,, C: star := 'star': := '': pstar := 'pstar': := '': eq := beta_erfc(star) = p: fsolve(eq, {star}, U): assig(%): := star: pstar := beta_cdf(): retr(pstar): ed proc: A iterval orieted variat of p p ca provide additioal iformatio Give two probability levels p ad p with p p, sppose we wish fid the probability that the size of a Type II error will be betwee these vales I this case, Property 9 allows s to calclate this probability A procedre for performig the ecessary calclatios ca be fod i [3] 5 Smmary This paper ivestigated the probability distribtio for Type II errors Several iterestig properties of the distribtio were obtaied These properties ca be sed to obtai the same iformatio as that obtaied sig other commoly sed methods I additio, the properties allow s to qatify several thory isses i precise ways The maer i which this ca be doe was discssed ad illstrated sig selected Maple procedres for workig with the distribtio 6 Ackowledgemets The athor is idebted to a aoymos referee ad to the staff for several sggestios which improved the expositio of this paper sigificatly 7 Refereces Figre 5 The srface p p, The followig procedre calclates depicts graphs of the fctios p p p Figre 4 p (the ble crves) vs p (the brow crve) for selected sigificace levels,,5, ad 5 The horizotal extet of each ble crve is the iterval, ; the vertical extet is the iterval, Figre 5 depicts a more detailed srface plot of p p, betastar := proc(p) [] D C Motgomery, Statistical Process Cotrol, Joh Wiley & Sos, 99 [] D S Wackerly, W Medehall, ad R Scheaffer, Mathematical Statistics with Applicatios, Dxbry Press, 996 [3] S Thompso, Maple Worksheets for Ivestigatig Type II Errors, ml [4] Maplesoft, Waterloo Maple Ic, Waterloo, Copyright SciRes