Sequentially Deciding Between Two Experiments for Estimating a Common Success Probability

Transcription

1 In Jounal Ameican Statistical Association 93 (998), Sequentially Deciding Between Two xeiments fo stimating a ommon Success Pobability Janis Hadwick Univesity of Michigan onnie Page Michigan State Univesity Quentin F. Stout Univesity of Michigan Abstact: To estimate a success obability, two exeiments ae available: individual Benoulli() tials o the oduct of individual Benoulli() tials. This oblem has its oots in eliability whee eithe single comonents can be tested o a system of identical comonents can be tested. A total of N exeiments can be efomed, and the oblem is to sequentially select some combination (allocation) of these two exeiments, along with an estimato of, to achieve low mean squae eo of the final estimate. This scenaio is simila to that of the betteknown gou testing oblem, but hee the goal is to estimate failue ates athe than to identify defective units. The oblem also aises in eidemiological alications such as estimating disease evalence. Infomation maximization consideations, and analysis of the asymtotic mean squae eo of seveal estimatos, lead to the following adative ocedue: use the maximum likelihood estimato to estimate, and if this estimato is below (above) the cut-oint a, then obseve an individual (oduct) tial at the next stage. In a Bayesian setting with squaed eo estimation loss and suitable egulaity conditions on the io distibution, this adative ocedue, elacing the maximum likelihood estimato with the Bayes estimato, will be asymtotically Bayes. xact comutational evaluations of the adative ocedue fo fixed samle sizes show that it behaves oughly as the asymtotics edict. The exact analyses also show aamete egions fo which the adative ocedue achieves negative eget, as well as egions fo which it achieves nomalized mean squaed eo sueio to that asymtotically ossible. An examle and a discussion of extensions conclude the wok. Keywods: gou testing, adative design, comosite samling, ooled testing, omniscient allocation, evalence estimation, eliability oyight cfl995, 998. ast modified: 23 Ail 998. Reseach suoted in at by National Science Foundation unde gants DMS and DMS

2 Intoduction onside a sequence of Benoulli tials with success obability. To estimate, two exeiments can be efomed: eithe an individual tial outcome can be obseved (the -exeiment) o the oduct of individual tial outcomes can be obseved (the -exeiment), whee is an intege 2. A total of N exeiments (tests) can be efomed, and the oblem is to select some combination (allocation) of these two exeiments along with an estimato of to achieve low mean squae eo of the teminal estimato. The exeiments can be selected sequentially, so that at each stage, infomation available at that stage can be used to detemine which exeiment to cay out at the next stage. Befoe continuing, some comments on the oigin and alication of this oblem ae in ode. The -exeiment is a slightly disguised vesion of the well-studied goued data exeiment with gous of size. In the goued data setting, the goal is to estimate the failue obability, q =, and using gous of sizes othe than can educe the cost of testing (Sobel and lashoff, 975) and can lowe the vaiance of the esulting estimato (hen and Swallow, 990). Reliability settings, in which comonents can be tested eithe individually o as a system of identical comonents in seies, ae ime examles of situations in which gou testing can be useful (asteling and Paiie, 97). Othe gou testing scenaios aise in envionmental monitoing whee samle units of soil o lant matte ae combined and tested fo toxins. In these settings, the tem gou testing is often elaced by comosite samling. See ancaste and Kelle-McNulty (996) fo a eview of comosite samling methods. A futhe alication, examined in some detail in Section 9, is that of estimating evalence. Gastwith and Hammick (989), fo examle, aly gou testing methods to estimate the evalence of HIV antibodies among suboulations. In sceening scenaios of this sot, gou testing is aticulaly desiable because it ovides fo dono ivacy, an issue of seious concen among individuals at isk fo HIV. The ooled testing of Tu, itvak and Pagano (995) is anothe examle in which gou testing is used to estimate HIV evalence. In the oblem consideed hee, it is assumed that a natual exists fo the gouing of items. In this case, the samling otions ae esticted to only the two exeiments: the -exeiment and the -exeiment. The eliability setting, in which a system equies that units in seies be tested, eesents a scenaio in which such an assumtion is clealy viable. Note, howeve, that it is sometimes useful to seek the otimal fo a secific alication. This oint is discussed biefly in Section 9 and addessed moe thooughly via the two-stage samling ocedues of Hughes-Olive and Swallow (994). In Section 2, notation is given and the oblem is ecisely defined. In Section 3, allocation that maximizes infomation is deived, and in Section 4, ooling data acoss exeiments is discussed and seveal estimatos ae analyzed. In Section 5, allocation that minimizes asymtotic mean squae eo is deived fo each of the estimatos. In Section 6, an ad-hoc adative allocation ocedue is oosed, and it is shown that not all of the estimatos ae consistent when combined with an abitay allocation ocedue. In Section 7, the oblem is laced in a Bayesian famewok with squaed eo estimation loss. Then, unde egulaity conditions, the adative allocation of Section 6, elacing the maximum likelihood estimato with the Bayes estimato, is shown to be asymtotically Bayes. In Section 8, an exact aoach is taken to evaluating and otimizing allocation olicies fo fixed values of N. Hee it is shown that the ad-hoc adative ule of Section 6 has negative eget with esect to the otimal best fixed samle size ule that can be geneated when the aamete is known. It is also shown that, fo some values of the aamete, the ad-hoc adative ule achieves a nomalized mean squaed eo that is smalle than the asymtotic

3 limit. Fo comaison uoses, the otimal adative allocation ule is also comuted, assuming that the aamete is known. In Section 9, an alication of the methods in Section 6 is discussed. Thee, the gou testing aoach of Gastwith and Hammick (989) is comaed with the individual testing aoach of Nusbache et al. (986) and with the methods oosed in the esent aticle. Finally, in Section 0 the extension fom 2 exeiments to abitaily many is biefly examined. 2 Notation and Poblem Statement The oblem is set u in its fully sequential fom although much of the develoment in the next sections will not use all of this notation. et X ;X 2 ;:::be a sequence of indeendent and identically distibuted Benoulli() andom vaiables that ae indeendent of X ;X 2 ;:::, indeendent and identically distibuted Benoulli( ) andom vaiables. The success obability is esticted to (0,) thoughout the ae. A total of N tests (exeiments) will be done, whee at each stage the decision to obseve an X o an X can be made based on ast obsevations. Moe ecisely, an allocation ule is a sequence d = (d ;:::;d N ) such that fo k = ;:::;N, d k takes values 0 o, and is measuable fz ;:::;Z k g,wheez i = d i X i +( d i )X i. Thus, d i indicates the oulation fom which the i th obsevation o test is samled, with indicating an X - obsevation o the -exeiment, and 0 indicating a X -obsevation o the -exeiment. A teminal estimato of must be measuable fz ;:::;Z N g. Finally, let m k = P k i= d i be the total numbe of obsevations taken fom the -exeiments at stage k, andletn k = k m k be the total numbe of obsevations taken fom the -exeiment at stage k, wheek =; 2;:::; N. Sometimes the k subscit will be doed. Since the final goal is estimation of, an allocation scheme and estimato will be evaluated as a ai by the mean squae eo of the teminal estimato. That is, the mean squae eo of using allocation d and estimato ^ is given by MS (d; ^) = (^ ) 2. In the Bayesian famewok, the oblem of selecting both the allocation ule and the estimato educes to selecting only the allocation ule and using the Bayes estimato. Howeve, in the non-bayesian famewok, the choice of estimato is not so obvious, and as will be noted in Section 6, the allocation ule and the estimato can inteact. 3 Maximum Infomation Allocation In this section, the oblem of selecting an estimato of is ignoed, and only allocation is consideed. The citeion used fo allocation will be maximizing the Fishe infomation, and the best nonandom allocation will be found. As is tyical with such otimal nonandom allocations, the ule will deend on the unknown aamete, but will suggest the fom of an adative ule, and the elationshi between the MS s of estimatos and Fishe infomation will make the adative ule efficient. In tyical sequential allocation oblems, the diffeent exeiments give infomation about diffeent aametes. Howeve, in this oblem both exeiments give infomation about the same aamete, although one exeiment will give moe infomation deending on the actual value of the aamete. In aticula, the Fishe infomation about contained in a single obsevation of the -exeiment is I X () = ; whee q = ; q 2

4 a Table : ut-oint Values and the Fishe infomation about contained in a single obsevation of the -exeiment is I X () = 2 2 : It is easy to show that I X () > I X () if and only if <a,wheea is the unique oot in (0,) of the equation in, ( 2 )+ 2 =0: Note that a is a function only of, which is known, and hence a can be detemined exlicitly. Poosition 3. If N tests ae available, then the allocation that maximizes the infomation about is 8 >< N (obseve all X s) if <a m N = 0 (obseve all X s) if >a >: abitay if = a (The infomations ae equal at = a ). 2 Thus, the maximum infomation allocation is to obseve only X s (the -exeiment) if <a, o only X s (the -exeiment) if > a. This will be denoted as a -cut allocation, and ecall that a deends only on. Howeve, the egion whee one exeiment is bette than the othe deends on the unknown aamete. Thus, the obvious adative ule is suggested whee is estimated at each stage, and the next obsevation is allocated deending the elationshi between the estimated and the cut a. Some of the values of a (fist eoted in oye, 983) ae noted in Table. As inceases to infinity, the cut-oint a tends to, and the egion ove which the -exeiment is bette shinks. This section is concluded by evaluating the need fo sequential allocation. This is done by comaing the infomation in each exeiment. onside the atio minfi X (); I X ()g maxfi X (); I X ()g ove the ange of. ee this atio bounded below by, say,.98, then using the nonotimal exeiment would neve esult in moe than a 2% loss of infomation, geatly limiting the woth of adative allocation. Since adative allocation is somewhat moe comlicated than fixed allocation, thee needs to be sufficient benefit to justify its utilization. Poosition 3.2 I X () maxfi X (); I X ()g = I X () maxfi X (); I X ()g = ( ( )= 2 q (which is =) ( 2 q =( ) (which tends to 0 as tends to 0) fo» a fo >a fo» a fo >a 3

5 Poof. This is simly algeba. 2 Poosition 3.2 indicates that if the -exeiment is used, then fo sufficiently small, the infomation obtained can be abitaily close to 0% of that ossible when the -exeiment is used. On the othe hand, if the -exeiment is used, then the infomation obtained neve falls below (00/)% of the maximal infomation obtainable, aoaching this bound when tends to. Thus, adative allocation can be wothwhile fo inceasing infomation. Also, using the -exeiment would be the moe consevative aoach since one neve loses moe than (00/)% of the maximal infomation obtainable. Note that the bound deceases as inceases, so that at = 2, no moe than 50% of the otimal can be lost, but at =0, one might get only 0% of the otimal. 4 stimatos of Since both exeiments give infomation about, and an adative allocation ocedue would tyically allocate to both exeiments, thee is the question of how to combine o ool data acoss exeiments. Seveal estimatos ae esented hee and thei oeties ae deived. Thoughout this section, m is the numbe of obsevations fom the -exeiment, n is the numbe of obsevations fom the -exeiment, and m + n = N is the fixed total numbe of exeiments. If m obsevations fom only the -exeiment ae used to estimate, then the best estimato (unifom minimum vaiance unbiased and the maximum likelihood) is the samle mean X μ P = mi= m X i, and its mean squae eo is equal to its vaiance, q=m. et^ x denote this estimato. If n obsevations fom only the -exeiment ae used to estimate, then thee is no unbiased estimato. Howeve, the maximum likelihood estimato is X μ =,theth oot of the samle mean, X μ P = nj= n X j,whichis equivalent to the usual estimato of in goued data exeiments using gous of size. et^ x denote this estimato. The MS of ^ x can be comuted fo diffeent values of and n using binomial obabilities, and as n tends to infinity, nms (^ x ) tends to =I X () =( )= 2 2. This has been noted and studied by Sobel and lashoff (975). Some comaisons of the exact mean squae and this asymtotic fom ae made in oye (983). Next, the maximum likelihood estimato is deived fo the geneal situation with obsevations of both exeiments. Theoem 4. Given m obsevations of the -exeiment and n obsevations of the -exeiment, the maximum likelihood estimato of, denoted ^ ml, is the unique oot in [0; ] of the equation whee x = P m i= X i and x = P n j= X j. (n + m) +(m x )( + :::+ ) (x + y) =0; Poof. Diffeentiating the logaithm of the joint likelihood function with esect to gives the equation above. It is then staightfowad to show the existence and uniqueness of a oot in [0,]. 2 Fo the case of =2, the maximum likelihood estimato can be given in closed fom: q a 2 ( ^ x ) 2 +4a^ x + 4( a)^ 2 x a( ^ x ) ^ ml = ; 2 4

6 whee a = m=(2n + m). Othe natual estimatos of ae weighted aveages of ^ x and ^ x, whee the weights could deend on m, n, and N. Two aticula weight choices ae consideed below: ffl The constant weights estimato: ^ ff =(ff)^ x +( ff)^ x. ffl The weight ootional to samle size estimato: ^ s =(m=n)^ x +(n=n)^ x. Finally, we mention biefly that atio estimatos such as ^ R = μ X = μ X have been consideed fo estimating, in the context of model validation (hen and Swallow, 990). Howeve, these estimatos ae neithe efficient no consistent (in the allocation sense), and thus, ae not included in this wok. Details on the oeties of such atio estimatos in this setting can be found in Hadwick, Page, and Stout (996). To detemine the MS s of the estimatos, note that fo the weighted aveage estimatos, since ^ x is unbiased fo,ms (^ ff )=ff 2 q=m +( ff) 2 MS (^ x ). Fo the othe estimatos, thei MS s can be comuted exactly fo any values of m, n,and, but thee is no aaent closed fom fo them. Howeve, thei asymtotic exessions ae vey tactable. Define the asymtotic mean squae eo of estimato ^ as AMS (^) =MS (^)+o(=n)+o(=m) fo in (0,). Theoem 4.2 The asymtotic MS s of the estimatos ae as follows: AMS (^ x ) = q m = MS (^ x ) AMS (^ x ) = AMS (^ ml ) = n 2 2 q( ) m( )+n 2 q AMS (^ ff ) = ff2 q m + ( ff)2 ( ) n 2 2 Poof: This is staightfowad. 2 AMS (^ s ) = mq + n( )= 2 2 N 2 To illustate the elative efomance of the estimatos when eithe degeneate o balanced allocations ae used, the values of the limit of N ΛAMSas N tends to infinity ae lotted in Figue fo = 2and 0. Note that m = N allocation is used fo ^ x, n = N is used fo ^ x,andm = n =0:5N is used fo ^ ml and ^ ff. 5 Allocation Minimizing Asymtotic Mean Squae o hile fixed allocations efomance of diffeent estimatos may be imotant in some instances, the oblem of inteest hee is the aiing of an estimato and an allocation ule to lowe the MS of the teminal estimato of. In this section, the estimatos oosed in Section 4 ae consideed. Since thei exact MS s ae analytically 5

7 =2 =0 N*AMS M w/all X s M w/all X s M ombined Data eighted alha=/2 N*AMS M w/all X s M w/all X s M ombined Data eighted alha=/ Allocation fo and is 50/50 on and. Figue : Nomalized N AMS s of stimatos, =2; 0. Note change in vetical scales. intactable while thei fist ode asymtotic foms ae simle, allocation ules that minimize the asymtotic MS among nonandom allocations ae consideed fo each estimato. As is tyical with otimal nonandom ules, the allocation will deend on the unknown, but will suggest an adative ule. Fo each estimato in Section 4, the following theoem gives a nonandom allocation, as a function of, that minimizes the AMS. Theoem 5. The AMS s of estimatos ae minimized as follows:. AMS (^ ml ) and AMS (^ s ) ae minimized by the a -cut allocation descibed in Section AMS (^ ff ) is minimized by allocating m=n in ootion fff=( ff)g q=( ). Poof: This is staightfowad algeba, fom Theoem Fo each estimato consideed, the allocations eoted in Theoem 5. will give the lowest AMS fo that estimato. These ae efeed to as asymtotic omniscient fixed allocations coesonding to the estimatos. To comae the estimatos based on the omniscient fixed allocations, the minimum AMS s ae needed. et Λ onthe AMS denote the minimum asymtotic mean squae eo when the allocations of Theoem 5. ae used. oollay 5.2 The nomalized AMS Λ values ae as follows: N AMS Λ (^ ml) = N AMS Λ (^ s) = minfq; ( )= 2 2 g» N AMS Λ (^ ff) = ff q 2 q +( ff) ( )=( 2 2 ) 6

8 Hence, fo in (0; ), fo any weight ff in (0; ), AMS Λ (^ ml) =AMS Λ (^ s)» AMS Λ (^ ff); with equality occuing only at = a. Poof. This follows easily fom Theoem oollay 5.2 shows that the AMS Λ fo the maximum likelihood and weighted aveage estimatos is the lowest among those consideed. Thus, accoding to the asymtotic MS citeion, the a -cut allocation, along with eithe the maximum likelihood estimato o the samle size weighted aveage estimato, should be used. Note that caution should be execised when using the AMS fomulas. These ae fist ode aoximations, ignoing tems o(=m) +o(=n), and thus ae valid when both m and n tend to infinity. Howeve, the oint of allocation hee is to eventually end u on the bette exeiment. As will be seen in Section 6, the AMS fo the maximum likelihood estimato and the samle size weighted aveage estimato emains valid if one samle size does not tend to infinity. Howeve, the constant weighted aveage is not even consistent if one of the samle sizes does not tend to infinity. 6 Adative Allocation As mentioned, the allocation ules consideed thus fa ae motivated by the nonandom otimal allocations of Sections 3 and 5. In Section 3, the maximum infomation ule was the a -cut ule. In Section 5, this same allocation ule came out of minimizing the asymtotic MS. Thus, the a -cut allocation ule is a natual choice fo an adative ule, whee an estimato of is inseted in the a -cut fom, and that estimato is udated at each stage fo sequential allocation. Othe esults in Section 5 indicated that the estimato of and the allocation should be comatible in the sense that the a -cut allocation should minimize the asymtotic MS of the estimato in use. To define an adative a -cut allocation, conside an estimato of. Hee, the tem estimato denotes a sequence of estimatos f^ k g k=,whee^ k is measuable fz ;:::;Z k g.fo k = ;:::;N, m k + n k = k, whee,atstage k, m k and n k ae the numbe of obsevations on the -exeiment and on the -exeiment, esectively. The estimatos suggested in Section 4 fill this equiement since they ae defined fo each ai (m; n). Once an estimato is selected, the ad-hoc a -cut allocation is defined in the obvious way: The a -cut allocation with estimated : At stage, take an obsevation fom the -exeiment. At stage k, <k» N, obseve fom the -exeiment if and only if ^ k» a. The tie at a has been decided in favo of the -exeiment because the wost of the consequences of a wong decision is less (see Poosition 3.2). The a -cut allocation aims eventually to allocate to the bette exeiment deending on the value of. This equies that the estimato of used with the allocation be consistent even if the numbe of obsevations on one of the exeiments does not tend to infinity. This consistency is called allocation consistency: stimato f^ k g is (stongly) allocation consistent if ^ k tends to a.s. as k tends to infinity, fo all in (0; ). 7

9 Theoem 6. stimatos ^ ml and ^ s ae allocation consistent, while all ^ ff estimatos ae not. Poof. See the Aendix. 2 The estimato used with the a -cut should be allocation consistent. Howeve, while use of a consistent estimato is udent, it does not guaantee low MS of the teminal estimato. That equies efficiency of an estimato, as descibed below. Ideally, an adative allocation should select the bette exeiment quickly (high ecision of the estimato), and once the bette exeiment is being used, the estimato should aoximate the individual maximum likelihood estimato fo that exeiment. That is, the goal is to use an estimato ^ with the oety that MS (^) ß minfms (^ x ); MS (^ x )g: Thus, ^ should aoximate ^ x ove the ange whee the -exeiment is bette, and should aoximate ^ x ove the ange whee the -exeiment is bette. Of couse, adative allocation equies that some of the N obsevations be used to identify the bette exeiment. Thus, the desied MS would not be attained. Howeve, the lage the N, the smalle the ootion exected on the infeio exeiment, and the MS would tend to be neae to the desied MS. Asymtotically, (as N tends to infinity), MS (^ x ) can be elaced by ( )=N 2 2, and the desied limiting MS can be given as MS (^) ß minfq; ( )= 2 2 g : () N Define H () = minfq; ( )= 2 2 g, and note that the MS should aoximate H ()=N fo N lage. In Section 8 the nomalized MS will be comaed to H (). Befoe this comaison is made, H () is shown in Section 7 to aise as the limit of nomalized osteio exected loss in a Bayesian setting. 7 Asymtotic Bayes Poeties The focus of this wok is fequentist, and both estimatos and allocations ae evaluated by MS. Howeve, the majo oles of the Fishe infomation and the maximum likelihood estimato lead to asymtotic Bayes oeties of the a cut ule (when used with the Bayes estimato), and to a limiting nomalized Bayes isk equal to H (). e set u the esent estimation oblem in a Bayesian famewok by assuming a io distibution f() on, squaed eo estimation loss (; ^) = ( ^) 2, and likelihood function to match the evious wok, m X μ ( ) m m X μ n X μ ( ) n n X μ. et N denote the osteio exected loss given Z ;:::Z N whee Z i = d i X i +( d i )X i as defined in Section 2. Unde sufficient egulaity conditions on the io (Kass, Tieney, and Kadane, 990), the osteio exected loss is aoximated by the eciocal of the Fishe infomation evaluated at the maximum likelihood estimato: N = I N (^ ml ) f+o(=n)g; whee I N () denotes the Fishe infomation afte N obsevations and ^ ml is the maximum likelihood estimato defined in Section 4. Note that when mx s and nx s ae obseved, then I N () =mi X () +ni X (), whee I X and I X ae given in Section 3. 8

10 Recall fom Section 6 that H () = minfq; ( )= 2 2 g, and note that H () = ( )I f»ag +( )= 2 2 I f>ag = I X () I f»ag + I X () I f>ag; whee I f g is the set indicato function. H () is a continuous bounded function of on [0,], and thus is unifomly continuous. Also, fo any, I N () = mi X () +ni X ()» N[I X ()I f»ag + I X ()I f>ag]: This imlies that NI N () [I X ()I f»ag + I X ()I f>ag] : But this last tem is equal to I X () I f»ag + I X () I f>ag = H () since set indicatos ae used. This oves the following lemma. emma 7. NI N (^ ml ) H (^ ml ). 2 Theoem 7.2 Unde sufficient egulaity conditions giving the aoximation of the osteio vaiance in tems of Fishe infomation, i. lim inf N! N N H () a.s. ii. lim inf N! N H (). Poof. Fo (i), note that by the aoximation and by emma 7., N N = NI N (^ ml ) f+o(=n)g H (^ ml )f +O(=N)g : But ^ ml conveges to a.s. and H is a unifomly continuous function, so thus the lowe bound tends a.s. to H (). Fo (ii), use the inequality above to get lim inf N lim inf H (^ ml ). Then note that H is bounded and aly the bounded convegence theoem to deduce the limit to be H (). 2 This theoem along with sufficient conditions to insue unifom integability imly that the a -cut ule used with the Bayes estimato will have limiting Bayes isk, H (), and thus, be asymtotically Bayes. 8 Fixed Samle Size Behavio U to this oint, the develoment and evaluation of allocation ules and estimatos has been based solely on asymtotic aguments. In this section, the estimatos and allocation ules ae examined fo thei behavio based on fixed samle sizes. Fo modeate samle sizes, some of the estimatos do not behave as exected, and thee ae seveal adjustments that need to be made when imlementing the estimatos and allocation ules. Desite these oblems, it will be shown that the a -cut allocation ule using the M estimato does vey well. 9

11 8. Adjusting ut-oints The a value was deived fom asymtotic consideations. If the exact MS fo the estimato ^ x is comaed with the exact MS fo ^ x using the same numbe of tests, the cut-oint below which the -exeiment does bette deends on N as well as. Detemining the cut-oint is a staightfowad comutation, but it does not have a simle closed fom. This cut-oint diffes fom a,andn may need to be quite lage (say 00) befoe it is aoximately equal to its asymtotic value of a. Fo examle, oye (983) showed that fo = 2 the value of a 2 is =3, but fo N = 20, the-exeiment is bette when < 0:445. Fo = 5, thevalueofa 5 is 0.536, but fo N = 20, the -exeiment is bette when < 0:729. Thus, if the total samle size is modeate, then the a -cut should be modified to account fo the lack of asymtotic fit. This adjustment is moe onounced the lage is. 8.2 xact Mean Squaed os of stimatos Using Fixed Allocations As noted ealie, the MS s of most of the estimatos unde consideation don t have convenient analytic foms. Howeve, given and N, the MS s can be calculated fo each estimato. In this section, efomance of the estimatos based on fixed allocations ae eviewed. Reesentative behavio is illustated in Figue 2 fo N = 40 and N = 00, fo =2. The labels and elative samle sizes used fo the diffeent estimatos ae the same as in Figue. The values in Figue 2 have been scaled by a facto of N, so they can be comaed acoss samle sizes as well as between estimatos. It seems clea fom both figues that the best estimatos fo small and lage values of, esectively, ae ^ x and ^ x. As noted, as the total samle size inceases, the oint at which ^ x begins to imove on ^ x moves towads the value a 2 ==3. Note that, even with edetemined allocations of N=2 obsevations fom each exeiment, the maximum likelihood estimato, ^ ml, does vey well acoss the entie ange of aamete values egadless of the total samle size. Also, while it may seem odd that the MS of ^ ff o ^ s could be lage than the MS s of eithe of the two comonents that comise the estimatos, the lots indicate that this is tue. Howeve, thee is no contadiction hee because the MS s fo ^ x and ^ x ae each based on N obsevations while the ^ x and ^ x comonents of the aveaged estimatos ae each based on factions of N, and thus ae not aveages of the X and X values shown in the lots. omaing Figue 2 to Figue, one sees a decease in the MS of ^ x and the estimatos that deend on it as N goes to infinity. Unfotunately, the MS s of these estimatos don t convege vey aidly to the AMS s. e examined MS s fo N = 200 and they wee not much close to the AMS s than ae the MS s fo N = 00. On the ositive side, the MS of the maximum likelihood estimato aidly conveges to its AMS. The MS s and AMS s fo ^ x ae, of couse, the same. 8.3 MS s of Adative Pocedues The efomance of the sequential a -cut allocation ule is illustated in Figue 3 fo N = 00, =2,anda 2 ==3. In this figue, denotes the exact MS of the adative a -cut ule and eesents the nomalized lowe enveloe fom quation () of Section 6, namely H (). Note that the cut-oint ule has essentially the same MS as the lowe enveloe of the AMS s, excet fo the 0

12 N=40, =2 N=00, =2 N * MS M w/all X s M w/all X s M ombined Data eighted alha=/2 N * MS M w/all X s M w/all X s M ombined Data eighted alha=/ Allocations fo and ae 50/50 on and. Figue 2: xact NΛMS Values Using Fixed Allocations. egion of aoximately 0:3 < < 0:4, as shown in the enlagement on the ight. In this inteval, the MS of ^ ml actually imoves uon the best values gotten using the AMS s. This behavio leads to questions about the comaison of the exact MS of the cut-oint adative ule with othe lowe enveloes deived fo fixed N. Fofixed N,theMS (^ ml ) can be comuted as a function of m and, then minimized as a function of m. all the allocation thus obtained omniscient fixed. This minimize is deendent on the unknown, but gives a lowe enveloe fo this MS. It also is inteesting to note that the minimizing m is not necessaily degeneate, i.e., equal to N o 0. Fo examle, at N = 00, = 2, and = 0:35, the omniscient fixed allocation to the -exeiment is m = 57. In Figue 3, fo N = 00 and = 2, the values of nomalized MS fo the omniscient fixed allocation ae denoted by F. The lot on the left ovides nomalized MS s ove the entie ange of and the lot on the ight is a blow-u of the aamete egion in which the MS of the omniscient fixed ule is bette than the asymtotic lowe enveloe. The egion in which the omniscient fixed ule imoves on the asymtotic lowe bound () is quite small and shinks to zeo width as N!. Also note that the cut-oint adative ule () efoms bette than the omniscient fixed ule in the aea of the cut-oint. This is a egion of negative eget, as discussed fo examle in oodoofe (977) and Matinsek (983), whee the ability to adat is so beneficial that it ovetakes allocation that has advanced knowledge of the aamete but which must be fixed in advance of any exeiments. Fo fixed N, the tue lowe bound fo the MS of all ules using the maximum likelihood estimato as teminal estimato can be comuted as a function of and N. This fully sequential ule, which we call omniscient adative, assumes knowledge of the aamete. Its nomalized MS, denoted by ffl, is shown in Figue 3. Aaently the only way to obtain the omniscient adative ule is though dynamic ogamming calculations.

13 N * MS F F F F F F F F F M w/all X s M w/all X s utoint Adat F Omni. Fixed Asym. nveloe Omni. Adat F F F F F F F N * MS F F F F F F F F F M w/all X s M w/all X s utoint Adat F Omni. Fixed Asym. nveloe Omni. Adat F F F F F Figue 3: Adative Allocation using the M, N = 00, =2, cut-oint = /3. Finally, comutations simila to Figue 3 ae shown in Figue 4 fo = 0 and N = 00. The cut-oint used was 0.76, athe than a 0 = 0:679, because this is the oint at which the 0 -exeiment is sueio to the -exeiment when N = 00. The basic behavio is simila to that seen fo = 2, but many of the diffeences between asymtotic behavio and that fo modeate N ae moe onounced. Fo examle, fo small, the extent to which the -exeiment is wose than the -exeiment is fa moe exteme. Fo the same value of N, the egion in which the omniscient fixed allocation ule is a mixtue of - and -exeiments is fa lage than it was fo =2, and the ability of the omniscient adative ule to imove uon the omniscient fixed ule is geatly enhanced. The MS of the omniscient adative ule is a moe iegula function of, and, though not evident in Figue 4, it is no longe unimodal. 9 xamle: stimating Pevalence To ovide some insight as to how the adative gou testing methods esented in Section 6 may be used, we aly them to the Gastwith and Hammick (989) eanalysis of a blood sceening study of Nusbache et al. (986). The authos of the latte wok examined whethe one could effectively inhibit HIV caies fom donating to a tansfusion blood ool. In thei study, blood donos who aticiated in high isk activities wee asked to designate thei donation to a eseach blood ool athe than to the usual tansfusion blood ool. Of the 627 donations to the eseach blood ool, wee found to cay HIV antibodies. The oblem of estimating the evalence of HIV antibodies motivated Gastwith and Hammick (989) to utilize gou testing methods on the eseach blood ool data. Howeve, since sceening tests tend to cost consideably less than confimatoy tests, these authos incooated the sensitivity and secificity of the sceening test into thei 2

14 N * MS F F F F F F F F F F F F M w/all X s M w/all X s F Omni. Fixed utoint Adat Asym. nveloe Omni. Adat F F F F N * MS F F F F F F F F F F F F M w/all X s M w/all X s F F F Omni. Fixed F utoint Adat F F Asym. nveloe F Omni. Adat Figue 4: Adative Allocation using the M, N = 00, = 0, cut-oint = estimatos. In this way, they wee able to do ealistic cost analyses of the gou testing aoach and the individual testing method. Futhe, while one of thei aims was to ovide an accuate evalence estimato, they also sought a testing method that would eseve the anonymity of the donos. Note that this latte goal is in oosition to the one that motivated Dofman (943) to oose gou testing methods in the fist lace. Dofman s objective was to educe the cost of detecting all ositive cases (see also Hwang (972)). Ou goal hee is to comae the accuacy of the adative cut-oint estimato fo evalence with those obtained fom individual testing and fixed gou size testing. To simlify comaisons, the stong assumtion that the sensitivity and secificity of the sceening test ae one is made, although adative cut-oint methods can be otimized fo moe geneal settings. It is also assumed that, fo each method examined, the obsevations ae samled fom a lage oulation in which the undelying evalence ate is =627 = 0:075, the ate obseved by Nusbache et al. (986). Gastwith and Hammick (989) used batches of size 0, but thee seems to be no aticula eason to believe that 0 is bette than some othe gou size. e conside what would haen if the adative cut-oint method is alied fo gou sizes of = 0 and s = 20, assuming, as did Gastwith and Hammick (989), that thee is no dilution effect. Note that this is equivalent to the =vesus s =2oblem consideed ealie, in the sense that the = obsevations ae samled fom a Benoulli oulation with success ate q 0. Taking a samle size of N = 63, as in Gastwith and Hammick (989), we obtain a MS of : To achieve the same MS using only batches of size 0, one would need a samle of size N = 0; and, using batches of size would equie a samle size of N = 020. These esults ae summaized in Figue 5. As noted in Gastwith and Hammick (989), thee ae significant advantages to using gous lage than, and an adative goued allocation ovides yet futhe advantages. Howeve, thee is the consideation that the numbe of individual samles equied inceases slightly. Gastwith and Hammick (989) addessed this concen 3

15 Batch Sizes No. Samles (No. Individuals) ost Advantage :4 s» d :3 s» d» :4 s Adat. 0, d» 0:3 s Figue 5: 3 Methods fo Achieving MS= : via cost analyses, and hee cost analyses ae caied out using a model consisting of two comonents. et s be the cost of one sceening test and let d be the cost of obtaining a single blood donation. Then the cost of achieving of an MS of : using only batches of size is 020( s + d ); the cost using only batches of size 0 is 0 s +00 d ; and the exected cost using the adative method is 63 s +250 d. The final column of Figue 5 shows the anges of elative s and d values fo which each method is the most cost-effective. Tyically it haens that d fi s, and in these situations, adative gou testing aeas to be significantly sueio. If total cost is the aoiate consideation, it can be diectly incooated into the adative cut-oint method. Fo examle, one could comae the asymtotic cost e unit of infomation using the -exeiment, vesus the cost using the -exeiment, to decide which to efom. That is, one would detemine the cut-oint by solving I X () s + d = I X () s + d Finally, one may wonde why =0and s =20wee selected fo the adative vesion of this examle. The only eason fo this is that it coesonds well to the main case, vesus 2, studied in this ae. The esent oblem was also solved when and s wee taken to be 0 and 00 esectively (which coesonds to the vesus 0 case). In this latte case, 63 samles esult in a significantly smalle MS of 3:2 0 7 fo ^. The fact that the 0 vesus 00 exeiment ovides an even geate eduction in MS fo this oblem leads one to wonde what the otimal gou sizes ae fo secific oblems. As mentioned ealie, Hughs-Olive and Swallow (994) conside this question using a two-stage aoach. One can extend the esent wok to include the fully sequential case in which one seeks to estimate not only, but also the value of that will otimize a gou testing scenaio. 0 xtensions The oblem consideed in the evious sections was to choose fom 2 exeiments, the -exeiment and the - exeiment, and was motivated by eliability alications. The esults can be extended to J exeiments and need not necessaily include the -exeiment. et the available exeiments be defined by integes ();(2);:::;(J) such that» () <(2) ::: < (J), whee the i th exeiment is a (i) -exeiment. As befoe, the Fishe infomation about contained in a -exeiment is I X = 2 2 =( ) : 4

16 Now, define G ;s () =I X () I Xs (), and note that G ;s () > 0 if and only if I X () > I Xs (), i.e., if and only if the -exeiment has moe infomation about than does the s -exeiment. 2 G ;s () =[(s 2 2 ) s s 2 s + 2 ] ( )( s ) ; so fo in (0,), the sign of G ;s () is detemined by the fist facto. Using deivatives, one can show that G ;s () =0 has a unique oot, a ;s, in (0,), and, fo <s, I X () > I Xs () if and only <a ;s. ut the unit inteval into J ats using cuts 0=a (0);() <a ();(2) < ::: <a (J );(J) <a (J);(J+) =; whee fo notational convenience we intoduce (0) = 0 and (J +) =. Notice that these cuts ae defined using the (i) in inceasing ode. Then it can be shown that the (i) -exeiment has maximum infomation when is in the i th inteval (a (i );(i) ;a (i);(i+) ),foi =; 2;:::;J. This motivates the vey simle adative ule which allocates to the (i) -exeiment at stage k +if the estimato of based on the data u to and including stage k is in the i th inteval. As befoe, this can also be modified by noting that the a ;s values ae based on an asymtotic analysis and can be adjusted fo given samle sizes. Thee ae othe useful extensions of the oblem examined in this ae. One is to allow the samle size to be a andom vaiable which deends on some stoing citeion. Anothe is to incooate unequal costs when samling fom the diffeent exeiments. Yet anothe is to take the Bayesian esective when samle sizes ae fixed. This latte oblem equies a significantly diffeent aoach than the one taken hee. 5

17 Aendix: Poof of Theoem 6. The lack of consistency of ^ ff occus when one of the samle sizes does not tend to infinity. That is, if, say,» n<n o <, thenx μ = does not convege in obability o a.s., and the weight ff does not tend to zeo. On the othe hand, the ^ s estimato has weights m=n and n=n and if eithe samle size is bounded as N tends to infinity, the weight tends to zeo, imlying consistency. The consistency of ^ ml equies moe wok. et x = P m i= X i and x = P n j= X j. Then the logaithm of the joint likelihood function is equal to h m;n () =f m ()+g n (), whee f m () =x log() +(m x ) log( ) and g n () =x log( )+(n x ) log( ). The deivative with esect to is h 0 m;n () =f 0 m ()+g0 n (), whee f 0 m () = x m x and g 0 n () = x (n x ). The maximum likelihood estimato is the unique oot in [0,] of h 0 m;n () = 0, and because f 0 m and g0 n ae both deceasing in, it follows that the oot of h 0 m;n is between the oots of f 0 m and gn. 0 Thus, minf^ x ; ^ x g»^ ml» maxf^ x ; ^ x g. Theefoe, if both m and n tend to infinity, then both ^ x and ^ x tend to a.s., and thus ^ ml tends to a.s. Now, suose that n» n o <. Then n o» g0 n()» n o, which in tun bounds h0 m;n by A () = x m x + n o q» h 0 m;n ()» x + n o m x q = A + (): But A () =0at = x m+n o and A + () =0at + = x +n o m+n o. Then,» ^ ml» +.But and + both tend to a.s. as m tends to infinity, and thus ^ ml tends to a.s. as m tends to infinity even though n» n o. A simila agument holds fo m» m o <. 2 6

18 Refeences [] hen,.. and Swallow,. H. (990), Using Gou Testing to stimate a Pootion, and to Test the Binomial Model, Biometics, 46, [2] Dofman, R. (943) The Detection of Defective Membes of age Poulations, Annals of Mathematical Statistics,4: [3] asteling, R. G. and Paiie, R. R. (97), ombining omonent and System Infomation, Technometics, 3, [4] Gastwith, J. and Hammick, P. (989), stimation of the Pevalence of a Rae Disease, Peseving the Anonymity of the Subjects by Gou Testing: Alication to stimating the Pevalence of AIDS Antibodies in Blood Donos, Jounal of Statistical Planning and Infeence, 22, [5] Hadwick, J., Page,. and Stout, Q. F. (996), Sequentially Deciding Between Two xeiments fo stimating a ommon Success Pobability, Technical eot, Michigan State Univesity, Deatment of Statistics and Pobability. [6] Hughs-Olive, J. and Swallow,. H. (994), A Two-Stage Adative Gou-Testing Pocedue fo stimating Small Pootions, Jounal of the Ameican Statistical Association, 89, [7] Hwang, F. K. (972), A Method fo Detecting all Defective Membes in a Poulation by Gou Testing, Jounal of the Ameican Statistical Association, 67, [8] Kass, R., Tieney,., and Kadane, J. (990), The Validity of Posteio xansions Based on alace s Method, Bayesian and ikelihood Methods in Statistics and conometics, S. Geisse, J. S. Hodges, S. J. Pess and A. Zellne (eds.), Amstedam: Noth-Holland. [9] ancaste, V. A. and Kelle-McNulty, S. (996), A Review of omosite Samling Methods, Technical Reot 96-0, Kansas State Univesity, Det. of Statistics. [0] oye, M.. (983), Bad Pobability, Good Statistics, and Gou Testing fo Binomial stimation, The Ameican Statistician, 37, [] Matinsek, A. (983), Second Ode Aoximation to the Risk of a Sequential Pocedue, Annals of Statistics,, [2] Noble,. (990), Fist Ode Allocation, Ph. D. Thesis, Michigan State Univesity. [3] Nusbache, J., hiavetta, J. et al. (986) valuation of a onfidential Method of xcluding Blood Donos xosed to Human Immunodeficiency Vius, Tansfusion, 27, [4] Sobel, M. and lashoff, R. M. (975), Gou Testing with a New Goal, stimation, Biometika, 62, [5] Tu, X. M., itvak,. and Pagano, M. (995), On the Infomativeness and Accuacy of Pooled Testing in stimating Pevalence of a Rae Disease: Alication to HIV Sceening, Biometika, 82,

19 [6] oodoofe, M. (977), Second Ode Aoximations fo Sequential Point and Inteval stimation, Annals of Statistics, 5,