Duke University. Duke Biostatistics and Bioinformatics (B&B) Working Paper Series. Randomized Phase II Clinical Trials using Fisher s Exact Test

Transcription

1 Duke University Duke Biostatistics and Bioinformatics (B&B) Working Paper Series Year 2010 Paper 7 Randomized Phase II Clinical Trials using Fisher s Exact Test Sin-Ho Jung sinho.jung@duke.edu This working paper is hosted by The Berkeley Electronic Press (bepress) and may not be commercially reproduced without the permission of the copyright holder. Copyright c 2010 by the author.

2 Randomized Phase II Clinical Trials using Fisher s Exact Test Sin-Ho Jung Abstract A typical phase II trial is conducted as a single-arm trial to compare the response probabilities between an experimental therapy and a historical control. Historical control data, however, often have a small sample size, are collected from a different patient population, or use a different response assessment method, so that a direct comparison between a historical control and an experimental therapy may be severely biased. Randomized phase II trials entering patients prospectively to both experimental and control arms have been proposed to avoid any bias in such case. In this paper, we propose two-stage randomized phase II trials based on Fisher s exact test. Through numerical studies, we observe that the proposed method controls the type I error accurately and maintains a high power. If we can specify the response probabilities of two arms under the alternative hypothesis accurately, we can identify good randomized phase II trial designs by adopting the Simon s minimax and optimal design concepts that were developed for single-arm phase II trials.

3 Randomized Phase II Clinical Trials using Fisher s Exact Test Sin-Ho Jung 1 SUMMARY A typical phase II trial is conducted as a single-arm trial to compare the response probabilities between an experimental therapy and a historical control. Historical control data, however, often have a small sample size, are collected from a different patient population, or use a different response assessment method, so that a direct comparison between a historical control and an experimental therapy may be severely biased. Randomized phase II trials entering patients prospectively to both experimental and control arms have been proposed to avoid any bias in such case. In this paper, we propose two-stage randomized phase II trials based on Fisher s exact test. Through numerical studies, we observe that the proposed method controls the type I error accurately and maintains a high power. If we can specify the response probabilities of two arms under the alternative hypothesis accurately, we can identify good randomized phase II trial designs by adopting the Simon s minimax and optimal design concepts that were developed for single-arm phase II trials. KEY WORDS: Minimax design, Optimal design, Sufficient statistic, Two-stage design, Unbalanced allocation 1 Department of Biostatistics and Bioinformatics, Duke University, Durham, North Carolina, 27710, U.S.A. ( sinho.jung@duke.edu) 1 Hosted by The Berkeley Electronic Press

4 1 Introduction A phase II cancer clinical trial is to investigate if an experimental therapy has a promising efficacy worth of further investigation. The most popular primary outcome is overall response, meaning the therapy shrinks the tumors, of the experimental therapy. As an effort to speed up this procedure, a phase II clinical trial usually recruits a small number of patients only to the experimental therapy arm to be compared to a historical control. So, traditional single-arm phase II trials are feasible only when reliable and valid data for an existing standard therapy are available for the same patient populations. Furthermore, the response assessment method in the historical control data should be identical to the one that will be used for a new study. If there exist no historical control data satisfying these conditions or the existing data are too small to represent the whole patient population, we have to consider a randomized phase II clinical trial with a prospective control to be compared with the experimental therapy under investigation. Cannistra [1] recommends a randomized phase II trial if a single-arm design is subject to any of these and other issues. Let p 1 and p 2 denote the response probabilities of an experimental and a control arms, respectively. In a randomized phase II trial, we want to test H 0 : p 1 p 2 against H 1 : p 1 > p 2. The null distribution of the binomial test statistic depends on the common response probability p 1 = p 2, see Jung [2]. Consequently, if the true response probabilities are different from the specified ones, the testing based on binomial distributions may not control the type I error accurately. In order to avoid this issue, Jung [2] proposes to control the type I error rate at p 1 = p 2 = 1/2. This results in a strong conservativeness when the true response probability is different from 50%. Asymptotic tests avoid specification of p 1 = p 2 by replacing them with their consistent estimators, but the sample sizes of phase II trials usually are not large enough for a good large sample approximation. Fisher s [3] exact test has been a popular testing method for comparing two sample binomial proportions with small sample sizes. In a randomized phase II trial setting, Fisher s exact test is based on the distribution of the number of responders on one arm conditioning on the total number of responders which is a sufficient statistic of p 1 = p 2 under H 0. Hence, the rejection value of Fisher s exact test does not require specification of the common response 2

5 probabilities p 1 = p 2 under H 0. In this paper, we propose two-stage randomized phase II trial designs based on Fisher s exact test. Using some example designs, we show that Fisher s exact test accurately controls the type I error over the wide range of true response values, and is more powerful than Jung s method based on binomial test if the true response probabilities are different from 50%. If we can project the true response probabilities accurately at the design stage, we can identify efficient designs by adopting the Simon s [4] optimal and minimax design concepts that were proposed for single-arm phase II trials. We provide tables of minimax and optimal two-stage designs under various practical design settings. In this paper, we limit our focus on randomized phase II trials for evaluating the efficacy of an experimental therapy compared to a prospective control. Other types I randomized phase II trial designs have been proposed by many investigators including Simon, Wittes and Ellenberg [5], Sargent and Goldberg [6], Thall, Simon and Ellenberg [7], Palmer [8], and Steinberg and Venzon [9]. Rubinstein et al. [10] discuss the strengths and weaknesses of some of these methods, and propose a method for randomized phase II screening designs based on large-sample approximation. 2 Single-Stage Design If patient accrual is fast or it takes much time (say, longer than 6 months) for response assessment, we may consider using a single-stage design. Suppose that n patients are randomized to each arm, and X and Y denote the number of responders in arms 1 (experimental) and 2 (control), respectively. Let q k = 1 p k for arm k(= 1, 2). Then the frequencies (and response probabilities in the parentheses) can be summarized as in Table 1. (Table 1 may be placed here.) At the design stage, n is prespecified. Fisher s exact test is based on the conditional distribution of X given the total number of responders Z = X + Y with a probability mass function f(x z, θ) = ( )( ) n n x z x θ x ( )( ) m+ n n i=m i z i θ i 3 Hosted by The Berkeley Electronic Press

6 for m x m +, where m = max(0, z n), m + = min(z, n), and θ = p 1 q 2 /(p 2 q 1 ) denotes the odds ratio. Suppose that we want to control the type I error rate below α. Given X + Y = z, we reject H 0 : p 1 = p 2 (i.e. θ = 1) in favor of H 1 : p 1 > p 2 (i.e. θ > 1) if X Y a, where a is the smallest integer satisfying P(X Y a z, H 0 ) = m + x=(z+a)/2 f(x z, θ = 1) α. Hence, the critical value a depends on the total number of responders z. Under H 1 : θ = θ 1 (> 1), the conditional power on X + Y = z is given by 1 β(z) = P(X Y a z, H 1 ) = m + x=(z+a)/2 f(x z, θ 1 ). We propose to choose n so that the marginal power is no smaller than a specified power level 1 β, i.e. E{1 β(z)} = 2n z=0 {1 β(z)}g(z) 1 β where g(z) is the probability mass function of Z = X + Y under H 1 : p 1 > p 2 that is given as g(z) = m + x=m ( ) ( ) n n p x x 1q1 n x p2 z x q2 n z+x z x for z = 0, 1,..., 2n. Note that the marginal type I error rate is controlled below α since the conditional type I error rate is controlled below α for any z value. Given a type I error rate and a power (α, 1 β ) and a specific alternative hypothesis H 1 : (p 1, p 2 ), we find a sample size n as follows. Algorithm for Single-Stage Design: 1. For n = 1, 2,..., (a) For z = 0, 1,..., 2n, find the smallest a = a(z) such that α(z) = P(X Y a z, θ = 1) α and calculate conditional power for the chosen a = a(z) 1 β(z) = P(X Y a z, θ 1 ). 4

7 (b) Calculate the marginal power 1 β = E{1 β(z)}. 2. Find the smallest n such that 1 β 1 β. Fisher s test that is based on the conditional distribution is valid under θ = 1 (i.e. controls the type I error rate exactly), and its conditional power depends only on the odds ratio θ 1 under H 1. However, the marginal power, and hence the sample size n, depends on (p 1, p 2 ), so that we need to specify (p 1, p 2 ) at the design stage. If (p 1, p 2 ) are mis-specified, the trial may be over- or under-powered but the type I error in data analysis will always be appropriately controlled. 3 Two-Stage Design For ethical and economical reasons, clinical trials are often conducted using multiple stages. Phase II trials usually enter small number of patients, so that the number of stages is mostly two at most. We consider most popular two-stage phase II trial designs with an early stopping when the experimental therapy has a low efficacy. Suppose that n l (l = 1, 2) patients are randomized to each arm during stage l(= 1, 2). Let n 1 + n 2 = n denote the maximal sample size for each arm, X l and Y l denote the number of responders during stage l in arms 1 and 2, respectively, X = X 1 + X 2 and Y = Y 1 + Y 2. At the design stage, n l are prespecified. Note that X 1 and X 2 are independent, and, given X l + Y l = z l, X l has the conditional probability mass function f l (x l z l, θ) = ( nl )( ) nl x l z l x l θ x l ( )( ) ml+ nl nl i=m l i z l i θ i for m l x l m l+, where m l = max(0, z l n l ) and m l+ = min(z l, n l ). We consider a two-stage randomized phase II trial whose rejection values are chosen conditional on z 1 and z 2 as follows. Stage 1: Randomize n 1 patients to each arm, and observe x 1 and y 1. 5 Hosted by The Berkeley Electronic Press

8 a. Given z 1 (= x 1 + y 1 ), find a stopping value a 1 = a 1 (z 1 ). b. If x 1 y 1 a 1, proceed to stage 2. c. Otherwise, stop the trial. Stage 2: Randomize n 2 patients to each arm, observe x 2 and y 2 (z 2 = x 2 + y 2 ). a. Given (z 1, z 2 ), find a rejection value a = a(z 1, z 2 ). b. Accept the experimental arm if x y a. Now, the question is how to choose rejection values (a 1, a) conditioning on (z 1, z 2 ). 3.1 How to choose a 1 and a In this section, we assume that n 1 and n 2 are given. We consider different options in choosing a 1. For example, We may want to stop the trial if the experimental arm is worse than the control. In this case, we choose a 1 = 0. This a 1 is constant with respect to z 1. We may choose a 1 so that the conditional probability of early termination given z 1 is no smaller than a level γ 0 (= 0.6 to 0.8) under H 0 : θ = 1, i.e. PET 0 (z 1 ) = P(X 1 X 2 < a H 0 ) = (a 1 +z 1 )/2 1 x 1 =m 1 f 1 (x 1 z 1, θ = 1) γ 0. We may choose a 1 so that the conditional probability of early termination given z 1 is no larger than a level γ 1 (= 0.02 to 0.1) under H 1 : θ = θ 1, i.e. PET 1 (z 1 ) = P(X 1 X 2 < a H 1 ) = (a 1 +z 1 )/2 1 x 1 =m 1 f 1 (x 1 z 1, θ 1 ) γ 1. Among these options, we propose to use a 1 = 0. Most of optimal two-stage phase II trials also stop early when the observed response probability from stage 1 is no larger than the specified response probability under H 0, refer to Simon [4] and Jung et al. [11] for single-arm trial cases and Jung [2] for randomized trial cases. 6

9 With a 1 fixed at 0, we choose the second stage rejection value a conditioning on (z 1, z 2 ). Given type I error rate α, a is chosen as the smallest integer satisfying We calculate α(z 1, z 2 ) by = m 1+ x 1 =m 1 α(z 1, z 2 ) P(X 1 Y 1 a 1, X Y a z 1, z 2, θ = 1) α. P(X 1 (a 1 + z 1 )/2, X 1 + X 2 (a + z 1 + z 2 )/2 z 1, z 2, θ = 1) m 2+ x 2 =m 2 I{x 1 (a 1 + z 1 )/2, x 1 + x 2 (a + z 1 + z 2 )/2}f 1 (x 1 z 1, 1)f 2 (x 2 z 2, 1), where I( ) is the indicator function. = Given z 1 and z 2, the conditional power under H 1 : θ = θ 1 is obtained by m 1+ x 1 =m 1 m 2+ 1 β(z 1, z 2 ) = P(X 1 Y 1 a 1, X Y a z 1, z 2, θ 1 ) x 2 =m 2 I{x 1 (a 1 + z 1 )/2, x 1 + x 2 (a + z 1 + z 2 )/2}f 1 (x 1 z 1, θ 1 )f 2 (x 2 z 2, θ 1 ). Note that, as in the single-stage case, the calculations of type I error rate α(z 1, z 2 ) and rejection values (a 1, a) do not require specification of the common response probability p 1 = p 2 under H 0, and the conditional power 1 β(z 1, z 2 ) requires specification of the odds ratio θ 1 under H 1, but not the response probabilities for two arms, p 1 and p How to choose n 1 and n 2 In this section we discuss how to choose sample sizes n 1 and n 2 at the design stage based on some criteria. Given (α, β ), we propose to choose n 1 and n 2 so that the marginal power is maintained above 1 β while controlling the conditional type I error rates for any (z 1, z 2 ) below α as described in Section 3.1. For stage l(= 1, 2), the marginal distribution of Z l = X l + Y l has a probability mass function g l (z l ) = m l+ x l =m l ( ) nl x l p x l 1 q n l x l 1 ( nl z l x l ) p z l x l 2 q n l z l +x l 2 for z l = 0,..., 2n l. Under H 0 : p 1 = p 2 = p 0, this is expressed as ) g 0l (z l ) = p z l 0 q 2n l z l 0 m l+ x l =m l ( )( nl nl x l z l x l. 7 Hosted by The Berkeley Electronic Press

10 Further, Z 1 and Z 2 are independent. Hence, we choose n 1 and n 2 so that the marginal power is no smaller than a specified level 1 β, i.e. 1 β 2n 1 2n 2 z 1 =0 z 2 =0 The marginal type I error is calculated by α 2n 1 {1 β(z 1, z 2 )}g 1 (z 1 )g 2 (z 2 ) 1 β. 2n 2 z 1 =0 z 2 =0 α(z 1, z 2 )g 01 (z 1 )g 02 (z 2 ). Since the conditional type I error rate is controlled below α for any (z 1, z 2 ), the marginal type I error rate is no larger than α. Although we do not have to specify the response probabilities for testing, we need to do so when choosing (n 1, n 2 ) at the design stage. If the specified response probabilities are different from the true ones, then the marginal power may be different from the expected one. But in this case our testing is still valid in the sense that it always controls the (both conditional and marginal) type I error rate below the specified level. Let PET 0 E{PET 0 (Z 1 ) H 0 } = 2n1 z 1 =0 PET 0 (z 1 )g 0 (z 1 ) denote the marginal probability of early termination under H 0. Then, among those (n 1, n 2 ) satisfying the (α, 1 β )-condition, the minimax and the optimal designs are chosen as follows. Minimax design chooses (n 1, n 2 ) with the smallest maximal sample size n(= n 1 + n 2 ). Optimal design chooses (n 1, n 2 ) with the smallest marginal expected sample size EN under H 0, where EN = n 1 PET 0 + n (1 PET 0 ). Tables 2 to 5 report the sample sizes (n, n 1 ) of the minimax and optimal two-stage designs for α = 0.15 or 0.2, 1 β = 0.8 or 0.85, and various combinations of (p 1, p 2 ) under H 1. For comparison, we also list the sample size n of the single-stage design under each setting. Note that the maximal sample size of the minimax is slightly smaller than or equal to the sample size of the single-stage design. If the experimental therapy is inefficacious, however, the expected sample sizes of minimax and optimal designs are much smaller than the sample size of the single-stage design. 8

11 4 Numerical Studies Jung [2] proposes a randomized phase II design method based on the binomial test, called MaxTest in this paper, by controlling the type I error rate at p 1 = p 2 = 50%. Since the type I error rate of the two-sample binomial test is maximized at p 1 = p 2 = 50%, this test will be conservative if the true response probability under H 0 is different from 50%. We want to compare the performance of our Fisher s test with that of MaxTest. Figure 1 displays the type I error rate and power in the range of 0 < p 2 < 1 for single-stage designs with n = 60 per arm, = p 1 p 2 = 0.15 or 0.2 under H 1 and α = 0.1, 0.15 or 0.2 under H 0 : p 1 = p 2. The solid lines are for Fisher s test and the dotted lines are for MaxTest, and the lower two lines are for type I error rate and the upper two lines are for power. As is well known, Fisher s test controls the type I error conservatively over the range of p 2. The conservativeness gets slightly stronger with small p 2 values close to 0. MaxTest controls the type I error accurately around p 2 = 0.5, but becomes more conservative for p 2 values far from 0.5, especially with small p 2 values. For α = 0.1, Fisher s test and MaxTest have similar power around 0.2 p except that MaxTest is slightly more powerful for p Otherwise, Fisher s test is more powerful. The difference in power between the two methods becomes larger with = We observe similar trends overall, but the difference in power becomes smaller with = 0.2, especially when combined with a large α(= 0.2). Figure 2 displays the type I error rate and power of two-stage designs with n 1 = n 2 = 30 per arm. We observe that, compared to MaxTest, Fisher s test controls the type I error more accurately in most range of p 2 values. If α = 0.1, Fisher s test is more powerful than MaxTest over the whole range of p 2 values. But with a larger α, such as 0.15 or 0.2, MaxTest is slightly more powerful for p As in the single-stage design case, the difference in power diminishes as and α increase. 5 Discussions We have proposed design and analysis methods for two-stage randomized phase II clinical trials based on Fisher s exact test. While the binomial test by Jung [2] requires specification 9 Hosted by The Berkeley Electronic Press

12 of the response probability of the control arm p 2 or conservatively controls the type I error rate at p 2 = 0.5, Fisher s exact test does not require specification of p 2. If p 1 and p 2 under H 1 can be accurately specified at the design stage, we can calculate the expected sample size under H 0 and the sample sizes (n 1, n 2 ) for the minimax and optimal two-stage designs. Even though p 1 and p 2 are mis-specified at the design, Fisher s test accurately control the type I error rate and maintains a higher power than the binomial test, especially if p 1 and p 2 are different from 50%. For two-stage Fisher s exact test, the rejection value of the first stage is fixed at a 1 = 0, but the rejection value of the second stage a is chosen depending on the total numbers of responders through two stages, (z 1, z 2 ). Hence, a design based on Fisher s exact test will be specified by the sample sizes (n 1, n 2 ) only, while Jung s [2] designs based on the binomial test are specified by the sample sizes and rejection values (n 1, n 2, a 1, a). The proposed method assumes an equal allocation between two arms. However, extension to unbalanced allocations is straightforward. Let n kl denote the sample size for arm k(= 1, 2) at stage l(= 1, 2). Then, we can find a design and conduct the statistical testing using f l (x l z l, θ) = ( n1l )( ) n1l x l z l x l θ x l ( )( ) ml+ n2l n2l i=m l i z l i θ i for m l x l m l+ and g l (z l ) = m l+ x l =m l ( ) n1l x l p x l 1 q n 2l x l 1 ( n2l z l x l ) p z l x l 2 q n 2l z l +x l 2 for z l = 0,..., n 1l + n 2l, where m l = max(0, z l n 2l ) and m l+ = min(z l, n 1l ). Even though the sample sizes (n 1, n 2 ) are determined at the design stage, the realized sizes when the study is completed may be slightly different from the pre-specified ones. This kind of discrepancy in sample sizes becomes no issue for our method by performing a Fisher s exact test conditioning on the realized sample sizes as well as the total number of responders. The Fortran program to find minimax and optimal designs are available from the author. 10

13 REFERENCES 1. Cannistra SA. (2009). Phase II trials in Journal of Clinical Oncology. Journal of Clinical Oncology. 27(19), Jung SH. (2008). Randomized phase II trials with a prospective control. Statistics in Medicine. 27, Fisher RA. (1935). The logic of inductive inference (with discussion). Journal of Royal Statistical Society. 98, Simon R. Optimal two-stage designs for phase II clinical trials. Controlled Clinical Trials 1989; 10: Simon R, Wittes RE, Ellenberg SS. Randomized phase II clinical trials. Cancer Treatment Reports 1985; 69: Sargent DJ, Goldberg RM. A flexible design for multiple armed screening trials. Statistics in Medicine 2001; 20: Thall PF, Simon R, Ellenberg SS. A two-stage design for choosing among several experimental treatments and a control in clinical trials. Biometrics 1989; 45: Palmer CR. A comparative phase II clinical trials procedure for choosing the best of three treatments. Statistics in Medicine 1991; 10: Steinberg SM, Venzon DJ. Early selection in a randomized phase II clinical trial. Statistics in Medicine 2002; 21: Rubinstein LV, Korn EL, Freidlin B, Hunsberger S, Ivy SP, Smith MA. Design issues of randomized phase II trials and a proposal for phase II screening trials. Journal of Clinical Oncology 2005; 23(28): Jung SH, Lee TY, Kim KM, George S. Admissible two-stage designs for phase II cancer clinical trials. Statistics in Medicine 2004; 23: Hosted by The Berkeley Electronic Press

14 Table 1. Frequencies (and response probabilities in the parentheses) of a single-stage randomized phase II trial Arm 1 Arm 2 Total Response Yes x (p 1 ) y (p 2 ) z No n x (q 1 ) n y (q 2 ) 2n z Total n n 12

15 Table 2. Single-stage designs, and minimax and optimal two-stage Fisher designs for (α, 1 β ) = (.15,.8) and balanced allocation (r = 1) Single-Stage Design Minimax Two-Stage Design Optimal Two-Stage Design p 2 p 1 θ n α 1 β (n, n 1 ) α 1 β EN (n, n 1 ) α 1 β EN (78, 40) (81, 26) (44, 17) (44, 17) (29, 11) (29, 11) (56, 25) (58, 19) (36, 16) (37, 12) (65, 36) (69.22) (41, 19) (42, 14) (74, 42) (79, 26) (46, 23) (49, 14) (81, 37) (84, 30) (47, 27) (50, 17) (85, 65) (95, 27) (49, 26) (52, 19) (86, 66) (95, 32) (54, 21) (55, 19) (87, 59) (94, 35) (54, 22) (56, 18) (87, 59) (94, 35) (54, 21) (55, 19) (86, 66) (95, 32) (49, 26) (52, 19) (85, 65) (96, 26) (47, 27) (50, 17) (81, 37) (84, 30) (46, 23) (50, 12) (74, 42) (81, 23) (41, 19) (43, 12) (65, 36) (69, 22) (36, 16) (38, 9) (56, 25) (59, 17) (29, 11) (30, 7) (44, 17) (46, 11) (78, 40) (83, 22) Hosted by The Berkeley Electronic Press

16 Table 3. Single-stage designs, and minimax and optimal two-stage Fisher designs for (α, 1 β ) = (.15,.85) and balanced allocation (r = 1) Single-Stage Design Minimax Two-Stage Design Optimal Two-Stage Design p 2 p 1 θ n α 1 β (n, n 1 ) α 1 β EN (n, n 1 ) α 1 β EN (92, 48) (94, 35) (51, 24) (52,18) (34, 11) (34, 11) (65, 37) (68, 24) (41, 21) (42, 16) (78, 48) (82, 29) (49, 23) (51, 17) (88, 43) (93, 32) (52, 29) (56, 18) (94, 68) (102, 37) (59, 30) (62, 19) (100, 55) (107, 39) (60, 39) (65, 22) (106, 79) (114, 39) (61, 37) (65, 24) (107, 74) (115, 45) (61, 38) (66, 23) (107, 74) (115, 45) (61, 37) (65, 24) (106, 79) (114, 39) (60, 39) (65, 22) (100, 55) (107, 39) (59, 30) (62, 19) (94, 68) (102, 37) (52, 29) (56, 18) (88, 43) (93, 32) (49, 23) (52, 15) (78, 48) (84, 26) (41, 21) (43, 14) (65, 37) (69, 22) (34, 11) (35, 7) (51, 24) (53, 15) (92, 48) (98, 27)

17 Table 4. Single-stage designs, and minimax and optimal two-stage Fisher designs for (α, 1 β ) = (.2,.8) and balanced allocation (r = 1) Single-Stage Design Minimax Two-Stage Design Optimal Two-Stage Design p 2 p 1 θ n α 1 β (n, n 1 ) α 1 β EN (n, n 1 ) α 1 β EN (65, 36) (68, 24) (38, 15) (38, 15) (25, 10) (25, 10) (47, 19) (47, 19) (30, 19) (31, 13) (54, 40) (60, 18) (34, 15) (35, 13) (62, 29) (65, 23) (39, 26) (40, 14) (67, 47) (73, 24) (40, 35) (45, 12) (68, 55) (75, 30) (41, 28) (45, 16) (69, 54) (76, 32) (42, 25) (46, 16) (70, 50) (77, 32) (42, 26) (45, 18) (70, 50) (77, 32) (42, 25) (46, 16) (69, 54) (76, 32) (41, 28) (45, 16) (68, 55) (77, 28) (41, 23) (45, 12) (67, 47) (73, 24) (39, 26) (40, 14) (62, 29) (65, 23) (34, 15) (36, 11) (54, 40) (60, 18) (30, 19) (33, 7) (47, 19) (48, 17) (25, 10) (26, 6) (38, 15) (40, 9) (65, 36) (70, 20) Hosted by The Berkeley Electronic Press

18 Table 5. Single-stage designs, and minimax and optimal two-stage Fisher designs for (α, 1 β ) = (.2,.85) and balanced allocation (r = 1) Single-Stage Design Minimax Two-Stage Design Optimal Two-Stage Design p 2 p 1 θ n α 1 β (n, n 1 ) α 1 β EN (n, n 1 ) α 1 β EN (78, 37) (81, 29) (44, 19) (44, 19) (30, 11) (30, 11) (56, 36) (59, 21) (35, 19) (37, 12) (65, 35) (69, 26) (42, 19) (43, 16) (74, 51) (80, 30) (45, 27) (48, 18) (78, 50) (84, 36) (46, 32) (50, 20) (87, 68) (92, 36) (50, 29) (52, 21) (89, 63) (93, 40) (53, 27) (55, 23) (89, 71) (96, 39) (53, 28) (57, 21) (89, 71) (96, 39) (53, 27) (55, 23) (89, 63) (92, 50) (50, 29) (52, 21) (87, 68) (92, 36) (46, 32) (50, 20) (78, 50) (85, 34) (45, 27) (48, 18) (74, 51) (82, 27) (42, 19) (44, 14) (65, 35) (69, 26) (35, 19) (37, 12) (56, 36) (59, 21) (30, 11) (31, 7) (44, 19) (46, 14) (78, 37) (83, 26)

19 Figure 1: Single-stage designs with n = 60 per arm: Type I error rate and power for Fisher s test (solid lines) and MaxTest (dotted lines) = 0.15, α= 0.1 = 0.15, α= 0.15 = 0.15, α= = 0.2, α= 0.1 = 0.2, α= 0.15 = 0.2, α= Hosted by The Berkeley Electronic Press

20 Figure 2: Two-stage designs with n 1 = n 2 = 30 per arm: Type I error rate and power for Fisher s test (solid lines) and MaxTest (dotted lines) = 0.15, α= 0.1 = 0.15, α= 0.15 = 0.15, α= = 0.2, α= 0.1 = 0.2, α= 0.15 = 0.2, α=