Adaptive Dunnett Tests for Treatment Selection

Size: px

Start display at page:

Download "Adaptive Dunnett Tests for Treatment Selection"

Vincent Leonard
6 years ago
Views:

1 s for Treatment Selection Franz König 1 Werner Brannath 1 Frank Bretz 2 Martin Posch 1 1 Section of Medical Statistics Medical University of Vienna 2 Novartis Pharma AG Basel Workshop Adaptive Designs 2006 Koenig et al (IMS) Adaptive Dunnett Tests for TS Workshop Adaptive Designs 06 1 / 18

2 Motivation Flexible vs Classical frequentist designs Flexible (Adaptive) Versus Frequentist Trial Classical frequentist trials details of design and analysis must be prefixed in advance (population, treatments, doses, main and secondary outcome variable(s), analysis strategy, sample sizes,...) lack of flexibility to react to information from inside or outside the trial Flexible (adaptive) design allow for mid-trial design modifications based on all internal and external information gathered at interim analyses without compromising the type I error rate To control the type I error rate, the design modifications need not be specified in advance. Koenig et al (IMS) Adaptive Dunnett Tests for TS Workshop Adaptive Designs 06 2 / 18

3 Seamless Designs Drug Development Flexible Trials and Drug Development Full flexibility (unscheduled adaptivity)= adapting design parameters without a (complete) specification of the adaptation rule. dealing with the unexpected dealing with the expected unpredictability of clinical trials Flexible designs allow to... integrate Phase II and Phase III trials into a single trial (adaptive seamless designs, e.g., Bauer & Kieser 1999, Bretz et al 2006,...) formally integrate the data of exploratory and confirmatory phases speed up the drug development process react flexibly to unexpected events Koenig et al (IMS) Adaptive Dunnett Tests for TS Workshop Adaptive Designs 06 3 / 18

4 Seamless Designs Treatment selection Flexible Trial with Treatment Selection (Phase II + III) Comparison of k treatments with a control. Treatments are selected during the trial. Selection rule need not be laid down in detail in advance. Scientific expert knowledge not known at the planning stage will influence the decision process. Selection rule is complex and unknown at planning stage. Treatment selection may be based on un-blinded interim data on the primary efficacy endpoint, secondary and safety endpoints, external information. Koenig et al (IMS) Adaptive Dunnett Tests for TS Workshop Adaptive Designs 06 4 / 18

5 Seamless Designs Treatment selection Flexible Trial with Treatment Selection (Phase II + III) Comparison of k treatments with a control. Treatments are selected during the trial. Selection rule can not? be laid down in detail in advance. Scientific expert knowledge not known at the planning stage will influence the decision process. Selection rule is complex and unknown at planning stage. Treatment selection may be based on un-blinded interim data on the primary efficacy endpoint, secondary and safety endpoints, external information. Koenig et al (IMS) Adaptive Dunnett Tests for TS Workshop Adaptive Designs 06 4 / 18

6 Seamless Designs Multiple type I error Treatment Selection & Multiple Testing Identify effective treatments while controlling the multiple type I error rate in the strong sense: Multiple type I error rate: The probability that at least one ineffective treatment is declared effective is controlled regardless of the number of (non-)effective treatments. Multiple testing procedures: Bonferroni, Dunnett... Test Designs with Treatment Selection Group Sequential Tests (Jennison & Turnbull, 2000) Tests with predefined selection rules (Thall et al. 1988, 1989, Stallard & Todd 2003) Combination Tests combined with the closed testing principle (Kieser et al. 1999, Bauer & Kieser 1999, Hommel 2001, Hommel & Kropf 2001,...) Koenig et al (IMS) Adaptive Dunnett Tests for TS Workshop Adaptive Designs 06 5 / 18

7 Seamless Designs Setup The Testing Problem Parallel group design with k = 2 treatments T 1 and T 2 and a control group T 0. Normal responses with common (known) variance (σ = 1). Testing the one sided hypotheses for the means at multiple level α = H 0,1 : µ 1 µ 0 = 0 vs H 1,1 : µ 1 µ 0 > 0 H 0,2 : µ 2 µ 0 = 0 vs H 1,2 : µ 2 µ 0 > 0 Koenig et al (IMS) Adaptive Dunnett Tests for TS Workshop Adaptive Designs 06 6 / 18

8 Classical Dunnett Test Treatment selection Treatment Selection with the Classical Dunnett Test Start with a classical step down Dunnett test to compare k treatments with a control. In (possibly unplanned) interim analyses treatments may be dropped based on all internal or external information. In the final analysis apply the critical boundaries of the original Dunnet test (correcting for k comparisons) to the remaining treatment control comparisons. Remarks Very Simple Approach No other adaptations (e.g. sample size reassessment) are possible The procedure is strictly conservative Koenig et al (IMS) Adaptive Dunnett Tests for TS Workshop Adaptive Designs 06 7 / 18

9 Classical Dunnett Test Closure Principle and the Dunnett Test The Closed Testing Principle with Dunnett Tests Dunnett test ϕ 12 at level α for H 0,1 H 0,2 max(z 1, Z 2) d2 YES NO H 0,1 and H 0,2 are accepted. Test ϕ 1 at level α for Test ϕ 2 at level α for Z H 0,1 H 0,2 1 d1 Z2 d1 Z i... z-score for treatment-control comparison of T i (i = 1, 2) d j... (Dunnett) critical boundary correcting for j comparisons.. Koenig et al (IMS) Adaptive Dunnett Tests for TS Workshop Adaptive Designs 06 8 / 18

10 Classical Dunnett Test Closure Principle and the Dunnett Test The Closed Testing Principle with Dunnett Tests Dunnett test ϕ 12 at level α for H 0,1 H 0,2 max(z 1, Z 2) d2 Test ϕ 1 at level α for Test ϕ 2 at level α for H 0,1 Z1 d1 H 0,2 Z2 d1 Z i... z-score for treatment-control comparison of T i (i = 1, 2) d j... (Dunnett) critical boundary correcting for j comparisons.. Koenig et al (IMS) Adaptive Dunnett Tests for TS Workshop Adaptive Designs 06 8 / 18

11 Classical Dunnett Test Closure Principle and the Dunnett Test The Closed Testing Principle with Dunnett Tests Dunnett test ϕ 12 at level α for H 0,1 H 0,2 max(z 1, Z 2) d2??? Test ϕ 1 at level α for Test ϕ 2 at level α for H 0,1 Z1 d1 H 0,2 Z2 d1 Z i... z-score for treatment-control comparison of T i (i = 1, 2) d j... (Dunnett) critical boundary correcting for j comparisons.. Koenig et al (IMS) Adaptive Dunnett Tests for TS Workshop Adaptive Designs 06 8 / 18

12 Classical Dunnett Test Closure Principle and the Dunnett Test The Closed Testing Principle with Dunnett Tests Dunnett test ϕ 12 at level α for H 0,1 H 0,2 max(z 1, Z 2 = ) d2!!! Test ϕ 1 at level α for Test ϕ 2 at level α for H 0,1 Z1 d1 H 0,2 Z2 d1 Z i... z-score for treatment-control comparison of T i (i = 1, 2) d j... (Dunnett) critical boundary correcting for j comparisons Note that d 2 > d 1. This is a strictly conservative test if only one treatment is selected! Koenig et al (IMS) Adaptive Dunnett Tests for TS Workshop Adaptive Designs 06 8 / 18

13 Procedure Treatment Selection with the Adaptive Dunnett Test Start with a classical step down Dunnett test to compare k treatments with a control. In (possibly unplanned) interim analyses treatments may be dropped based on all internal or external information. In the final analysis apply the closure principle with conditional error rates of Dunnett tests for the intersection hypotheses. Remarks The procedure controls the multiple level. Improves the Classical Dunnett test uniformly in the case of treatment selection. Further adaptations are possible Koenig et al (IMS) Adaptive Dunnett Tests for TS Workshop Adaptive Designs 06 9 / 18

14 Procedure Treatment Selection with the Adaptive Dunnett Test Start with a classical step down Dunnett test to compare k treatments with a control. In (possibly unplanned) interim analyses treatments may be dropped based on all internal or external information. In the final analysis apply the closure principle with conditional error rates of Dunnett tests for the intersection hypotheses. Remarks The procedure controls the multiple level. Improves the Classical Dunnett test uniformly in the case of treatment selection. Further adaptations are possible Koenig et al (IMS) Adaptive Dunnett Tests for TS Workshop Adaptive Designs 06 9 / 18

15 Conditional Error The Conditional Error Rate Principle Consider a test ϕ to test a hypothesis H 0. The conditional error rate principle (Müller & Schäfer 2001) At an interim analysis we compute: A(interim data) = P H0 ( ϕ rejects H0 interim data ) Continue with an independent second stage and a test for H 0 at level A(interim data). In the final analysis reject H 0 iff q A(interim data), where q is the p-value from the second stage data The resulting procedure controls the level α. Koenig et al (IMS) Adaptive Dunnett Tests for TS Workshop Adaptive Designs / 18

16 Closure principle and the conditional error The Adaptive Dunnett Test The Closed Testing Principle and Conditional Error Dunnett test ϕ 12 at level α for H 0,1 H 0,2 Test ϕ 1 at level α H 0,1 Test ϕ 2 at level α H 0,2 After selecting treatment T 1 only: reject H 0,1 iff q 1 min{a 12, A 1 } Koenig et al (IMS) Adaptive Dunnett Tests for TS Workshop Adaptive Designs / 18

17 Closure principle and the conditional error The Adaptive Dunnett Test The Closed Testing Principle and Conditional Error Dunnett test ϕ 12 at level α for H 0,1 H 0,2 max(z 1, Z 2) d2 Test ϕ 1 at level α Test ϕ 2 at level α H 0,1 H 0,2 Z1 d1 Z2 d1 After selecting treatment T 1 only: reject H 0,1 iff q 1 min{a 12, A 1 } Koenig et al (IMS) Adaptive Dunnett Tests for TS Workshop Adaptive Designs / 18

18 Closure principle and the conditional error The Adaptive Dunnett Test The Closed Testing Principle and Conditional Error Dunnett test ϕ 12 at level α for H 0,1 H 0,2 q12 A 12(interim data T 0,T 1 & T 2) Test ϕ 1 at level α Test ϕ 2 at level α H 0,1 H 0, q1 A 1(interim data T 0,T 1) q A (interim data T,T ) After selecting treatment T 1 only: reject H 0,1 iff q 1 min{a 12, A 1 } Koenig et al (IMS) Adaptive Dunnett Tests for TS Workshop Adaptive Designs / 18

19 Closure principle and the conditional error The Adaptive Dunnett Test Selecting only Treatment 1 Dunnett test ϕ 12 at level α for H 0,1 H 0,2 q 1 A 12(interim data T 0,T1 &T 2) Test ϕ 1 at level α H 0,1 q1 A 1(interim data T 0,T 1) Test ϕ 2 at level α H 0,2 q2 A 2(interim data T 0,T 2) After selecting treatment T 1 only: reject H 0,1 iff q 1 min{a 12, A 1 } Koenig et al (IMS) Adaptive Dunnett Tests for TS Workshop Adaptive Designs / 18

20 Uniform Improvement: Conditional Error Uniform improvement after selecting only T 1 Adaptive Dunnett Test A 12 = P H0 (max(z 1, Z 2 ) d 2 i.d.) A 1 = P H0 (Z 1 d 1 interim data) Reject H 0,1 iff q 1 min{a 12, A 1 } Classical Dunnett Test A 12 := P H 0 (Z 1 d 2 interim data) Rejects H 0,1 iff q 1 A 12 A 12 > A 12 A 1 > A 12 (since d 1 < d 2 ) min{a 12, A 1 } > A 12 for all interim data (i.d.) of T 0, T 1, T 2. The adaptive Dunnett test may borrow strength from the dropped treatments to test the intersection hypotheses. Koenig et al (IMS) Adaptive Dunnett Tests for TS Workshop Adaptive Designs / 18

21 Uniform Improvement: Conditional Error Uniform improvement after selecting only T 1 Adaptive Dunnett Test A 12 = P H0 (max(z 1, Z 2 ) d 2 i.d.) A 1 = P H0 (Z 1 d 1 interim data) Reject H 0,1 iff q 1 min{a 12, A 1 } Classical Dunnett Test A 12 := P H 0 (Z 1 d 2 interim data) Rejects H 0,1 iff q 1 A 12 A 12 > A 12 A 1 > A 12 (since d 1 < d 2 ) min{a 12, A 1 } > A 12 for all interim data (i.d.) of T 0, T 1, T 2. The adaptive Dunnett test may borrow strength from the dropped treatments to test the intersection hypotheses. Koenig et al (IMS) Adaptive Dunnett Tests for TS Workshop Adaptive Designs / 18

22 Uniform Improvement: Conditional Error Uniform improvement after selecting only T 1 Adaptive Dunnett Test A 12 = P H0 (max(z 1, Z 2 ) d 2 i.d.) A 1 = P H0 (Z 1 d 1 interim data) Reject H 0,1 iff q 1 min{a 12, A 1 } Classical Dunnett Test A 12 := P H 0 (Z 1 d 2 interim data) Rejects H 0,1 iff q 1 A 12 A 12 > A 12 A 1 > A 12 (since d 1 < d 2 ) min{a 12, A 1 } > A 12 for all interim data (i.d.) of T 0, T 1, T 2. The adaptive Dunnett test may borrow strength from the dropped treatments to test the intersection hypotheses. Koenig et al (IMS) Adaptive Dunnett Tests for TS Workshop Adaptive Designs / 18

23 Uniform Improvement: Conditional Error Uniform improvement after selecting only T 1 Adaptive Dunnett Test A 12 = P H0 (max(z 1, Z 2 ) d 2 i.d.) A 1 = P H0 (Z 1 d 1 interim data) Reject H 0,1 iff q 1 min{a 12, A 1 } Classical Dunnett Test A 12 := P H 0 (Z 1 d 2 interim data) Rejects H 0,1 iff q 1 A 12 A 12 > A 12 A 1 > A 12 (since d 1 < d 2 ) min{a 12, A 1 } > A 12 for all interim data (i.d.) of T 0, T 1, T 2. The adaptive Dunnett test may borrow strength from the dropped treatments to test the intersection hypotheses. Koenig et al (IMS) Adaptive Dunnett Tests for TS Workshop Adaptive Designs / 18

24 Simulation results A Simulation Study Parallel group design with k = 2 treatments with control. Normal responses with common (known) variance (σ = 1). Testing the one sided hypotheses for the means at multiple level α = Remark for simulations Maximum sample size n for each group: such that the single stage test at the unadjusted level has power 80% at standardized treatment effect of alternative µ 1 µ 0 = 0.5 (σ = 1). interim analysis after n 1 = n/2 observations per treatment group simulation runs for each parameter combination Koenig et al (IMS) Adaptive Dunnett Tests for TS Workshop Adaptive Designs / 18

25 Simulation results Selecting the treatment with larger effect at interim µ 2 µ 0 = 0.5 Koenig et al (IMS) Adaptive Dunnett Tests for TS Workshop Adaptive Designs / 18

26 Simulation results Selecting the treatment with larger effect at interim µ 2 µ 0 = 0.5 Koenig et al (IMS) Adaptive Dunnett Tests for TS Workshop Adaptive Designs / 18

27 Summary adaptive Dunnett test Adaptive Dunnett Test The Conditional error in combination with the closed test principle reduce the conservatism of the Dunnett test with treatment selection If all treatments are selected (and no other adaptations) then the adaptive Dunnett test coincides with the conventinal step down Dunnett test. One can incorporate early rejection and acceptance boundaries at interim analysis. Full flexibility Reassessment or reallocation of sample size, e.g.: reallocate the sample size of a dropped treatment equally to all treatments still under investigation. Koenig et al (IMS) Adaptive Dunnett Tests for TS Workshop Adaptive Designs / 18

28 Simulation results: Reallocation Selecting the treatment with larger effect at interim and sample size reallocation µ 2 µ 0 = 0.5 Koenig et al (IMS) Adaptive Dunnett Tests for TS Workshop Adaptive Designs / 18

29 Simulation results: Reallocation Selecting the treatment with larger effect at interim and sample size reallocation µ 2 µ 0 = 0.5 Koenig et al (IMS) Adaptive Dunnett Tests for TS Workshop Adaptive Designs / 18

30 Simulation results: Reallocation Selecting the treatment with larger effect at interim and sample size reallocation µ 2 µ 0 = 0.5 Koenig et al (IMS) Adaptive Dunnett Tests for TS Workshop Adaptive Designs / 18

31 Conlusion Conclusion: Adaptive Dunnett Test Take Home Message Start with a Dunnett Test For full flexiblity: conditional error together with the closed test Properties In principle, after every observation adaptations can be performed The interim analysis need not to be preplanned. The adaptation rules need to to be prefixed. The multiple type I error rate is controlled. Back to Reality Nobody in a clinical trial realistically would calculate the conditional error rate after every observation. However, the general principle show that an interim analysis can be performed whenever it seems appropriate without any preplanning. Koenig et al (IMS) Adaptive Dunnett Tests for TS Workshop Adaptive Designs / 18

32 Conlusion Selected References Koenig, F., Brannath, W., Bretz, F., Bauer, P., and Posch, M. Adaptive Dunnett tests for treatment selection. Working Paper Bretz F, Schmidli H, Koenig F, Racine A, Maurer W. Confirmatory seamless phase II/III clinical trials with hypotheses selection at interim: General concepts. Biometrical Journal 2006; 48(4): Hommel G. Adaptive modifications of hypotheses after an interim analysis. Biometrical Journal 2001; 43: Müller HH, Schäfer H. Adaptive group sequential designs for clinical trials: Combining the advantages of adaptive and of classical group sequential approaches. Biometrics 2001; 57: Bauer P, Kieser M. Combining different phases in the development of medical treatments within a single trial. Statistics in Medicine 1999; 18: Koenig et al (IMS) Adaptive Dunnett Tests for TS Workshop Adaptive Designs / 18

Type I error rate control in adaptive designs for confirmatory clinical trials with treatment selection at interim

Type I error rate control in adaptive designs for confirmatory clinical trials with treatment selection at interim Frank Bretz Statistical Methodology, Novartis Joint work with Martin Posch (Medical University