Overrunning in Clinical Trials: a Methodological Review

Transcription

1 Overrunning in Clinical Trials: a Methodological Review Dario Gregori Unit of Biostatistics, Epidemiology and Public Health Department of Cardiac, Thoracic and Vascular Sciences dario.gregori@unipd.it Nicola Soriani, Ileana Baldi Unit of Biostatistics, Epidemiology and Public Health Department of Cardiac, Thoracic and Vascular Sciences Beatrice Barbetta, Paola Vaghi, Giampaolo Giacovelli Rottapharm Madaus Paola Berchialla Department of Clinical and Biological Sciences, University of Torino

2 Overrunning in group sequential trials (GSD) or Adaptive Designs (AD) occurs when data continue to be collected also if a stopping criterion has been reached at IA n 1 n 2 t 0 t 1 t 2 Pre-planned I Interim Analysis t 1+ Pre-planned II Interim Analysis n + Analysis Starts Stop Decision is taken Reasons the time delay between the subject recruitment and the actual evaluation of the primary outcome. sponsor s decision to create more robust safety database

3 If a trial is to be terminated as a result of an interim analysis it is always important to carry out an additional analysis including all of these further patients that did not contribute to the interim analysis. It may be that when this analysis is carried out, the null hypothesis can no longer be rejected and apparently decision making may depend on whether or not these so called overrunning patients are included or excluded from the analysis. In such a situation, it is accepted regulatory practice to base decision making on the final results of the trial (not the interim analysis). This is also in accordance with the intention to treat principle that all randomised patients should be analysed.

4 General notation A two-arms parallel trial, experimental (E) and a control (C) treatment groups. The advantage on a major endpoint between E and C is expressed by a parameter θ type I (α) and type II (β) errors The power is given by 1 β. H 0 effect is absent (θ = 0) H 1 there s an effect (θ 0). At the k th IA, the response on the primary endpoint is summarized by the pair of statistics (Z k, V k ). Z k represents the efficient score statistics for the superiority of E respect to C, and V k denotes the Fisher s observed information. It is assumed that, conditional to V k Z k ~N θv k, V k.

5 Overrunning In Group Sequential Design/Adaptive, stopping criterion is determined by group sequential test, where the sequence of p-values for (Z 1, Z 2, Z K ), computed recursively by Fairbacks and Madsen ordering and according to the trial design, are compared with respect to an opportune sequence (α 1, α 2,, α K ) of significance levels, chosen to control the type I error probability. When overrunning occurs after that the trial is stopped at the k th IA, then the analysis that includes the overrunning data will have associated the statistics Z k+1 and V k+1. The contribution of the overrunning data to the analysis could be determined by the quantities Z O = Z k+1 Z k and V O = V k+1 V k.

6 Deletion Method Whitehead (1992) The IA that led to fulfillment stopping criterion is deleted and replaced with a new one that includes the overrunning data. This means that if the trial reached the stopping criterion at the k th IA, then deletion method reduces to re-compute the p-value on the statistics (Z k+1, V k+1 ) instead of on the statistics (Z k, V k ).

7 Combining p-values Hall 2008 The method of combining p-values distinguishes and analyzes separately the sequential (until recruitment termination) and the overrunning portions of the trial. P θ = 1 Φ w 1 Φ 1 P T (θ) + w 2 Φ 1 P O (θ) Random weights: are related to the Fisher s information in the two portions of data Fixed weights are related to the expected value under the null hypothesis of the sample sizes in the sequential and overrunning portions of the trial. This choice of weights requires that the value of the expected overrunning size should be expressed a- priori, in the trial design description Observed weigths where the observed overrunning size substitutes its expected value w 1 = w 1 = w 1 = V T V T + V O E[n T ; H 0 E[n T ; H 0 + E[n O ; H 0 E[n T ; H 0 E[n T ; H 0 + n O

8 Repeated Confidence Interval Jenninson and Turnbull (1989) The repeated confidence intervals (RCIs) method, for a trial with K IAs, leads to a 1 α level sequence of confidence intervals I k : k = 1,, K for the parameter θ, where each I k is built from the information available at analysis k P θ θ I k : k = 1,, K = 1 α. If the study is stopped at the k th analysis and overrunning occurs then the repeated confidence interval (I k ) is recomputed considering also the overrunning portion of the data. I k = θ 0 : Z k θ 0 V k < c k V k Z k c k V k = ; Z k + c k V k, V k V k the stopping criterion is to stop the study at the k th analysis if θ 0 I k θ

9 Simulated Studies Superiority Trial Superiority of an experimental calcium channel blocker over a placebo control in the immediate treatment of patients accusing an acute ischemic stroke (ASCLEPIOS). The trial is designed assuming a reduction in 90- day death rate from 15% in the control arm to 9% in experimental treatment arm, corresponding to a log-odds ratio of θ = Two sided alpha at 0.05 level. According to O Brien and Fleming sequential test with three equally-spaced IAs, a sample size of 1248 subjects (416 for each IA and 624 for treatment arm) is needed to ensure a power of 0.9, given a two-sided 0.05 level. First Interim planned at 299. Non Inferiority Trial Multicenter phase III trial. The trial was designed as a double-blind, randomized, parallel-group study for noninferiority of a Test drug compared to a Control drug. The target was an increase in success rate from 45% in control to 50% in test drug, with a non-inferiority margin of 15%, that corresponds to a log-odds ratio θ= The power was set to 0.80 to detect θ= 0.20 as significant at one-side 2.5% level. The sample size of 198 (99 patients in each treatment group was obtained by O Brien and Fleming design. First Interim planned at 66 UBSEPH

10 Simulation Study First IA Superiority n 1 =229 Non Inferiority n 1 =66 Rejection close to boundary n 0 additional patients 5%, 10%, 15%, 20%, 25%, 50%, 75% of n 1 Under H 0 and under H 1 Expected confirmation rate after OR analysis Maximum P-value at Interim compatible with a pre-specified confirmation rate

11 Superiority Trial H 0 is rejected at first IA (n 1 =229) OR data simulated under H 0 % OR Deletion Fixed RCI Weights 5% % % % % % % Confirmation rates (%) of decision taken at first IA UBSEPH

12 Superiority Trial H 0 is rejected at first IA (n 1 =229) OR data simulated under H 1 % OR Deletion Fixed RCI Weights 5% % % % % % % Confirmation rates (%) of decision taken at first IA

13 Superiority Trial H 0 is rejected at first IA (n 1 =229) OR data simulated under H 1 O'Brien and Fleming alpha for stopping % OR 5% 10% 15% 20% 25% 50% 75% Deletion 3.99E E E E E E E-05 Fixed W RCI 2.26E E E E E E E-05 Maximum p-values requested at IA to ensure a 90% chances of trial stopping confirmation after OR UBSEPH

14 Superiority Trial Robustness of Fixed weight method for different a-priori choices of Expected OR rates vs Observed OR Superiority Trial H 0 is rejected at first IA (n 1 =229) OR sample in favour of H 0 OR sample in favour of H 1 5% 10% 15% 20% 25% 50% 5% 10% 15% 20% 25% 50% Fixed 5% Fixed 10% Fixed 15% Fixed 20% Fixed 25% Fixed 50% Confirmation rates (%) of decision taken at first IA

15 Non-Inferiority Trial H 0 is rejected at first IA (n 1 =66) OR data simulated under H 0 % OR Deletion Fixed Weights RCI 5% % % % % % % Confirmation rates (%) of decision taken at first IA UBSEPH

16 Non-Inferiority Trial H 0 is rejected at first IA (n 1 =66) OR data simulated under H 1 % OR Deletion Fixed Weights RCI 5% % % % % % % Confirmation rates (%) of decision taken at first IA

17 Non Inferiority Trial H 0 is rejected at first IA (n 1 =66) OR data simulated under H 1 O'Brien and Fleming alpha for stopping % OR 5% 10% 15% 20% 25% 50% 75% Deletion 1.82E E E E E E E-05 Fixed Weights E RCI 1.82E E E E E E E-05 Maximum p-values requested at IA to ensure a 90% chances of trial stopping confirmation after OR UBSEPH

18 Non Inferiority Trial Robustness of Fixed weight method for different a-priori choices of Expected OR rates vs Observed OR Non Inferiority Trial H 0 is rejected at first IA (n 1 =66) OR sample in favour of H 0 OR sample in favour of H 1 5% 10% 15% 20% 25% 50% 5% 10% 15% 20% 25% 50% Fixed 5% Fixed 10% Fixed 15% Fixed 20% Fixed 25% Fixed 50% Confirmation rates (%) of decision taken at first IA

19 Operational issue: OR analysis is performed knowing the result of the trial. Source of bias? All methods biased toward the NULL Higher sensitivity to OR sample Deletion method RCI Lower sensitivity to OR sample Fixed weights Robust to a-priori scenarios