4. Issues in Trial Monitoring

Transcription

1 4. Issues in Trial Monitoring 4.1 Elements of Trial Monitoring Monitoring trial process and quality Infrastructure requirements and DSMBs 4.2 Interim analyses: group sequential trial design 4.3 Group sequential design families 4.4 Frequentist evaluation of sequential trials 4.5 On the use of stochastic curtailment What is stochastic curtailment? (a) Stopping decisions based on conditional power (b) Stopping decisions based on predictive power Example: sepsis trial 4.6 Implementing a monitoring plan 4. Trial Monitoring 4.5 Stochastic Curtailment 3 April / 18 1

2 4.5 On the use of stochastic curtailment: What is it? Consider the sepsis trial: Suppose you observe a 5% higher mortality rate with antibody treatment at the first interim analysis (ˆθ = 0.05). You are considering stopping for lack of effect. It might seem relevant to ask: if I were to continue, would I change my mind? More formally, what is the probability of reversing the current decision: P θ (ˆθ J < d J ˆθ 1 = 0.05)? This is a form of power. If it is small, then you should stop. The probability of changing your mind based on the data is known as stochastic curtailment. There are two types of approaches: (a) Conditional power: Power based on some relevant value for θ. (b) Predictive power: Mean power integrated over a posterior distribution for θ Trial Monitoring 4.5 Stochastic Curtailment 3 April / 18

3 4.5(a) Conditional power (Example): Recall the sepsis trial: According to the monitoring plan, the trial would stop for benefit if ˆθ J < d J = at N J = 1700 patients. If ˆθ 1 = 0.05, then what is the chance (power) that ˆθ J < d J? If we observe ˆθj = x, then the increment between the jth and Jth analysis is normally distributed: N J ˆθJ N j x N [(N J N j )θ, (N J N j )V ] where V = = Trial Monitoring 4.5 Stochastic Curtailment 3 April / 18 3

4 4.5(a) Conditional power (Example): It follows that: P θ (ˆθ J < d J ˆθ j = x) < α N J d J N jx (N J N j )θ) < z α (NJ N j )V ] x > N 1 j [N J d J (N J N j )θ z α NJ N j That is, we would be very unlikely (probability less than α) to reverse a futility decision if ˆθ j > d j where: ] d j = N 1 j [N J d J (N J N j )θ z α NJ N j 4. Trial Monitoring 4.5 Stochastic Curtailment 3 April / 18 4

5 4.5(a) Conditional power (Example): Suppose in the sepsis trial we used the following stopping criteria: Final critical value is d J = Decide futility as long as the power for changing our mind is smaller than 10% (z α = 1.28). Calculate this power under θ = 2 d J = The following futility stopping criteria result from the above calculations: d 1 = d 2 = d 3 = d 4 = Notice the degree conservatism. Is it ethical? 5 4. Trial Monitoring 4.5 Stochastic Curtailment 3 April / 18

6 4.5(a) Conditional power (Example): Suppose we instead use zα = 0; i.e., a 50% chance of changing our mind. Then: d 1 = d 2 = d 3 = d 4 = Notice the degree conservatism. Is it ethical? Do these stopping rules look familiar? Is it reasonable to use z α = 0? 4. Trial Monitoring 4.5 Stochastic Curtailment 3 April / 18 6

7 4.5(a) Conditional power (design family): It is instructive to construct a conditional futility design family in the standardized scale: V/NJ Standardized mean: δ = θ θ 0 Standardized statistic: ˆδj N (δ, 1 Decision criteria: Π j ) δ j > d j Decide for superiority δ j < a j Decide for lack of superiority Let d J = a J = G. 4. Trial Monitoring 4.5 Stochastic Curtailment 3 April / 18 7

8 4.5(a) Conditional power (design family): Conditional power decision rules: Stop for efficacy if P δ=0 (ˆδ J < G δ j = d j ) = α, which implies d j = (G + z 1 α 1 Πj )Π 1 j Stop for futility if P δ=δ+ (ˆδ J > G δ j = a j ) = α, which implies: a j = 2G (G + z 1 α 1 Πj )Π 1 j As in other families, G can be selected to control operating characteristics (see R-code in file lctsec4-5.r). 4. Trial Monitoring 4.5 Stochastic Curtailment 3 April / 18 8

9 4.5(a) Conditional power (design family): Example: Suppose Π j = 0.25, 0.5, 0.75, 1.0. The values of G satisfying: are: J P (ˆδ j > d j δ = 0) = (type I error) j=1 J P (ˆδ j < a j δ = 2G) = (type II error) j=1 Values of d j for selected α Π j α = 0.10 α = 0.35 α = Trial Monitoring 4.5 Stochastic Curtailment 3 April / 18

10 4.5(a) Conditional power (design family): Stopping boundaries: CF.10 CF.35 CF.50 Difference in Means Sample size Trial Monitoring 4.5 Stochastic Curtailment 3 April / 18

11 4.5(a) Conditional power (design family): Sample Size: Average Sample Size 75th percentile Sample Size CF.10 CF.35 CF.50 Sample Size CF.10 CF.35 CF Difference in Means Difference in Means 4. Trial Monitoring 4.5 Stochastic Curtailment 3 April / 18 11

12 4.5(a) Conditional power (design family): Power: CF.10 CF.35 CF.50 Power (Upper) Theta Trial Monitoring 4.5 Stochastic Curtailment 3 April / 18

13 4.5(a) Conditional power (Sepsis example) Comparing inference at lower (efficacy) boundary: Design: DSMB2 Bias-adjusted inference IA (N) a j ˆδj 95% CI p-value 1 (N = 425) (-0.228, ) (N = 850) (-0.132, ) (N = 1275) (-0.097, ) (N = 1700) (-0.086, ) Design: Conditional Futility (α = 0.1): Bias-adjusted inference IA (N) a j ˆδj 95% CI p-value 1 (N = 425) (-0.319, ) (N = 850) (-0.168, ) (N = 1275) (-0.114, ) (N = 1700) (-0.085, ) Trial Monitoring 4.5 Stochastic Curtailment 3 April / 18

14 4.5(a) Conditional power (Sepsis example) Comparing inference at upper (futility) boundary: Design: DSMB2 Bias-adjusted inference IA (N) d j ˆδj 95% CI p-value 1 (N = 425) ( 0.003, 0.141) (N = 850) (-0.060, 0.044) (N = 1275) (-0.079, 0.010) (N = 1700) (-0.086, ) Design: Conditional Futility (α = 0.1): Bias-adjusted inference IA (N) d j ˆδj 95% CI p-value 1 (N = 425) ( 0.096, 0.234) (N = 850) (-0.022, 0.083) (N = 1275) (-0.060, 0.028) (N = 1700) (-0.085, ) Trial Monitoring 4.5 Stochastic Curtailment 3 April / 18

15 4.5(a) Conditional power (Summary) When compared with an O Brien-Fleming design, the conditional power family has the following characteristics: Nearly identical power Much lower stopping probability at early interim analyses Much larger ASN (loss of efficiency) Extreme conservatism in making any early decision Resulting questions: Is it ethical to continue in the presence of overwhelming evidence of harm (or benefit)? Does conditional power obfuscate the essential clinical/scientific issues in deciding whether to terminate a trial? 4. Trial Monitoring 4.5 Stochastic Curtailment 3 April / 18 15

16 4.5(a) Conditional power (Summary) Note: There are fixes for the problems with conditional power: Calculate conditional power under a different value of δ; e.g.,: Calculate P δ=δ+ /2(ˆδ J < G ˆδ j = d j) instead of P δ=0 (ˆδ J < G ˆδ j = d j). Calculate P δ=δ+ /2(ˆδ J > G ˆδ j = a j) instead of P δ=δ+ (ˆδ J > G ˆδ j = a j). Calculate conditional power under the MLE ˆδj : Calculate P δ=ˆδj (ˆδ J < G ˆδ j = d j) instead of P δ=0 (ˆδ J < G ˆδ j = d j). Calculate P δ=ˆδj (ˆδ J > G ˆδ j = a j) instead of P δ=δ+ (ˆδ J > G ˆδ j = a j). (Consider predictive power.) 4. Trial Monitoring 4.5 Stochastic Curtailment 3 April / 18 16

17 4.5(b) Predictive power Conditional power relied on a specific choice for δ: Efficacy: Pδ=0 (ˆδ J < G ˆδ j = d j ). Futility: P δ=δ+ (ˆδ J > G ˆδ j = a j ). Predictive power integrates over a posterior distribution for δ: Let λ0 (δ) denote a prior distribution for δ Let λ(δ ˆδ j ) denote the posterior distribution of δ at the jth interim analysis. Predictive probability: P (ˆδ J > G ˆδ j ) = P δ=u (ˆδ J > G ˆδ j )λ(u ˆδ j )du u Decision criteria can be defined based on the magnitude of the predictive probability. 4. Trial Monitoring 4.5 Stochastic Curtailment 3 April / 18 17

18 4.5 Concluding remarks There are foundational issues with both conditional and predictive power. Neither frequentist nor Bayesian foundations construct inference around the probability of changing our mind (see paper). Underlying principles illustrated with this discussion of stochastic curtailment: All designs can be expressed as conditions on the observed estimate. Always consider statistical inference on the boundary when evaluating stopping criteria. 4. Trial Monitoring 4.5 Stochastic Curtailment 3 April / 18 18