Multiple Testing in Group Sequential Clinical Trials

Transcription

1 Multiple Testing in Group Sequential Clinical Trials Tian Zhao Supervisor: Michael Baron Department of Mathematical Sciences University of Texas at Dallas 7/2/213

2 1 Sequential statistics Problem 2 Bonferroni approach 3 Truncated Sequential Probability Ratio Test 4

3 Sequential statistics Sequential statistics Problem Multiple testing in clinical trials Multiple endpoints ( efficacy and safety ) Multiple treatment arms or doses of a drug Overall, get an answer to several questions

4 Sequential statistics Sequential statistics Problem Multiple testing in clinical trials Multiple endpoints ( efficacy and safety ) Multiple treatment arms or doses of a drug Overall, get an answer to several questions Sequential / group sequential clinical trials Interim analysis (Flector trial: stop for futility) Sequential clinical trials always have a restriction on the maximum number of sampled groups (K)

5 Our problem Sequential statistics Problem Test multiple hypotheses based on sequentially observed data, achieving a decision on each individual test Group sequential sampling

6 Our problem Sequential statistics Problem Test multiple hypotheses based on sequentially observed data, achieving a decision on each individual test Group sequential sampling Formulation

7 Our problem Sequential statistics Problem Test multiple hypotheses based on sequentially observed data, achieving a decision on each individual test Group sequential sampling Formulation Goal

8 Group sequential methods Sequential statistics Problem Group sequential sampling A number of group sequential procedures have been proposed for testing ONE hypothesis. Pocock (1977) applies repeated significance tests to group sequential trials with equal-size groups and derives a constant critical value on the standardized normal Z scale across all stages that control Type I error. O Brien and Fleming (1979) propose a sequential procedure that has boundary values decrease over the stages on the standardized normal Z scale

9 Formulation Sequential statistics Problem Observe a sequence of iid random vectors X n, n = 1, 2,... X n = (X (1) 1,...X n (d) ) R d is a set of measurements on unit n. X (j) i f j (x θ (j) ) Multiple hypotheses Test d hypotheses: H (j) : θ (j) = θ (j) vs H (j) A : θ(j) = θ (j) 1, for j = 1, 2,..., d Let m = constant be the group size and T be a stopping time Tests are based on a group sequential experiment. Data are collected in groups of m patients.

10 Goal Sequential statistics Problem Control Familywise Type I error rate = FWER-I = P( At least one Type I error ) α Familywise Type II error rate = FWER-II = P( At least one Type II error) β P(T K) = 1, T is stopping time, K = given integer = max allowed number of groups Do it at low expected cost

11 Bonferroni approach Bonferroni approach Bonferroni approach : control of both error rates. Choose α j (, 1), αj = α; β j (, 1), βj = β Test H (j) at level α j and power (1 β j ) by any known sequential procedure d P j=1 d P j=1 Type I error on H (j) Type II error on H (j) A ( P = α j = α ( P = β j = β Type I error on H (j) Type II error on H (j) A ) )

12 of Bonferroni Methods Bonferroni inequality is very crude for moderate to large d, extend to Holm s stepwise approach. Order the log-likelihood ratio (LLR) statistics in decreasing (step-down)/increasing (step-up) Choose stopping boundaries for the ordered LLR Define the stopping rule and decision rules This requires analysis of the group sequential test Boundaries of Pocock, O Brien-Fleming, Wang-Tsiatis tests are computed numerically, not tractable to analyze Derive equations for Whitehead triangular test and Truncated SPRT

13 Group sequential tests Z k Pocock test Z k O Brien-Fleming test k k Z k Whitehead test k Λ k Truncated SPRT k

14 Derive Truncated SPRT SPRT Choose stopping boundaries a, b to control α and β where, a = 1 β α b = β 1 α SPRT control both α, β among all sequential tests that have P(Type I error) α P(Type II error) β

15 Wald s SPRT a Λ k R e j e c t H k T b A c c e p t H

16 Truncated SPRT Stopping time and Decision rule Following SPRT approach for single hypothesis testing, truncated SPRT is based on log-likelihood rations Stopping time Λ n = ln f (x θ 1) f (x θ ) T = min(k, min {n : Λ n / (b, a)}) K Decision rule, { reject H if Λ δ = T a or Λ T c T = K accept H if Λ T b or Λ T < c T = K

17 Truncated SPRT (cont d) a c b Λ k R e j e c t H K A c c e p t H A c c e p t H Reject H k

18 Truncated SPRT (cont d) Plan Evaluate performance of TSPRT, probabilities of Type I and Type II errors. Find equations of the stopping boundaries and the required group size to control P{Type I error} and P{Type II error}. Then develop Holm-type stepwise sequential procedures to test d hypotheses controlling familywise error rates I and II.

19 Truncated SPRT. Performance evaluation. a c b Evaluate probabilities of Type I and Type II errors by comparison between TSPRT and SPPRT. Λ k Both tests reject H R e j e c t H k K A c c e p t H

20 Truncated SPRT. Performance evaluation. a c b Λ k R e j e c t H TSPRT rejects H but SPRT accepts it k K A c c e p t H

21 Truncated SPRT. Performance evaluation. a Λ k R e j e c t H Both tests accept H c b k K A c c e p t H

22 Truncated SPRT. Performance evaluation. a c b Λ k TSPRT accepts H R e j e c t H but SPRT rejects it k A c c e p t H K

23 Probability of Type I Error Choose c to attain P(Type I error TSPRT ) α. Let Λ K = Km j=1 ln f (x j θ 1 ) f (x j θ ). For a given α 1=level for SPRT, P(Type I error TSPRT ) = P H (Type I error SPRT ) +P H ({Λ 1,...Λ K 1 (b, a)} {Λ K [c, a)} {Λ n b before Λ n a}) P H ({Λ 1,...Λ K 1 (b, a)} {Λ K (b, c)} {Λ n a before Λ n b}) α 1 + P(A) P(B) We need to make P(A) α α 1

24 Type I error P(A) P H (Λ K [c, a))p H (Λ n b before Λ n a Λ K c) P H (Λ K c)p H (Λ n b before Λ n a Λ K c) P H (Λ K c)p H (Λ n b before Λ n a Λ K = c)

25 Truncated SPRT (cont d) Derivations Assumption: ( General ) Exponential family Test H : θ = θ vs H A : θ = θ 1. Exponential family is written in the canonical form with η = canonical parameter as follows, f (x η) = f (x )e ηx ψ(η) where, ψ() = and µ(η) = EX = ψ (η), VarX = ψ (η)

26 Truncated SPRT (cont d) Chernoff Inequality In general case, we have P( n i=1 Z j ny) e n sup t {ty ψz (t)} here, Z = ln f (x θ 1) f (x θ ), c = ny

27 Truncated SPRT (cont d) Normal as example Observe X 1, X 2,..., X n N(θ, σ 2 ), we can write it as f (x θ, σ 2 ) = 1 2πσ e x2 2σ 2 e θx σ 2 θ2 2σ 2 here, η = θ σ 2, f (x ) = 1 2πσ e x2 2σ 2, ψ x (η) = σ2 η 2 2

28 Truncated SPRT (cont d) Normal as example By Chernoff Inequality, Therefore, sup{ty ψ z (t)} attained at ˆt = [ψ z] 1 (y) t P( n Z j ny) e n{ i=1 σ 2 y 2 2(θ 1 θ ) y+ (θ 1 θ )2 8σ 2 }

29 Type I error P(A) P H (Λ K [c, a))p H (Λ n b before Λ n a Λ K c) Hence, we have P H (Λ K c)p H (Λ n b before Λ n a Λ K c) P H (Λ K c)p H (Λ n b before Λ n a Λ K = c) P(A) 1 α 1e c 1 α 1 β e n{ σ 2 c 2 2n 2 (θ 1 θ ) 2 + c 2n + (θ 1 θ )2 8σ 2 } α α1

30 Type II error For α 1 =level for SPRT (, α), β 1 = power for SPRT (, β) Let a = ln α 1, and b = ln β 1 Choose c 1 to control Type I error: 1 α 1 e c 1 e n{ 1 α 1 β 1 σ 2 c 2 1 2n 2 (θ 1 θ ) 2 + c 1 2n + (θ 1 θ )2 8σ 2 } α α1 Choose c 2 to control Type II error: 1 β 1 e c 2 e n{ 1 α 1 β 1 σ 2 c 2 2 2n 2 (θ 1 θ ) 2 c 2 2n + (θ 1 θ )2 8σ 2 } β β1

31 Type II error Therefore, if θ 1 θ σ c 1 >, we have, ( ) 2 n ln (α α 1)(1 α 1 β 1 ) (θ 1 θ ) n(θ 1 θ ) 2σ σ ( ) 2 c 2 n ln (β β 1)(1 α 1 β 1 ) + (θ 1 θ ) 2σ Minimize n to control both Type I error and Type II error n(θ 1 θ ) σ ( 2σ 2 ln (α α1 )(1 α 1 β 1 ) + ln (β β 1 )(1 α 1 β 1 ) n = (θ 1 θ ) 2 ) 2

32 Stepwise method for Truncated SPRT Test d hypotheses: H (j) : θ (j) = θ (j) vs H (j) A : θ(j) = θ (j) 1 j = 1, 2,..., d Test statistics Λ (j) k = ln f j (X (j) i θ (j) 1 ) f j (X (j) i θ (j) ) Order test statistics Λ [1] k... Λ [d] k, let H(j) be the corresponding tested null hypotheses arranged in the same order. a j = ln α α 1 d j+1, b j = ln β β 1 j

33 Stepwise method for Truncated SPRT(cont d) Decision rule (reject one by one, accept all) If Λ [1] k b 1 stop, accept H [1],..., H[d] If Λ [1] k a 1 reject H [1], go to H[2] If Λ [1] k (b 1, a 1 ), continue sampling...

34 Stepwise method for Truncated SPRT(cont d) At k = K, If Λ [j] K / (b j, a j ), stop If Λ [j] K (b j, a j ), Λ [j] K > c j stop, reject Λ [j] K < c j stop, accept This stepwise method can control FWER-I and FWER-II

35 Pocock Test Pocock derives the constant boundary [ C p, C p ] on the standardized Z scale, if Z / [ C p, C p ], stop and reject H, otherwise, keep sampling, after K groups, if Z / [ C p, C p ], stop and reject H, otherwise, stop and accept H Fallback method for Pocock: recycle β j on all H (j) that were rejected, then at k = K, test remaining H (j) at higher β j β j (1) β j (K) = 1 i:h (i) was rejected β i Testing with higher β means that we can accept more often, expand the acceptance region. P(reject H ) reduces, we can shrink boundary [ C p, C p ] to meet the α level, and then reduce m.

36 Use stepwise approach to multiple hypotheses starting from various standard group sequential methods Thank you!

37 Use stepwise approach to multiple hypotheses starting from various standard group sequential methods Evaluate performance of fallback procedures, compare fallback and stepwise testing approaches Thank you!

38 Use stepwise approach to multiple hypotheses starting from various standard group sequential methods Evaluate performance of fallback procedures, compare fallback and stepwise testing approaches Composite hypotheses and nuisance parameters. Sequential t-test. Thank you!

39 Use stepwise approach to multiple hypotheses starting from various standard group sequential methods Evaluate performance of fallback procedures, compare fallback and stepwise testing approaches Composite hypotheses and nuisance parameters. Sequential t-test. Examples of clinical data Thank you!