August 01, 2012
Disclaimer: This presentation reflects the views of the author and should not be construed to represent the views or policies of the U.S. Food and Drug Administration
Introduction We describe a new class of multiple testing procedures for addressing multiplicity problems arising in clinical trials with multiple objectives grouped into families. The families may correspond to equally important sets of objectives, e.g., co-primary endpoints, or ordered sets of objectives, e.g., primary and secondary endpoints. The procedures, termed superchain procedures, serve as an extension of several classes of other multiple testing procedures, including chain procedures and parallel gatekeeping procedures.
Example 1: Two-family problem Clinical trial where experimental treatment is evaluated versus placebo at two dose levels (Dose 1, Dose 2) with respect to two endpoints (Endpoint 1, Endpoint 2). Four null hypotheses grouped into 2 families. Family Hypotheses Endpoint Comparison F 1 H 1 Endpoint 1 Dose 1 vs Placebo F 1 H 2 Endpoint 1 Dose 2 vs Placebo F 2 H 3 Endpoint 2 Dose 1 vs Placebo F 2 H 4 Endpoint 2 Dose 2 vs Placebo If two families may be treated as co-primary families, they can be tested simultaneously rather than sequentially.
procedure for two families Hypothesis testing problem with n null hypotheses grouped into 2 families: F 1 and F 2. The non-negative weights w 1 and w 2 with w 1 + w 2 =1 are assigned to F 1 and F 2 to quantify relative importance of the two families. At each step of the procedure, families F i, i =1, 2are tested simultaneously by component procedures P i at the α i level (to be specified). Component procedures P i, i =1,...,2, are derived using the closure principle and control the local FWER within F i.
procedure for two families At Step 1, families F i are tested at respective significance levels α i = w i α. Whenever one or more null hypotheses are rejected in a family, a certain fraction of FWER is released and can be transferred to other family. The transition parameters g ij 0 specify how the error rate is distributed. w 1 F 1 g 12 g 21 w 2 F 2
procedure for two families Truncated component procedures Assume that P i are truncated Hochberg (Holm or Hommel) procedures with truncation parameters 0 γ i 1. Truncated procedure is defined by computing a convex combination of the critical values of the original procedure with the critical values of the Bonferroni procedure based on the truncation parameter. Power of truncated procedures P i increases monotonically with the increase of γ i. Denote by γ i,0 the initial values of the truncation parameters for P i
procedure for two families Monotonicity (power pumping) Each null hypothesis that is rejected in F j, j i pumps the significance level for testing F i. The component procedures used in family F i, i =1, 2at each step of the algorithm, depend on the set of rejected null hypotheses in the other family denoted by R j, j i: P 1 P 1 [R 2 ], P 2 P 2 [R 1 ]. The truncation parameters γ i of P i grow monotonically as more null hypotheses are rejected in F j, j i. Thus, more powerfull tests are applied to all families at subsequent steps and the sets of testable hypotheses shrink from one step to the next.
Example 1 (continued) Clinical trial with four null hypotheses H 1, H 2, H 3, H 4 grouped into two families. Components are Hochberg based truncated Hochberg procedures Let 0 <γ i,0 < 1 denote the initial values of the truncation parameters for P i s Family Hypotheses Component Initial γ i F 1 H 1 H 2 Hochberg(γ 1 ) γ 1,0 =1/2 F 2 H 3 H 4 Hochberg(γ 2 ) γ 2,0 =1/2
Example 1 Consider case of equally weighted endpoints (Endpoints 1 and 2 are equally important): w 1 =1/2, w 2 =1/2. The families are logically related through the following graph with the transition weights g 12 and g 21 equal to 1. 1 2 F 1 1 1 1 2 F 2
Example 1 Decision rules for truncated Hochberg Let ordered p-values in F 1 be p (1) < p (2). Hochberg (γ 1 ) rejects intersection hypothesis H 1 H 2 locally if p (1) < (γ 1 /2+(1 γ 1 )/2)α 1 = α 1 /2 or p (2) < (γ 1 +(1 γ 1 )/2)α 1 =(1+γ 1 )α 1 /2. Thus, within family F 1, both null hypotheses are rejected if p (2) (1 + γ 1 )α/2; only H (1) is rejected if p (1) α/2 andp (2) > (1 + γ 1 )α/2.
Example 1 Suppose the raw p-values for the null hypotheses are given by Hypotheses H 1 H 2 H 3 H 4 p-values 0.0057 0.0110 0.0021 0.0202 The global FWER to be controlled at a one-sided α =0.025
Example 1 Step 1. Test F 1 by Hochberg(1/2) at level α 1 = α/2 =0.0125. Since p 1 α 1 /2, and p 2 > (1 + γ 1,0 )α 1 /2=3α 1 /4, only H 1 is rejected, and, thus, R 1 = {H 1 }. Test F 2 by Hochberg(1/2) at level α 2 = α/2 =0.0125. Since p 3 α 2 /2, and p 4 > (1 + γ 2,0 )α 2 /2=3α 2 /4, only H 3 is rejected. Thus, R 2 = {H 2 }.
Example 1 Updating significance levels, and truncation parameters ( α 1 = κ 1 α = w 1 + (1 γ ) 2,0) R 2 w 2 α = 5α n 2 8 Similarly, γ 1 = w 1 κ 1 γ 1,0 + κ 1 w 1 κ 1 1= 3 5. α 2 = 5α 8, γ 2 = 3 5.
Example 1 Step 2. Retest F 1 by Hochberg(3/5) at significance level α 1 =5α/8 =0.0156. Since p 2 < (1 + γ 1 )α 1 /2=0.0125, both, H 1 and H 2 are rejected. Thus, R 1 = {H 1, H 2 }. Retest F 2 by Hochberg(3/5) at level α 2 =5α/8 =0.0156. Since p 4 > (1 + γ 2 )α 2 /2=0.0125, no new rejections in F 2,andR 2 = {H 3 }.
Example 1 Step 3. Since all hypotheses in F 1 are rejected, retest F 2 by Hochberg(1) at full significance level α = 0.025. p 4 < 0.025, and, thus, both hypotheses are rejected in F 2 The final set of null hypotheses rejected by the superchain procedure includes H 1, H 2, H 3, H 4.
procedure for multi-family problem Hypothesis testing problem with n null hypotheses grouped into m families: F 1,...,F m. Logical restrictions or connections among families F i are defined by a directed graph G where each node corresponds to exactly one family, and each directed edge represents a connection. The families F i are connected with each other to account for clinically relevant relationship among the families.
Graph G m nodes (one node for each family) w i 0 - family/node weights with w i 1, i g ij 0 - connection/edge weights with g ij 1. j
Example of a three-family graph w 1 F 1 g 21 g 13 g 12 g 31 w 2 g w 3 23 F 2 F 3 g 32
Overview At each step of the procedure, families are tested by component procedures. As long as the number of rejected hypotheses increases, the method allows to iterate retesting of the families with increasingly more powerful component procedures. If no additional null hypotheses are rejected at a step, the algorithm stops
Strong control of type I error rate Proposition The superchain procedure controls overall type I error rate at level α. Shown by constructing dominating closed testing procedure (mixture procedure) where each intersection is tested at an error rate at most α. Every null hypothesis that is rejected by the superchain procedure is also rejected by the corresponding mixture procedure.
Bretz, F., Maurer, W., Brannath, W., Posch, M. (2009). A graphical approach to sequentially rejective multiple test procedure. Statistics in Medicine. 28, 586-604., A., Tamhane, A.C., Wiens, B. (2008). General multistage gatekeeping procedures. Biometrical Journal. 50, 667-677., A.,, G., Tamhane, A.C. (2011). Multistage and mixture parallel gatekeeping procedures in clinical trials. Journal of Biopharmaceutical Statistics. Volume 21, Issie 4, 726-747.