Optimizing trial designs for targeted therapies - A decision theoretic approach comparing sponsor and public health perspectives

Transcription

1 Optimizing trial designs for targeted therapies - A decision theoretic approach comparing sponsor and public health perspectives Thomas Ondra Sebastian Jobjörnsson Section for Medical Statistics, Medical University of Vienna Chalmers University of Technology Seminar der Wiener Biometrische Sektion der Internationalen Biometrischen Gesellschaft Region Österreich Schweiz Vienna, 11 February 2016 Thomas Ondra, Sebastian Jobjörnsson, Robert Beckman, Carl-Fredrik Burman, Franz König, Nigel Stallard and Martin Posch This project has received funding from the European Union's 7th Framework Programme for research, technological development and demonstration under the IDEAL Grant Agreement no , and the InSPiRe Grant Agreement no / 33

2 Framework Full Population F Subgroup S Overall treatment eect Complement S = F \ S δ F = δ S + (1 )δ S where is the prevalence of subgroup S. We assume δ S δ S. Allows for investigating the hypotheses H F : δ F 0 and H S : δ S 0. 2 / 33

3 Enrichment, classical and stratication design Enrichment Design: Randomize only patients of subgroup S (say Biomarker +). Patients of the complement S are excluded from the trial (Biomarker ). Classical Design: Recruit from the full population F. No Biomarker is determined. Stratication Design: Include Biomarker + and Biomarker patients. Stratify randomization by biomarker status. With the enrichment design one can test H S, i.e., for a treatment eect in the subpopulation. With the classical design one can test H F. With the stratication design one can test H S and H F. 3 / 33

4 Testing problems Parallel group comparison of the means of normal distributions. Enrichment Design: Classical Design: Stratication Design: Test H S with a z-test. H F with a z-test. Test H S and H F with a closed test, based on the Spiessens-Debois test for testing the global null hypothesis H F H S (Song and Chi, 2007; Spiessens and Debois, 2010). 4 / 33

5 Testing strategy in the stratied design The stratied design allows for investigating H S and H F. Closed Testing principle If η = (η HS, η HF, η HS H F ) are local level alpha tests for H = {H S, H F, H S H F }, then the closed test ψ S = min{η HS, η HS H F } and ψ F = min{η HF, η HS H F } controls the FWER in the strong sense. Local level α tests: Test statistic for H S : based on a z-test (η HS ). Test statistic for H F : stratied z-test (η HF ). The test η HS H F for the global null hypothesis H F H S will be based on the SpiessensDebois test. 5 / 33

6 Testing the global null hypothesis H F H S. For adjusted signicance levels α F, α S we reject H F H S if p F α F or p S α S, where p F, p S are the p-values of the z-tests for H F and H S. Some remarks: For xed α F and α, the level α S is chosen such that P HF H S (p F < α F or p S < α S ) = α. For xed α F the level α S increases with the prevalence because the correlation of the test statistics increases. Formulas well known from group sequential tests. 6 / 33

7 Modied testing procedure A signicant eect in F might be totally driven by the eect in the subgroup. To account for that we ask in addition to a signicant eect in the full population that the eect in the complement (and the subgroup) show a positive trend. Let (ψ F, ψ S ) denote the closed test based on the Spiessens-Debois test. We dene the modied testing procedure test via ψ S := ψ S ψ F := ψ F 1 {ps τ S } 1 {ps τ S } 7 / 33

8 Optimizing trial designs Traditionally power arguments can be the basis for determining the best trial design. An alternative is to apply a utility based approach (Graf et al., 2015; Beckman et al., 2011). We model the sponsors/public health gain and costs of a particular trial design. Best trial design is determined by maximizing the sponsors/public health's prot. In particular we optimize the following aspects of a clinical trial: Which type of design (Enrichment Design/Classical Design/Stratied Design) to choose? Which sample size? Which signicance levels α F and α S for H F and H S in the weighted multiple test for the stratied design are optimal? Which thresholds τ S, τ S are optimal (optimized in public health view only). 8 / 33

9 The utility function U k (d) = ψ F,d ϕ k F,d + (1 ψ F,d ) ψ S,d ϕ k S,d C(d). k {Sponsor, Public Health}. ψ F,d modied Spiessens-Debois test for H F (= 0 for enrichment trials). ϕ k F,d measure of revenue if drug is licensed in F. ψ S,d modied Spiessens-Debois test for H S (= 0 for classical trials). ϕ k S,d measure of revenue if drug is licensed in S only. C(d) cost for the trial. 9 / 33

10 The revenue measures We assume that the revenue measures ϕ Sponsor F,d, ϕ Sponsor S,d depend on the data via the observed eect sizes ˆδ F,d and ˆδ S,d : where ϕ Sponsor F,d = N r F (ˆδ F,d µ F ) + ϕ Sponsor S,d = N r S (ˆδ S,d µ S ) + N denotes the number of future patients (market size). r F, r S are revenue parameters. µ F, µ S denote clinically relevant thresholds. The revenue measures for the public health view are given by Public health ϕf,d Public health ϕs,d Public health = ϕf = N r F (δ F µ F ) Public health = ϕs = N r S (δ S µ S ) 10 / 33

11 The costs of a trial The costs of a trial are the same for the sponsor and the public health view. Classical Trial Stratied Trial C(d) = c setup + 2nc per-patient. C(d) = c setup +c Biomarker development +2n(c per-patient +c screening ). Enrichment Trial C(d) = c setup +c Biomarker development +2n(c per-patient + c screening ). 11 / 33

12 The optimal trial design The optimal design is dened via: d argmax d D E π [U(d)], where E π [U(d)] = E [U(d)]π( ), 12 / 33

13 Results - overview Plan: 1 How to compute the expected utilities? 2 Description of the dierent cases studied. 3 Presentation and discussion of plots for the dierent cases. 4 Some conclusions from the case studies. Recall, the design parameters are: All designs: sample size. Stratied design: signicance levels for the multiplicity adjustment procedure. Stratied design for public health view: the additional threshold for an eect in S that is required for approval in F (p S < τ S ). 13 / 33

14 Expected utility for enrichment design For the enrichment design ψ F,En = 0, so that U S (E n ) = Nr S ψ S,En (ˆδS,En µ S ) + C(En). The expected utility given eect sizes is E[U S (E n ) ] = Nr S ((1 Φ(κ))(δ S µ S ) + κ = [ n 2σ 2σ 2 max (z 2 α n, µ S Similarly, for the public health view ( E[U PH (E n ) ] = Nr S (δ S µ S ) 1 Φ ( ) 2σ 2 n φ(κ) ) δ S ] z α δ S /. C(E n), )) 2σ 2 C(E n). n 14 / 33

15 Expected utility for classical design For the classical design ψ S,Cn = 0, so that U S (C n ) = Nr F ψ F,Cn (ˆδ F µ F ) + C(Cn), The expected utility given eect sizes is ) E[U S (C n ) ] = Nr F ((1 Φ(κ))(δ F µ F ) + V (ˆδ F )φ(κ) V (ˆδ F ) = ( 2σ 2 + (1 )(δ S δ S ) 2) /n, ) ] κ = V (ˆδ F ) [max (z 1/2 2σ 2 α n, µ F δ F. C(C n), Similarly, for the public health view ( )) E[U PH (C n ) ] = Nr F (δ F µ F ) 1 Φ (z α δ F V (ˆδ F ) 1/2 C(C n). 15 / 33

16 Expected utility for stratied design The expected utility given the eect sizes is given by [ ) ] + E[U S (S n,αs ) ] = Nr F E ψ F (ˆδ F µ F [ ( + Nr S E 1 ψ ) ) ] + F ψs (ˆδS µ S C(S n,αs ). It can be computed using numerical integration, as can the corresponding expression for the public health view. 16 / 33

17 Scenarios and cases Each particular situation is dened as follows: 1 Fix all parameters except the form of the prior, the market size and the biomarker costs. 2 Choose a scenario dening the form of the prior. 3 Choose a case dening the market size and the biomarker costs. 17 / 33

18 Fixed parameters for case studies Minimum sample size required by regulator: n min = 50. Sample variance for each observation: σ = 1. One-sided signicance level when testing: α = Minimal clinically relevant thresholds for regulatory approval: µ S = µ F = 0.1. Thresholds in the multiple test for the stratied design: τ S = τ S = 0.3. Fixed setup cost for the trial: c setup = 1 MUSD. Marginal cost per patient included: c per-patient = USD. 18 / 33

19 Scenarios A, B and C We have considered priors π δs,i,δ S,i on a grid (δ S,i, δ S,i), i = 1,..., K, of eect sizes. Scenario A A point prior with π δs,0 = 1 for δ S = 0.3. Scenario B A prior with K = 3. δ S = 0.3 and π j δs, K 1 δ = 1 S 3, j = 0,... K 1. Scenario C A point prior with π δs,δ S = 1 for δ S = / 33

20 Parameters for cases 1, 2 and 3 Reward and cost parameters in the utility function: Case 1 Large market and negligible biomarker costs. Nr F = Nr S = MUSD per unit of ecacy and c screening = c Biomarker development = 0. Case 2 Small market and negligible biomarker costs. Nr F = Nr S = 1000 MUSD per unit of ecacy and c screening = c Biomarker development = 0. Case 3 Small market with biomarker and screening costs. Nr F = Nr S = 1000 MUSD per unit of ecacy. c screening = 5000 USD per patient and c Biomarker development = 10 MUSD. 20 / 33

21 Plots for cases 1, 2 and 3 For each case, we'll look at 1 Optimal expected utility and sample size vs. [0.04, 0.94]. 2 Optimal signicance levels of the multiple test for the stratied design, and the optimal threshold τ S vs. [0.04, 0.94]. 3 The power vs. [0.04, 0.94]. Optimization of sample size is done for n / 33

22 Optimal EU and Sample size (Case 1) Sponsor Public Health Utility (Mio. $) Enrichment design Classical design Stratified design Utility (Mio. $) Sample Size Sample Size / 33

23 Optimal sig levels for stratied design (Case 1) Sponsor Public health Optimal significance levels Significance level for H S Significance level for H F Optimal significance levels Significance level for H S Significance level for H F Optimized licensing Threshold / 33

24 Power for the designs (Case 1) Sponsor Public health Power of optimized designs Reject H S Reject H F Reject H S or H F Reject H S (Enrichment) Reject H F (Classical) Power of optimized designs Reject H S Reject H F Reject H S or H F Reject H S (Enrichment) Reject H F (Classical) 24 / 33

25 Optimal EU and Sample size (Case 2) Sponsor Public Health Utility (Mio. $) Enrichment design Classical design Stratified design Utility (Mio. $) Sample Size Sample Size / 33

26 Optimal signicance levels for stratied design (Case 2) Sponsor Public health Optimal significance levels Significance level for H S Significance level for H F Optimal significance levels Significance level for H S Significance level for H F Optimized licensing Threshold / 33

27 Power for the designs (Case 2) Sponsor Public health Power of optimized designs Reject H S Reject H F Reject H S or H F Reject H S (Enrichment) Reject H F (Classical) Power of optimized designs Reject H S Reject H F Reject H S or H F Reject H S (Enrichment) Reject H F (Classical) 27 / 33

28 Optimal EU and Sample size (Case 3) Sponsor Public Health Utility (Mio. $) Enrichment design Classical design Stratified design Utility (Mio. $) Sample Size Sample Size / 33

29 Optimal signicance levels for stratied design (Case 3) Sponsor Public health Optimal significance levels Significance level for H S Significance level for H F Optimal significance levels Significance level for H S Significance level for H F Optimized licensing Threshold / 33

30 Power for the designs (Case 3) Sponsor Public health Power of optimized designs Reject H S Reject H F Reject H S or H F Reject H S (Enrichment) Reject H F (Classical) Power of optimized designs Reject H S Reject H F Reject H S or H F Reject H S (Enrichment) Reject H F (Classical) 30 / 33

31 Optimality regions for the designs in the (δ S, S )-plane Case 1 Case 2 Case 3 Sponsor δ S δ S δ S Public health δ S δ S δ S 31 / 33

32 Summary of results from case studies The optimal design depends heavily on the particular parameter conguration. However, our case studies indicates that Optimal expected utilities are larger for the sponsor. Optimal sample sizes are larger for the public health view. Since the public health decision maker optimises utility as a function of the true eects, it sometimes decides not to perform a study that a (commercial) sponsor would nd attractive. Typically, this can be observed for priors corresponding to a belief in low eect sizes. Either the classical or stratied design tends to be optimal for the sponsor, while the enrichment design is sometimes optimal for the public health view. The relative size of the costs associated with the trial and the costs associated with biomarker testing has a strong impact on the optimality regions for the dierent designs. 32 / 33

33 References Beckman, R. A., J. Clark, and C. Chen (2011). Integrating predictive biomarkers and classiers into oncology clinical development programmes. Nature Reviews Drug Discovery 10(10), Graf, A. C., M. Posch, and F. Koenig (2015, January). Adaptive designs for subpopulation analysis optimizing utility functions. Biometrical Journal 57, Song, Y. and G. Y. H. Chi (2007, August). A method for testing a prespecied subgroup in clinical trials. Statistics in Medicine 26(19), Spiessens, B. and M. Debois (2010, November). Adjusted signicance levels for subgroup analyses in clinical trials. Contemporary Clinical Trials 31(6), / 33