Robust design in model-based analysis of longitudinal clinical data Giulia Lestini, Sebastian Ueckert, France Mentré IAME UMR 1137, INSERM, University Paris Diderot, France PODE, June 0 016
Context Optimal design in Nonlinear Mixed Effects Models (NLMEM) Choosing a good design for a planned study is essential Number of subjects Number of sampling times for each subject Sampling times (allocation in time) Optimal design depends on prior information (model and parameters) Adaptive design Robust design (robustness on parameters) Atkinson, Optimum Experimental Designs. 1995 Dodds, Hooker and Vicini, J Pharmacokinet Pharmacodyn. 005 Foo L-K, Duffull S. J Biopharm Stat. 010
Objectives Objectives To compare various robust design criteria in NLMEM for two examples: Pharmacokinetic/Pharmacodynamic (PKPD) model with continuous data Longitudinal binary model using a new method for the evaluation of the Fisher information matrix (FIM) for NLMEM with discrete data 3
Method Basic mixed effect model Individual model (one continuous response) y i =f(φ i, ξ i ) + ε i vector of n i observations ξ i : individual sampling times t ij j=1, n i φ i : individual parameters (size p) f: nonlinear function defining the structural model ε i : gaussian zero mean random error var (ε i ) = diag (σ inter + σ slope f(φ i,ξ i )) combined error model Random-effects model φ i = μ exp(b i ) or μ + b i b i ~N 0, Ω here Ω diagonal: ω k = Var(b ik ) Population parameters: Ψ (size P) μ (fixed effects) unknowns in Ω (variance of random effects) σ inter and/or σ slope (error model parameters) 4
Method Basic population design Assumption N individuals i same elementary design ξ in all N subjects (ξ i = ξ) with t 1,, t n sampling times n tot = N n Population design Ξ = ξ, N 5
Method Fisher Information Matrix (FIM) in NLMEM Elementary FIM: M F Ψ, ξ = E L(y;Ψ) Ψ Ψ T No analytical expression for FIM Continuous data FO approximation o M F is implemented in the R function «PFIM» 1 Discrete data New method: Adaptive Gaussian Quadrature (AGQ) and Quasi Random Monte Carlo,3 (QRMC) o M F is implemented in an R program. Population FIM for one group design M F Ψ, Ξ = N M F Ψ, ξ Standard D- criterion: M F Ψ, Ξ 1/P where Ψ = Ψ (fixed values) 1 www.pfim.biostat.fr Ueckert S, Mentré F. Computational and Methodological Statistics (CMStatistics). 015 3 Ueckert S, Mentré F. Population Optimum Design of Experiments (PODE). 015 6
Method Criteria for optimal robust designs For robust design, a distribution for Ψ, p Ψ, is assumed Optimal robust designs ξ DE ξ ED ξ EID ξ ELD ξ MM = argmax ξ Criteria E Ψ M F Ψ, ξ E Ψ M F Ψ, ξ E Ψ M F Ψ, ξ E Ψ log M F Ψ, ξ min Ψ M F Ψ, ξ 1 1 7
Method Criteria for robust optimal designs Criteria are computed by Monte Carlo simulations (MC) K= Total number of MC samples Ψ 1... Ψ k... Ψ K M F Ψ 1, ξ... M F Ψ k, ξ... M F Ψ K, ξ Compute Robust Criteria 8
Outline Part I Part Comparison of robust design criteria in NLMEM with continuous data Designs were evaluated using (i) D-criterion and predicted Relative Standard Errors (RSE) (ii) Relative Root Mean Squared Errors (RRMSE) derived from Clinical Trial Simulations (CTS) Comparison of robust design criteria in NLMEM with discrete data (NEW way to compute FIM) Designs were evaluated using (i) D-criterion and predicted Relative Standard Errors (RSE) 9
Objectives Objectives To compare various robust design criteria in NLMEM for Pharmacokinetic/Pharmacodynamic (PKPD) model with continuous data 10
Method Part I - Example: PKPD model with continuous data responses model, for a biomarker in oncology (TGF β) PK: concentration f PK (φ, t) = DOSE V k = CL V k a k a k e kt e k at, ka Concentration CL/V Parameters: k a, V, CL PD: relative inhibition of TGF-β df PD (φ, t) dt = k out I max f PK (φ, t) f PK (φ, t)+ic 50 k out f PD (φ, t), ksyn Effect kout I max = 1 Parameters: k out, IC 50 The model is implemented in the DDMoRe model repository Gueorguieva et al., Comput Methods Programs Biomed. 007 Bueno et al., Eur J Cancer. 008 Gueorguieva et al., Br J Clin Pharmacol. 014 Lestini et al., Pharm Res. 015 11
Method Part I - Example: PKPD model with continuous data PK Parameters Ψ* p(ψ) μ ka h 1 μ V L 100 100 μ CL Lh 1 10 10 ω V 0.49 0.49 ω CL 0.49 0.49 σ slope,pk 0. 0. PD Parameters Ψ* p(ψ) μ kout h 1 0. logn log(0.), 0.8 μ IC50 mg/l 0.3 logn log(0.3), 0.8 ω kout 0.49 0.49 0.49 0.49 ω IC50 σ inter,pd 0. 0. 1
Method For each optimal design, and for a fixed equispaced design ξ ES compute D-criterion 13 and predicted RSE(%) for each set simulated parameters Ψ k, k = 1,.., K Design optimization Constraints N = 50 patients n = 3 observations per patient For PK, times fixed to 0.1, 4, 1 h For PD, 3 sampling times among possible times: 1,, 3, 4, 6, 9,10,15,, 3, 4 h 11 3 = 165 elementary designs FIM Computation FIM was computed in PFIM with FO approximation Optimization Find the D-optimal design ξ D using D-criterion for Ψ* Find optimal robust designs ξ DE, ξ ED, ξ EID, ξ ELD, ξ MM using MC with K=1000 Evaluation
Results D-criterion for optimal designs Robust designs Non-Robust designs Best design with highest median: ξ ELD 6 9 4 6 3 4 1 4 15 3 9 4 1 3 15 4 6 1 10 4 Whiskers of boxplot: 10 th and 90 th percentiles 14
Results Predicted Relative Standard Errors (RSE) μ kout μ IC 50 Robust designs Non-Robust designs Whiskers of boxplot: 10 th and 90 th percentiles 15
Results Predicted Relative Standard Errors (RSE) ω k out ω IC 50 Robust designs Non-Robust designs Whiskers of boxplot: 10 th and 90 th percentiles 16
Method Simulation Study For each design ξ Ψ 1 Y 1 Ψ 1... Ψ k... K=1000 Simulations N=50 subjects... Y k...... Ψ k... Compute Relative Root Mean Squared Error (RRMSE) Ψ K Y K Ψ K ξ = ξ DE, ξ ED, ξ EID, ξ ELD, ξ MM, ξ D, ξ ES Parameter estimation: SAEM algorithm in MONOLIX 4.3 5 chains, initial estimates: Ψ* 17
Results Relative Root Mean Squared Errors (RRMSE) = Means across parameters of RRMSEs standardized to ξ ELD RRMSEs 18
Conclusion Conclusion (Part 1) All criteria led to various designs rather different ξ ELD performed globally the best in terms of median of D-criteria across the 1000 MC simulations RRMSE obtained from the CTS study confirm the results, showing ξ ELD being globally the more robust design of the 1000 simulations 19
Objectives Objectives To compare various robust design criteria in NLMEM for Longitudinal binary model using a new method for the evaluation of the Fisher information matrix (FIM) for NLMEM with discrete data 0
Method Part - Example: NLMEM with binary data Logistic model for repeated binary response with treatment increasing the slope of the logit of the response with time logit π = β 1 + β (1 + μ 3 δ)t where β = g μ, b = μ + b; π is the probability of success treatment groups (δ = 0 & δ = 1) Parameters Ψ* p(ψ) μ 1 - - μ (months) 0.09 N 0.09, 0. μ 3 5 N 5, 0.49 0.49 ω (months) 0.3 0.3 ω 1 Ogungbenro K, Aarons L.. J Pharmacokinet Pharmacodyn. 011 Ueckert S, Mentré F. Computational and Methodological Statistics (CMStatistics). 015 1
Method Design optimization Constraints N = N T (treatment) + N C (control) = 100 patients n = 4 observations per patient including 0 and 1 months Possible intermediate times: 1,, 3, 4, 5, 6, 7, 8, 9, 10, 11 months 11 = 55 elementary designs (first and last times are fixed) FIM Computation o FIM was computed with the new method by Ueckert and Mentré, 015, based on AGQ and QRMC, with 3 nodes and 500 integrations samples Optimization Find the D-optimal design ξ D using D-criterion for Ψ* Find optimal robust designs ξ DE, ξ ED, ξ EID, ξ ELD, ξ MM using MC with K=1000 Evaluation For each optimal design, and for a fixed equispaced design ξ ES compute D- criterion and predicted RSE(%) for each set simulated parameters Ψ k, k = 1,.., K
Results D-criterion distribution for selected designs Robust designs Non-Robust designs Best designs with highest median: ξ EID and ξ ELD Whiskers of boxplot: 10 th and 90 th percentiles 0 7 1 0 7 1 0 1 1 0 1 3 1 0 3 8 1 0 3 1 0 4 8 1 3
Results Predicted Relative Standard Errors (RSE) Robust designs Non-Robust designs μ 1 μ μ 3 Whiskers of boxplot: 10 th and 90 th percentiles 4
Results Predicted Relative Standard Errors (RSE) ω 1 ω Robust designs Non-Robust designs Whiskers of boxplot: 10 th and 90 th percentiles 5
Conclusion Conclusion (Part ) The evaluation of FIM with the new approach is rather fast, allowing for the first time robust design optimization for discrete longitudinal models From median of 1000 simulated D-criteria, ξ ELD and ξ EID performed globally the best ξ ES has efficiency of 0.8 compare to ξ D. When prior uncertainty is assumed, the efficiency is 0.66 compared to ξ ELD 6
Conclusion General conclusions All these robust criteria were never systematically compared in NLMEM Different criteria led to various optimal designs, with different impact on predicted RSE From median of 1000 simulated D-criteria For PKPD model: ELD led to the best designs For longitudinal binary model: ELD and EID led to the best designs (which are very close) Perspectives Perform CTS for binary data example (Part ) Perform robust and adaptive design Perform model averaging 7
Thank You! Thank you for your attention! The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement n 115156, resources of which are composed of financial contributions from the European Union's Seventh Framework Programme (FP7/007-013) and EFPIA companies in kind contribution. The DDMoRe project is also financially supported by contributions from Academic and SME partners 8
Back-up 9
Method Criteria for robust optimal designs For robust design, a distribution for Ψ, p Ψ, is assumed Optimal designs Criteria Compute Criteria ξ DE E Ψ M F Ψ, ξ 1 K K k=1 M F Ψ k, ξ 1 K ξ ED E Ψ M F Ψ, ξ K M F Ψ k, ξ k=1 = argmax ξ EID ξ E Ψ M F Ψ, ξ 1 1 1 K K M F Ψ k, ξ 1 k=1 ξ ELD E Ψ log M F Ψ, ξ 1 K K log M F Ψ k, ξ k=1 ξ MM min Ψ M F Ψ, ξ min k(k=1, K) M F Ψ k, ξ 1 30