Comparing Adaptive Interventions Using Data Arising from a SMART: With Application to Autism, ADHD, and Mood Disorders

Transcription

1 Comparing Adaptive Interventions Using Data Arising from a SMART: With Application to Autism, ADHD, and Mood Disorders Daniel Almirall, Xi Lu, Connie Kasari, Inbal N-Shani, Univ. of Michigan, Univ. of California Los Angeles University of Chicago May 1, 2015 Almirall, Lu, Kasari, N-Shani Design and Analysis of SMART May 1, / 35

2 Outline Background: Adaptive Interventions and SMARTs (Autism) Marginal Model and Efficiency Considerations in Estimation (Autism) Modeling Longitudinal Outcomes in a SMART (Autism and ADHD) Adaptive Implementation Interventions (Mood Disorders) Other SMART Designs in Mental Health (Autism, Adolescent Depression) Almirall, Lu, Kasari, N-Shani Design and Analysis of SMART May 1, / 35

3 Background: Adaptive Interventions and SMART Almirall, Lu, Kasari, N-Shani Design and Analysis of SMART May 1, / 35

4 Sequential, Individualized Treatment is Often Needed Management of many health disorders often entails a sequential, individualized approach treatment is adapted and re-adapted over time This type of sequential decision-making is necessary when there is high level of individual heterogeneity in response to treatment. great treatment effect heterogeneity, no widely effective treatment, there are widely effective treatments but they are burdensome, costly, or carry side effects. Adaptive Interventions (AI) provide one way to operationalize the strategies (e.g., continue, augment, switch, step-down) leading to individualized sequences of treatment. Almirall, Lu, Kasari, N-Shani Design and Analysis of SMART May 1, / 35

5 Definition of an Adaptive Intervention A sequence of decision rules that specify whether, how, when (timing), and based on which measures, to alter the dosage (duration, frequency or amount), type, or delivery of treatment(s) at decision stages in the course of care. aka: dynamic treatment regime/regimen, adaptive treatment strategy, treatment policy, treatment algorithms, medication algorithms, etc. Almirall, Lu, Kasari, N-Shani Design and Analysis of SMART May 1, / 35

6 Example of an Adaptive Intervention in Autism (PI: Kasari) For older, minimally verbal children with autism spectrum disorder Stage One JASP for 12 weeks; Stage Two At the end of week 12, determine early sign of response: IF slow responder: Augment JASP with AAC for 12 weeks; ELSE IF responder: Maintain JASP for 12 weeks. Almirall, Lu, Kasari, N-Shani Design and Analysis of SMART May 1, / 35

7 Example of an Adaptive Intervention in Autism (PI: Kasari) For older, minimally verbal children with autism spectrum disorder Stage One JASP for 12 weeks; Stage Two At the end of week 12, determine early sign of response: IF slow responder: Augment JASP with AAC for 12 weeks; ELSE IF responder: Maintain JASP for 12 weeks. JASP First stage Treatment (Weeks 1 12) Responders Slow Responders End of Week 12 Responder Status Continue: JASP Augment: JASP + AAC Second stage Treatment (Weeks 13 24) Almirall, Lu, Kasari, N-Shani Design and Analysis of SMART May 1, / 35

8 But AAC s are costly (time and money). So there are important unanswered questions when building an Adaptive Intervention with AAC. For example... Is it better to provide AAC from the start? How long to wait before declaring a child a slow responder to JASP? Can we identify which children would benefit from AAC from the start vs those who would benefit from delayed AAC? For slow responders, what is the effect of providing the AAC vs intensifying JASP (not providing AAC)? Sequential Multiple Assignment Randomized Trials (SMARTs) can be used to address such questions empirically, using experimental design principles. Almirall, Lu, Kasari, N-Shani Design and Analysis of SMART May 1, / 35

9 What is a Sequential Multiple Assignment Randomized Trial (SMART)? A type of multi-stage randomized trial design. At each stage, subjects randomized to a set of feasible/ethical treatment options. Treatment options latter stages may be restricted by early response status SMARTs were developed explicitly for the purpose of building a high-quality Adaptive Intervention. Almirall, Lu, Kasari, N-Shani Design and Analysis of SMART May 1, / 35

10 Example of a SMART in Autism Research (N = 61) PI: Kasari (UCLA). (ages 5-8; planned N = 98 but recruitment difficult, despite multi-site. Wk12 response rates much higher than anticipated.) Almirall, Lu, Kasari, N-Shani Design and Analysis of SMART May 1, / 35

11 Scientific Aim is to compare the 3 embedded AIs. (a 1, a 2 ) = (1, 1) (JASP,JASP+) Subgroups A+C (a 1, a 2 ) = (1, 1) (JASP,AAC) Subgroups A+B (a 1, a 2 ) = ( 1,.) (AAC,AAC+) Subgroups D+E Almirall, Lu, Kasari, N-Shani Design and Analysis of SMART May 1, / 35

12 A Marginal Model for Y (Social Communicative Utterances) and Efficiency Considerations in Estimation Other work includes work by Robins (1995, 2008, etc.), Orellana et al. (2010), Hirano et al. (2003), Chafee and van der Laan (2012), Bembom and van der Laan (2007, 2008), Williamson et al. (2013), Wang et al. (2012) and others... Almirall, Lu, Kasari, N-Shani Design and Analysis of SMART May 1, / 35

13 The marginal model for comparing AIs. E[Y (a 1, a 2 ) X ] = β 0 + η T X + β 1 a 1 + β 2 I (a 1 = 1)a 2 e.g., 2β 1 + β 2 = (AAC,AAC+) vs (JASP,JASP+) But how do we estimate this with observed data {X i, A 1i, R i, A 2i, Y i }? Regressing Y on [1, X, A 1, I (A 1 = 1)A 2 ] does not work. Why? By design, there is an under-representation of slow responders to JASP who get JASP+ (or JASP+AAC). How do we account for the fact that responders to JASP are consistent with two of the embedded AIs? Almirall, Lu, Kasari, N-Shani Design and Analysis of SMART May 1, / 35

14 Recycling-and-Weighting Regression (RWR) Estimator Almirall, Lu, Kasari, N-Shani Design and Analysis of SMART May 1, / 35

15 Recycling-and-Weighting Regression Estimator (RWR) Statistical foundation found in work by Orellana, Rotnitzky and Robins: Robins JM, Orellana L, Rotnitzky A. Estimation and extrapolation in optimal treatment and testing strategies. Statistics in Medicine Jul; 27: Orellana L, Rotnitzky A, Robins JM. Dynamic Regime Marginal Structural Mean Models for Estimation of Optimal Dynamic Treatment Regimes, Parts I and II. Int J Biostat. 2010; 6(2): Article 8 and 9. Very nicely explained and implemented with SMART data in: Nahum-Shani I, Qian M, Almirall D, et al. Experimental design and primary data analysis methods for comparing adaptive interventions. Psychol Methods Dec; 17(4): Almirall, Lu, Kasari, N-Shani Design and Analysis of SMART May 1, / 35

16 Improving the Efficiency of the RWR by Estimating the Known Weights with Covariates By design, we know the true weights. Since Pr(A 1 ) = 1/2 and Pr(A 2 = 1 A 1 = 1, R = 0) = 1/2, then W = 4I {A 1 = 1, R = 0} + 2I { everyone else }. Almirall, Lu, Kasari, N-Shani Design and Analysis of SMART May 1, / 35

17 Improving the Efficiency of the RWR by Estimating the Known Weights with Covariates By design, we know the true weights. Since Pr(A 1 ) = 1/2 and Pr(A 2 = 1 A 1 = 1, R = 0) = 1/2, then W = 4I {A 1 = 1, R = 0} + 2I { everyone else }. Robins and colleagues (1995), Hirano et al (2003) and many others describe potential gains in statistical efficiency by estimating the known weights using auxiliary baseline (L 1 ) and time-varying (L 2 ) covariate information. Almirall, Lu, Kasari, N-Shani Design and Analysis of SMART May 1, / 35

18 Improving the Efficiency of the RWR by Estimating the Known Weights with Covariates By design, we know the true weights. Since Pr(A 1 ) = 1/2 and Pr(A 2 = 1 A 1 = 1, R = 0) = 1/2, then W = 4I {A 1 = 1, R = 0} + 2I { everyone else }. Robins and colleagues (1995), Hirano et al (2003) and many others describe potential gains in statistical efficiency by estimating the known weights using auxiliary baseline (L 1 ) and time-varying (L 2 ) covariate information. Example using the Autism SMART data: The observed data is now {L 1i, X i, A 1i, R i, L 2i, A 2i, Y i } Use logistic regression to get p 1 = Pr(A 1 L 1, X ) Use logistic regression to get p 2 = Pr(A 2 L 1, X, A 1 = 1, R = 0, L 2 ). Use Ŵ = I {A 1 = 1, R = 0}/( p 1 p 2 ) + I { everyone else }/ p 1. Almirall, Lu, Kasari, N-Shani Design and Analysis of SMART May 1, / 35

19 Improving the Efficiency of the RWR by Estimating the Known Weights with Covariates By design, we know the true weights. Since Pr(A 1 ) = 1/2 and Pr(A 2 = 1 A 1 = 1, R = 0) = 1/2, then W = 4I {A 1 = 1, R = 0} + 2I { everyone else }. Robins and colleagues (1995), Hirano et al (2003) and many others describe potential gains in statistical efficiency by estimating the known weights using auxiliary baseline (L 1 ) and time-varying (L 2 ) covariate information. Example using the Autism SMART data: The observed data is now {L 1i, X i, A 1i, R i, L 2i, A 2i, Y i } Use logistic regression to get p 1 = Pr(A 1 L 1, X ) Use logistic regression to get p 2 = Pr(A 2 L 1, X, A 1 = 1, R = 0, L 2 ). Use Ŵ = I {A 1 = 1, R = 0}/( p 1 p 2 ) + I { everyone else }/ p 1. The key is to choose L t s that are highly correlated with Y! Almirall, Lu, Kasari, N-Shani Design and Analysis of SMART May 1, / 35

20 Sim: Relative RMSE for (AAC,AAC+) vs (JASP,JASP+) Almirall, Lu, Kasari, N-Shani Design and Analysis of SMART May 1, / 35

21 Results from an Analysis of the Autism SMART Y = is Socially Communicative Utterances at Week 24 (SD=34.6). W (known) Ŵ (est.) ESTIMAND EST SE PVAL EST SE PVAL (AAC,AAC+) < < 0.01 (JASP,AAC) < < 0.01 (JASP,JASP+) < < 0.01 (AAC,AAC+) vs (JASP,JASP+) < < 0.01 (AAC,AAC+) vs (JASP,AAC) (JASP,AAC) vs (JASP,JASP+) Apparent greater efficiency was observed in the analysis of the autism SMART (about 4% - 21% apparent improvement). Almirall, Lu, Kasari, N-Shani Design and Analysis of SMART May 1, / 35

22 Analysis of Longitudinal Outcomes in a SMART Other work in this space includes work by Li (2014) and Miyahara and Wahed (2012). Almirall, Lu, Kasari, N-Shani Design and Analysis of SMART May 1, / 35

23 Methodological Challenges 1 Modeling: The intermixing of repeated measures and sequential randomizations requires new modeling considerations to account for the fact that embedded AIs will share paths/trajectories at different time points (...more difficult the more complex the SMART is...) 2 Statistical: Develop an estimator that takes advantage of the within person correlation in the outcome over time (...think GEE...) Almirall, Lu, Kasari, N-Shani Design and Analysis of SMART May 1, / 35

24 An Example Marginal Model for Longitudinal Outcomes Y t : # Socially Communicative Utterances at week t. t = 0, 12, 24, 36 Ideally, the longitudinal model should 1 permit deflections/changes in trajectories at week 12, 2 respect temporal ordering, and 3 respect shared trajectory paths An example is the following piece-wise linear model: E[Y t (a 1, a 2 ) X ] = β 0 + η T X + 1 t 12 {β 1 t + β 2 ta 1 } + 1 t>12 {12β β 2 a 1 + β 3 (t 12) + β 4 (t 12)a 1 + β 5 (t 12)a 1 a 2 } where X s are mean-centered baseline covariates. Almirall, Lu, Kasari, N-Shani Design and Analysis of SMART May 1, / 35

25 Analysis of Longitudinal Outcomes in the Autism SMART Average level of spoken communication over 36 weeks (i.e., AUC/36) for each AI AI Estimate 95% CI (AAC,AAC+) 51.4 [45.6, 57.3] (JASP,AAC) 40.7 [34.5, 46.8] (JASP,JASP+) 39.3 [32.6, 46.0] Almirall, Lu, Kasari, N-Shani Design and Analysis of SMART May 1, / 35

26 Child Attention Deficit Hyperactivity Disorder (ADHD) PI: Pelham (FIU) (N = 153; ages 6-12; 8 month study; monthly non-response based on two teacher ratings (ITB < 0.75 and IRS > 1 domain) Almirall, Lu, Kasari, N-Shani Design and Analysis of SMART May 1, / 35

27 Analysis of Longitudinal Outcomes in the ADHD SMART Y t = Classroom performance over the school year under each AI Classroom performance DTR AI (MED, MED+) BMOD.INT BMOD.MED (MED, MED.BMOD MED+BMD) (BMD,BMD+MED) MED.INT (BMD,BMD+) Color Purple Blue Green Red Time (months) Almirall, Lu, Kasari, N-Shani Design and Analysis of SMART May 1, / 35

28 Methodological Challenges 1 Modeling: The intermixing of repeated measures and sequential randomizations requires new modeling considerations to account for the fact that embedded AIs will share paths/trajectories at different time points (...more difficult the more complex the SMART is...) 2 Statistical: Develop an estimator that takes advantage of the within person correlation in the outcome over time (...think GEE...) Almirall, Lu, Kasari, N-Shani Design and Analysis of SMART May 1, / 35

29 Statistical: RWR Estimator for Longitudinal Outcomes Obtain estimates of (η, β) by solving the following estimating equation: 0 = 1 N N I i (a 1, a 2 )D i (X i, a 1, a 2 )V 1 i W i (Y i µ i (X i, a 1, a 2 ; β, η)), i=1 (a 1,a 2 ) where I i (a 1, a 2 ): indicates if i s txt sequence is consistent with (a 1, a 2 ) Y i : a vector of longitudinal outcomes, i.e. (Y i,0, Y i,12, Y i,24, Y i,36 ) T ; µ i a vector of corresponding conditional means; ( µi (X i,āi ;β,η) ) T D i : the design matrix, i.e., µ i (X i,āi ;β,η) β T η ; T W i : a diagonal matrix containing IPW (known or estimated); V i : working covariance matrix for Y i. Almirall, Lu, Kasari, N-Shani Design and Analysis of SMART May 1, / 35

30 Statistical: RWR Estimator for Longitudinal Outcomes We implemented it using the following estimating equation: 0 = 1 M M D i (X i, Ā i )V 1 i W i (Y i µ i (X i, Ā i ; β, η)), i=1 where M = N+ # JASP Slow Responders Y i : a vector of longitudinal outcomes, i.e. (Y i,0, Y i,12, Y i,24, Y i,36 ) T ; µ i a vector of corresponding conditional means; ( µi (X i,ā i ;β,η) ) T D i : the design matrix, i.e., µ i (X i,ā i ;β,η) β T η ; T W i : a diagonal matrix containing IPW (known or estimated); V i : working covariance matrix for Y i. This facilitates using existing software (SEs requires more work). Almirall, Lu, Kasari, N-Shani Design and Analysis of SMART May 1, / 35

31 Statistical: The choice of V and the form of W For the case of V = I (independence working covariance structure), W may have the form W i = w w where w i = 4I {A 1 = 1, R = 0} + 2I { everyone else }. However, for V I (e.g., autoregressive or exchangeable structure), W must have the form w W i = 0 w w w But why? Almirall, Lu, Kasari, N-Shani Design and Analysis of SMART May 1, / 35

32 Why W = diag(1/(pr(a 1 )Pr(A 2 A 1, R))) when V I? Some intuition: 0 = 1 M M D i (X i, Ā i )V 1 i W i (Y i µ i (X i, Ā i ; β, η)) i=1 =... + D i,t=24 (X i, A i1, A i2 ) v 12,24 w 12 (Y i,12 µ i,12 (X i, A i1 ; β, η)) +... This cross-product term Is not mean zero if w 12 = 2 = 1/Pr(A 1 ), leading to bias Is mean zero if w 12 = 1/(Pr(A 1 )Pr(A 2 A 1, R)) Pepe and Anderson (1994, cautioning about GEE s with time-varying covariates, in general; Stijn Vansteelandt (2007, cautioning about MSMs) Almirall, Lu, Kasari, N-Shani Design and Analysis of SMART May 1, / 35

33 An Exciting and SMART Development in Mental Health Implementation Science Almirall, Lu, Kasari, N-Shani Design and Analysis of SMART May 1, / 35

34 Adaptive Implementation Intervention in Mental Health PI: Kilbourne; Co-I: Almirall (CO/AR/MI; Aim is to improve uptake of psychosocial intervention for mood disorders; primary aim compared initial REP+EF vs REP+EF+IF.)

35 Other SMART Designs in Mental Health Almirall, Lu, Kasari, N-Shani Design and Analysis of SMART May 1, / 35

36 Adaptive Treatment Strategies for Adolescent Depression PI: Gunlicks-Stoessel (Univ of Minnesota); Co-I: Almirall (Univ Michigan)

37 Interventions for Minimally Verbal Children with Autism PI: Kasari(UCLA), Kaiser(Vanderbilt), Smith(Rochester), Lord(Cornell), Almirall(Mich)

38 Interventions for Minimally Verbal Children with Autism PI: Kasari(UCLA), Kaiser(Vanderbilt), Smith(Rochester), Lord(Cornell), Almirall(Mich) R JASP (joint attention and social play) Non-Responders (Parent training no feasible) Responders (Blended txt unnecessary) R R JASP + DTT Continue JASP Continue JASP JASP + Parent Training DTT (discrete trials training) Non-Responders (Parent training not feasible) R JASP + DTT Continue DTT Responders (Blended txt unnecessary) R Continue DTT DTT + Parent Training

39 Thank you! Questions? More About SMART: More papers and these slides on my website (Daniel Almirall): dalmiral/ me with questions about this presentation: Daniel Almirall: Thanks to NIDA and NIMH for Funding this Research: P50DA10075, R03MH , RC4MH Almirall, Lu, Kasari, N-Shani Design and Analysis of SMART May 1, / 35