Randomization-Based Inference With Complex Data Need Not Be Complex! JITAIs JITAIs Susan Murphy 07.18.17 HeartSteps JITAI JITAIs
Sequential Decision Making Use data to inform science and construct decision making policies Mobile Health: determine, repeatedly over time, whether a mobile device/wearable should deliver a treatment &, if so, which type of treatment to deliver. 2
Data On each individual:,,,,,,, t: Decision point (possible treatment delivery times) : All (sensor, self report) observations up to and at t th decision point (high dimensional) : Treatment at t th decision point (actions, controls) : Proximal response (e.g., reward, utility) 3
Micro-Randomized Trial 4
Micro-Randomized Trial If a participant is available at each decision point, t, randomize between treatments. Pre-specified algorithm for the randomization probability: P[A t =a H t, I t =1] = p t (H t ) I t =1 if available, I t =0 if not H t denotes data on participant through t 5
Causal Effects Assess causal effects of tailored activity suggestion on subsequent 30 min. step count. Above is a very vague statement! 6
Potential Outcomes =,,, (random treatments), =,,, (realizations of treatments) is one potential proximal response is one potential available for treatment indicator is one potential history vector is a vector of features of history 7
Marginal, Causal Excursion Effects Excursion Effect at decision point t:,1,0 = 1, Effect is marginal over any,,, < not in ---over all variables not in. Effect is conditional on availability; only concerns the subpopulation of individuals available at decision t 8
B,1,0 = 1, = = Consistency & Micro-Randomized A t = 1, = 1, = 0, = 1, = 1,! 1 1! = 1, (! is randomization probability) 9
Marginal Excursion Effect Model Excursion Effect: = 1, = 1, = 0, = 1, = 1, Assume low dimensional structure for above effect: " # We aim to conduct inference about #! is participant s data up to and at time t is a vector of data summaries and time, t, ( ) indicator of availability 10
Centered and Weighted Least Squares Estimation Simple method for complex data Enables unbiased inference for a causal treatment effect (the β s) Inference for treatment effect is not biased by how we use covariates to reduce the noise variance in Y t+1 https://arxiv.org/abs/1601.00237 11
Estimation Steps 1. Select probabilities:!' ( 0,1 2. Form weights: * = +', -, +,., /, +', -, +,., /, 3. Center treatment actions:!' 4. Form working model: = 1, 2 " 3 5. Minimize over #, 3 : b 4 " 2 " 3!' " # 6 * 4 is empirical distribution over individuals. 12
" Minimize 4 7 2 " 3!' 6 " # * 4 is expectation with respect to empirical distribution Incorrect intuition: We are *not* assuming that, = 1,2, = 2 " 3 +!' " # 13
Minimize " 4 7 2 " 3!' 6 " # * If!' depends, at most, on features in and = 1, = 1, = 0, = 1, = 1, = " # < then, under moment conditions, #= is consistent for # < and > #= # < has a limiting normal distribution. 14
Choice of Weights If simple model, " # <, does not hold, then choice of!' determines the estimand. Example: = 1,!' =!'. Resulting #= is an estimator of where " " 7 #? 7 6 6 # = = 1, = 1, = 0, = 1, = 1 15
Gains from Randomization Causal inference for a marginal treatment effect Inference on treatment effect is robust to **working** model: = 1, 2 " 3 2 is a vector of features from the history, 16
On each of n=37 participants: Tailored activity suggestion Provide a suggestion with probability.6 Do nothing with probability=.4 5 times per day * 42 days= 210 randomizations per participant 17
Conceptual Models = 1, = 1, = 0, = 1, = 1 = β 0 = 1, = 1, = 0, = 1, = 1 = β 0 + β 1 d t t=1, T=210 Y t+1 = log-transformed step count in the 30 minutes after the t th decision point, A t = 1 if an activity suggestion is delivered at the t th decision point; A t = 0, otherwise, Z t = log-transformed step count in the 30 minutes prior to the t th decision point, (use to reduce variance) d t =days in study; takes values in (0,1,.,41) 18
Pilot Study Analysis = 1, = 1, = 0, = 1, = 1 = β 0 & = 1, = 1, = 0, = 1, = 1 = β 0 + β 1 d t Causal Effect Term Estimate 95% CI p-value β 0 A t (effect of an activity suggestion) β 0 A t + β 1 A t d t (time trend in effect of an activity suggestion) β@ < =.13 (-0.01, 0.27).06 β@ < =.51 (.20,.81) <.01 β@ = -.02 (-.03, -.01) <.01 19
Non-stationarity The data indicates that there is a causal effect of the activity suggestion vs no activity suggestion on step count in the succeeding 30 minutes. This effect deteriorates with time The walking activity suggestion initially increases step count over succeeding 30 minutes by 171 steps but by day 21 this increase is only 35 steps. 20
Discussion Problematic Analyses GLM & GEE analyses Random effects models & analyses Machine Learning Generalizations: Partially linear, single index models & analysis Varying coefficient models & analysis --These analyses do not take advantage of the microrandomization. Can accidentally eliminate the advantages of randomization for estimating causal effects-- 21
Discussion Randomization enhances: Causal inference based on minimal structural assumptions Challenge: How to include random effects which reflect scientific understanding ( person-specific effects) yet not destroy causal inference? 22
Collaborators! 23