Advantages of Mixed-effects Regression Models (MRM; aka multilevel, hierarchical linear, linear mixed models) 1. MRM explicitly models individual

Transcription

1 Advantages of Mixed-effects Regression Models (MRM; aka multilevel, hierarchical linear, linear mixed models) 1. MRM explicitly models individual change across time 2. MRM more flexible in terms of repeated measures (a) need not have same number of obs per subject (b) time can be continuous, rather than a fixed set of points 3. Flexible specification of the covariance structure among repeated measures methods for testing specific determinants of this structure 4. MRM can be extended to higher-level models repeated observations within individuals within clusters 5. Generalizations for non-normal data 1

2 2-level model for longitudinal data y i n i 1 = X i n i p β p 1 + Z i n i r υ i r 1 + ε i n i 1 y i = n i 1 response vector for individual i i = 1... N individuals j = 1... n i observations for individual i X i = n i p design matrix for the fixed effects β = p 1 vector of unknown fixed parameters Z i = n i r design matrix for the random effects υ i = r 1 vector of unknown random effects N (0, Σ υ ) ε i = n i 1 residual vector N (0, σ 2 I ni ) 2

3 Random-intercepts Model each subject is parallel to their group trend y = T ime + Grp + (Grp T ime) + Subj + Error y ij = β 0 + β 1 T ij + β 2 G i + β 3 (G i T ij ) + υ 0i + ε ij υ 0i N (0, σ 2 υ) ε ij N (0, σ 2 ) 3

4 Random Intercepts and Trend Model subjects deviate in terms of both intercept & slope y = T ime + Grp + (G T ) + Subj + (S T ) + Error y ij = β 0 + β 1 T ij + β 2 G i + β 3 (G i T ij ) + υ 0i + υ 1i T ij + ε ij υ 0i υ 1i N 0 0, σ 2 υ 0 σ υ0 υ 1 σ υ0 υ 1 σ 2 υ 1 ε ij N (0, σ 2 ) 4

5 Within-Unit / Between-Unit representation Within-subjects model - level 1 (j = 1,..., n i ) y ij = b 0i + b 1i X 1ij b p1i X p1ij + ε ij Between-subjects model - level 2 (i = 1,..., N) slopes as outcomes model b 0i = β 0 + β 0(2) x i + υ 0i b 1i = β 1 + β 1(2) x i + υ 1i. =... b p1i = β p1 + β p1(2) x i β = β 0 β 1... β p1 β 0(2) β 1(2)... β p1(2) intercept level-1 level-2 cross-level 5

6 Matrix form of model for individual i y i1 y i2... y ini y i n i 1 = 1 T ime i1 Group i Grp i T i1 1 T ime i2 Group i Grp i T i T ime ini Group i Grp i T ini X i n i p β 0 β 1 β 2 β 3 β p T ime i1 1 T ime i T ime ini Z i n i r υ 0i υ 1i υ i r 1 + ε i1 ε i2... ε ini ε i n i 1 Time might be years or months, and could differ for each subject 6

7 The conditional variance-covariance matrix is now of the form: Σy i = Z i Σ υ Z i + σ 2 I ni For example, with r = 2, n = 3, and Z i = the conditional variance-covariance Σy i = σ 2 I ni + σ 2 υ 0 σ 2 υ 0 + σ υ0 υ 1 σ 2 υ 0 + 2σ υ0 υ 1 σ 2 υ 0 + σ υ0 υ 1 σ 2 υ 0 + 2σ υ0 υ 1 + σ 2 υ 1 σ 2 υ 0 + 3σ υ0 υ 1 + 2σ 2 υ 1 σ 2 υ 0 + 2σ υ0 υ 1 σ 2 υ 0 + 3σ υ0 υ 1 + 2σ 2 υ 1 σ 2 υ 0 + 4σ υ0 υ 1 + 4σ 2 υ 1 variances and covariances change across time More general models allow autocorrelated errors, ε i N (0, σ 2 Ω i ), where Ω might represent AR or MA process 7

8 Estimation - EM algorithm opposite process of I do cocaine so I can work more, so I can do more cocaine, so I can work more, etc., Effect of increasing cocaine use Cocaine Work Health do cocaine work more declines do more cocaine work more declines more do even more cocaine work even more declines even more do a ton of cocaine always working death Effect of EM estimation of parameters M-Step (ML) E-Step (EB) Estimation starting values β, σ 2, Σ υ estimate υ i Σ υ yi improves re-estimate β, σ 2, Σ υ re-estimate υ i Σ υ yi improves more re-re-estimate β, σ 2, Σ υ re-re-estimate υ i Σ υ yi improves even more RE-estimate β, σ 2, Σ υ RE-estimate υ i Σ υ yi convergence EM is better than cocaine since EM leads to convergence and not death 8

9 EM solution - random intercepts model E-step (expectation - Expected A Posteriori or Empirical Bayes) 1 n i υ i = ρ ni n y i ij x ijβ j=1 n i σ 2 υ y i = σ 2 υ(1 ρ ni n i ) where ρ ni n i = M-step (maximization - Maximium Likelihood ) ˆβ = N i X ix i 1 N i X i(y i 1 i υ i ) n i r 1 + (n i 1)r and r = σ2 υ σ 2 υ + σ 2 ˆσ 2 υ = 1 N N i υ i 2 + σ 2 υ y i ˆσ 2 = ( N i n i) 1 N i (y i X i ˆβ 1i υ i ) (y i X i ˆβ 1i υ i ) + n i σ 2 υ y i provide starting values for β, σ 2 υ, and σ 2 perform E-step, perform M-step, repeat early and often (until convergence) 9

10 Example: Drug Plasma Levels and Clinical Response Riesby and associates (Riesby et al., 1977) examined the relationship between Imipramine (IMI) and Desipramine (DMI) plasma levels and clinical response in 66 depressed inpatients (37 endogenous and 29 non-endogenous) Drug-Washout day0 day7 day14 day21 day28 day35 wk 0 wk 1 wk 2 wk 3 wk 4 wk 5 Hamilton Depression HD 1 HD 2 HD 3 HD 4 HD 5 HD 6 Diagnosis Dx IMI IMI 3 IMI 4 IMI 5 IMI 6 DMI DMI 3 DMI 4 DMI 5 DMI 6 n

11 outcome variable Hamilton Depression Scores (HD) independent variables Dx, IMI and DMI Dx - endogenous (=1) or non-endogenous (=0) IM I (imipramine) drug-plasma levels (µg/l) antidepressant given 225 mg/day, weeks 3-6 DM I (desipramine) drug-plasma levels (µg/l) metabolite of imipramine 11

12 Descriptive Statistics Observed HDRS Means, n, and sd Washout wk 0 wk 1 wk 2 wk 3 wk 4 wk 5 Endog n Non-Endog n pooled sd

13 Correlations: n = 46 and 46 n 66 wk 0 wk 1 wk 2 wk 3 wk 4 wk 5 week week week week week week

14 increasing variance across time general linear decline over time 14

15 Examination of HD across all weeks HD i1 HD i2... HD ini y i n i 1 = 1 W EEK i1 1 W EEK i W EEK ini X i n i p β 0 β 1 β p W EEK i1 1 W EEK i W EEK ini υ 0i υ 1i + ε i1 ε i2... ε ini Z i υ i ε i n i r r 1 n i 1 where max(n i ) = 6, and X i = Z i =

16 Within-subjects and between-subjects components Within-subjects model HD ij = b 0i + b 1i T ime ij + RESID ij y ij = b 0i + b 1i x ij + ε ij b 0i b 1i = week 0 HD level for patient i = weekly change in HD for patient i i = patients j = 1... n i observations (max = 6) for patient i Between-subjects models b 0i = β 0 + υ 0i b 1i = β 1 + υ 1i β 0 β 1 υ 0i υ 1i = average week 0 HD level = average HD weekly improvement = individual deviation from average intercept = individual deviation from average improvement 16

17 parameter ML estimate se z p < β β σ 2 υ σ υ0 υ σ 2 υ σ log L = χ 2 2 = 66.1, p <.0001 for H 0 : σ υ0 υ 1 = συ 2 1 = 0 σ υ0 υ 1 as corr between intercept and slope = Wald tests are dubious for variance parameters, likelihood-ratio tests are preferred (though divide p-value by 2) Wald z-statistics sometimes expressed as χ 2 1 (by squaring z-value) 17

18 Observed and estimated means (= X ˆβ) wk 0 wk 1 wk 2 wk 3 wk 4 wk 5 n obs est

19 Obs. (pairwise) and est. variance-covariance matrix Σy = ˆΣy = Z ˆΣ υ Z + ˆσ 2 I Z = = ˆΣυ = note: from random-int model: ˆσ 2 υ = and ˆσ2 =

20 Empirical Bayes estimates of Subject Trends 20

21 Empirical Bayes estimates of Subject Trends 21

22 Examination of HD across all weeks by diagnosis HD i1 HD i2... HD ini y i n i 1 = 1 W EEK i1 Dx i Dx i W k i1 1 W EEK i2 Dx i Dx i W k i W EEK ini Dx i Dx i W k ini X i n i p β 0 β 1 β 2 β 3 β p W EEK i1 1 W EEK i W EEK ini υ 0i υ 1i + ε i1 ε i2... ε ini Z i υ i ε i n i r r 1 n i 1 where max(n i ) = 6, Z i = , Dx i = 0 for NE 1 for E 22

23 Within-subjects and between-subjects components Within-subjects model HD ij = b 0i + b 1i T ime ij + RESID ij b 0i b 1i = week 0 HD level for patient i = weekly change in HD for patient i Between-subjects models b 0i = β 0 + β 2 Dx i + υ 0i b 1i = β 1 + β 3 Dx i + υ 1i β 0 = average week 0 HD level for NE patients (Dx i = 0) β 1 = average HD weekly improvement for NE patients (Dx i = 0) β 2 = average week 0 HD difference for E patients β 3 = average HD weekly improvement difference for endogenous patients = individual deviation from average intercept = individual deviation from average improvement υ 0i υ 1i 23

24 parameter ML estimate se z p < NE int β NE slope β E int diff β E slope diff β σ 2 υ σ υ0 υ σ 2 υ σ log L = χ 2 2 = 4.1, p ns, compared to model with β 2 = β 3 = 0 σ β0 β 1 as corr between intercept and slope =

25 Riesby data - model fit by diagnosis 25

26 Examination of HD across all weeks - quadratic trend HD i1 HD i2... HD ini y i n i 1 = 1 W EEK i1 W EEKi1 2 1 W EEK i2 W EEKi W EEK ini W EEKin 2 i X i n i p β 0 β 1 β 2 β p W EEK i1 W EEKi1 2 1 W EEK i2 W EEKi W EEK ini W EEKin 2 i υ 0i υ 1i υ 2i + ε i1 ε i2... ε ini Z i υ i ε i n i r r 1 n i 1 where max(n i ) = 6, and X i = Z i =

27 Within-subjects and between-subjects components Within-subjects model HD ij y ij = b 0i + b 1i T ime ij + b 2i T ime 2 ij + RESID ij = b 0i + b 1i x ij + b 2i x 2 ij + ε ij b 0i b 1i b 2i = week 0 HD level for patient i = weekly linear change in HD for patient i = weekly quadratic change in HD for patient i Between-subjects models b 0i = β 0 + υ 0i b 1i = β 1 + υ 1i b 2i = β 2 + υ 2i β 0 β 1 β 2 υ 0i υ 1i υ 2i = average week 0 HD level = average HD weekly linear change = average HD weekly quadratic change = individual deviation from average intercept = individual deviation from average linear change = individual deviation from average quadratic change 27

28 parameter ML estimate se z p < β β β συ σ υ0 υ συ σ υ0 υ σ υ1 υ συ σ log L = χ 2 4 = 11.4, p < 0.025, compared to model with β 2 = σ 2 υ 2 = σ υ0 υ 2 = σ υ1 υ 2 = 0 χ 2 3 = 11.0, p < 0.02, compared to model with σ2 υ 2 = σ υ0 υ 2 = σ υ1 υ 2 = 0 σ υ1 υ 2 as corr between linear and quadratic terms =

29 Average linear and individual quadratic trends 29

30 Observed (pairwise) and estimated variance-covariance matrix Σy = ˆΣy = Z ˆΣ υ Z + ˆσ 2 I = where Z = ˆΣ υ =

31 Empirical Bayes estimates of Subject Trends 31

32 SAS PROC MIXED formulation y n 1 = X n p β p 1 + Z n Sr υ Sr 1 + ε n 1 i = 1... S individuals n = ( S i n i ) total number of observations y = vector obtained by stacking y i vectors X = matrix obtained by stacking X i matrices υ = vector obtained by stacking υ i vectors ε = vector obtained by stacking ε i vectors Z = block diagonal matrix with Z i on diagonal Z = Z Z Z S 32

33 V ar υ ε = G 0 0 R V ar(y) = Z G Z + R n n n Sr Sr Sr Sr n n n G = Σ υ Σ υ Σ υs Can model variance/covariance of y in terms of: G - random-effects only (R = σ 2 I n ) R - variance/covariance structures unstructured, or AR(1), or Toeplitz (banded) G and R - random-effects with correlated errors 33

34 Example 4a: Analysis of Riesby dataset using MRM. This example has a few different PROC MIXED specifications, and includes a grouping variable and curvilinear effect of time. (SAS code and output) hedeker/riesbym.txt 34

35 SAS MIXED code - RIESBYM.SAS TITLE1 analysis of riesby data - hdrs scores across time ; DATA one; INFILE c:\mixdemo\riesby.dat ; INPUT id hamd intcpt week endog endweek ; PROC FORMAT; VALUE endog 0= nonendog 1= endog ; VALUE week 0= week 0 1= week 1 2= week 2 3= week 3 4= week 4 5= week 5 ; PROC MIXED METHOD=ML COVTEST; CLASS id; MODEL hamd = week /SOLUTION; RANDOM INTERCEPT /SUB=id TYPE=UN G; TITLE2 random intercepts model: compound symmetry structure ; PROC MIXED METHOD=ML COVTEST; CLASS id; MODEL hamd = week /SOLUTION; RANDOM INTERCEPT week /SUB=id TYPE=UN G GCORR; TITLE2 random trend model ; 35

36 PROC MIXED METHOD=ML COVTEST; CLASS id; MODEL hamd = week endog endweek /SOLUTION; RANDOM INTERCEPT week /SUB=id TYPE=UN G GCORR; TITLE2 random trend model with group effects ; PROC MIXED METHOD=ML COVTEST; CLASS id; MODEL hamd = week week*week /SOLUTION; RANDOM INTERCEPT week week*week /SUB=id TYPE=UN G GCORR; TITLE2 random quadratic trend model ; RUN; 36

37 Riesby.dat - data from a few subjects

38 Example 4b: Analysis of Riesby dataset. This handout shows how empirical Bayes estimates can be output to a dataset in order to calculate estimated individual scores at all timepoints. (SAS code and output) hedeker/riesbym2.txt 38

39 SAS MIXED code - RIESBYM2.SAS TITLE1 analysis of riesby data - empirical bayes estimates ; DATA one; INFILE c:\mixdemo\riesby.dat ; INPUT id hamd intcpt week endog endweek ; PROC FORMAT; VALUE endog 0= nonendog 1= endog ; VALUE week 0= week 0 1= week 1 2= week 2 3= week 3 4= week 4 5= week 5 ; PROC MIXED METHOD=ML; CLASS id; MODEL hamd = week /SOLUTION; RANDOM INTERCEPT week /SUB=id TYPE=UN G S; ODS LISTING EXCLUDE SOLUTIONR; ODS OUTPUT SOLUTIONR=randest; TITLE2 random trend model ; /* print out the estimated random effects dataset */ PROC PRINT DATA=randest; 39

40 /* get a printout of the data in multivariate form */ PROC SORT DATA=one; BY id; DATA t0;set one; IF week=0; hamd 0 = hamd; DATA t1;set one; IF week=1; hamd 1 = hamd; DATA t2;set one; IF week=2; hamd 2 = hamd; DATA t3;set one; IF week=3; hamd 3 = hamd; DATA t4;set one; IF week=4; hamd 4 = hamd; DATA t5;set one; IF week=5; hamd 5 = hamd; DATA comp (KEEP=id hamd 0-hamd 5); MERGE t0 t1 t2 t3 t4 t5; BY id; PROC PRINT DATA=comp; VAR id hamd 0-hamd 5; 40

41 /* extract the intercepts and slopes for each person */ /* and compute the estimated hamd values across time */ PROC SORT DATA=randest; BY id; DATA randest2 (KEEP=id intdev slopedev int slope hdest 0-hdest 5); ARRAY y(2) intdev slopedev; DO par = 1 TO 2; SET randest; BY id; y(par) = ESTIMATE; IF par = 2 THEN DO; int = intdev; slope = slopedev; hdest 0 = int; hdest 1 = int + slope; hdest 2 = int + 2*slope; hdest 3 = int + 3*slope; hdest 4 = int + 4*slope; hdest 5 = int + 5*slope; END; IF LAST.id THEN RETURN; END; 41

42 PROC PRINT DATA=randest2; VAR id hdest 0-hdest 5; PROC PLOT DATA=randest2; PLOT intdev * slopedev; PLOT int * slope; TITLE2 plot of individual intercepts versus slopes ; RUN; 42

43 Time-varying Covariates - WS and BS effects Section in Hedeker & Gibbons (2006), Longitudinal Data Analysis, Wiley. 43

44 Examination of HD across 4 weeks by plasma drug-levels HD i1 HD i2... HD ini y i n i 1 = 1 W EEK i1 lnimi i1 lndmi i1 1 W EEK i2 lnimi i2 lndmi i W EEK ini lnimi ini lndmi ini X i n i p β 0 β 1 β 2 β 3 β p W EEK i1 1 W EEK i W EEK ini υ 0i υ 1i + ε i1 ε i2... ε ini Z i υ i ε i n i r r 1 n i 1 where max(n i ) = 4, and Z i =

45 Within-subjects and between-subjects components Within-subjects model HD ij = b 0i + b 1i T ij + b 2i ln IMI ij + b 3i ln DMI ij + Res ij b 0i = week 2 HD level for patient i with both ln IMI and ln DMI = 0 b 1i = weekly change in HD for patient i b 2i = change in HD due to ln IMI = change in HD due to ln DMI b 3i Between-subjects models b 0i = β 0 + υ 0i b 1i = β 1 + υ 1i b 2i = β 2 b 3i = β 3 45

46 β 0 β 1 β 2 β 3 υ 0i υ 1i = average week 2 HD level for drug-free patients = average HD weekly improvement = average HD difference for unit change in ln IMI = average HD difference for unit change in ln DMI = individual intercept deviation from model = individual slope deviation from model Here, week 2 is the actual study week (i.e., one week after the drug washout period), which is coded as 0 in this analysis of the last four study timepoints 46

47 parameter ML estimate se z p < int β slope β ln IMI β ln DMI β σ 2 υ σ υ0 υ σ 2 υ σ log L = σ υ0 υ 1 as corr between intercept and slope =

48 parameter estimate se p < HD total score intercept β slope β Baseline HD β ln IMI β ln DMI β συ σ υ0 υ συ σ HD change from baseline intercept β ns slope β ln IMI β ns ln DMI β συ σ υ0 υ συ σ

49 Correlation between HD scores and plasma levels (ln units) HD total score week 2 week 3 week 4 week 5 IMI DMI HD change from baseline week 2 week 3 week 4 week 5 IMI DMI p <

50 Model with time-varying covariate X ij Within-subjects model Y ij = b 0i + b 1i T ij + b 2i X ij + E ij Between-subjects models b 0i = β 0 + υ 0i b 1i = β 1 + υ 1i b 2i = β 2 Is the effect of X ij purely within-subjects? What about X ij = X ij + X i X i = X i + (X ij X i ) X i is between-subjects component of X X ij X i is within-subjects component of X 50

51 Time-varying covariate effects: purely between-subjects 51

52 Time-varying covariate effects: purely within-subjects 52

53 Model with decomposition of time-varying covariate X ij Within-subjects model Y ij = b 0i + b 1i T ij + b 2i (X ij X i ) + E ij Between-subjects models b 0i = β 0 + β BS Xi + υ 0i b 1i = β 1 + υ 1i b 2i = β W S Notice, effect of X is now β BS Xi + β W S (X ij X i ) β BS = effect of Xi on Ȳi BS or cross-sectional β W S = effect of (X ij X i ) on (Y ij Ȳi) WS or longitudinal 53

54 Model with only X ij assumes equal BS and WS effects (β BS = β W S ) suppose β BS = β W S = β, then in the model with decomposition, the effect of X ij = β Xi + β (X ij X i ) = β X ij precisely what the model with only X ij assumes Equal WS and BS effects of X ij? can be a dubious assumption needs to be tested (by comparing two models via LR test) there is no guarantee that β BS and β W S even agree on sign 54

55 Time-varying covariate effects: opposite sign WS and BS effects 55

56 parameter estimate se p < assuming BS=WS drug effects intercept ns slope ln IMI ns ln DMI deviance = relaxing BS=WS drug effects intercept ns slope ln IMI BS ns ln DMI BS ln IMI WS ns ln DMI WS ns deviance = X 2 2 = = 3 Accept H 0 : β BS = β W S 56

57 Example 4c: Analysis of Riesby dataset. This handout has the analysis considering the time-varying drug plasma levels, separating the within-subjects from the between-subjects effects for these time-varying covariates. (SAS code and output) hedeker/riesbsws.txt 57

58 SAS MIXED code - RIESBSWS.SAS TITLE1 partitioning BS and WS effects of drug levels ; DATA one; INFILE c:\mixdemo\riesbyt4.dat ; INPUT id hamdelt intcpt week sex endog lnimi lndmi ; PROC SORT; BY id; PROC MEANS NOPRINT; CLASS id; VAR lnimi lndmi; OUTPUT OUT = two MEAN = mlnimi mlndmi; DATA three; MERGE one two; BY id; lnidev = lnimi - mlnimi; lnddev = lndmi - mlndmi; PROC MIXED METHOD=ML COVTEST; CLASS id; MODEL hamdelt = week lnimi lndmi /SOLUTION; RANDOM INTERCEPT week /SUB=id TYPE=UN G GCORR; TITLE2 assuming bs=ws drug effects ; PROC MIXED METHOD=ML COVTEST; CLASS id; MODEL hamdelt = week mlnimi mlndmi lnidev lnddev /SOLUTION; RANDOM INTERCEPT week /SUB=id TYPE=UN G GCORR; TITLE2 relaxing bs=ws drug effects ; 58