Statistical Analysis of Hierarchical Data. David Zucker Hebrew University, Jerusalem, Israel

Transcription

1 Statistical Analysis of Hierarchical Data David Zucker Hebrew University, Jerusalem, Israel

2 Unit 1 Linear Mixed Models 1

3 Examples of Hierarchical Data Hierarchical data = subunits within units Students within schools Eyes within patients Measurements within subjects in repeated measures data (longitudinal data) 2

4 Examples of Linear Mixed Models 1. One-way ANOVA With Random Effect Y ij = µ + a i + ɛ ij a i N(0, σ 2 a), ɛ ij N(0, σ 2 ɛ ) 2. A Fixed Group Effect and a Random Cluster Effect Y ij = µ + βx i + a i + ɛ ij X i = 1 for treatment, X i = 0 for control 3. More General Regression Effects Y ij = µ + p k=1 β kx ijk + a i + ɛ ij 4. More Than One Level Y ijk = µ + p l=1 β jx ijkl + a i + b ij + ɛ ijk 3

5 Examples of Linear Mixed Models Continued 5. Random Line With a Group Effect Y ij = γ 1i + γ 2i t ij + ɛ ij γ 1i = β (1) 0 + β (1) 1 X i + b 1i γ 2i = β (2) 0 + β (2) 1 X i + b 2i b N(0, G) Y ij = [β (1) 0 + β (1) 1 X i] + [β (2) 0 + β (2) 1 X i]t ij + [b 1i + b 2i t ij ] + ɛ ij 4

6 General Form of Linear Mixed Model Y ij = x T ijβ + z T ijb i + ɛ ij b i N(0, G(θ (1) )) ɛ i N(0, R i (θ (2) )) in matrix form Y i = X i β + Z i b i + ɛ i Y i N(X i β, V i (θ)) V i (θ) = Z i G(θ (1) )Z T i + R i (θ (2) ) 5

7 Random Line Example in General LMM Form random line model Y ij = [β (1) 0 + β (1) 1 X i] + [β (2) 0 + β (2) 1 X i]t ij + [b 1i + b 2i t ij ] + ɛ ij representation in general LMM form Y ij = x T ijβ + z T ijb i + ɛ ij x T ij = [1, X i, t ij, X i t ij ] β T = [β (1) 0, β(1) 1, β(2) 0, β(2) 1 ] z T ij = [1, t ij ] b i = [b 1i, b 2i ] 6

8 Statistical Inference for Parameters Estimation Maximum Likelihood Large Sample Distribution φ = (β, θ) ˆφ φ N(0, Ω( ˆφ)) Ω(φ) = inverse Fisher information matrix 7

9 Inference for Individual Fixed-Effect Coefficients Regard ˆβ j β j Ω ββ jj as being approximately normally distributed, or approximately t-distributed with d d.f. Choice of d: Various methods available DDFM option in SAS PROC MIXED DDFM=SAT uses Satterthwaite approximation the SPSS MIXED procedure also uses the Satterthwaite approximation 8

10 Inference for Individual Coefficients Continued Testing H 0 : β j = 0: Reject if ˆβ j Ω ββ jj t d (1 α 2 ) Confidence interval for β j ˆβ j ± t d (1 α 2 ) Ω ββ jj 9

11 Testing a Single Linear Combination of Fixed Effects Look at ψ = c T β ˆψ = c T ˆβ N(ψ, c T Ω (ββ) c) Test H 0 : ψ = 0 using t-test with d d.f. Confidence interval for ψ ˆψ ± t d (1 α 2 ) c T Ω (ββ) c) Examples: 1. Expected response for a given set of covariate values 2. Treatment contrast 10

12 Testing Several Linear Combinations Look at φ = L T β, L T = m p matrix ˆφ = L T ˆβ N(φ, L T Ω (ββ) L) test H 0 : ψ = 0 using F -test with d.f. m and d F = ˆφ T (L T Ω (ββ) L) 1 ˆφ/m Example: Test for overall group effect in multiple group study 11

13 REML Estimation of Covariance Parameters Idea: Adjust the estimator of θ to take into account the estimation of β Similar to The Method: ˆσ 2 = 1 n n i=1 (Y i Ȳ )2 vs. ˆσ 2 = 1 n 1 n (Y i Ȳ )2 Let U be an n (n p) matrix of full rank such that U T X = 0 Then base estimation of θ on T = U T Y We have T N(0, U T V(θ)U) i=1 12

14 Software SAS: PROC MIXED SPSS: MIXED R: several procedures lme function it nlme package lmer function in lme4 package lmertest package for Satterthwaite degrees of freedom 13

15 Example Three groups of rats given different growth treatments and then followed over time Analysis in SAS PROC MIXED First model: proc mixed method=reml ratio covtest; class group; model response = group time group*time / solution ddfm=sat; random intercept time / subject=subject type=un; run; Second model: proc mixed method=reml ratio covtest; class group; model response = group time group*time / solution ddfm=sat; random intercept / subject=subject type=un; run; 14

16 1 GROUP=1 RESPONSE SUBJECT TIME

17 2 GROUP=2 RESPONSE SUBJECT TIME

18 3 GROUP=3 RESPONSE SUBJECT TIME

19 The SAS System 1 The Mixed Procedure First Analysis Model Information Data Set Dependent Variable Covariance Structure Subject Effect Estimation Method Residual Variance Method Fixed Effects SE Method Degrees of Freedom Method WORK.INDAT RESPONSE Unstructured SUBJECT REML Profile Model-Based Satterthwaite Class Level Information Class Levels Values GROUP Dimensions Covariance Parameters 4 Columns in X 8 Columns in Z per Subject 2 Subjects 50 Max Obs per Subject 7 Number of Observations Number of Observations Read 350 Number of Observations Used 252 Number of Observations Not Used 98 Iteration History Iteration Evaluations -2 Res Log Like Criterion Convergence criteria met.

20 The SAS System 2 The Mixed Procedure Covariance Parameter Estimates Cov Parm Subject Ratio Estimate Standard Error Z Value Pr Z UN(1,1) SUBJECT UN(2,1) SUBJECT UN(2,2) SUBJECT Residual <.0001 Fit Statistics -2 Res Log Likelihood AIC (Smaller is Better) AICC (Smaller is Better) BIC (Smaller is Better) Null Model Likelihood Ratio Test DF Chi-Square Pr > ChiSq <.0001 Solution for Fixed Effects Effect GROUP Estimate Standard Error DF t Value Pr > t Intercept <.0001 GROUP GROUP GROUP TIME <.0001 TIME*GROUP TIME*GROUP TIME*GROUP Type 3 Tests of Fixed Effects Effect Num DF Den DF F Value Pr > F GROUP TIME <.0001 TIME*GROUP

21 The SAS System 1 The Mixed Procedure Second Analysis Model Information Data Set WORK.INDAT Dependent Variable RESPONSE ` Covariance Structure Subject Effect Estimation Method Residual Variance Method Fixed Effects SE Method Degrees of Freedom Method Unstructured SUBJECT REML Profile Model-Based Satterthwaite Class Level Information Class Levels Values GROUP Dimensions Covariance Parameters 2 Columns in X 8 Columns in Z per Subject 1 Subjects 50 Max Obs per Subject 7 Number of Observations Number of Observations Read 350 Number of Observations Used 252 Number of Observations Not Used 98 Iteration History Iteration Evaluations -2 Res Log Like Criterion Convergence criteria met.

22 The SAS System 2 The Mixed Procedure Covariance Parameter Estimates Cov Parm Subject Ratio Estimate Standard Error Z Value Pr > Z UN(1,1) SUBJECT <.0001 Residual <.0001 Fit Statistics -2 Res Log Likelihood AIC (Smaller is Better) AICC (Smaller is Better) BIC (Smaller is Better) Null Model Likelihood Ratio Test DF Chi-Square Pr > ChiSq <.0001 Solution for Fixed Effects Effect GROUP Estimate Standard Error DF t Value Pr > t Intercept <.0001 GROUP GROUP GROUP TIME <.0001 TIME*GROUP TIME*GROUP TIME*GROUP Type 3 Tests of Fixed Effects Effect Num DF Den DF F Value Pr > F GROUP TIME <.0001 TIME*GROUP

23 Example Analysis in SPSS MIXED RESPONSE BY GROUP WITH TIME /FIXED=GROUP TIME GROUP*TIME SSTYPE(3) /METHOD=REML /RANDOM=INTERCEPT SUBJECT(SUBJECT) COVTYPE(VC) /PRINT=SOLUTION. 22

24 DATASET ACTIVATE DataSet1. MIXED RESPONSE BY GROUP WITH TIME /FIXED=GROUP TIME GROUP*TIME SSTYPE(3) /METHOD=REML /RANDOM=INTERCEPT SUBJECT(SUBJECT) COVTYPE(VC) /PRINT=SOLUTION. Mixed Model Analysis Notes Output Created Comments Input Missing Value Handling Syntax Resources [DataSet1] Active Dataset Filter Weight Split File N of Rows in Working Data File Definition of Missing Cases Used Processor Time Elapsed Time DataSet1 <none> <none> <none> 10-FEB :02: User-defined missing values are treated as missing. Statistics are based on all cases with valid data for all variables in the model. MIXED RESPONSE BY GROUP WITH TIME /FIXED=GROUP TIME GROUP*TIME SSTYPE(3) /METHOD=REML /RANDOM=INTERCEPT SUBJECT(SUBJECT) COVTYPE (VC) /PRINT=SOLUTION. 00:00: :00:00.23 Page 1

25 Fixed Effects Random Effects Residual Total Intercept GROUP TIME GROUP * TIME Intercept b Model Dimension a Number of Levels Covariance Structure Number of Parameters Variance Components 9 8 Subject Variables 1 SUBJECT 1 a. Dependent Variable: RESPONSE. b. As of version 11.5, the syntax rules for the RANDOM subcommand have changed. Your command syntax may yield results that differ from those produced by prior versions. If you are using version 11 syntax, please consult the current syntax reference guide for more information. Information Criteria a -2 Restricted Log Likelihood Akaike's Information Criterion (AIC) Hurvich and Tsai's Criterion (AICC) Bozdogan's Criterion (CAIC) Schwarz's Bayesian Criterion (BIC) The information criteria are displayed in smaller-is-better forms. a. Dependent Variable: RESPONSE. Fixed Effects Type III Tests of Fixed Effects a Source Intercept GROUP TIME GROUP * TIME Numerator df Denominator df F Sig a. Dependent Variable: RESPONSE. Page 2

26 Estimates of Fixed Effects a Parameter Estimate Std. Error df t Sig. 95%... Lower Bound Intercept [GROUP=1] [GROUP=2] [GROUP=3] 0 b TIME [GROUP=1] * TIME [GROUP=2] * TIME [GROUP=3] * TIME 0 b Estimates of Fixed Effects a Parameter Intercept [GROUP=1] [GROUP=2] [GROUP=3] TIME [GROUP=1] * TIME [GROUP=2] * TIME [GROUP=3] * TIME 95%... Upper Bound a. Dependent Variable: RESPONSE. b. This parameter is set to zero because it is redundant. Covariance Parameters Estimates of Covariance Parameters a Parameter Estimate Std. Error Residual Intercept [subject = SUBJECT] Variance a. Dependent Variable: RESPONSE Page 3

27 Example Analysis in R library(lme4) library(lmertest) grp1 = (GROUP==1) grp2 = (GROUP==2) as.numeric(grp1) as.numeric(grp2) grp1tim = grp1*time grp2tim = grp2*time reres1= lmer(response ~ grp1 + grp2 + TIME + grp1tim + grp2tim + (1 SUBJECT), REML=TRUE) summary(reres1, ddf="satterthwaite") 26

28 Linear mixed model fit by REML t-tests use Satterthwaite approximations to degrees of freedom [mermodlmertest] Formula: RESPONSE ~ grp1 + grp2 + TIME + grp1tim + grp2tim + (1 SUBJECT) REML criterion at convergence: 1082 Scaled residuals: Min 1Q Median 3Q Max Random effects: Groups Name Variance Std.Dev. SUBJECT (Intercept) Residual Number of obs: 252, groups: SUBJECT, 50 Fixed effects: Estimate Std. Error df t value Pr(> t ) (Intercept) <2e-16 *** grp1true grp2true TIME <2e-16 *** grp1tim grp2tim Signif. codes: 0 *** ** 0.01 * Correlation of Fixed Effects: (Intr) g1true g2true TIME grp1tm grp1true grp2true TIME grp1tim grp2tim

29 Example - Continued Examining Contrasts proc mixed; class group; model response = group time group*time / solution ddfm=sat; random intercept / subject=subject; estimate lincomb group / cl alpha=0.05; contrast lincomb1 group ; contrast lincomb2 group , group ; run; 28

30 Estimates Standard Label Estimate Error DF t Value Pr > t lincomb Alpha Lower Upper Contrasts Num Den Label DF DF F Value Pr > F lincomb lincomb

31 Contrasts in R Direct Computation Using Formulas reres1= lmer(response ~ grp1 + grp2 + TIME + grp1tim + grp2tim + (1 SUBJECT), REML=TRUE) beta = fixef(reres1) omega = vcov(reres1) l1 = c(0,-0.5,-0.5,0,0,0) psi1 = t(l1) %*% beta vpsi1 = t(l1) %*% omega %*% l1 sdpsi1 = sqrt(vpsi1) cbind(psi1,sdpsi1) l2t = rbind(c(0,1,0,0,0,0),c(0,0,1,0,0,0)) phi = l2t %*% beta mdlmat = solve(l2t %*% omega %*% t(l2t)) f = t(phi) %*% mdlmat %*% phi / nrow(l2t) Output: > ans psi1 sdpsi > f 1 x 1 Matrix of class "dgematrix" [,1] [1,]

32 Inference for Covariance Parameters Recall: ˆθ θ N(0, Ω θθ ) Wald Test of H 0 : θ j = 0: Z W ald j = ˆθ j / Wald CI: ˆθ j ± z 1 α/2 Ω θθ jj Ω θθ jj Handling Positivity Constraints on Covariance Parameters Satterthwaite approach: Use the approximation ˆθ j χ2 ν θ j ν with ν = 2(Zj W ald ) 2 (matching distn of ˆθ j on first two moments) Transformation and delta method (Taylor approximation): log ˆθ j N(log θ j, Ω θθ jj/ˆθ 2 j) 31

33 Covariance Structures Recall the model Y i = X i β + Z i b i + ɛ i b i N(0, G(θ (1) )) ɛ i N(0, R i (θ (2) )) Need to specify the forms of the matrices G(θ (1) ) and R(θ (2) ) RANDOM statement specifies structure of G(θ (1) ) TYPE=VC means the elements of b i are independent TYPE=UN means that G(θ (1) ) is unstructured (this is the usual choice) REPEATED statement specifies structure of R(θ (2) ) Default structure of R(θ (2) ) is σ 2 I Other structures are available for longitudinal or spatial data 32

34 Structures for R(θ (2) ) for Longitudinal Data Many structures offered in PROC MIXED are appropriate only for data sets where all individuals are measured at the same set of times (except for missing entries). E.g. VC structure: ɛ ij N(0, σj 2 ) independent across j Some available structures (referred to as spatial ) are appropriate for more general measurement schedules E.g., SP(EXP) structure: Cov(ɛ ij, ɛ ik ) = σ 2 exp( d ijk /θ) In time series terms, this is an AR(1) structure 33

35 Example of Analysis with SP(EXP) Structure proc mixed covtest cl; class group; model response = group time group*time / solution ddfm=sat; random intercept / subject=subject; repeated / subject=subject type=sp(exp)(time); run; 34

36 The SAS System 1 The Mixed Procedure Model Information Data Set Dependent Variable Covariance Structures Subject Effects Estimation Method Residual Variance Method Fixed Effects SE Method Degrees of Freedom Method WORK.INDAT RESPONSE Variance Components, Spatial Exponential SUBJECT, SUBJECT REML Profile Model-Based Satterthwaite Class Level Information Class Levels Values GROUP Dimensions Covariance Parameters 3 Columns in X 8 Columns in Z per Subject 1 Subjects 50 Max Obs per Subject 7 Number of Observations Number of Observations Read 350 Number of Observations Used 252 Number of Observations Not Used 98 Iteration History Iteration Evaluations -2 Res Log Like Criterion

37 The SAS System 2 The Mixed Procedure Iteration History Iteration Evaluations -2 Res Log Like Criterion Convergence criteria met. Covariance Parameter Estimates Cov Parm Subject Estimate Standard Error Z Value Pr > Z Alpha Lower Upper Intercept SUBJECT SP(EXP) SUBJECT Residual < Fit Statistics -2 Res Log Likelihood AIC (Smaller is Better) AICC (Smaller is Better) BIC (Smaller is Better) Solution for Fixed Effects Effect GROUP Estimate Standard Error DF t Value Pr > t Intercept <.0001 GROUP GROUP GROUP TIME <.0001 TIME*GROUP TIME*GROUP TIME*GROUP Type 3 Tests of Fixed Effects Effect Num DF Den DF F Value Pr > F GROUP TIME <.0001 TIME*GROUP

38 Heterogeneous Variances Allow variance of random effect to depend on group Allow variance of error term to depend on group proc mixed cl covtest; class group; model response = group time group*time / solution ddfm=sat; random intercept / subject=subject group=group; repeated / subject=subject group=group; run; 37

39 The SAS System 1 The Mixed Procedure Model Information Data Set Dependent Variable Covariance Structure Subject Effects Group Effects Estimation Method Residual Variance Method Fixed Effects SE Method Degrees of Freedom Method WORK.INDAT RESPONSE Variance Components SUBJECT, SUBJECT GROUP, GROUP REML None Model-Based Satterthwaite Class Level Information Class Levels Values GROUP Dimensions Covariance Parameters 6 Columns in X 8 Columns in Z per Subject 3 Subjects 50 Max Obs per Subject 7 Number of Observations Number of Observations Read 350 Number of Observations Used 252 Number of Observations Not Used 98 Iteration History Iteration Evaluations -2 Res Log Like Criterion Convergence criteria met.

40 The SAS System 2 The Mixed Procedure Covariance Parameter Estimates Cov Parm Subject Group Estimate Standard Error Z Value Pr > Z Alpha Lower Upper Intercept SUBJECT GROUP Intercept SUBJECT GROUP Intercept SUBJECT GROUP Residual SUBJECT GROUP < Residual SUBJECT GROUP < Residual SUBJECT GROUP < Fit Statistics -2 Res Log Likelihood AIC (Smaller is Better) AICC (Smaller is Better) BIC (Smaller is Better) Solution for Fixed Effects Effect GROUP Estimate Standard Error DF t Value Pr > t Intercept <.0001 GROUP GROUP GROUP TIME <.0001 TIME*GROUP TIME*GROUP TIME*GROUP Type 3 Tests of Fixed Effects Effect Num DF Den DF F Value Pr > F GROUP TIME <.0001 TIME*GROUP

41 Comparing Models Can compare models using likelihood ratio test: 1. Run larger model, extract 2 log L 2. Run smaller model, extract 2 log L 3. Compute G 2 = difference in the value of 2 log L 4. Test significance based on the chi-square distn with r d.f., r = the number of added parameters in the larger model Important note: If we are changing fixed effects, LR test is valid only with ML If we are changing only random effects, can use ML or REML Example: Model with homogenous variance structure: 2 log L = Model with heterogeneous variance structure: 2 log L = G 2 = 5.8, r = 4, p-value =

42 Nested Random Effects Example 2 groups (fixed effect) 4 schools per group (1st level random effect) 2 classes per school (2nd level random effect) 5 students per class proc mixed ratio covtest; class group school class; model score = group / solution ddfm=sat; random intercept / subject=school v; random intercept / subject=class(school); run; 41

43 The SAS System 1 The Mixed Procedure Model Information Data Set Dependent Variable Covariance Structure Subject Effects Estimation Method Residual Variance Method Fixed Effects SE Method Degrees of Freedom Method WORK.INDAT score Variance Components school, class(school) REML Profile Model-Based Satterthwaite Class Level Information Class Levels Values group school class Dimensions Covariance Parameters 3 Columns in X 3 Columns in Z per Subject 3 Subjects 8 Max Obs per Subject 10 Number of Observations Number of Observations Read 80 Number of Observations Used 80 Number of Observations Not Used 0 Iteration History Iteration Evaluations -2 Res Log Like Criterion Convergence criteria met.

44 The SAS System 2 The Mixed Procedure Estimated V Matrix for school 1 Row Col1 Col2 Col3 Col4 Col5 Col6 Col7 Col8 Col9 Col Covariance Parameter Estimates Cov Parm Subject Ratio Estimate Standard Error Z Value Pr > Z Intercept school Intercept class(school) Residual <.0001 Fit Statistics -2 Res Log Likelihood AIC (Smaller is Better) AICC (Smaller is Better) BIC (Smaller is Better) Solution for Fixed Effects Effect group Estimate Standard Error DF t Value Pr > t Intercept group group Type 3 Tests of Fixed Effects Effect Num DF Den DF F Value Pr > F group

45 Estimation of Random Effects Y i b i N X iβ 0, V i GZ T i Z i G G b i Y i N(D i (Y i X i β), Q i ) ˆb i = D i (Y i X i β) Without the normality assumption, the above predictor is Best Linear Unbiased Predictor (BLUP), where best means minimum mean square error With estimates substituted for unknown parameters (as above), the term Empirical BLUP (EBLUP) is used 44

46 Estimation of Random Effects Continued Ignoring the estimation error in estimating the random effects parameters θ, we can write ˆβ β N(0, C) ˆb b C = C 11 C 12 C 21 C 22 where the C rs are matrices that can be written down 45

47 Prediction Intervals L T ˆβ β ˆb b N(0, L T CL) Prediction interval for L T β b is given by L T ˆβ ˆb ± z 1 α/2 [L T CL] 1/2 46

48 Example Gaucher s Disease Study Study of growth of children with Gaucher s disease Measurements are standardized height values Y ij = β 0 + β 1 t ij + β 3 X i + β 4 X i t ij + b i1 + b i2 t ij + ɛ j X i = group indicator (0/1) 47

49 proc mixed method=reml ratio covtest; model y = sex t sex*t / solution covb ddfm=satterth outpred=op; random int t / subject=id type=un; estimate pred1 intercept 1 t 1.5 sex 1 sex*t 1.5 intercept 1 t 1.5 / e sub ; 48

50 estimate pred2 intercept 1 t 3.5 sex 1 sex*t 3.5 intercept 1 t 3.5 / e sub ; run; proc print data=op; run; 49

51 The SAS System 1 Obs id t sex y

52 The SAS System 2 Obs id t sex y

53 The SAS System 3 Obs id t sex y

54 The SAS System 4 Obs id t sex y

55 The SAS System 5 The Mixed Procedure Model Information Data Set Dependent Variable Covariance Structure Subject Effect Estimation Method Residual Variance Method Fixed Effects SE Method Degrees of Freedom Method WORK.INDAT y Unstructured id REML Profile Model-Based Satterthwaite Dimensions Covariance Parameters 4 Columns in X 4 Columns in Z per Subject 2 Subjects 28 Max Obs per Subject 12 Number of Observations Number of Observations Read 136 Number of Observations Used 136 Number of Observations Not Used 0 Iteration History Iteration Evaluations -2 Res Log Like Criterion Convergence criteria met.

56 The SAS System 6 The Mixed Procedure Covariance Parameter Estimates Cov Parm Subject Ratio Estimate Standard Error Z Value Pr Z UN(1,1) id UN(2,1) id UN(2,2) id Residual <.0001 Fit Statistics -2 Res Log Likelihood 38.4 AIC (Smaller is Better) 46.4 AICC (Smaller is Better) 46.8 BIC (Smaller is Better) 51.8 Null Model Likelihood Ratio Test DF Chi-Square Pr > ChiSq <.0001 Solution for Fixed Effects Effect Estimate Standard Error DF t Value Pr > t Intercept sex t sex*t Covariance Matrix for Fixed Effects Row Effect Col1 Col2 Col3 Col4 1 Intercept sex t sex*t

57 The SAS System 7 The Mixed Procedure Type 3 Tests of Fixed Effects Effect Num DF Den DF F Value Pr > F sex t sex*t Coefficients for pred1 Effect Subject Row1 Intercept 1 sex 1 t 1.5 sex*t 1.5 Intercept 1 1 t Intercept 2 t 2 Intercept 3 t 3 Intercept 4 t 4 Intercept 5 t 5 Intercept 6 t 6 Intercept 7 t 7 Intercept 8 t 8 Intercept 9 t 9 Intercept 10 t 10 Intercept 11 t 11 Intercept 12 t 12 Intercept 13

58 The SAS System 8 The Mixed Procedure Coefficients for pred1 Effect Subject Row1 t 13 Intercept 14 t 14 Intercept 15 t 15 Intercept 16 t 16 Intercept 17 t 17 Intercept 18 t 18 Intercept 19 t 19 Intercept 20 t 20 Intercept 21 t 21 Intercept 22 t 22 Intercept 23 t 23 Intercept 24 t 24 Intercept 25 t 25 Intercept 26 t 26 Intercept 27 t 27 Intercept 28 t 28

59 The SAS System 9 The Mixed Procedure Coefficients for pred2 Effect Subject Row1 Intercept 1 sex 1 t 3.5 sex*t 3.5 Intercept 1 1 t Intercept 2 t 2 Intercept 3 t 3 Intercept 4 t 4 Intercept 5 t 5 Intercept 6 t 6 Intercept 7 t 7 Intercept 8 t 8 Intercept 9 t 9 Intercept 10 t 10 Intercept 11 t 11 Intercept 12 t 12 Intercept 13 t 13 Intercept 14 t 14 Intercept 15 t 15 Intercept 16

60 The SAS System 10 The Mixed Procedure Coefficients for pred2 Effect Subject Row1 t 16 Intercept 17 t 17 Intercept 18 t 18 Intercept 19 t 19 Intercept 20 t 20 Intercept 21 t 21 Intercept 22 t 22 Intercept 23 t 23 Intercept 24 t 24 Intercept 25 t 25 Intercept 26 t 26 Intercept 27 t 27 Intercept 28 t 28 Estimates Label Estimate Standard Error DF t Value Pr > t pred pred

61 The SAS System 11 Obs id t sex y Pred StdErrPred DF Alpha Lower Upper Resid

62 The SAS System 12 Obs id t sex y Pred StdErrPred DF Alpha Lower Upper Resid

63 The SAS System 13 Obs id t sex y Pred StdErrPred DF Alpha Lower Upper Resid

64 The SAS System 14 Obs id t sex y Pred StdErrPred DF Alpha Lower Upper Resid

65 Example Fay-Herriot Model for Sample Surveys Y i = µ + b i + ɛ i b i N(0, τ 2 ), ɛ i N(0, σi 2 ), σi 2 known n n ˆµ = c i Y i / c i, c i = τ 2 /(τ 2 + σi 2 ) i=1 i=1 ˆbi = c i (Y i ˆµ) Ŷ i = ˆµ + ˆb i = (1 c i )ˆµ + c i Y i 64

66 Fay-Herriot Model Continued data fh_example; input area y sig2; sig = sqrt(sig2); cards; [SNIP] proc mixed; model y = / solution ddfm=bw; random intercept sig / subject=area type=vc solution; parms (1) (1) (1e-6) / hold=2,3; run; data newdat; set fh_example; muhat = ; tau2hat = ; wt = tau2hat/(tau2hat+sig2); bhat = wt*(y-muhat); ypred = (1-wt)*muhat + wt*y; proc print; run; 65

67 ' 1 The Mixed Procedure Model Information Data Set Dependent Variable Covariance Structure Subject Effect Estimation Method Residual Variance Method Fixed Effects SE Method Degrees of Freedom Method WORK.FH_EXAMPLE y Variance Components area REML Parameter Model-Based Between-Within Dimensions Covariance Parameters 3 Columns in X 1 Columns in Z per Subject 2 Subjects 22 Max Obs per Subject 1 Number of Observations Number of Observations Read 22 Number of Observations Used 22 Number of Observations Not Used 0 Parameter Search CovP1 CovP2 CovP3 Res Log Like -2 Res Log Like E Iteration History Iteration Evaluations -2 Res Log Like Criterion Convergence criteria met.

68 ' 2 The Mixed Procedure Covariance Parameter Estimates Cov Parm Subject Estimate Intercept area sig area Residual 1E-6 Fit Statistics -2 Res Log Likelihood 13.8 AIC (Smaller is Better) 15.8 AICC (Smaller is Better) 16.0 BIC (Smaller is Better) 16.9 PARMS Model Likelihood Ratio Test DF Chi-Square Pr > ChiSq <.0001 Solution for Fixed Effects Effect Estimate Standard Error DF t Value Pr > t Intercept <.0001 Solution for Random Effects Effect Subject Estimate Std Err Pred DF t Value Pr > t Intercept sig Intercept sig Intercept sig Intercept sig Intercept sig Intercept sig Intercept

69 ' 3 The Mixed Procedure Solution for Random Effects Effect Subject Estimate Std Err Pred DF t Value Pr > t sig Intercept sig Intercept sig Intercept sig Intercept sig Intercept sig Intercept sig Intercept sig Intercept sig Intercept sig Intercept sig Intercept sig Intercept sig Intercept sig Intercept sig Intercept sig

70 ' 4 Obs area y x sig2 sig muhat tau2hat wt bhat ypred

71 Unit 2 Nonlinear Mixed Models 70

72 Example 1 Hospital Comparisons Want to compare hospitals with respect to surgical mortality rates Response variable: alive or dead Have data on a number of hospitals Want to analyze variation across hospitals Want to account for patient-level factors ( case mix ) 71

73 Example 2 Toxicology Study female mice assigned to different doses of a chemical fetuses examined for malformations Example 3 Booster Seat Study (HU Public Health School) 11 neighborhoods, average of 50 children sampled per neighborhood Response variable: child uses booster seat (yes/no) interested in effect of neighborhood-level variables effect of child-level variables variation across neighborhoods 72

74 Logistic Regression With Random Effect The Setup i = cluster (i = 1,..., n) j = individual within cluster (j = 1,..., J i ) Y ij = 0 1 binary response X ijk = value of explanatory variable k for individual ij (k = 1,..., p) We regard the X ijk s as fixed quantities. Clusters are assumed independent. The Concept Random effect b i used to express within-cluster dependence Responses within each cluster assumed conditionally independent given the b i s 73

75 Software SAS: PROC NLMIXED PROC GLMMIX NLMIXED is more flexible SPSS: GENLINMIXED Estimation is not by maximum likelihood, but instead by penalized quasi-likelihood, which is an inferior method R: glmer function in lme4 package The SAS procedures are superior to the R functions in terms of the options offered. 74

76 Model Specification Leads to p(x ij, β, b i ) = Pr(Y ij = 1 b i ) = exp(βt x ij + b i ) 1 + exp(β T x ij + b i ) f Yi b i (y i b i ) = Pr(Y ij = y ij, j = 1,..., J i b i ) = J i j=1 p(x ij, β, b i ) y ij (1 p(x ij, β, b i )) 1 y ij f bi (b i ) = σ 1 b φ(b i /σ b ) (i.e. b i N(0, σ 2 b)) f Yi (y i ) = Pr(Y ij = y ij, j = 1,..., J i ) J i = p(x ij, β, b i ) y ij (1 p(x ij, β, b i )) 1 y ij σ 1 b φ(b i /σ b )db i = j=1 J i p(x ij, β, σ b ξ i ) y ij (1 p(x ij, β, σ b ξ i )) 1 y ij φ(ξ i )dξ i j=1 75

77 Computation of Integrals Laplace Approximation: valid if J i is large for all i Adaptive Gaussian Quadrature (AGQ) Simulation (offered only by SAS and not by R) 76

78 Statistical Inference for Parameters Estimation of φ = (β, σb 2 ): Maximum Likelihood (or REML) Large Sample Distribution Types of Inference ˆφ φ N(0, Ω( ˆφ)) Ω(φ) = inverse Fisher information matrix Inference for Individual Fixed-Effect Coefficients Testing a Single Linear Combination of Fixed Effects Testing Several Linear Combinations Similar to LMM case, with similar syntax in SAS PROC NLMIXED Also: Wald test for significance of σ 2 b Confidence interval for σ 2 b 77

79 Estimation of Random Effects By Bayes theorem, the conditional density of b i given Y i is given by f bi Y i (b i Y i ) = f Y i b i (Y i b i )f bi (b i ) f Yi (Y i ) Expressions for the quantities on the right side have been given previously. In principle, we could estimate b i using bi f Yi b ˆbi = E[b i Y i ] = i (Y i b i )f bi (b i )db i f Yi (Y i ) and we could use f bi Y i (b i Y i ) to compute an empirical Bayes 95% credible interval for b i. SAS PROC NLMIXED and the R package lme4 use an approximate method, valid for large J i. 78

80 Estimation of Random Effects Continued Let b i be the value of b that maximizes f b i Y i (b Y i ) (posterior mode). Define ḡ i (b) = J 1 i log(f bi (b)f Yi b i (Y i b)) Finally, define v i = ḡ i (b i ) 1 The approximation is then b i Y i N(b i, J 1 i v i ) 79

81 Estimation of Random Effects Continued Now, b i = b i (ψ). In practice, we have to substitute ˆψ for ψ. SAS PROC MIXED offers a correction for this. The R function lmer4 does not. Defining c = b i / ψ, the correction is as follows: Cov ˆψ Y i = Ω Ωc b i c T Ω Ji 1 v i + c T Ωc Using this result and the delta method, we can compute credible intervals for functions of ψ and b i, such as p(x, β, b i ) for a given x. 80

82 Example 1 Mortality rates in 12 hospitals performing cardiac surgery in babies proc nlmixed; parms beta=-2.5 sigb2=1; bounds sigb2 >=0; odds = exp(beta+bi); p = odds/(1+odds); model d ~ binomial(n,p); random bi ~ normal(0,sigb2) subject=hospital out=odat1; predict p out=odat2; run; proc print data=odat1; proc print data=odat2; run; 81