36-720: Linear Mixed Models

Transcription

1 36-720: Linear Mixed Models Brian Junker October 8, 2007 Review: Linear Mixed Models (LMM s) Bayesian Analogues Facilities in R Computational Notes Predictors and Residuals Examples [Related to Christensen Chapter 3; Borrows from: Peter Dalgaard s short course athttp://staffpubhealthkudk/ pd/mixed-jan2006/; Brian Ripley slme4 notes athttp://wwwstatsoxacuk/ ripley/] October 8, 2007 The Model Laird & Ware (982): Review: Linear Mixed Models (LMM s) y i = X i β+z i u+ε i, i=,,n where u N(0,Ψ q q ),ε N(0,σ 2 εi); β (p vector) are the fixed effects; u (q vector) are the random effects; X i is a known n i p matrix; Z i is a known n i q matrix Usually the X i and Z i are strung together into a fixed effects design matrix X, and a random effects design matrix Z In many cases, the components of u are uncorrelated with each other and withε (in particular,ψwould be diagonal) October 8, 2007

2 Examples: Measurement error model Batches of i =,, 3 widgets each come off of j=,,3 different assembly lines There is too much variation between the widgets to justify the null model y i j =β 0 +ε i j, but not enough to justify the one-way ANOVA model y i j =β 0 +β j +ε i j An intermediate model is the measurement error model y i =β 0 + u j +ε i j where u iid iid i N(0,ψ) andε i j N(0,σ 2 ε ) Here, Y= Xβ+Zu+ε takes the form y 0 0 ε y ε 2 y 3 y 2 y 22 y 32 = β u u 2 u 3 + ε 3 ε 2 ε 22 ε 32 y ε 3 y 23 y ε 23 ε October 8, 2007 Growth curve models Toxins accumulate in muscles over time as the muscles are used There is an overall shape to the growth (say, linear) but due to individual variations no one individual follows the overall trend exactly Again, there may be too much variability among individuals to have a single linear regression, but not enough to estimate fully separate slopes and intercepts for each individual A compromise model would be y i j = (β 0 +β t j )+(u 0i + u i t j )+ε i j = (β 0 + u 0i )+(β + u i )t j +ε i j for individual i at time t j Thus, each individual gets his or her own growth curve, with slope and intercept sampled from a distribution whose mean is the overal trend lineβ 0 +β t Here, Y= Xβ+Zu+ε where y y J = X= t t J β 0 β + t t J u u 2 + ε ε J y I y IJ t t J t t J u I u 2I ε I ε IJ October 8, 2007

3 Bayesian Analogues There is always a hierarchical Bayes model equivalent to a mixed model The model y i j =β 0 + u j +ε i j is equivalent to Level 3: Level 2: Level : β 0 (flat hyperprior) µ j iid N(β 0,ψ) y i j indep N(µ j,σ 2 ε) The model y it = (β 0 + u 0i )+(β + u i )t+ε it is equivalent to Level 3: Level 2: Level : β 0,β (flat hyperpriors) µ 0i iid N(β 0,ψ 0 ),µ i iid N(β,ψ ) y i j indep N(µ 0 j +µ i t j,σ 2 ε ) Thus the results of mixed model analysis and computational Bayes posterior (direct, Laplace, MCMC, etc posterior computation) can be compared, or one used when the other is inconvenient, etc October 8, 2007 Facilities in R R (and Splus) provide two default packages for LMM analysis library(nlme) provides lme() for working with y i = X i β+z i u+ε i nlme() for working with y i = f (X i, Z i ;β, u i )+ε i These can be complex to use, but are more flexible for modeling random effects, and correlations among and between the u s andε s library(lme4) is a rewrite ofnlme that provideslmer(), for both LMM s and GLMM s lmer() has a simpler modeling language, but less well-developed methods functions for extracting fits, residuals, plots, etc As with all things, R is great for breadboarding, and for analyses of problems of moderate size For larger problems, SAS provides PROC MIXED and PROC NLMIXED, that provide approximately the same functionality aslme, nlme andlmer October 8, 2007

4 Computational Notes Consider the general LMM formulation Y= Xβ+Zu+ε Usuallyβis constant over subjects,εis iid between subjects, and the variance-covariance matrixψ=var (u) depends on only a few free parameters ω Assuming Cov (u, ε) = 0, Y V(ω) N(Xβ, V(ω)) = Var (ε)+zψ(ω)z T so 2 log(likelihood) is (Y Xβ) T V (ω)(y Xβ)+log V(ω) ( ) To find MLE s we can iterate betwen minimizing inωgivenβ, and minimizing inβgivenω; the latter is generalized least-squares (indeed,library(nlme) also providesgls andngls functions) October 8, 2007 Restricted Maximum Likelihood (REML) To reduce the amount of iteration for ML, we can compute a linear transformation AY whose distribution is independent ofβ, eg AY= Y ˆβ OLS X (ignoring the error structure) Then we calculate ˆω REML by minimizing (AY) T V (ω)ay+ log V(ω) and re-polish ˆβ REML by GLS in ( ) This turns out to be invariant to choice of A Ripley slme4 notes observe that the REML estimates minimize (Y Xβ) T V (ω)(y Xβ)+log V(ω) +log X T V(ω)X Since REML is a bit faster, it is often the default method However for LR and AIC/BIC comparisons, full MLE s must be computed October 8, 2007

5 Predictors and Residuals Best linear unbiased predictors (BLUP s) of the data are Ŷ BLUP = X ˆβ ML + Zû post where û post = Ê[u Y] These are linear functions of the data, and hence are unbiased (and minimum variance) Residuals can be computed at different levels in the model For example in the model y i j =β 0 + u j +ε i j, Level residuals would be ri 0 j = y i j y BLUP i j ; standardized or Pearson level residuals would be r 0 i j /SE(r0 i j ); Level 0 residuals would be ri j = y i j ˆβ 0 ; standardized or Pearson level 0 residuals would be ri j /SE(r i j ) Note that fitted values and residuals are numbered in the opposite order from Bayesian hierarchical levels [higher levels nearer the data, not farther from it] fitted() andresiduals() understand these things forlme() Forlmer(), they only understand how to compute the raw residuals nearest the data October 8, 2007 Examples A measurement error model for batches of dyes A growth curve model for spinal bone marrow density [See R notes in class] These examples only scratch the surface For more details, see Dalgaard s or Ripley s notes; see also the appendix to Fox s Applied Linear Regression text, onlme (a pdf of this is stored with these class notes online for the class) October 8, 2007