Introduction to Mixed Models in R

Transcription

1 Introduction to Mixed Models in R Galin Jones School of Statistics University of Minnesota galin March 2011

2 Second in a Series Sponsored by Quantitative Methods Collaborative. Previous workshop An Introduction to R was given on 3/7 by Professor Sanford Weisberg. Upcoming talk on 4/11 by Chris Winship of Harvard University on causal inference.

3 Goals Brief review of first workshop. Definition of mixed models and why they may be useful. Using mixed models in R through two simple case studies.

4 Review R is free software and can be downloaded at A package is a collection of functions designed for specific tasks. For example the package lme4 fits many mixed models. When you install R on your computer you get the base distribution. This includes many useful packages but others, such as lme4, you need to install separately. The R search engine is extremely useful Getting data into R (or any other software package) can be challenging. Many R packages include built-in data sets and we will use two of these today.

5 Mixed Models Mixed models are a large and complex topic, we will only just barely get started with them today. There are many varieties of mixed models: Linear mixed models (LMM) Nonlinear mixed models (NLM) Generalized linear mixed models (GLMM) Our focus will be on linear mixed models. Much more discussion of this material can be found in the following books. Extending the Linear Model with R by Julian Faraway Mixed-Effects Models in S and S-PLUS by José Pinheiro and Douglas Bates

6 Factors, Effects and Treatments Suppose apple slices are treated with five preservative compounds (A, B, C, D, E) with the goal of extending shelf life. Response: Shelf life Factor: Preservative Treatments = Levels of a factor: A, B, C, D, E Effect: Impact of compound on shelf life µ=population mean shelf life µ A = population mean shelf life with treatment A effect of A = µ A µ

7 Fixed or Random? Factor effects are either fixed or random. Fixed: The levels in the study represent all levels of interest Random: The levels in the study represent only a sample of the levels of interest. Mixed models have both fixed and random effects. In our example, preservative is fixed since A, B, C, D, E are the only levels of interest.

8 Block: Blocking A group of units formed so that units within the group are as homogeneous as possible. Reduces the effects of variation among experimental units. Treatmets are randomly assigned to blocks. Lets add to the example: Suppose 10 individual fruit are randomly chosen from a population of fruit and the 5 preservatives are randomly assigned to 5 portions of fruit. Preservative is a fixed effect. Fruit is a (random) block effect.

9 Mixed models Mixed models contain both fixed and random effects This has several ramifications: Using random effects broadens the scope of inference. That is, inferences can be made on a statistical basis to the population from which the levels of the random factor have been drawn. Naturally incorporates dependence in the model. Observations that share the same level of the random effects are being modeled as correlated. Using random factors often gives more accurate estimates. Sophisticated estimation and fitting methods must be used.

10 Rail Data The Rail data is a built-in R data set. It can be found in the MEMSS and the NLME packages. The commands > library(nlme) > data(rail) >?Rail will make the data available to us and produce a description.

11 Rail Data Evaluation of Stress in Railway Rails Description The Rail data frame has 18 rows and 2 columns. Format This data frame contains the following columns: Rail an ordered factor identifying the rail on which the measurement was made. travel a numeric vector giving the travel time for ultrasonic head-waves in the rail (nanoseconds). The value given is the original travel time minus 36,100 nanoseconds.

12 Rail Data Summary Six rails were chosen from a group of rails. Each rail was tested 3 times. Measured the time it takes an ultrasonic wave to travel the length of a rail. Rail is a factor (fixed or random?) and travel is the response.

13 > Rail Grouped Data: travel ~ 1 Rail Rail travel Rail Data

14 Rail Data > summary(rail) Rail travel 2:3 Min. : :3 1st Qu.: :3 Median : :3 Mean : :3 3rd Qu.: :3 Max. : > with(rail, tapply(travel, Rail, mean)) > pdf(file="railplot1.pdf") > with(rail, plot(travel, Rail, xlab="travel time")) > dev.off()

15 Rail Data Rail Travel time

16 Rail Data Between-rail variability is greater than within-rail variability. Within-rail variability is not constant. Mean travel time appears different for some rails We clearly need to account for the classification factor (Rail) in the analysis.

17 Rail as a fixed effect Rail Data where y ij = β i + e ij i = 1,..., 6 j = 1, 2, 3 y ij is the observed travel time for observation j on rail i. β i is the population mean travel time of rail i e ij are independent and identically normally distributed with mean 0 and variance σ 2 or e ij iid N(0, σ 2 ). This asumption means y ij ind N(β i, σ 2 ) > #Rail as a fixed effect > r1.lm<-lm(travel ~ Rail - 1, data=rail)

18 Rail Data > r1.lm Call: lm(formula = travel ~ Rail - 1, data = Rail) Coefficients: Rail2 Rail5 Rail1 Rail6 Rail3 Rail This means that the oredered estimates of the β i are β 2 = 31.67,..., β 4 = 96.00

19 Rail Data > summary(r1.lm) Coefficients: Estimate Std. Error t value Pr(> t ) Rail e-08 ***.. Rail e-14 *** Residual standard error: on 12 degrees of freedom Multiple R-squared: ,Adjusted R-squared: F-statistic: on 6 and 12 DF, p-value: 2.971e-15 Note that the F-statistic and p-value are testing for any differences between the rail effects. Also, the estimate of σ 2 is on 12 degrees of freedom.

20 Rail Data > with(rail, bwplot(rail ~ residuals(r1.lm))) residuals(r1.lm)

21 Rail Data The fixed effects model gives a good summary of the data but the main interest is in the population of rails. Also, we don t believe the model is accurate since the 3 observations on each rail are clearly not independent. Rail as a random effect y ij = β + b i + e ij where y ij is the observed travel time for observation j on rail i. β is the population mean travel time b i is the deviation from β for the ith rail

22 Rail Data e ij is the deviation for observation j on rail i from the mean travel time for rail i b i iid N(0, σ 2 b ) e ij iid N(0, σ 2 ). Our assumptions imply that observations on the same rail are correlated, in fact, corr = σ2 b σ 2 b + σ2.

23 Rail Data > #Rail as random effect > r2.lme<-lmer(travel ~ 1 + (1 Rail), REML=FALSE, data=rail) This notation takes some getting used to. Specifically, (1 Rail) means that there is a single random factor which is constant within each level and its levels are given by the grouping variable Rail.

24 Rail Data > summary(r2.lme) Linear mixed model fit by maximum likelihood Formula: travel ~ 1 + (1 Rail) Random effects: Groups Name Variance Std.Dev. Rail (Intercept) Residual Number of obs: 18, groups: Rail, 6 Our best guess at σ 2 and σ 2 b are ˆσ 2 b = ˆσ2 = and the estimated correlation between observations on a rail is ĉorr = =

25 Fixed effects: Estimate Std. Error t value (Intercept) Rail Data The estimate of β is ˆβ = For a new rail drawn for the population this is our best guess at travel time while the predicted values for a new observation on each of the same 6 rails are > fixef(r2.lme)+ranef(r2.lme)$rail (Intercept)

26 Rail Data > qqnorm(resid(r2.lme), main="") Sample Quantiles Theoretical Quantiles

27 Rail Data > plot(fitted(r2.lme), resid(r2.lme), xlab="fitted", ylab="residuals") Residuals Fitted

28 Multilevel Models Multilevel models are special cases of mixed models, are useful for data with a hierarchical structure, and can be implemented in a variety of R packages.

29 Joint Schools Project > library(faraway) > data(jsp) >?jsp Description Example Dataset from "Practical Regression and Anova" Format See for yourself Source See Reference References Reference details may be found in "Practical Regression and Anova" by Julian Faraway

30 Joint Schools Project > str(jsp) data.frame : 3236 obs. of 9 variables: $ school : Factor w/ 49 levels "1","2","3","4",..: $ class : Factor w/ 4 levels "1","2","3","4": $ gender : Factor w/ 2 levels "boy","girl": $ social : Factor w/ 9 levels "1","2","3","4",..: $ raven : num $ id : Factor w/ 1192 levels "1","2","3","4",..: $ english: num $ math : num $ year : num

31 Joint Schools Project > summary(jsp) school class gender social raven 48 : 206 1:1949 boy : :1225 Min. : : 131 2: 987 girl: : 484 1st Qu.: : 131 3: : 424 Median : : 107 4: : 288 Mean : : : 270 3rd Qu.: : : 221 Max. :36.00 (Other):2458 (Other): 324 id english math year 1 : 3 Min. : 0.00 Min. : 1.00 Min. : : 3 1st Qu.: st Qu.: st Qu.: : 3 Median :54.00 Median :28.00 Median : : 3 Mean :52.49 Mean :26.66 Mean : : 3 3rd Qu.: rd Qu.: rd Qu.: : 3 Max. :98.00 Max. :40.00 Max. : (Other):3218

32 Joint Schools Project #Subset data to focus on Year=2 > jsp.year2<-jsp[jsp$year==2,] > plot(jitter(math) ~ jitter(raven), xlab="raven Score", ylab="math Score", data=jsp.year2) > boxplot(math ~ social, xlab="social Class", ylab="math Score", data=jsp.year2) > boxplot(math ~ gender, xlab="gender", ylab="math Score", data=jsp.year2)

33 Joint Schools Project Raven Score Math Score There is clearly correlation between math score and raven score.

34 Joint Schools Project Math Score Social Class There are differences in math scores between the levels of social class.

35 Joint Schools Project boy girl Gender Math Score Math scores are not different between the levels of gender.

36 Joint Schools Project Center raven since we would otherwise be comparing to zero. > jsp.y2$ctrraven<-jsp.y2$raven-mean(jsp.y2$raven) > jsp1.lme<-lmer(math ~ ctrraven*social*gender + (1 school) + (1 school:class), data=jsp.y2) > qqnorm(resid(jsp1.lme),main="") > plot(fitted(jsp1.lme) ~ resid(jsp1.lme), xlab="fitted", ylab="residuals")

37 Joint Schools Project Theoretical Quantiles Sample Quantiles There isn t much of concern here.

38 Joint Schools Project Fitted Residuals There is some evidence of non-constant variance. Transformation?

39 Joint Schools Project > anova(jsp1.lme) Analysis of Variance Table Df Sum Sq Mean Sq F value ctrraven social gender ctrraven:social ctrraven:gender social:gender ctrraven:social:gender

40 Joint Schools Project The p-values associated with the F -statistics are approximate and can be too small when the number of cases is small. However, this data set is probably large enough to overcome this limitation. The parametric bootstrap can be used if this is a concern. > nrow(jsp.y2) - sum(anova(jsp1.lme)[,1]) - 1 [1] 917 > round(pf(anova(jsp1.lme)[,4], anova(jsp1.lme)[,1], 917, lower.tail=false),4) [1]

41 Joint Schools Project > #Remove Gender > jsp2.lme<-lmer(math ~ ctrraven*social + (1 school) + (1 school:class), data=jsp.y2) > qqnorm(resid(jsp2.lme),main="") > plot(fitted(jsp2.lme) ~ resid(jsp2.lme), xlab="fitted", ylab="residuals") > qqnorm(ranef(jsp2.lme)$"school:class"[[1]], main="school Effects") > qqnorm(ranef(jsp2.lme)$"school:class"[[1]], main="class Effects")

42 Joint Schools Project Theoretical Quantiles Sample Quantiles Still not much of concern.

43 Joint Schools Project Fitted Residuals Constant variance is still a problem.

44 Joint Schools Project School Effects Theoretical Quantiles Sample Quantiles

45 Joint Schools Project Class Effects Theoretical Quantiles Sample Quantiles

46 Joint Schools Project > anova(jsp2.lme) Analysis of Variance Table Df Sum Sq Mean Sq F value ctrraven social ctrraven:social > nrow(jsp.y2) - sum(anova(jsp2.lme)[,1]) - 1 [1] 935 > round(pf(anova(jsp2.lme)[,4], anova(jsp2.lme)[,1], 935, lower.tail=false),4) [1]

47 Joint Schools Project > sch.effects<-ranef(jsp2.lme)$school[[1]] > summary(sch.effects) Min. 1st Qu. Median Mean 3rd Qu. Max > raw.sch.effects<-coef(lm(math ~ school-1,jsp.y2)) > raw.sch.effects<-raw.sch.effects-mean(raw.sch.effects) > plot(raw.sch.effects,sch.effects) > sint<-c(9,14,29) > text(raw.sch.effects[sint],sch.effects[sint]+0.2, c("9","15","30"))

48 Joint Schools Project raw.sch.effects sch.effects