HW 2 due March 6 random effects and mixed effects models ELM Ch. 8 R Studio Cheatsheets In the News: homeopathic vaccines

Transcription

1 Today HW 2 due March 6 random effects and mixed effects models ELM Ch. 8 R Studio Cheatsheets In the News: homeopathic vaccines STA 2201: Applied Statistics II March 4, /35

2 A general framework y γ = Xβ + Z γ + ɛ, ɛ N(0, σ 2 Λ) γ: q vector of random effects assumption γ N(0, σ 2 D) β: p-vector of fixed effects marginal distribution y N(Xβ, σ 2 (Λ + ZDZ T )) = N(Xβ, σ 2 V ), say applications multi-level models repeated measures longitudinal data components of variance STA 2201: Applied Statistics II March 4, /35

3 Illustration SM Example 9.16 STA 2201: Applied Statistics II March 4, /35

4 Estimation y N(Xβ, σ 2 (Λ + ZDZ T )) = N(Xβ, σ 2 V ) l(β; y) = n 2 log(σ2 ) 1 1 log V 2 2σ 2 (y Xβ)T V 1 (y Xβ) V may have one or more unknown parameters Example 9.16: γ N 3 (0, σ 2 b I), ɛ N(0, σ2 I) Λ+ZDZ T = 1 + σb/σ 2 2 σb/σ σb/σ σb/σ σb/σ σb/σ 2 2 σb/σ σb/σ σb/σ 2 2 ˆβψ = (X T V 1 X) 1 X T V 1 y STA 2201: Applied Statistics II March 4, /35

5 ... estimation ˆβ ψ = (X T V 1 X) 1 X T V 1 y profile log-likelihood l p (σ 2, ψ) = n 2 log σ2 1 2 log V ψ 1 2σ 2 (y X ˆβ ψ ) T V 1 ψ (y X ˆβ ψ ) to get better divisors properly adjust for degrees of freedom modified profile log-likelihood l mp (σ 2, ψ) = n p 2 also called restricted profile log-likelihood log σ log V ψ 1 2 log X T V 1 ψ X 1 2σ 2 (y X ˆβ ψ ) T V 1 ψ (y X ˆβ ψ ) estimation of σ 2, and ψ (parameters in V ) available in most software by the name REML STA 2201: Applied Statistics II March 4, /35

6 Example: Growth Data SM Example 9.18 STA 2201: Applied Statistics II March 4, /35

7 ... growth data y y week week data(rat.growth, library="smpracticals")} with(rat.growth, plot(week, y, type="l", col = levels(rat))) qplot(week, y, data = rat.growth, geom = "path", colour = rat) last_plot() + theme(legend.position = "none") STA 2201: Applied Statistics II March 4, /35

8 Example 9.18 repeated measurements on the 30 individuals, at 5 time points fixed effects model: y jt = µ + γ j + β 1 x jt + ɛ jt, t = 1,..., 5 x jt = x t takes values 0, 1, 2, 3, 4 for t = 1, 2, 3, 4, 5 or even y jt = µ + γ j + α t + ɛ jt rats as blocks, time as treatment random effects model y jt = β 0 + γ 0 j + (β 1 + γ 1 j )x jt + ɛ jt, t = 1,..., 5 (γ 0 j, γ 1 j ). N 2 (0, σ 2 D), ɛ jt. N(0, σ 2 ) independent two fixed parameters β 0, β 1 four variance/covariance parameters: σ 2 g0, σ2 g1, σ g01, σ 2 STA 2201: Applied Statistics II March 4, /35

9 ... Example 9.18 maximum likelihood estimates of fixed effects: ˆβ 0 = (2.16), ˆβ 1 = 43.27(0.73) weight in week 1 is estimated to be about 156 units, and average increase per week estimated to be there is large variability between rats: estimated standard deviation of for intercept, 3.53 for slope there is little correlation between the intercepts and slopes separate.lm = lm(y week + factor(rat)+ week:factor(rat), data = rat.growth) # fit separate linear models to each set of 5 observations library(lme4) rat.mixed = lmer(y week + (week rat), data = rat.growth) # REML is the default summary(rat.mixed) # Fixed effects: Estimate Std. Error t value (Intercept) week STA 2201: Applied Statistics II March 4, /35

10 ... Example 9.18 there is large variability between rats estimated standard deviation of for intercept, 3.53 for slope there is little correlation between the intercepts and slopes summary(rat.mixed) # Random effects: Groups Name Variance Std.Dev. Corr rat (Intercept) week Residual Number of obs: 150, groups: rat, 30 var(γ 0 j ) = ; var(γ 1 j ) = σ 2 = STA 2201: Applied Statistics II March 4, /35

11 Example: Panel Study of Income Dynamics ELM, income year library(lattice) xyplot(income year person, data = psid, type="l", subset = (person < 21), strip = F) STA 2201: Applied Statistics II March 4, /35

12 ... PSID log(income + 100) F M year psid$cyear = psid$year head(psid) age educ sex income year person cyear M M M M : Applied Statistics March 2015 STA II 4, 12/35

13 ... PSID > mmod = lmer(log(income) cyear*sex + age + educ + + (cyear person), data=psid) log(income) ij = µ + γ 0 j + α year i + γ 1 j year i + β sex j + αβ (year i sex j ) + β 2 educ j + β 3 age j + ɛ ij, ɛ ij N(0, σ 2 ), γ j N 2 (0, σ 2 D) we could fit separate lines for each subject as with rat growth data this would give us 85 slopes and 85 intercepts we could compare these slopes and intercepts between genders two-sample test analysis of derived responses is often simple, but sometimes limited see p.188 STA 2201: Applied Statistics II March 4, /35

14 ... PSID using lmer compare random effects model to fixed effects model: > mmod = lmer(log(income) cyear*sex + age + educ + + (cyear person), data=psid) Fixed effects: Estimate Std. Error t value (Intercept) cyear sexm age educ cyear:sexm > lmod = lm(log(income) cyear*sex + age + educ, data = paid) Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) <2e-16 *** cyear <2e-16 *** sexm <2e-16 *** age educ <2e-16 *** cyear:sexm * Residual standard error: on 1655 degrees of freedom coefficients the same; standard errors for lm much smaller 1655 degrees of freedom? all observations treated as independent STA 2201: Applied Statistics II March 4, /35

15 Inference for fixed effects log(income) ij = µ + γ 0 j + α year i + γ 1 j year i + β sex j + αβ (year i sex j ) + β 2 educ j + β 3 age j + ɛ ij, ɛ ij N(0, σ 2 ), γ j N 2 (0, σ 2 D) ˆβ = (X T ˆV 1 X) 1 X T ˆV 1 y, σ 2 by REML s.e.( ˆβ j ) = { σ 2 (X T ˆV 1 X) 1 jj } educ coefficient estimate , e = 1.11, 11% increase in income per year of education sexm coefficient estimate 1.15, e 1.15 = 3.16, 3 higher at baseline for males slope for females approximately 9% per year; for males approximately 6% per year STA 2201: Applied Statistics II March 4, /35

16 ... PSID using lme(nlme) glmm faq > mmod2 = lme(log(income) cyear*sex + age + educ, random = 1 + cyear person, data=psid) Fixed effects: log(income) cyear * sex + age + educ Value Std.Error DF t-value p-value (Intercept) cyear sexm age educ cyear:sexm Random effects: Formula: 1 + cyear person Structure: General positive-definite, Log-Cholesky parametrization StdDev Corr (Intercept) (Intr) cyear Residual > summary(mmod) # using lmer Fixed effects: Estimate Std. Error t value (Intercept) cyear sexm age educ cyear:sexm STA 2201: Applied Statistics II March 4, /35

17 Inference for random effects Random effects: # using lmer Groups Name Variance Std.Dev. Corr person (Intercept) cyear Residual Number of obs: 1661, groups: person, 85 Random effects: # using lme Formula: 1 + cyear person Structure: General positive-definite, Log-Cholesky parametriza StdDev Corr (Intercept) (Intr) cyear Residual standard deviation of slopes estimated to be variation within subjects (0.68) 2 larger than between subjects (0.53) 2 STA 2201: Applied Statistics II March 4, /35

18 Random effects estimates (predictions) of b 0i, b 1i available Y = Xβ + Zb + ɛ; b N(0, σ 2 Ω b ), ɛ N(0, σ 2 Ω j ) Y N(Xβ, (Ω + Z Ω b Z T )) b = (Z T ˆΩ 1 Z + ˆΩ 1 b ) 1 Z T Ω 1 (y Xβ) y X ˆβ = Z b + y X ˆβ Z b = Z b + {I n Z (Z T ˆΩ 1 Z + ˆΩ 1 b ) 1 Z T ˆΩ 1 }(y X ˆβ) }{{} new residual STA 2201: Applied Statistics II March 4, /35

19 pieces of lmer > methods(class="mermod") [1] anova.mermod* as.function.mermod* coef.mermod* [4] confint.mermod deviance.mermod* drop1.mermod* [7] extractaic.mermod* family.mermod* fitted.mermod* [10] fixef.mermod* formula.mermod* isglmm.mermod* [13] islmm.mermod* isnlmm.mermod* isreml.mermod* [16] loglik.mermod* model.frame.mermod* model.matrix.mermod* [19] nobs.mermod* plot.mermod* predict.mermod* [22] print.mermod* profile.mermod* ranef.mermod* [25] refit.mermod* refitml.mermod* residuals.mermod* [28] sigma.mermod* simulate.mermod* summary.mermod* [31] terms.mermod* update.mermod* VarCorr.merMod* [34] vcov.mermod weights.mermod* > ranef(mmod) $person (Intercept) cyear > length(residuals(mmod)) [1] 1661 STA 2201: Applied Statistics II March 4, /35

20 Example: Acuity of Vision ELM, acuity npower > xyplot(acuity npower subject, data=vision, + type="l", groups=eye, lty=1:2, layout = c(4,2)) STA 2201: Applied Statistics II March 4, /35

21 ... vision > head(vision) acuity power eye subject npower /6 left /18 left /36 left /60 left /6 right /18 right 1 2 > eyemod <- lmer(acuity power + (1 subject) + + (1 subject:eye), data = vision) y ijk = µ + p j + s i + e ik + ɛ ijk, i = 1,..., 7; j = 1,... 4; k = 1, 2 s i N(0, σ 2 s), e ik N(0, σ 2 e), ɛ ijk N(0, σ 2 ) STA 2201: Applied Statistics II March 4, /35

22 ... vision > summary(eyemod) Linear mixed model fit by REML [ lmermod ] Formula: acuity power + (1 subject) + (1 subject:eye) Data: vision REML criterion at convergence: Random effects: Groups Name Variance Std.Dev. subject:eye (Intercept) subject (Intercept) Residual Number of obs: 56, groups: subject:eye, 14; subject, 7 Fixed effects: Estimate Std. Error t value (Intercept) power6/ power6/ power6/ STA 2201: Applied Statistics II March 4, /35

23 Part 2 in Rstudio

24 Non-specific effects C&D 7.2 example: a clinical trial involves several or many centres an agricultural field trial repeated at a number of different farms, and over a number of different growing seasons a sociological study repeated in broadly similar form in a number of countries laboratory study uses different sets of analytical apparatus, imperfectly calibrated such factors are non-specific how do we account for them on an appropriate scale, a parameter represents a shift in outcome more complicated: the primary contrasts of concern vary across centres i.e. treatment-center interaction STA 2201: Applied Statistics II March 4, /35

25 ... non-specific effects suppose no treatment-center interaction example: logit{pr(y ci = 1)} = α c + x T ci β should α c be?fixed? or?random? effective use of a random-effects representation will require estimation of the variance component corresponding to the centre effects even under the most favourable conditions the precision achieved in that estimate will be at best that from estimating a single variance from a sample of a size equal to the number of centres very fragile unless there are at least, say, 10 centres and preferably considerably more STA 2201: Applied Statistics II March 4, /35

26 ... non-specific effects if centres are chosen by an effectively random procedure from a large population of candidates,... the random-effects representation has an attractive tangible interpretation. This would not apply, for example, to the countries of the EU in a social survey some general considerations in linear mixed models: in balanced factorial designs, the analysis of treatment means is unchanged in other cases, estimated effects will typically be shrunk, and precision improved representation of the nonspecific effects as random effects involves independence assumptions which certainly need consideration and may need some empirical check STA 2201: Applied Statistics II March 4, /35

27 ... non-specific effects if estimates of effect of important explanatory variables are essentially the same whether nonspecific effects are ignored, or are treated as fixed constants, then random effects model will be unlikely to give a different result it is important in applications to understand the circumstances under which different methods give similar or different conclusions in particular, if a more elaborate method gives an apparent improvement in precision, what are the assumptions on which that improvement is based, and are they reasonable? STA 2201: Applied Statistics II March 4, /35

28 ... non-specific effects if there is an interaction between an explanatory variable [e.g. treatment] and a nonspecific variable i.e. the effects of the explanatory variable change with different levels of the nonspecific factor the first step should be to explain this interaction, for example by transforming the scale on which the response variable is measure or by introducing a new explanatory variable example: two medical treatments compared at a number of centres show different treatment effects, as measured by an ratio of event rates possible explanation: the difference of the event rates might be stable across centres possible explanation: the ratio depends on some characteristic of the patient population, e.g. socio-economic status an important special application of random-effect models for interactions is in connection with overviews, that is, assembling of information from different studies of essentially the same effect STA 2201: Applied Statistics II March 4, /35

29 In the News Globe & Mail, March 3: U of T investigates instructor over anti-vaccine course materials Globe & Mail, Feb 18: Health experts criticize government approval of homeopathic vaccines British Homeopathic Association: In line with the Department of Health s advice, the BHA recommends that immunization should be carried out in the normal way using the conventional tested and approved vaccines Faculty of Homeopathy: randomized controlled trials in homeopathy STA 2201: Applied Statistics II March 4, /35

30 Generalized linear mixed models f (y j θ j, φ) = exp{ y jθ j b(θ j ) φa j + c(y j ; φa j )} b (θ j ) = µ j random effects g(µ j ) = x T j β + z T j b, b N(0, Ω b) likelihood n L(β, φ; y) = j=1 f (y j β, b, φ)f (b; Ω b )db STA 2201: Applied Statistics II March 4, /35

31 ... generalized linear mixed models likelihood L(β, φ; y) = n j=1 f (y j β, b, φ)f (b; Ω b )db doesn t simplify unless f (y j b) is normal solutions proposed include numerical integration, e.g. by quadrature integration by MCMC Laplace approximation to the integral penalized quasi-likelihood reference: MASS library and book ( 10.4): glmmnq, GLMMGibbs, glmmpql, all in library(mass) glmer in library(lme4) STA 2201: Applied Statistics II March 4, /35

32 Example: Balance experiment Faraway, 10.1 effects of surface and vision on balance; 2 levels of surface; 3 levels of vision surface: normal or foam vision: normal, eyes closed, domed 20 males and 20 females tested for balance, twice at each of 6 combinations of treatments auxiliary variables age, height, weight Steele 1998, OzDASL linear predictor: Sex + Age + Weight + Height + Surface + Vision + Subject(?) response measured on a 4 point scale; converted by Faraway to binary (stable/not stable) analysed using linear models at OzDASL STA 2201: Applied Statistics II March 4, /35

33 ... balance > balance <- glmer(stable Sex + Age + Height + Weight + Surface + Vision + + (1 Subject), family = binomial, data = ctsib) # Subject effect is random > summary(balance) Generalized linear mixed model fit by maximum likelihood [ glmermod ]... Random effects: Groups Name Variance Std.Dev. Subject (Intercept) Number of obs: 480, groups: Subject, 40 Fixed effects: Estimate Std. Error z value Pr(> z ) (Intercept) Sexmale Age Height Weight Surfacenorm < 2e-16 *** Visiondome Visionopen e-14 *** --- STA 2201: Applied Statistics II March 4, /35

34 ... balance > library(mass) > balance2 <- glmmpql(stable Sex + Age + Height + Weight + Surface + Vision, + random = 1 Subject, family = binomial, data = ctsib) > summary(balance2) Random effects: Formula: 1 Subject (Intercept) Residual StdDev: Variance function: Structure: fixed weights Formula: invwt Fixed effects: stable Sex + Age + Height + Weight + Surface + Vision Value Std.Error DF t-value p-value (Intercept) Sexmale Age Height Weight Surfacenorm Visiondome Visionopen STA 2201: Applied Statistics II March 4, /35

35 ... balance > balance4 <- glmer(stable Sex + Age + Height + Weight + Surface + Vision + + (1 Subject), family = binomial, data = ctsib, nagq = 9) > summary(balance4) Random effects: Groups Name Variance Std.Dev. Subject (Intercept) Number of obs: 480, groups: Subject, 40 Fixed effects: Estimate Std. Error z value Pr(> z ) (Intercept) Sexmale Age Height Weight Surfacenorm < 2e-16 *** Visiondome Visionopen e-14 *** STA 2201: Applied Statistics II March 4, /35