Lecture 9 STK3100/4100

Transcription

1 Lecture 9 STK3100/ October 2014 Plan for lecture: 1. Linear mixed models cont. Models accounting for time dependencies (Ch. 6.1) 2. Generalized linear mixed models (GLMM, Ch ) Examples R code GLMM - general formulation of the model Likelihood and estimation p. 1

2 Models accounting for time dependencies (Ch. 6.1) Ex: Abundance of a bird species at Hawaii (square root transformed) Moorhen abundance on Kauai Many subsequent observations above or below trend Year p. 2

3 Linear regression y s = α+β 1 Rainfall s +β 2 Year s +ε s ε s N(0,σ 2 ) > fit = gls(birds Rainfall+Year,na.action=na.omit,data=Hawaii) > summary(fit) Generalized least squares fit by REML Model: Birds Rainfall + Year Data: Hawaii AIC BIC loglik Coefficients: Value Std.Error t-value p-value (Intercept) Rainfall Year Residual standard error: Degrees of freedom: 45 total; 42 residual Can not trust p values if residuals are dependent Positive dependencies give too little uncertainties and too small p values p. 3

4 Time dependencies in residuals Residuals Still many subsequent observations above or below trend Year p. 4

5 Time dependency Uncorrelated (= independence if Gaussian) if 1 ifs = t cov[ε s,ε t ] = σ 2 0 ifs t Extension: cov[ε s,ε t ] = σ 2 1 ifs = t h(s,t) ifs t We will typical assume h(s, t) = h( s t ) p. 5

6 Autocorrelation function (ACF) 1 n v n v s=1 ĥ(v) = (y s ȳ)(y s+v ȳ) ˆσ 2 Can be calculated and plotted by the R function acf p. 6

7 Ex: ACF for Bird residuals M0<-gls(Birds Rainfall+Year,na.action=na.omit,data=Hawaii) E <- residuals(m0, type = "normalized") I1 <-!is.na(hawaii$birds) Efull <- rep(na,length(hawaii$birds)) Efull[I1] <- E acf(efull, na.action = na.pass, main = "Auto-correlation plot for residuals") Auto correlation plot for residuals ACF Lag p. 7

8 Models accounting for time dependency: Compound symmetry Marginal model cov[ε s,ε t ] = σ 2 1 ifs = t φ ifs t M1<-gls(Birds Rainfall + Year, na.action = na.omit, correlation = corcompsymm(form = Year), data=hawaii) Same covariance structure as within a group in a random intercept model p. 8

9 Residual plot for model with compound symmetry Auto correlation plot for residuals ACF Covariance matrix with compound symmetry does not remove positive autocorrelation Lag p. 9

10 Models accounting for time dependency: AR(1) Assume ε s = φε s 1 +η s uif η s N(0,σ 2 ) 1 < φ < 1 This is called an autoregressive model of order 1, AR(1) 1 ifs = t cov[ε s,ε t ] = σ 2 φ s t ifs t p. 10

11 AR(1) - R code M2<-gls(Birds Rainfall + Year, na.action = na.omit, correlation = corar1(form = Year), data = Hawaii) Auto correlation plot for residuals ACF Correlation structure is removed! Lag p. 11

12 AR(1) - R code > summary(m2) Generalized least squares fit by REML Model: Birds Rainfall + Year Data: Hawaii AIC BIC loglik Correlation Structure: ARMA(1,0) Formula: Year Parameter estimate(s): Phi Coefficients: Value Std.Error t-value p-value (Intercept) Rainfall Year Residual standard error: Degrees of freedom: 45 total; 42 residual ˆφ = ˆ cor[ε s,ε s+1 ] = 0.77 ˆφ 2 = cor[ε ˆ,ε ] = p. 12

13 Autoregressive (AR) models AR(1): ε s = φε s 1 +η s uif η s N(0,σ 2 ) AR(p): cor[ε s,ε t ] = φ s t ε s = φ 1 ε s 1 +φ 2 ε s 2 + +φ p ε s p +η s uif η s N(0,σ 2 ) p. 13

14 Moving average (MA) models MA(1): ε s = θ 1 η s 1 +η s θ 1 s t = 1 cor[ε s,ε t ] = 0 s t > 1 MA(q): ε s = θ 1 η s 1 +θ 2 η s 2 + +θ q η s q +η s p. 14

15 ARMA models ARMA(p,q): ε s =φ 1 ε s 1 +φ 2 ε s 2 + +φ p ε s p + θ 1 η s 1 +θ 2 η s 2 + +θ q η s q +η s p. 15

16 Fitting different models M0 = gls(birds Rainfall+Year,na.action=na.omit,data=Hawaii) M1<-gls(Birds Rainfall + Year, na.action = na.omit, correlation = corcompsymm(form = Year),data=Hawaii) M2 = gls(birds Rainfall + Year, na.action = na.omit, correlation = corar1(form= Year), data = Hawaii) arma2 = corarma(c(.2,.2),p=2,q=0,form= Year) Marma20<-gls(Birds Rainfall + Year, na.action = na.omit, correlation = arma2, data = Hawaii) arma21 = corarma(c(.2,.2,.2),p=2,q=1,form= Year) Marma21<-gls(Birds Rainfall + Year, na.action = na.omit, correlation = arma21, data = Hawaii) arma22 = corarma(c(.2,.2,.2,.2),p=2,q=2,form= Year) Marma22<-gls(Birds Rainfall + Year, na.action = na.omit, correlation = arma22, data = Hawaii) arma3 = corarma(c(.2,.2,.2),p=3,q=0,form= Year) Marma30<-gls(Birds Rainfall + Year, na.action = na.omit, correlation = arma3, data = Hawaii) p. 16

17 Model selection among several models > AIC(M0,M1,M2,Marma20,Marma21,Marma22,Marma30) df AIC M M M Marma Marma Marma Marma p. 17

18 GLMM related to other model classes Normal Exponential family lm glm Fixed effects lmm GLMM Fixed and random effects glmm = Generalized Linear Mixed Models p. 18

19 GLMM Allow dependencies between observations Model structure similar to linear mixed models Theory and methods still under development Many approaches for estimation Documentation is rather technical p. 19

20 Ex: Species richness No. species RIKZ measured at 9 beaches/areas 5 observations at each beach Want to explain variation in RIKZ by NAP Exposure When we analysed these data by a linear mixed model, we assumed that the response was normal distributed However, the response are count data, and a Poisson distribution is a more natural assumption p. 20

21 Ex. Species richness: Quasi Poisson > Mglm2 = glm(richness NAP,family=quasipoisson,data = RIKZ) > summary(mglm2) Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) < 2e-16 *** NAP e-05 *** --- Signif. codes: 0 *** ** 0.01 * (Dispersion parameter for quasipoisson family taken to be ) Null deviance: on 44 degrees of freedom Residual deviance: on 43 degrees of freedom AIC: NA Ignore dependencies within beaches If we do quasipoisson, we can account for over dispersion, but will still ignore dependencies between groups of observations p. 21

22 Ex. Species richness: GLMM and glmmpql function > library(mass) > MglmmPQL = glmmpql(richness NAP,random= 1 fbeach,family=poisson,data > summary(mglmmpql) Linear mixed-effects model fit by maximum likelihood Random effects: Formula: 1 fbeach (Intercept) Residual StdDev: Variance function: Structure: fixed weights Formula: invwt Fixed effects: Richness NAP Value Std.Error DF t-value p-value (Intercept) e-11 NAP e-06 Estimates and standard errors differs from quasi Poisson p. 22

23 GLMM Y ij b i (Y ij conditioned onb i ) are independent and from the same distribution from the exponential family E[Y ij b i ] = µ ij g(µ ij ) = X ij β +Z ij b i b i uif N(0,D) p. 23

24 Ex: E. cervi L1 in deer Ecervi.01: 1 if a deer has E. cervi L1, 0 if not fsex: sex of deer Length: length of deer Farm: Farm (24 different farms) p. 24

25 Ex. E. cervi L1 in deer: Ordinary logistic regression > DE.glm<-glm(Ecervi.01 CLength * fsex, data = DeerEcervi, + family = binomial) > summary(de.glm) Coefficients: Estimate Std. Error z value Pr(> z ) (Intercept) e-09 *** CLength e-06 *** fsex CLength:fSex * Data are repeated measurements within same farm But the model ignore dependencies within farms p. 25

26 Ex. E. cervi L1 in deer: Ordinary logistic regression including farm as a fixed effect factor > DE.glm<-glm(Ecervi.01 CLength * fsex+ffarm, data = DeerEcervi, + family = binomial) > anova(de.glm,test="chisq") Df Deviance Resid. Df Resid. Dev P(> Chi ) NULL CLength e-16 *** fsex ffarm < 2.2e-16 *** CLength:fSex ** Problems ffarm clearly significant, but uses 23 parameters Can show that interaction between ffarm and CLength is also significant, and uses additional 22 parameters How can we predict for a farm without data? p. 26

27 Ex. E. cervi L1 in deer: GLM vs. GLMM GLM GLMM Y ij Bin(1,p ij ) logit(p ij ) =α+β 1 Length ij +β 2 Sex ij +β 3 Length ij Sex ij +α Farm i Y ij Bin(1,p ij ) logit(p ij ) =α+β 1 Length ij +β 2 Sex ij +β 3 Length ij Sex ij +a i a i N(0,σ 2 a) The size ofσ 2 a indicates the importance of Farm p. 27

28 Ex. E. cervi L1 in deer: GLMM and glmmpql glmmpql is one out of several R functions for estimating a GLMM: > library(mass) > DE.PQL<-glmmPQL(Ecervi.01 CLength * fsex, + random = 1 ffarm, family = binomial, data = DeerEcervi) > summary(de.pql) Random effects: Formula: 1 ffarm (Intercept) Residual StdDev: Variance function: Structure: fixed weights Formula: invwt Fixed effects: Ecervi.01 CLength * fsex Value Std.Error DF t-value p-value (Intercept) CLength fsex CLength:fSex p. 28

29 glmmpql - interpretation of output Random effects: Formula: 1 ffarm (Intercept) Residual StdDev: ˆσ a 2 = = 2.14 Residual StdDev: Standard deviation of working residuals. Does not correspond directly to a parameter in the model! p. 29

30 GLMM and likelihood GLM : Likelihood can be written directly LMM : Y i multivariate normal, likelihood can be written directly GLMM : Likelihood contribution from observations from i-th group is f(y i β,θ) = f(y i b i,β)f(b i D)db i b i = f(y ij b i,β)f(b i D)db i b i Difficult to compute the integral Must in addition optimise j L(β,θ) = i f(y i β,θ) wrt. β,θ whered = D(θ) Very complicated numerical problem p. 30

31 Estimation methods Maximum likelihood REML - difficult, not well understood, probably not much used in practice yet Penalised quasi-likelihood optimise a function that is simpler than the likelihood quasi-likelihood has another meaning than previously in the course MCMC and Bayesian methods - not in this course In addition numerical approximations within these methods: Laplace approximation Gauss-Hermite integration - approximates integrals by sums p. 31

32 Ex: Comparison of estimation results #Penalized quasi-likelihood > library(mass) > DE.PQL<-glmmPQL(Ecervi.01 CLength * fsex, + random = 1 ffarm, family = binomial, data = DeerEcervi) #ML: Laplace approximation with lmer > library(lme4) > DE.lme4<-lmer(Ecervi.01 CLength * fsex +(1 ffarm), + family = binomial, data = DeerEcervi) #ML: Laplace approx with glmmml > library(glmmml) > DE.glmmML<-glmmML(Ecervi.01 CLength * fsex, + cluster = ffarm,family=binomial, data = DeerEcervi) #ML: Gauss-Hermite with glmmml > DE.glmmML2<-glmmML(Ecervi.01 CLength * fsex,method="ghq", + cluster = ffarm,family=binomial, data = DeerEcervi) Note: None of these use REML p. 32

33 Ex: Comparison of results Default Gauss-Hermite (20) GLM Intercept Estimates SE Estimates SE Length Sex Length Sex glmmpql Intercept Length Sex Length Sex lmer Intercept Length Sex Length Sex glmmml Intercept Length Sex Length Sex p. 33