13. October p. 1
|
|
- Preston Caldwell
- 5 years ago
- Views:
Transcription
1 Lecture 8 STK3100/4100 Linear mixed models 13. October 2014 Plan for lecture: 1. The lme function in the nlme library 2. Induced correlation structure 3. Marginal models 4. Estimation - ML and REML 5. Model selection 6. Model validation p. 1
2 Linear mixed model with all assumptions Y i =X i β +Z i b i +ε i,i = 1,...,N b i N(0,D) ε i N(0,Σ i ) b 1,..,b N,ε 1,...,ε N independent Often: Simplified structures ond,σ i D = d 2 I,Σ i = σ 2 I p. 2
3 Ex. Species richness: Model Simplified model with only NAP Y ij =α+b i +βnap ij +ε ij yieldsβ = (α,β) T, b i = b i,σ = σ 2 I,D = d 2 and 1 NAP i1 1 1 NAP i2 1 X i = 1 NAP i3, Z i = 1 1 NAP i4 1 1 NAP i5 1 p. 3
4 Ex. Species richness: Output from lme > RIKZ$fBeach <- factor(rikz$beach) > Mlme1 <- lme(richness NAP, random = 1 fbeach,data=rikz) > summary(mlme1) Linear mixed-effects model fit by REML Data: RIKZ AIC BIC loglik Random effects: Formula: 1 fbeach (Intercept) Residual StdDev: Fixed effects: Richness NAP Value Std.Error DF t-value p-value (Intercept) NAP Here is ˆd 2 = = and ˆσ 2 = = p. 4
5 Ex. Species richness: Coefficients > Mlme1 <- lme(richness NAP, random = 1 fbeach,data=rikz) > summary(mlme1)$coef $fixed (Intercept) NAP $random $random$fbeach (Intercept) ˆbi = E[b i data, estimated parameters]. p. 5
6 Ex. Species richness: Fitted values Y i =X i β +Z i b i +ε i Two options for fitted values: ˆµ i =X iˆβi Level 0 µ i =X iˆβi +Z i ˆbi Level 1 Mlme1 <- lme(richness NAP, random = 1 fbeach,data=rikz) F0<-fitted(Mlme1,level=0) F1<-fitted(Mlme1,level=1) p. 6
7 Ex. Species richness: Fitted values NAP Richness Level 0 Level 1 p. 7
8 Random intercept and slope model Species richness Y ij =α+b 1i +(β +b 2i )NAP ij +ε ij Intercept α+b 1i N(α,d 2 11) Slopeβ +b 2i N(β,d 2 22) Can also have non-zero Cov[b 1i,b 2i ] = d 12 = d 21 p. 8
9 R code for random intercept and slope > Mlme2 <- lme(richness 1+NAP, + random = 1 + NAP fbeach, data = RIKZ) > summary(mlme2) Random effects: Formula: 1 + NAP fbeach Structure: General positive-definite, Log-Cholesky parametrization StdDev Corr (Intercept) (Intr) NAP Residual Fixed effects: Richness NAP Value Std.Error DF t-value p-value (Intercept) e+00 NAP e-04 Correlation: (Intr) NAP p. 9
10 Fitted values: up:separate regressions low: random intercept and slope RIKZ$Richness RIKZ$NAP RIKZ$Richness RIKZ$NAP p. 10
11 Model on matrix form yields Y i =X i β +Z i b i +ε i b i N(0,D) ε i N(0,Σ i ) Y i N(X i β,v i ) V i =Z i D Z T i +Σ i p. 11
12 Induced correlation structure in random intercept and slope model Y ij =α+b 1i +(β +b 2i )NAP ij +ε ij Var[Y ij ] =d NAP ij d 12 + NAP 2 ij d σ 2 Cov[Y ij,y ik ] =d (NAP ij + NAP ik ) d 12 + (NAP ij NAP ik ) d σ 2 p. 12
13 Intraclass correlation and effective sample size Simple situation: Y i = (Y i1,...,y in ), E[Y ij ] =µ, Var[Y ij ] = σ 2, Cov[Y ij,y ik ] = ρσ 2 ˆµ =Ȳ Var[Ȳ] = 1 n 2[ n j=1 Var[Y ij ]+ k j Cov[Y ij,y ik ]] = 1 n 2[nσ2 +n(n 1)ρσ 2 ] = σ2 n [1+(n 1)ρ] Variance increased by 1 +(n 1)ρ. Design effect Ex: n = 5, ˆρ = 0.48 yields1+(n 1)ρ = Effective sample size: N effective = N n design effect = = p. 13
14 Marginal model/likelihood Y i N(X i β,v i ) V i =Z i DZ T i +Σ i dependent on parameters ψ L i =f(y i ;β,ψ) 1 = (2π) n i/2 V i 1/2 exp{ 1(Y 2 i X i β) T V 1 i (Y i X i β)} l i = n i log(2π) 1 log V 2 2 i 1(Y 2 i X i β) T V 1 i (Y i X i β) N l(β,ψ) = i=1 l i p. 14
15 Direct specification of marginal model V i = τ 2 φ φ φ φ φ τ 2 φ φ φ φ φ τ 2 φ φ φ φ φ τ 2 φ φ φ φ φ τ 2 Compound symmetric structure, corresponds to Σ i =σ 2 I,Z i = 1 T,D = d 2 = φ,τ 2 = σ 2 +d 2 p. 15
16 Direct specification of marginal model cont. V i = τ 2 c 21 c 31 c 41 c 51 c 21 τ 2 c 32 c 42 c 52 c 31 c 32 τ 2 c 43 c 53 c 41 c 42 c 43 τ 2 c 54 c 51 c 52 c 53 c 54 τ 2 general covariance matrix p. 16
17 Ex. Species richness Mixed model > M.mixed <- lme(richness NAP, random = 1 fbeach, data = RIKZ) > summary(m.mixed)$coef$fixed (Intercept) NAP Marginal model can be estimated by the gls function > M.gls <- gls(richness NAP, correlation = corcompsymm(form = 1 fbeach), data = RIKZ) > coef(m.gls) (Intercept) NAP p. 17
18 Estimation ML estimation: yields ˆµ = ȳ and Y i uif N(µ,σ 2 ) ˆσ 2 = 1 n n (y i ȳ) 2 Biased i=1 We often prefer ˆσ 2 = 1 n 1 n (y i ȳ) 2 Unbiased i=1 Bias in MLE because Uncertainty inµis ignored p. 18
19 ML estimation V i =V i (ψ) N l(β,ψ) = [ n i log(2π) 1 log V 2 2 i i=1 Can be maximised numerically 1 (Y 2 i X i β) T V 1 i (Y i X i β)] Gives biased estimates for variances Not so important in ordinary regression, because observations are independent and get the same weight More important in mixed models, because the weight or importance of observations in the estimation process depends on the variances p. 19
20 REML= Restricted maximum likelihood REML is an alternative strategy which gives unbiased estimates Illustrated by the ordinary linear regression model: Y i = X i β +ε i, ε i N(0,σ 2 ) Idea: Transform data such thatβ disappears A is ann (n p) matrix of full rank such thata T X = 0 Gives (n p) 1 {}}{ A T Y i =A T X i β +A T ε i N(0,σ 2 A T A) REML: Estimate σ 2 by ML based ona T Y i p. 20
21 REML cont. L REML = l REML = N 2 Yields exp{ Y T A[A T A] 1 A T Y} (2π) N/2 σ 2 A T A 1/2 2σ 2 n p log(2π) logσ log AT A Y T A[A T A] 1 A T Y 2σ 2 ˆσ 2 = 1 n p YT A[A T A] 1 A T Y p. 21
22 Example Y i uif N(µ,σ 2 ) X =1 T N = (1 T N 1 1) A = I N 1 1 T N 1 A T X =I N 1 1 N 1 1 N 1 = 0 A T Y =(y 1 y n,...,y n 1 y n ) T ˆσ 2 = 1 n (y i ȳ) 2 n 1 i=1 Derivation of ˆσ 2 includes some calculations p. 22
23 REML and mixed models Model Y i N(X i β,v i ), V i = Z i D Z T i +Σ i combined Y N(Xβ,V) Define A such thata T X = 0. This yields A T Y N(0,A T VA) Estimate parameters invby ML estimation based ona T Y Note: The result does not depend on how we specifya. Gives unbiased estimates for elements inv! p. 23
24 Estimation ofβ by REML REML transformation gives estimate forv i. β can then be estimated by ML conditioned on the estimates of the V i s p. 24
25 Example in R table(rikz$exposure) RIKZ$fExp<-RIKZ$Exposure RIKZ$fExp[RIKZ$fExp==8]<-10 RIKZ$fExp<-factor(RIKZ$fExp,levels=c(10,11)) M0.ML <- lme(richness NAP, data = RIKZ, random = 1 fbeach, method = "ML") M0.REML <-lme(richness NAP, data = RIKZ, random = 1 fbeach, method = "REML") M1.ML <- lme(richness NAP+fExp, data = RIKZ, random = 1 fbeach, method = "ML") M1.REML <- lme(richness NAP+fExp, data = RIKZ, random = 1 fbeach, method = "REML") p. 25
26 Estimates p. 26
27 Ex. Species richness: individual fixed intercepts + random slope (N=NAP,Ex=Exposure) Y ij =α i +(η +τex i +b i )N ij +ε ij =α i +ηn ij +τex i N ij +b i N ij +ε ij yieldsβ = (α 1,...,α 9,η,τ) T,b i = b i,σ = σ 2 I,D = d 2 and i {}}{ N i1 Ex i N i N i2 Ex i N i2 X i = N i3 Ex i N, Z i = i N i4 Ex i N i N i5 Ex i N i5 N i1 N i2 N i3 N i4 N i5 p. 27
28 Model selection in mixed models Two parts of the model: Fixed effects - explanatory variables Random effects - correlation structure These will interact Must use different methods for model selection p. 28
29 Protocol for model selection in mixed models Main idea: Want to explain as much as possible from the fixed effects 1. Start with the model with all explanatory variables (if possible) and as many interactions as possible - beyond optimal model 2. Find optimal structure on random effects using REML 3. Find optimal structure for fixed effects using REML for t- and F-tests using ML for likelihood ratio tests 4. Estimate the final model using REML p. 29
30 Model selection methods Two main strategies Information criteria: AIC, BIC Hypothesis testing on parameters in nested models t test (Wald test) F test (multiple parameters) Likelihood ratio test p. 30
31 AIC/BIC and ML AIC = 2 l(ˆθ)+2 p BIC = 2 l(ˆθ)+log(n) p Misprint in the book p: Number of parameters in the model, bothβ-s andσ-s n = N i=1 n i AIC: tend to give large models, especially with many observations BIC: gives smaller models, but perhaps to small for smalln Can be used directly in ML estimation l(ˆθ) is the ML log likelihood p. 31
32 AIC/BIC and REML AIC = 2 l(ˆθ)+2 p BIC = 2 l(ˆθ)+log(n p) p Misprint in the book p: Number of parameters in the model, bothβ-s andσ-s l(ˆθ) is now the REML log likelihood Can show L REML (θ) = N i=1 X T i V 1 i X i 1/2 L ML (θ) p. 32
33 Ex: The wrong approach (the good is presented later) Species richness: Start with only NAP as explanatory variable - this is a bad choice! Find optimal structure for random effects, three possibilities: No random effects Random intercept for each area Random intercept and slope for each area Compare models by REML Note: lm use ML, and lme can only be used for random effect models, so we must use gls for the model without random effects p. 33
34 Ex. The wrong approach: Step 2, by AIC/BIC > Wrong1 <- gls(richness 1 + NAP, method = "REML", data = RIKZ) > Wrong2 <- lme(richness 1 + NAP, random = 1 fbeach, method = "REML", data = RIKZ) > Wrong3 <- lme(richness 1 + NAP, method = "REML", random = 1 + NAP fbeach, data = RIKZ) > cbind(aic(wrong1,wrong2,wrong3),bic(wrong1,wrong2,wrong3)) df AIC df BIC Wrong Wrong Wrong AIC: Best to include both random intercept and slope BIC: Best to includerandom intercept, but fixed slope p. 34
35 LR tests for random effects Likelihood ratio test by the anova function: > anova(wrong1,wrong2,wrong3) Model df AIC BIC loglik Test L.Ratio p-value Wrong Wrong vs Wrong vs Problem: Test H 0 : σ? 2 parameter space Gives wrong p values! = 0 which is on the boundary of the p. 35
36 LR tests for random effects cont. H 0 : θ Θ 0 mot H 1 : θ Θ a LR test: Ordinary asymptotic theory requiresθ 0 to be in the inner of Θ = Θ 0 Θ a. Have then 2LR χ 2 q a q 0. P value =Pr(χ 2 q a q 0 > 2LR). Here: TestsH 0 : d 2 11 = 0. On the boundary of Θ : {d2 11 0}. Can show: P values too large. Conservative test More precise: Assume k random effects underh 0, k +1 random effects under H a. T = 2LR Pr(T > c) = 0.5 [Pr(χ 2 k > c)+pr(χ2 k+1 > c)] Especially: k = 0: Pr(T > c) = 0.5 Pr(χ 2 1 > c) p. 36
37 Ex. The wrong approach: Step 2, by corrected LRT Wrong1 vs. Wrong2: > 0.5*(1-pchisq(T,1)) [1] Wrong2 vs. Wrong3 > T = anova(wrong1,wrong2,wrong3)[3,8] > 1-0.5*(pchisq(T,1)+pchisq(T,2)) [1] Choose model Wrong3 including both random intercept and slope p. 37
38 Ex. The wrong approach: Step 3, fixed effects > summary(wrong3) Fixed effects: Richness 1 + NAP Value Std.Error DF t-value p-value (Intercept) e+00 NAP e-04 NAP significant, we include now Exposure and interaction > RIKZ$fExp<-RIKZ$Exposure > RIKZ$fExp[RIKZ$fExp==8]<-10 > RIKZ$fExp<-factor(RIKZ$fExp,levels=c(10,11)) > Wrong4 <- lme(richness 1 + NAP * fexp,random = 1 + NAP fbeach, method = "REML", data = RIKZ) Error in lme.formula(richness 1 + NAP * fexp, random= 1+NAP fbeach, : nlminb problem, convergence error code = 1 > lmc <- lmecontrol(niterem = 2200, msmaxiter = 2200) > Wrong4 = lme(richness 1 + NAP * fexp,random = 1 + NAP fbeach, method = "REML", data = RIKZ,control=lmc) p. 38
39 Ex. The wrong approach: Step 3 cont. - F and t tests based on REML > anova(wrong4) numdf dendf F-value p-value (Intercept) <.0001 NAP fexp NAP:fExp > summary(wrong4) Fixed effects: Richness 1 + NAP * fexp Value Std.Error DF t-value p-value (Intercept) NAP fexp NAP:fExp Exposure seems to be significant, but interaction not significant p. 39
40 Ex. The wrong approach: Step 3 cont. Excluding interaction - F and t tests based on REML > Wrong4.2 <- lme(richness 1 + NAP + fexp,random = 1 + NAP fbeach, method = "REML", data = RIKZ) > anova(wrong4.2) numdf dendf F-value p-value (Intercept) <.0001 NAP fexp > summary(wrong4.2) Fixed effects: Richness 1 + NAP + fexp Value Std.Error DF t-value p-value (Intercept) NAP fexp Exposure is bordeline significant at 5 % level Final model so far: Y ij =α+b 1i +(β +b 2i )NAP ij +ε ij p. 40
41 Degree of freedom (df) Are used intandf tests Explanatory variables divided into two groups Level 1: Variables with different values for observations within each group df = Total no obs. - no groups - no level 1 variables Ex: NAP, df=45-9-1=35 Level 2: Variables with equal values within each group df = No groups - no level 2 variables (including constant term) Ex: Exposure, df=9-2=7 p. 41
42 Ex. The wrong approach: Step 3 cont. Likelihood ratio tests based on ML > lmc <- lmecontrol(niterem = 5200, msmaxiter = 5200) > Wrong4A <- lme(richness 1 + NAP, method="ml", control = lmc, data = RIKZ, random = 1+NAP fbeach) > Wrong4B <- lme(richness 1 + NAP + fexp, random = 1 + NAP fbeach, method="ml", data = RIKZ,control = lmc) > Wrong4C <- lme(richness 1 + NAP * fexp, random = 1 + NAP fbeach, data = RIKZ, method = "ML", control = lmc) > anova(wrong4a, Wrong4B, Wrong4C) Model df AIC BIC loglik Test L.Ratio p-value Wrong4A Wrong4B vs Wrong4C vs LRT gives optimal model: Only NAP but interaction term bordeline significant, so not totally clear p. 42
43 Ex. The wrong approach: Step 4, final model > Wrong5 <- lme(richness 1 + NAP,random = 1 + NAP fbeach, + method = "REML", data = RIKZ) > summary(wrong5) Random effects: Formula: 1 + NAP fbeach Structure: General positive-definite, Log-Cholesky parametrization StdDev Corr (Intercept) (Intr) NAP Residual Fixed effects: Richness 1 + NAP Value Std.Error DF t-value p-value (Intercept) e+00 NAP e-04 Number of Observations: 45 Number of Groups: 9 p. 43
44 Ex. The good method: Step 1 and 2 Includes both variables + interaction > B1=gls(Richness 1+NAP*fExp,method="REML",data=RIKZ) > B2=lme(Richness 1+NAP*fExp,data=RIKZ,random= 1 fbeach,method="reml") > B3=lme(Richness 1+NAP*fExp,data=RIKZ,random= 1+NAP fbeach,method="reml > AIC(B1,B2,B3) df AIC B B B Best model: B2 - random intercept p. 44
45 Ex. The good method: Step 3 > summary(b2) Fixed effects: Richness 1 + NAP * fexp Value Std.Error DF t-value p-value (Intercept) NAP fexp NAP:fExp Drop the interaction term p. 45
46 Ex. The good method: Step 4, final model > B2B=lme(Richness 1+NAP+fExp,data=RIKZ,random= 1 fbeach,method="reml") > summary(b2b) Linear mixed-effects model fit by REML Data: RIKZ AIC BIC loglik Random effects: Formula: 1 fbeach (Intercept) Residual StdDev: Fixed effects: Richness 1 + NAP + fexp Value Std.Error DF t-value p-value (Intercept) NAP fexp Final model: Y ij =α+b i +β 1 NAP ij +β 2 Exposure i +ε ij p. 46
47 Model validation Model choice: Choose among a set of models Does the final model describe the data well? Model validation: Check the final model Goodness-of-fit test Residual plot p. 47
48 Model validation - residuals Response residuals (default in R): Y ij ŷ ij Standardised residuals (Pearson residuals) Y ij ŷ ij sd[y ij ] Both can be computed at different levels: Level 0, population level: ŷ ij = X T i ˆβ. Level 1, within group: ŷ ij = X T i ˆβ +Z T i ˆb i. (Default) sd[y ij ] adjusted according to level Note: Level 0 residuals are dependent! p. 48
49 Residual plot for RIKZ data - level 0 plot(fitted(b2b,level=0),residuals(b2b,level=0)) residuals(b2b, level = 0) fitted(b2b, level = 0) p. 49
50 Residual plot for RIKZ data - level 1 plot(fitted(b2b,level=1),residuals(b2b,level=1)) residuals(b2b, level = 1) fitted(b2b, level = 1) p. 50
Lecture 9 STK3100/4100
Lecture 9 STK3100/4100 27. October 2014 Plan for lecture: 1. Linear mixed models cont. Models accounting for time dependencies (Ch. 6.1) 2. Generalized linear mixed models (GLMM, Ch. 13.1-13.3) Examples
More informationThese slides illustrate a few example R commands that can be useful for the analysis of repeated measures data.
These slides illustrate a few example R commands that can be useful for the analysis of repeated measures data. We focus on the experiment designed to compare the effectiveness of three strength training
More informationExploring Hierarchical Linear Mixed Models
Exploring Hierarchical Linear Mixed Models 1/49 Last time... A Greenhouse Experiment testing C:N Ratios Sam was testing how changing the C:N Ratio of soil affected plant leaf growth. He had 3 treatments.
More information20. REML Estimation of Variance Components. Copyright c 2018 (Iowa State University) 20. Statistics / 36
20. REML Estimation of Variance Components Copyright c 2018 (Iowa State University) 20. Statistics 510 1 / 36 Consider the General Linear Model y = Xβ + ɛ, where ɛ N(0, Σ) and Σ is an n n positive definite
More informationHierarchical Linear Models (HLM) Using R Package nlme. Interpretation. 2 = ( x 2) u 0j. e ij
Hierarchical Linear Models (HLM) Using R Package nlme Interpretation I. The Null Model Level 1 (student level) model is mathach ij = β 0j + e ij Level 2 (school level) model is β 0j = γ 00 + u 0j Combined
More informationModel comparison and selection
BS2 Statistical Inference, Lectures 9 and 10, Hilary Term 2008 March 2, 2008 Hypothesis testing Consider two alternative models M 1 = {f (x; θ), θ Θ 1 } and M 2 = {f (x; θ), θ Θ 2 } for a sample (X = x)
More informationIntroduction to General and Generalized Linear Models
Introduction to General and Generalized Linear Models Mixed effects models - II Henrik Madsen, Jan Kloppenborg Møller, Anders Nielsen April 16, 2012 H. Madsen, JK. Møller, A. Nielsen () Chapman & Hall
More informationIntroduction and Background to Multilevel Analysis
Introduction and Background to Multilevel Analysis Dr. J. Kyle Roberts Southern Methodist University Simmons School of Education and Human Development Department of Teaching and Learning Background and
More informationStat 579: Generalized Linear Models and Extensions
Stat 579: Generalized Linear Models and Extensions Linear Mixed Models for Longitudinal Data Yan Lu April, 2018, week 14 1 / 64 Data structure and Model t1 t2 tn i 1st subject y 11 y 12 y 1n1 2nd subject
More informationWorkshop 9.3a: Randomized block designs
-1- Workshop 93a: Randomized block designs Murray Logan November 23, 16 Table of contents 1 Randomized Block (RCB) designs 1 2 Worked Examples 12 1 Randomized Block (RCB) designs 11 RCB design Simple Randomized
More informationOutline for today. Maximum likelihood estimation. Computation with multivariate normal distributions. Multivariate normal distribution
Outline for today Maximum likelihood estimation Rasmus Waageetersen Deartment of Mathematics Aalborg University Denmark October 30, 2007 the multivariate normal distribution linear and linear mixed models
More informationRepeated measures, part 2, advanced methods
enote 12 1 enote 12 Repeated measures, part 2, advanced methods enote 12 INDHOLD 2 Indhold 12 Repeated measures, part 2, advanced methods 1 12.1 Intro......................................... 3 12.2 A
More informationA brief introduction to mixed models
A brief introduction to mixed models University of Gothenburg Gothenburg April 6, 2017 Outline An introduction to mixed models based on a few examples: Definition of standard mixed models. Parameter estimation.
More informationBIO5312 Biostatistics Lecture 13: Maximum Likelihood Estimation
BIO5312 Biostatistics Lecture 13: Maximum Likelihood Estimation Yujin Chung November 29th, 2016 Fall 2016 Yujin Chung Lec13: MLE Fall 2016 1/24 Previous Parametric tests Mean comparisons (normality assumption)
More informationOutline. Statistical inference for linear mixed models. One-way ANOVA in matrix-vector form
Outline Statistical inference for linear mixed models Rasmus Waagepetersen Department of Mathematics Aalborg University Denmark general form of linear mixed models examples of analyses using linear mixed
More informationMISCELLANEOUS TOPICS RELATED TO LIKELIHOOD. Copyright c 2012 (Iowa State University) Statistics / 30
MISCELLANEOUS TOPICS RELATED TO LIKELIHOOD Copyright c 2012 (Iowa State University) Statistics 511 1 / 30 INFORMATION CRITERIA Akaike s Information criterion is given by AIC = 2l(ˆθ) + 2k, where l(ˆθ)
More informationRestricted Maximum Likelihood in Linear Regression and Linear Mixed-Effects Model
Restricted Maximum Likelihood in Linear Regression and Linear Mixed-Effects Model Xiuming Zhang zhangxiuming@u.nus.edu A*STAR-NUS Clinical Imaging Research Center October, 015 Summary This report derives
More informationIntroduction to Within-Person Analysis and RM ANOVA
Introduction to Within-Person Analysis and RM ANOVA Today s Class: From between-person to within-person ANOVAs for longitudinal data Variance model comparisons using 2 LL CLP 944: Lecture 3 1 The Two Sides
More informationSimple Linear Regression
Simple Linear Regression In simple linear regression we are concerned about the relationship between two variables, X and Y. There are two components to such a relationship. 1. The strength of the relationship.
More informationStat 579: Generalized Linear Models and Extensions
Stat 579: Generalized Linear Models and Extensions Linear Mixed Models for Longitudinal Data Yan Lu April, 2018, week 15 1 / 38 Data structure t1 t2 tn i 1st subject y 11 y 12 y 1n1 Experimental 2nd subject
More informationCovariance Models (*) X i : (n i p) design matrix for fixed effects β : (p 1) regression coefficient for fixed effects
Covariance Models (*) Mixed Models Laird & Ware (1982) Y i = X i β + Z i b i + e i Y i : (n i 1) response vector X i : (n i p) design matrix for fixed effects β : (p 1) regression coefficient for fixed
More informationCO2 Handout. t(cbind(co2$type,co2$treatment,model.matrix(~type*treatment,data=co2)))
CO2 Handout CO2.R: library(nlme) CO2[1:5,] plot(co2,outer=~treatment*type,layout=c(4,1)) m1co2.lis
More informationEstimation: Problems & Solutions
Estimation: Problems & Solutions Edps/Psych/Stat 587 Carolyn J. Anderson Department of Educational Psychology c Board of Trustees, University of Illinois Fall 2017 Outline 1. Introduction: Estimation of
More informationStat/F&W Ecol/Hort 572 Review Points Ané, Spring 2010
1 Linear models Y = Xβ + ɛ with ɛ N (0, σ 2 e) or Y N (Xβ, σ 2 e) where the model matrix X contains the information on predictors and β includes all coefficients (intercept, slope(s) etc.). 1. Number of
More informationRandom and Mixed Effects Models - Part III
Random and Mixed Effects Models - Part III Statistics 149 Spring 2006 Copyright 2006 by Mark E. Irwin Quasi-F Tests When we get to more than two categorical factors, some times there are not nice F tests
More informationEstimation: Problems & Solutions
Estimation: Problems & Solutions Edps/Psych/Soc 587 Carolyn J. Anderson Department of Educational Psychology c Board of Trustees, University of Illinois Spring 2019 Outline 1. Introduction: Estimation
More informationLecture 15. Hypothesis testing in the linear model
14. Lecture 15. Hypothesis testing in the linear model Lecture 15. Hypothesis testing in the linear model 1 (1 1) Preliminary lemma 15. Hypothesis testing in the linear model 15.1. Preliminary lemma Lemma
More informationHomework 3 - Solution
STAT 526 - Spring 2011 Homework 3 - Solution Olga Vitek Each part of the problems 5 points 1. KNNL 25.17 (Note: you can choose either the restricted or the unrestricted version of the model. Please state
More informationProblems. Suppose both models are fitted to the same data. Show that SS Res, A SS Res, B
Simple Linear Regression 35 Problems 1 Consider a set of data (x i, y i ), i =1, 2,,n, and the following two regression models: y i = β 0 + β 1 x i + ε, (i =1, 2,,n), Model A y i = γ 0 + γ 1 x i + γ 2
More informationMA 575 Linear Models: Cedric E. Ginestet, Boston University Midterm Review Week 7
MA 575 Linear Models: Cedric E. Ginestet, Boston University Midterm Review Week 7 1 Random Vectors Let a 0 and y be n 1 vectors, and let A be an n n matrix. Here, a 0 and A are non-random, whereas y is
More informationStatistics 203: Introduction to Regression and Analysis of Variance Course review
Statistics 203: Introduction to Regression and Analysis of Variance Course review Jonathan Taylor - p. 1/?? Today Review / overview of what we learned. - p. 2/?? General themes in regression models Specifying
More informationShort Course Applied Linear and Nonlinear Mixed Models* Introduction
Short Course Applied Linear and Nonlinear Mixed Models* Introduction Mixed-effect models (or simply, mixed models ) are like classical ( fixedeffects ) statistical models, except that some of the parameters
More informationParametric Modelling of Over-dispersed Count Data. Part III / MMath (Applied Statistics) 1
Parametric Modelling of Over-dispersed Count Data Part III / MMath (Applied Statistics) 1 Introduction Poisson regression is the de facto approach for handling count data What happens then when Poisson
More informationSTAT3401: Advanced data analysis Week 10: Models for Clustered Longitudinal Data
STAT3401: Advanced data analysis Week 10: Models for Clustered Longitudinal Data Berwin Turlach School of Mathematics and Statistics Berwin.Turlach@gmail.com The University of Western Australia Models
More informationIntruction to General and Generalized Linear Models
Intruction to General and Generalized Linear Models Mixed Effects Models IV Henrik Madsen Anna Helga Jónsdóttir hm@imm.dtu.dk April 30, 2012 Henrik Madsen Anna Helga Jónsdóttir (hm@imm.dtu.dk) Intruction
More informationMS&E 226: Small Data
MS&E 226: Small Data Lecture 15: Examples of hypothesis tests (v5) Ramesh Johari ramesh.johari@stanford.edu 1 / 32 The recipe 2 / 32 The hypothesis testing recipe In this lecture we repeatedly apply the
More informationFall 2017 STAT 532 Homework Peter Hoff. 1. Let P be a probability measure on a collection of sets A.
1. Let P be a probability measure on a collection of sets A. (a) For each n N, let H n be a set in A such that H n H n+1. Show that P (H n ) monotonically converges to P ( k=1 H k) as n. (b) For each n
More informationPart III: Linear mixed effects models
Part III: Linear mixed effects models 159 BIO 245, Spring 2018 Dental growth data Females Males Length, mm 16 18 20 22 24 26 28 30 32 11 11 4 3 2 11 4 7 11 4 3 3 2 8 4 8 5 8 58 7 1 7 5 6 9 57 2 1 9 12
More informationSCHOOL OF MATHEMATICS AND STATISTICS. Linear and Generalised Linear Models
SCHOOL OF MATHEMATICS AND STATISTICS Linear and Generalised Linear Models Autumn Semester 2017 18 2 hours Attempt all the questions. The allocation of marks is shown in brackets. RESTRICTED OPEN BOOK EXAMINATION
More informationCorrelation and Regression
Correlation and Regression October 25, 2017 STAT 151 Class 9 Slide 1 Outline of Topics 1 Associations 2 Scatter plot 3 Correlation 4 Regression 5 Testing and estimation 6 Goodness-of-fit STAT 151 Class
More informationMultivariate Statistics in Ecology and Quantitative Genetics Summary
Multivariate Statistics in Ecology and Quantitative Genetics Summary Dirk Metzler & Martin Hutzenthaler http://evol.bio.lmu.de/_statgen 5. August 2011 Contents Linear Models Generalized Linear Models Mixed-effects
More informationWorkshop 9.1: Mixed effects models
-1- Workshop 91: Mixed effects models Murray Logan October 10, 2016 Table of contents 1 Non-independence - part 2 1 1 Non-independence - part 2 11 Linear models Homogeneity of variance σ 2 0 0 y i = β
More informationSTK4900/ Lecture 10. Program
STK4900/9900 - Lecture 10 Program 1. Repeated measures and longitudinal data 2. Simple analysis approaches 3. Random effects models 4. Generalized estimating equations (GEE) 5. GEE for binary data (and
More informationOn the Behavior of Marginal and Conditional Akaike Information Criteria in Linear Mixed Models
On the Behavior of Marginal and Conditional Akaike Information Criteria in Linear Mixed Models Thomas Kneib Department of Mathematics Carl von Ossietzky University Oldenburg Sonja Greven Department of
More information36-720: Linear Mixed Models
36-720: Linear Mixed Models Brian Junker October 8, 2007 Review: Linear Mixed Models (LMM s) Bayesian Analogues Facilities in R Computational Notes Predictors and Residuals Examples [Related to Christensen
More informationMixed effects models
Mixed effects models The basic theory and application in R Mitchel van Loon Research Paper Business Analytics Mixed effects models The basic theory and application in R Author: Mitchel van Loon Research
More informationCh 2: Simple Linear Regression
Ch 2: Simple Linear Regression 1. Simple Linear Regression Model A simple regression model with a single regressor x is y = β 0 + β 1 x + ɛ, where we assume that the error ɛ is independent random component
More informationIntroduction to Estimation Methods for Time Series models Lecture 2
Introduction to Estimation Methods for Time Series models Lecture 2 Fulvio Corsi SNS Pisa Fulvio Corsi Introduction to Estimation () Methods for Time Series models Lecture 2 SNS Pisa 1 / 21 Estimators:
More information2018 2019 1 9 sei@mistiu-tokyoacjp http://wwwstattu-tokyoacjp/~sei/lec-jhtml 11 552 3 0 1 2 3 4 5 6 7 13 14 33 4 1 4 4 2 1 1 2 2 1 1 12 13 R?boxplot boxplotstats which does the computation?boxplotstats
More informationPractical Econometrics. for. Finance and Economics. (Econometrics 2)
Practical Econometrics for Finance and Economics (Econometrics 2) Seppo Pynnönen and Bernd Pape Department of Mathematics and Statistics, University of Vaasa 1. Introduction 1.1 Econometrics Econometrics
More informationMixed Model Theory, Part I
enote 4 1 enote 4 Mixed Model Theory, Part I enote 4 INDHOLD 2 Indhold 4 Mixed Model Theory, Part I 1 4.1 Design matrix for a systematic linear model.................. 2 4.2 The mixed model.................................
More informationPh.D. Qualifying Exam Friday Saturday, January 6 7, 2017
Ph.D. Qualifying Exam Friday Saturday, January 6 7, 2017 Put your solution to each problem on a separate sheet of paper. Problem 1. (5106) Let X 1, X 2,, X n be a sequence of i.i.d. observations from a
More informationLinear Mixed Models. One-way layout REML. Likelihood. Another perspective. Relationship to classical ideas. Drawbacks.
Linear Mixed Models One-way layout Y = Xβ + Zb + ɛ where X and Z are specified design matrices, β is a vector of fixed effect coefficients, b and ɛ are random, mean zero, Gaussian if needed. Usually think
More informationRandom Intercept Models
Random Intercept Models Edps/Psych/Soc 589 Carolyn J. Anderson Department of Educational Psychology c Board of Trustees, University of Illinois Spring 2019 Outline A very simple case of a random intercept
More informationTopic 25 - One-Way Random Effects Models. Outline. Random Effects vs Fixed Effects. Data for One-way Random Effects Model. One-way Random effects
Topic 5 - One-Way Random Effects Models One-way Random effects Outline Model Variance component estimation - Fall 013 Confidence intervals Topic 5 Random Effects vs Fixed Effects Consider factor with numerous
More informationRandomized Block Designs with Replicates
LMM 021 Randomized Block ANOVA with Replicates 1 ORIGIN := 0 Randomized Block Designs with Replicates prepared by Wm Stein Randomized Block Designs with Replicates extends the use of one or more random
More informationR Output for Linear Models using functions lm(), gls() & glm()
LM 04 lm(), gls() &glm() 1 R Output for Linear Models using functions lm(), gls() & glm() Different kinds of output related to linear models can be obtained in R using function lm() {stats} in the base
More informationEconometrics of Panel Data
Econometrics of Panel Data Jakub Mućk Meeting # 4 Jakub Mućk Econometrics of Panel Data Meeting # 4 1 / 30 Outline 1 Two-way Error Component Model Fixed effects model Random effects model 2 Non-spherical
More informationPeter Hoff Linear and multilinear models April 3, GLS for multivariate regression 5. 3 Covariance estimation for the GLM 8
Contents 1 Linear model 1 2 GLS for multivariate regression 5 3 Covariance estimation for the GLM 8 4 Testing the GLH 11 A reference for some of this material can be found somewhere. 1 Linear model Recall
More informationLecture 2. The Simple Linear Regression Model: Matrix Approach
Lecture 2 The Simple Linear Regression Model: Matrix Approach Matrix algebra Matrix representation of simple linear regression model 1 Vectors and Matrices Where it is necessary to consider a distribution
More informationOn the Behavior of Marginal and Conditional Akaike Information Criteria in Linear Mixed Models
On the Behavior of Marginal and Conditional Akaike Information Criteria in Linear Mixed Models Thomas Kneib Institute of Statistics and Econometrics Georg-August-University Göttingen Department of Statistics
More informationOutline. Mixed models in R using the lme4 package Part 3: Longitudinal data. Sleep deprivation data. Simple longitudinal data
Outline Mixed models in R using the lme4 package Part 3: Longitudinal data Douglas Bates Longitudinal data: sleepstudy A model with random effects for intercept and slope University of Wisconsin - Madison
More informationTopic 17 - Single Factor Analysis of Variance. Outline. One-way ANOVA. The Data / Notation. One way ANOVA Cell means model Factor effects model
Topic 17 - Single Factor Analysis of Variance - Fall 2013 One way ANOVA Cell means model Factor effects model Outline Topic 17 2 One-way ANOVA Response variable Y is continuous Explanatory variable is
More informationDe-mystifying random effects models
De-mystifying random effects models Peter J Diggle Lecture 4, Leahurst, October 2012 Linear regression input variable x factor, covariate, explanatory variable,... output variable y response, end-point,
More informationStat 579: Generalized Linear Models and Extensions
Stat 579: Generalized Linear Models and Extensions Mixed models Yan Lu March, 2018, week 8 1 / 32 Restricted Maximum Likelihood (REML) REML: uses a likelihood function calculated from the transformed set
More informationThe consequences of misspecifying the random effects distribution when fitting generalized linear mixed models
The consequences of misspecifying the random effects distribution when fitting generalized linear mixed models John M. Neuhaus Charles E. McCulloch Division of Biostatistics University of California, San
More informationAMS-207: Bayesian Statistics
Linear Regression How does a quantity y, vary as a function of another quantity, or vector of quantities x? We are interested in p(y θ, x) under a model in which n observations (x i, y i ) are exchangeable.
More informationChapter 4: Constrained estimators and tests in the multiple linear regression model (Part III)
Chapter 4: Constrained estimators and tests in the multiple linear regression model (Part III) Florian Pelgrin HEC September-December 2010 Florian Pelgrin (HEC) Constrained estimators September-December
More informationSTAT5044: Regression and Anova. Inyoung Kim
STAT5044: Regression and Anova Inyoung Kim 2 / 47 Outline 1 Regression 2 Simple Linear regression 3 Basic concepts in regression 4 How to estimate unknown parameters 5 Properties of Least Squares Estimators:
More informationChapter 3: Maximum Likelihood Theory
Chapter 3: Maximum Likelihood Theory Florian Pelgrin HEC September-December, 2010 Florian Pelgrin (HEC) Maximum Likelihood Theory September-December, 2010 1 / 40 1 Introduction Example 2 Maximum likelihood
More informationIntroduction to Hierarchical Data Theory Real Example. NLME package in R. Jiang Qi. Department of Statistics Renmin University of China.
Department of Statistics Renmin University of China June 7, 2010 The problem Grouped data, or Hierarchical data: correlations between subunits within subjects. The problem Grouped data, or Hierarchical
More informationFor more information about how to cite these materials visit
Author(s): Kerby Shedden, Ph.D., 2010 License: Unless otherwise noted, this material is made available under the terms of the Creative Commons Attribution Share Alike 3.0 License: http://creativecommons.org/licenses/by-sa/3.0/
More information13.2 Example: W, LM and LR Tests
13.2 Example: W, LM and LR Tests Date file = cons99.txt (same data as before) Each column denotes year, nominal household expenditures ( 10 billion yen), household disposable income ( 10 billion yen) and
More informationSimple and Multiple Linear Regression
Sta. 113 Chapter 12 and 13 of Devore March 12, 2010 Table of contents 1 Simple Linear Regression 2 Model Simple Linear Regression A simple linear regression model is given by Y = β 0 + β 1 x + ɛ where
More information22s:152 Applied Linear Regression. Returning to a continuous response variable Y...
22s:152 Applied Linear Regression Generalized Least Squares Returning to a continuous response variable Y... Ordinary Least Squares Estimation The classical models we have fit so far with a continuous
More informationSpatial inference. Spatial inference. Accounting for spatial correlation. Multivariate normal distributions
Spatial inference I will start with a simple model, using species diversity data Strong spatial dependence, Î = 0.79 what is the mean diversity? How precise is our estimate? Sampling discussion: The 64
More informationAdvanced Quantitative Methods: maximum likelihood
Advanced Quantitative Methods: Maximum Likelihood University College Dublin March 23, 2011 1 Introduction 2 3 4 5 Outline Introduction 1 Introduction 2 3 4 5 Preliminaries Introduction Ordinary least squares
More information22s:152 Applied Linear Regression. In matrix notation, we can write this model: Generalized Least Squares. Y = Xβ + ɛ with ɛ N n (0, Σ)
22s:152 Applied Linear Regression Generalized Least Squares Returning to a continuous response variable Y Ordinary Least Squares Estimation The classical models we have fit so far with a continuous response
More informationModels for Clustered Data
Models for Clustered Data Edps/Psych/Soc 589 Carolyn J Anderson Department of Educational Psychology c Board of Trustees, University of Illinois Spring 2019 Outline Notation NELS88 data Fixed Effects ANOVA
More informationMASM22/FMSN30: Linear and Logistic Regression, 7.5 hp FMSN40:... with Data Gathering, 9 hp
Selection criteria Example Methods MASM22/FMSN30: Linear and Logistic Regression, 7.5 hp FMSN40:... with Data Gathering, 9 hp Lecture 5, spring 2018 Model selection tools Mathematical Statistics / Centre
More informationModels for Clustered Data
Models for Clustered Data Edps/Psych/Stat 587 Carolyn J Anderson Department of Educational Psychology c Board of Trustees, University of Illinois Fall 2017 Outline Notation NELS88 data Fixed Effects ANOVA
More information36-463/663: Hierarchical Linear Models
36-463/663: Hierarchical Linear Models Lmer model selection and residuals Brian Junker 132E Baker Hall brian@stat.cmu.edu 1 Outline The London Schools Data (again!) A nice random-intercepts, random-slopes
More informationOn the Behavior of Marginal and Conditional Akaike Information Criteria in Linear Mixed Models
On the Behavior of Marginal and Conditional Akaike Information Criteria in Linear Mixed Models Thomas Kneib Department of Mathematics Carl von Ossietzky University Oldenburg Sonja Greven Department of
More informationIntroductory Econometrics
Based on the textbook by Wooldridge: : A Modern Approach Robert M. Kunst robert.kunst@univie.ac.at University of Vienna and Institute for Advanced Studies Vienna November 23, 2013 Outline Introduction
More informationRandom and Mixed Effects Models - Part II
Random and Mixed Effects Models - Part II Statistics 149 Spring 2006 Copyright 2006 by Mark E. Irwin Two-Factor Random Effects Model Example: Miles per Gallon (Neter, Kutner, Nachtsheim, & Wasserman, problem
More informationLinear Models and Estimation by Least Squares
Linear Models and Estimation by Least Squares Jin-Lung Lin 1 Introduction Causal relation investigation lies in the heart of economics. Effect (Dependent variable) cause (Independent variable) Example:
More informationECON 4160, Autumn term Lecture 1
ECON 4160, Autumn term 2017. Lecture 1 a) Maximum Likelihood based inference. b) The bivariate normal model Ragnar Nymoen University of Oslo 24 August 2017 1 / 54 Principles of inference I Ordinary least
More informationA Very Brief Summary of Statistical Inference, and Examples
A Very Brief Summary of Statistical Inference, and Examples Trinity Term 2009 Prof. Gesine Reinert Our standard situation is that we have data x = x 1, x 2,..., x n, which we view as realisations of random
More informationConfidence Intervals, Testing and ANOVA Summary
Confidence Intervals, Testing and ANOVA Summary 1 One Sample Tests 1.1 One Sample z test: Mean (σ known) Let X 1,, X n a r.s. from N(µ, σ) or n > 30. Let The test statistic is H 0 : µ = µ 0. z = x µ 0
More informationOne-stage dose-response meta-analysis
One-stage dose-response meta-analysis Nicola Orsini, Alessio Crippa Biostatistics Team Department of Public Health Sciences Karolinska Institutet http://ki.se/en/phs/biostatistics-team 2017 Nordic and
More informationNow consider the case where E(Y) = µ = Xβ and V (Y) = σ 2 G, where G is diagonal, but unknown.
Weighting We have seen that if E(Y) = Xβ and V (Y) = σ 2 G, where G is known, the model can be rewritten as a linear model. This is known as generalized least squares or, if G is diagonal, with trace(g)
More informationy it = α i + β 0 ix it + ε it (0.1) The panel data estimators for the linear model are all standard, either the application of OLS or GLS.
0.1. Panel Data. Suppose we have a panel of data for groups (e.g. people, countries or regions) i =1, 2,..., N over time periods t =1, 2,..., T on a dependent variable y it and a kx1 vector of independent
More informationStatement: With my signature I confirm that the solutions are the product of my own work. Name: Signature:.
MATHEMATICAL STATISTICS Homework assignment Instructions Please turn in the homework with this cover page. You do not need to edit the solutions. Just make sure the handwriting is legible. You may discuss
More informationLecture 4. Random Effects in Completely Randomized Design
Lecture 4. Random Effects in Completely Randomized Design Montgomery: 3.9, 13.1 and 13.7 1 Lecture 4 Page 1 Random Effects vs Fixed Effects Consider factor with numerous possible levels Want to draw inference
More informationGeneralized Linear Models
Generalized Linear Models Lecture 3. Hypothesis testing. Goodness of Fit. Model diagnostics GLM (Spring, 2018) Lecture 3 1 / 34 Models Let M(X r ) be a model with design matrix X r (with r columns) r n
More information8. Hypothesis Testing
FE661 - Statistical Methods for Financial Engineering 8. Hypothesis Testing Jitkomut Songsiri introduction Wald test likelihood-based tests significance test for linear regression 8-1 Introduction elements
More informationMixed-Models. version 30 October 2011
Mixed-Models version 30 October 2011 Mixed models Mixed models estimate a vector! of fixed effects and one (or more) vectors u of random effects Both fixed and random effects models always include a vector
More informationRegression: Main Ideas Setting: Quantitative outcome with a quantitative explanatory variable. Example, cont.
TCELL 9/4/205 36-309/749 Experimental Design for Behavioral and Social Sciences Simple Regression Example Male black wheatear birds carry stones to the nest as a form of sexual display. Soler et al. wanted
More informationMa 3/103: Lecture 25 Linear Regression II: Hypothesis Testing and ANOVA
Ma 3/103: Lecture 25 Linear Regression II: Hypothesis Testing and ANOVA March 6, 2017 KC Border Linear Regression II March 6, 2017 1 / 44 1 OLS estimator 2 Restricted regression 3 Errors in variables 4
More informationLecture 2: Linear Models. Bruce Walsh lecture notes Seattle SISG -Mixed Model Course version 23 June 2011
Lecture 2: Linear Models Bruce Walsh lecture notes Seattle SISG -Mixed Model Course version 23 June 2011 1 Quick Review of the Major Points The general linear model can be written as y = X! + e y = vector
More information