13. October p. 1

Size: px
Start display at page:

Download "13. October p. 1"

Transcription

1 Lecture 8 STK3100/4100 Linear mixed models 13. October 2014 Plan for lecture: 1. The lme function in the nlme library 2. Induced correlation structure 3. Marginal models 4. Estimation - ML and REML 5. Model selection 6. Model validation p. 1

2 Linear mixed model with all assumptions Y i =X i β +Z i b i +ε i,i = 1,...,N b i N(0,D) ε i N(0,Σ i ) b 1,..,b N,ε 1,...,ε N independent Often: Simplified structures ond,σ i D = d 2 I,Σ i = σ 2 I p. 2

3 Ex. Species richness: Model Simplified model with only NAP Y ij =α+b i +βnap ij +ε ij yieldsβ = (α,β) T, b i = b i,σ = σ 2 I,D = d 2 and 1 NAP i1 1 1 NAP i2 1 X i = 1 NAP i3, Z i = 1 1 NAP i4 1 1 NAP i5 1 p. 3

4 Ex. Species richness: Output from lme > RIKZ$fBeach <- factor(rikz$beach) > Mlme1 <- lme(richness NAP, random = 1 fbeach,data=rikz) > summary(mlme1) Linear mixed-effects model fit by REML Data: RIKZ AIC BIC loglik Random effects: Formula: 1 fbeach (Intercept) Residual StdDev: Fixed effects: Richness NAP Value Std.Error DF t-value p-value (Intercept) NAP Here is ˆd 2 = = and ˆσ 2 = = p. 4

5 Ex. Species richness: Coefficients > Mlme1 <- lme(richness NAP, random = 1 fbeach,data=rikz) > summary(mlme1)$coef $fixed (Intercept) NAP $random $random$fbeach (Intercept) ˆbi = E[b i data, estimated parameters]. p. 5

6 Ex. Species richness: Fitted values Y i =X i β +Z i b i +ε i Two options for fitted values: ˆµ i =X iˆβi Level 0 µ i =X iˆβi +Z i ˆbi Level 1 Mlme1 <- lme(richness NAP, random = 1 fbeach,data=rikz) F0<-fitted(Mlme1,level=0) F1<-fitted(Mlme1,level=1) p. 6

7 Ex. Species richness: Fitted values NAP Richness Level 0 Level 1 p. 7

8 Random intercept and slope model Species richness Y ij =α+b 1i +(β +b 2i )NAP ij +ε ij Intercept α+b 1i N(α,d 2 11) Slopeβ +b 2i N(β,d 2 22) Can also have non-zero Cov[b 1i,b 2i ] = d 12 = d 21 p. 8

9 R code for random intercept and slope > Mlme2 <- lme(richness 1+NAP, + random = 1 + NAP fbeach, data = RIKZ) > summary(mlme2) Random effects: Formula: 1 + NAP fbeach Structure: General positive-definite, Log-Cholesky parametrization StdDev Corr (Intercept) (Intr) NAP Residual Fixed effects: Richness NAP Value Std.Error DF t-value p-value (Intercept) e+00 NAP e-04 Correlation: (Intr) NAP p. 9

10 Fitted values: up:separate regressions low: random intercept and slope RIKZ$Richness RIKZ$NAP RIKZ$Richness RIKZ$NAP p. 10

11 Model on matrix form yields Y i =X i β +Z i b i +ε i b i N(0,D) ε i N(0,Σ i ) Y i N(X i β,v i ) V i =Z i D Z T i +Σ i p. 11

12 Induced correlation structure in random intercept and slope model Y ij =α+b 1i +(β +b 2i )NAP ij +ε ij Var[Y ij ] =d NAP ij d 12 + NAP 2 ij d σ 2 Cov[Y ij,y ik ] =d (NAP ij + NAP ik ) d 12 + (NAP ij NAP ik ) d σ 2 p. 12

13 Intraclass correlation and effective sample size Simple situation: Y i = (Y i1,...,y in ), E[Y ij ] =µ, Var[Y ij ] = σ 2, Cov[Y ij,y ik ] = ρσ 2 ˆµ =Ȳ Var[Ȳ] = 1 n 2[ n j=1 Var[Y ij ]+ k j Cov[Y ij,y ik ]] = 1 n 2[nσ2 +n(n 1)ρσ 2 ] = σ2 n [1+(n 1)ρ] Variance increased by 1 +(n 1)ρ. Design effect Ex: n = 5, ˆρ = 0.48 yields1+(n 1)ρ = Effective sample size: N effective = N n design effect = = p. 13

14 Marginal model/likelihood Y i N(X i β,v i ) V i =Z i DZ T i +Σ i dependent on parameters ψ L i =f(y i ;β,ψ) 1 = (2π) n i/2 V i 1/2 exp{ 1(Y 2 i X i β) T V 1 i (Y i X i β)} l i = n i log(2π) 1 log V 2 2 i 1(Y 2 i X i β) T V 1 i (Y i X i β) N l(β,ψ) = i=1 l i p. 14

15 Direct specification of marginal model V i = τ 2 φ φ φ φ φ τ 2 φ φ φ φ φ τ 2 φ φ φ φ φ τ 2 φ φ φ φ φ τ 2 Compound symmetric structure, corresponds to Σ i =σ 2 I,Z i = 1 T,D = d 2 = φ,τ 2 = σ 2 +d 2 p. 15

16 Direct specification of marginal model cont. V i = τ 2 c 21 c 31 c 41 c 51 c 21 τ 2 c 32 c 42 c 52 c 31 c 32 τ 2 c 43 c 53 c 41 c 42 c 43 τ 2 c 54 c 51 c 52 c 53 c 54 τ 2 general covariance matrix p. 16

17 Ex. Species richness Mixed model > M.mixed <- lme(richness NAP, random = 1 fbeach, data = RIKZ) > summary(m.mixed)$coef$fixed (Intercept) NAP Marginal model can be estimated by the gls function > M.gls <- gls(richness NAP, correlation = corcompsymm(form = 1 fbeach), data = RIKZ) > coef(m.gls) (Intercept) NAP p. 17

18 Estimation ML estimation: yields ˆµ = ȳ and Y i uif N(µ,σ 2 ) ˆσ 2 = 1 n n (y i ȳ) 2 Biased i=1 We often prefer ˆσ 2 = 1 n 1 n (y i ȳ) 2 Unbiased i=1 Bias in MLE because Uncertainty inµis ignored p. 18

19 ML estimation V i =V i (ψ) N l(β,ψ) = [ n i log(2π) 1 log V 2 2 i i=1 Can be maximised numerically 1 (Y 2 i X i β) T V 1 i (Y i X i β)] Gives biased estimates for variances Not so important in ordinary regression, because observations are independent and get the same weight More important in mixed models, because the weight or importance of observations in the estimation process depends on the variances p. 19

20 REML= Restricted maximum likelihood REML is an alternative strategy which gives unbiased estimates Illustrated by the ordinary linear regression model: Y i = X i β +ε i, ε i N(0,σ 2 ) Idea: Transform data such thatβ disappears A is ann (n p) matrix of full rank such thata T X = 0 Gives (n p) 1 {}}{ A T Y i =A T X i β +A T ε i N(0,σ 2 A T A) REML: Estimate σ 2 by ML based ona T Y i p. 20

21 REML cont. L REML = l REML = N 2 Yields exp{ Y T A[A T A] 1 A T Y} (2π) N/2 σ 2 A T A 1/2 2σ 2 n p log(2π) logσ log AT A Y T A[A T A] 1 A T Y 2σ 2 ˆσ 2 = 1 n p YT A[A T A] 1 A T Y p. 21

22 Example Y i uif N(µ,σ 2 ) X =1 T N = (1 T N 1 1) A = I N 1 1 T N 1 A T X =I N 1 1 N 1 1 N 1 = 0 A T Y =(y 1 y n,...,y n 1 y n ) T ˆσ 2 = 1 n (y i ȳ) 2 n 1 i=1 Derivation of ˆσ 2 includes some calculations p. 22

23 REML and mixed models Model Y i N(X i β,v i ), V i = Z i D Z T i +Σ i combined Y N(Xβ,V) Define A such thata T X = 0. This yields A T Y N(0,A T VA) Estimate parameters invby ML estimation based ona T Y Note: The result does not depend on how we specifya. Gives unbiased estimates for elements inv! p. 23

24 Estimation ofβ by REML REML transformation gives estimate forv i. β can then be estimated by ML conditioned on the estimates of the V i s p. 24

25 Example in R table(rikz$exposure) RIKZ$fExp<-RIKZ$Exposure RIKZ$fExp[RIKZ$fExp==8]<-10 RIKZ$fExp<-factor(RIKZ$fExp,levels=c(10,11)) M0.ML <- lme(richness NAP, data = RIKZ, random = 1 fbeach, method = "ML") M0.REML <-lme(richness NAP, data = RIKZ, random = 1 fbeach, method = "REML") M1.ML <- lme(richness NAP+fExp, data = RIKZ, random = 1 fbeach, method = "ML") M1.REML <- lme(richness NAP+fExp, data = RIKZ, random = 1 fbeach, method = "REML") p. 25

26 Estimates p. 26

27 Ex. Species richness: individual fixed intercepts + random slope (N=NAP,Ex=Exposure) Y ij =α i +(η +τex i +b i )N ij +ε ij =α i +ηn ij +τex i N ij +b i N ij +ε ij yieldsβ = (α 1,...,α 9,η,τ) T,b i = b i,σ = σ 2 I,D = d 2 and i {}}{ N i1 Ex i N i N i2 Ex i N i2 X i = N i3 Ex i N, Z i = i N i4 Ex i N i N i5 Ex i N i5 N i1 N i2 N i3 N i4 N i5 p. 27

28 Model selection in mixed models Two parts of the model: Fixed effects - explanatory variables Random effects - correlation structure These will interact Must use different methods for model selection p. 28

29 Protocol for model selection in mixed models Main idea: Want to explain as much as possible from the fixed effects 1. Start with the model with all explanatory variables (if possible) and as many interactions as possible - beyond optimal model 2. Find optimal structure on random effects using REML 3. Find optimal structure for fixed effects using REML for t- and F-tests using ML for likelihood ratio tests 4. Estimate the final model using REML p. 29

30 Model selection methods Two main strategies Information criteria: AIC, BIC Hypothesis testing on parameters in nested models t test (Wald test) F test (multiple parameters) Likelihood ratio test p. 30

31 AIC/BIC and ML AIC = 2 l(ˆθ)+2 p BIC = 2 l(ˆθ)+log(n) p Misprint in the book p: Number of parameters in the model, bothβ-s andσ-s n = N i=1 n i AIC: tend to give large models, especially with many observations BIC: gives smaller models, but perhaps to small for smalln Can be used directly in ML estimation l(ˆθ) is the ML log likelihood p. 31

32 AIC/BIC and REML AIC = 2 l(ˆθ)+2 p BIC = 2 l(ˆθ)+log(n p) p Misprint in the book p: Number of parameters in the model, bothβ-s andσ-s l(ˆθ) is now the REML log likelihood Can show L REML (θ) = N i=1 X T i V 1 i X i 1/2 L ML (θ) p. 32

33 Ex: The wrong approach (the good is presented later) Species richness: Start with only NAP as explanatory variable - this is a bad choice! Find optimal structure for random effects, three possibilities: No random effects Random intercept for each area Random intercept and slope for each area Compare models by REML Note: lm use ML, and lme can only be used for random effect models, so we must use gls for the model without random effects p. 33

34 Ex. The wrong approach: Step 2, by AIC/BIC > Wrong1 <- gls(richness 1 + NAP, method = "REML", data = RIKZ) > Wrong2 <- lme(richness 1 + NAP, random = 1 fbeach, method = "REML", data = RIKZ) > Wrong3 <- lme(richness 1 + NAP, method = "REML", random = 1 + NAP fbeach, data = RIKZ) > cbind(aic(wrong1,wrong2,wrong3),bic(wrong1,wrong2,wrong3)) df AIC df BIC Wrong Wrong Wrong AIC: Best to include both random intercept and slope BIC: Best to includerandom intercept, but fixed slope p. 34

35 LR tests for random effects Likelihood ratio test by the anova function: > anova(wrong1,wrong2,wrong3) Model df AIC BIC loglik Test L.Ratio p-value Wrong Wrong vs Wrong vs Problem: Test H 0 : σ? 2 parameter space Gives wrong p values! = 0 which is on the boundary of the p. 35

36 LR tests for random effects cont. H 0 : θ Θ 0 mot H 1 : θ Θ a LR test: Ordinary asymptotic theory requiresθ 0 to be in the inner of Θ = Θ 0 Θ a. Have then 2LR χ 2 q a q 0. P value =Pr(χ 2 q a q 0 > 2LR). Here: TestsH 0 : d 2 11 = 0. On the boundary of Θ : {d2 11 0}. Can show: P values too large. Conservative test More precise: Assume k random effects underh 0, k +1 random effects under H a. T = 2LR Pr(T > c) = 0.5 [Pr(χ 2 k > c)+pr(χ2 k+1 > c)] Especially: k = 0: Pr(T > c) = 0.5 Pr(χ 2 1 > c) p. 36

37 Ex. The wrong approach: Step 2, by corrected LRT Wrong1 vs. Wrong2: > 0.5*(1-pchisq(T,1)) [1] Wrong2 vs. Wrong3 > T = anova(wrong1,wrong2,wrong3)[3,8] > 1-0.5*(pchisq(T,1)+pchisq(T,2)) [1] Choose model Wrong3 including both random intercept and slope p. 37

38 Ex. The wrong approach: Step 3, fixed effects > summary(wrong3) Fixed effects: Richness 1 + NAP Value Std.Error DF t-value p-value (Intercept) e+00 NAP e-04 NAP significant, we include now Exposure and interaction > RIKZ$fExp<-RIKZ$Exposure > RIKZ$fExp[RIKZ$fExp==8]<-10 > RIKZ$fExp<-factor(RIKZ$fExp,levels=c(10,11)) > Wrong4 <- lme(richness 1 + NAP * fexp,random = 1 + NAP fbeach, method = "REML", data = RIKZ) Error in lme.formula(richness 1 + NAP * fexp, random= 1+NAP fbeach, : nlminb problem, convergence error code = 1 > lmc <- lmecontrol(niterem = 2200, msmaxiter = 2200) > Wrong4 = lme(richness 1 + NAP * fexp,random = 1 + NAP fbeach, method = "REML", data = RIKZ,control=lmc) p. 38

39 Ex. The wrong approach: Step 3 cont. - F and t tests based on REML > anova(wrong4) numdf dendf F-value p-value (Intercept) <.0001 NAP fexp NAP:fExp > summary(wrong4) Fixed effects: Richness 1 + NAP * fexp Value Std.Error DF t-value p-value (Intercept) NAP fexp NAP:fExp Exposure seems to be significant, but interaction not significant p. 39

40 Ex. The wrong approach: Step 3 cont. Excluding interaction - F and t tests based on REML > Wrong4.2 <- lme(richness 1 + NAP + fexp,random = 1 + NAP fbeach, method = "REML", data = RIKZ) > anova(wrong4.2) numdf dendf F-value p-value (Intercept) <.0001 NAP fexp > summary(wrong4.2) Fixed effects: Richness 1 + NAP + fexp Value Std.Error DF t-value p-value (Intercept) NAP fexp Exposure is bordeline significant at 5 % level Final model so far: Y ij =α+b 1i +(β +b 2i )NAP ij +ε ij p. 40

41 Degree of freedom (df) Are used intandf tests Explanatory variables divided into two groups Level 1: Variables with different values for observations within each group df = Total no obs. - no groups - no level 1 variables Ex: NAP, df=45-9-1=35 Level 2: Variables with equal values within each group df = No groups - no level 2 variables (including constant term) Ex: Exposure, df=9-2=7 p. 41

42 Ex. The wrong approach: Step 3 cont. Likelihood ratio tests based on ML > lmc <- lmecontrol(niterem = 5200, msmaxiter = 5200) > Wrong4A <- lme(richness 1 + NAP, method="ml", control = lmc, data = RIKZ, random = 1+NAP fbeach) > Wrong4B <- lme(richness 1 + NAP + fexp, random = 1 + NAP fbeach, method="ml", data = RIKZ,control = lmc) > Wrong4C <- lme(richness 1 + NAP * fexp, random = 1 + NAP fbeach, data = RIKZ, method = "ML", control = lmc) > anova(wrong4a, Wrong4B, Wrong4C) Model df AIC BIC loglik Test L.Ratio p-value Wrong4A Wrong4B vs Wrong4C vs LRT gives optimal model: Only NAP but interaction term bordeline significant, so not totally clear p. 42

43 Ex. The wrong approach: Step 4, final model > Wrong5 <- lme(richness 1 + NAP,random = 1 + NAP fbeach, + method = "REML", data = RIKZ) > summary(wrong5) Random effects: Formula: 1 + NAP fbeach Structure: General positive-definite, Log-Cholesky parametrization StdDev Corr (Intercept) (Intr) NAP Residual Fixed effects: Richness 1 + NAP Value Std.Error DF t-value p-value (Intercept) e+00 NAP e-04 Number of Observations: 45 Number of Groups: 9 p. 43

44 Ex. The good method: Step 1 and 2 Includes both variables + interaction > B1=gls(Richness 1+NAP*fExp,method="REML",data=RIKZ) > B2=lme(Richness 1+NAP*fExp,data=RIKZ,random= 1 fbeach,method="reml") > B3=lme(Richness 1+NAP*fExp,data=RIKZ,random= 1+NAP fbeach,method="reml > AIC(B1,B2,B3) df AIC B B B Best model: B2 - random intercept p. 44

45 Ex. The good method: Step 3 > summary(b2) Fixed effects: Richness 1 + NAP * fexp Value Std.Error DF t-value p-value (Intercept) NAP fexp NAP:fExp Drop the interaction term p. 45

46 Ex. The good method: Step 4, final model > B2B=lme(Richness 1+NAP+fExp,data=RIKZ,random= 1 fbeach,method="reml") > summary(b2b) Linear mixed-effects model fit by REML Data: RIKZ AIC BIC loglik Random effects: Formula: 1 fbeach (Intercept) Residual StdDev: Fixed effects: Richness 1 + NAP + fexp Value Std.Error DF t-value p-value (Intercept) NAP fexp Final model: Y ij =α+b i +β 1 NAP ij +β 2 Exposure i +ε ij p. 46

47 Model validation Model choice: Choose among a set of models Does the final model describe the data well? Model validation: Check the final model Goodness-of-fit test Residual plot p. 47

48 Model validation - residuals Response residuals (default in R): Y ij ŷ ij Standardised residuals (Pearson residuals) Y ij ŷ ij sd[y ij ] Both can be computed at different levels: Level 0, population level: ŷ ij = X T i ˆβ. Level 1, within group: ŷ ij = X T i ˆβ +Z T i ˆb i. (Default) sd[y ij ] adjusted according to level Note: Level 0 residuals are dependent! p. 48

49 Residual plot for RIKZ data - level 0 plot(fitted(b2b,level=0),residuals(b2b,level=0)) residuals(b2b, level = 0) fitted(b2b, level = 0) p. 49

50 Residual plot for RIKZ data - level 1 plot(fitted(b2b,level=1),residuals(b2b,level=1)) residuals(b2b, level = 1) fitted(b2b, level = 1) p. 50

Lecture 9 STK3100/4100

Lecture 9 STK3100/4100 Lecture 9 STK3100/4100 27. October 2014 Plan for lecture: 1. Linear mixed models cont. Models accounting for time dependencies (Ch. 6.1) 2. Generalized linear mixed models (GLMM, Ch. 13.1-13.3) Examples

More information

These slides illustrate a few example R commands that can be useful for the analysis of repeated measures data.

These slides illustrate a few example R commands that can be useful for the analysis of repeated measures data. These slides illustrate a few example R commands that can be useful for the analysis of repeated measures data. We focus on the experiment designed to compare the effectiveness of three strength training

More information

Exploring Hierarchical Linear Mixed Models

Exploring Hierarchical Linear Mixed Models Exploring Hierarchical Linear Mixed Models 1/49 Last time... A Greenhouse Experiment testing C:N Ratios Sam was testing how changing the C:N Ratio of soil affected plant leaf growth. He had 3 treatments.

More information

20. REML Estimation of Variance Components. Copyright c 2018 (Iowa State University) 20. Statistics / 36

20. REML Estimation of Variance Components. Copyright c 2018 (Iowa State University) 20. Statistics / 36 20. REML Estimation of Variance Components Copyright c 2018 (Iowa State University) 20. Statistics 510 1 / 36 Consider the General Linear Model y = Xβ + ɛ, where ɛ N(0, Σ) and Σ is an n n positive definite

More information

Hierarchical Linear Models (HLM) Using R Package nlme. Interpretation. 2 = ( x 2) u 0j. e ij

Hierarchical Linear Models (HLM) Using R Package nlme. Interpretation. 2 = ( x 2) u 0j. e ij Hierarchical Linear Models (HLM) Using R Package nlme Interpretation I. The Null Model Level 1 (student level) model is mathach ij = β 0j + e ij Level 2 (school level) model is β 0j = γ 00 + u 0j Combined

More information

Model comparison and selection

Model comparison and selection BS2 Statistical Inference, Lectures 9 and 10, Hilary Term 2008 March 2, 2008 Hypothesis testing Consider two alternative models M 1 = {f (x; θ), θ Θ 1 } and M 2 = {f (x; θ), θ Θ 2 } for a sample (X = x)

More information

Introduction to General and Generalized Linear Models

Introduction to General and Generalized Linear Models Introduction to General and Generalized Linear Models Mixed effects models - II Henrik Madsen, Jan Kloppenborg Møller, Anders Nielsen April 16, 2012 H. Madsen, JK. Møller, A. Nielsen () Chapman & Hall

More information

Introduction and Background to Multilevel Analysis

Introduction and Background to Multilevel Analysis Introduction and Background to Multilevel Analysis Dr. J. Kyle Roberts Southern Methodist University Simmons School of Education and Human Development Department of Teaching and Learning Background and

More information

Stat 579: Generalized Linear Models and Extensions

Stat 579: Generalized Linear Models and Extensions Stat 579: Generalized Linear Models and Extensions Linear Mixed Models for Longitudinal Data Yan Lu April, 2018, week 14 1 / 64 Data structure and Model t1 t2 tn i 1st subject y 11 y 12 y 1n1 2nd subject

More information

Workshop 9.3a: Randomized block designs

Workshop 9.3a: Randomized block designs -1- Workshop 93a: Randomized block designs Murray Logan November 23, 16 Table of contents 1 Randomized Block (RCB) designs 1 2 Worked Examples 12 1 Randomized Block (RCB) designs 11 RCB design Simple Randomized

More information

Outline for today. Maximum likelihood estimation. Computation with multivariate normal distributions. Multivariate normal distribution

Outline for today. Maximum likelihood estimation. Computation with multivariate normal distributions. Multivariate normal distribution Outline for today Maximum likelihood estimation Rasmus Waageetersen Deartment of Mathematics Aalborg University Denmark October 30, 2007 the multivariate normal distribution linear and linear mixed models

More information

Repeated measures, part 2, advanced methods

Repeated measures, part 2, advanced methods enote 12 1 enote 12 Repeated measures, part 2, advanced methods enote 12 INDHOLD 2 Indhold 12 Repeated measures, part 2, advanced methods 1 12.1 Intro......................................... 3 12.2 A

More information

A brief introduction to mixed models

A brief introduction to mixed models A brief introduction to mixed models University of Gothenburg Gothenburg April 6, 2017 Outline An introduction to mixed models based on a few examples: Definition of standard mixed models. Parameter estimation.

More information

BIO5312 Biostatistics Lecture 13: Maximum Likelihood Estimation

BIO5312 Biostatistics Lecture 13: Maximum Likelihood Estimation BIO5312 Biostatistics Lecture 13: Maximum Likelihood Estimation Yujin Chung November 29th, 2016 Fall 2016 Yujin Chung Lec13: MLE Fall 2016 1/24 Previous Parametric tests Mean comparisons (normality assumption)

More information

Outline. Statistical inference for linear mixed models. One-way ANOVA in matrix-vector form

Outline. Statistical inference for linear mixed models. One-way ANOVA in matrix-vector form Outline Statistical inference for linear mixed models Rasmus Waagepetersen Department of Mathematics Aalborg University Denmark general form of linear mixed models examples of analyses using linear mixed

More information

MISCELLANEOUS TOPICS RELATED TO LIKELIHOOD. Copyright c 2012 (Iowa State University) Statistics / 30

MISCELLANEOUS TOPICS RELATED TO LIKELIHOOD. Copyright c 2012 (Iowa State University) Statistics / 30 MISCELLANEOUS TOPICS RELATED TO LIKELIHOOD Copyright c 2012 (Iowa State University) Statistics 511 1 / 30 INFORMATION CRITERIA Akaike s Information criterion is given by AIC = 2l(ˆθ) + 2k, where l(ˆθ)

More information

Restricted Maximum Likelihood in Linear Regression and Linear Mixed-Effects Model

Restricted Maximum Likelihood in Linear Regression and Linear Mixed-Effects Model Restricted Maximum Likelihood in Linear Regression and Linear Mixed-Effects Model Xiuming Zhang zhangxiuming@u.nus.edu A*STAR-NUS Clinical Imaging Research Center October, 015 Summary This report derives

More information

Introduction to Within-Person Analysis and RM ANOVA

Introduction to Within-Person Analysis and RM ANOVA Introduction to Within-Person Analysis and RM ANOVA Today s Class: From between-person to within-person ANOVAs for longitudinal data Variance model comparisons using 2 LL CLP 944: Lecture 3 1 The Two Sides

More information

Simple Linear Regression

Simple Linear Regression Simple Linear Regression In simple linear regression we are concerned about the relationship between two variables, X and Y. There are two components to such a relationship. 1. The strength of the relationship.

More information

Stat 579: Generalized Linear Models and Extensions

Stat 579: Generalized Linear Models and Extensions Stat 579: Generalized Linear Models and Extensions Linear Mixed Models for Longitudinal Data Yan Lu April, 2018, week 15 1 / 38 Data structure t1 t2 tn i 1st subject y 11 y 12 y 1n1 Experimental 2nd subject

More information

Covariance Models (*) X i : (n i p) design matrix for fixed effects β : (p 1) regression coefficient for fixed effects

Covariance Models (*) X i : (n i p) design matrix for fixed effects β : (p 1) regression coefficient for fixed effects Covariance Models (*) Mixed Models Laird & Ware (1982) Y i = X i β + Z i b i + e i Y i : (n i 1) response vector X i : (n i p) design matrix for fixed effects β : (p 1) regression coefficient for fixed

More information

CO2 Handout. t(cbind(co2$type,co2$treatment,model.matrix(~type*treatment,data=co2)))

CO2 Handout. t(cbind(co2$type,co2$treatment,model.matrix(~type*treatment,data=co2))) CO2 Handout CO2.R: library(nlme) CO2[1:5,] plot(co2,outer=~treatment*type,layout=c(4,1)) m1co2.lis

More information

Estimation: Problems & Solutions

Estimation: Problems & Solutions Estimation: Problems & Solutions Edps/Psych/Stat 587 Carolyn J. Anderson Department of Educational Psychology c Board of Trustees, University of Illinois Fall 2017 Outline 1. Introduction: Estimation of

More information

Stat/F&W Ecol/Hort 572 Review Points Ané, Spring 2010

Stat/F&W Ecol/Hort 572 Review Points Ané, Spring 2010 1 Linear models Y = Xβ + ɛ with ɛ N (0, σ 2 e) or Y N (Xβ, σ 2 e) where the model matrix X contains the information on predictors and β includes all coefficients (intercept, slope(s) etc.). 1. Number of

More information

Random and Mixed Effects Models - Part III

Random and Mixed Effects Models - Part III Random and Mixed Effects Models - Part III Statistics 149 Spring 2006 Copyright 2006 by Mark E. Irwin Quasi-F Tests When we get to more than two categorical factors, some times there are not nice F tests

More information

Estimation: Problems & Solutions

Estimation: Problems & Solutions Estimation: Problems & Solutions Edps/Psych/Soc 587 Carolyn J. Anderson Department of Educational Psychology c Board of Trustees, University of Illinois Spring 2019 Outline 1. Introduction: Estimation

More information

Lecture 15. Hypothesis testing in the linear model

Lecture 15. Hypothesis testing in the linear model 14. Lecture 15. Hypothesis testing in the linear model Lecture 15. Hypothesis testing in the linear model 1 (1 1) Preliminary lemma 15. Hypothesis testing in the linear model 15.1. Preliminary lemma Lemma

More information

Homework 3 - Solution

Homework 3 - Solution STAT 526 - Spring 2011 Homework 3 - Solution Olga Vitek Each part of the problems 5 points 1. KNNL 25.17 (Note: you can choose either the restricted or the unrestricted version of the model. Please state

More information

Problems. Suppose both models are fitted to the same data. Show that SS Res, A SS Res, B

Problems. Suppose both models are fitted to the same data. Show that SS Res, A SS Res, B Simple Linear Regression 35 Problems 1 Consider a set of data (x i, y i ), i =1, 2,,n, and the following two regression models: y i = β 0 + β 1 x i + ε, (i =1, 2,,n), Model A y i = γ 0 + γ 1 x i + γ 2

More information

MA 575 Linear Models: Cedric E. Ginestet, Boston University Midterm Review Week 7

MA 575 Linear Models: Cedric E. Ginestet, Boston University Midterm Review Week 7 MA 575 Linear Models: Cedric E. Ginestet, Boston University Midterm Review Week 7 1 Random Vectors Let a 0 and y be n 1 vectors, and let A be an n n matrix. Here, a 0 and A are non-random, whereas y is

More information

Statistics 203: Introduction to Regression and Analysis of Variance Course review

Statistics 203: Introduction to Regression and Analysis of Variance Course review Statistics 203: Introduction to Regression and Analysis of Variance Course review Jonathan Taylor - p. 1/?? Today Review / overview of what we learned. - p. 2/?? General themes in regression models Specifying

More information

Short Course Applied Linear and Nonlinear Mixed Models* Introduction

Short Course Applied Linear and Nonlinear Mixed Models* Introduction Short Course Applied Linear and Nonlinear Mixed Models* Introduction Mixed-effect models (or simply, mixed models ) are like classical ( fixedeffects ) statistical models, except that some of the parameters

More information

Parametric Modelling of Over-dispersed Count Data. Part III / MMath (Applied Statistics) 1

Parametric Modelling of Over-dispersed Count Data. Part III / MMath (Applied Statistics) 1 Parametric Modelling of Over-dispersed Count Data Part III / MMath (Applied Statistics) 1 Introduction Poisson regression is the de facto approach for handling count data What happens then when Poisson

More information

STAT3401: Advanced data analysis Week 10: Models for Clustered Longitudinal Data

STAT3401: Advanced data analysis Week 10: Models for Clustered Longitudinal Data STAT3401: Advanced data analysis Week 10: Models for Clustered Longitudinal Data Berwin Turlach School of Mathematics and Statistics Berwin.Turlach@gmail.com The University of Western Australia Models

More information

Intruction to General and Generalized Linear Models

Intruction to General and Generalized Linear Models Intruction to General and Generalized Linear Models Mixed Effects Models IV Henrik Madsen Anna Helga Jónsdóttir hm@imm.dtu.dk April 30, 2012 Henrik Madsen Anna Helga Jónsdóttir (hm@imm.dtu.dk) Intruction

More information

MS&E 226: Small Data

MS&E 226: Small Data MS&E 226: Small Data Lecture 15: Examples of hypothesis tests (v5) Ramesh Johari ramesh.johari@stanford.edu 1 / 32 The recipe 2 / 32 The hypothesis testing recipe In this lecture we repeatedly apply the

More information

Fall 2017 STAT 532 Homework Peter Hoff. 1. Let P be a probability measure on a collection of sets A.

Fall 2017 STAT 532 Homework Peter Hoff. 1. Let P be a probability measure on a collection of sets A. 1. Let P be a probability measure on a collection of sets A. (a) For each n N, let H n be a set in A such that H n H n+1. Show that P (H n ) monotonically converges to P ( k=1 H k) as n. (b) For each n

More information

Part III: Linear mixed effects models

Part III: Linear mixed effects models Part III: Linear mixed effects models 159 BIO 245, Spring 2018 Dental growth data Females Males Length, mm 16 18 20 22 24 26 28 30 32 11 11 4 3 2 11 4 7 11 4 3 3 2 8 4 8 5 8 58 7 1 7 5 6 9 57 2 1 9 12

More information

SCHOOL OF MATHEMATICS AND STATISTICS. Linear and Generalised Linear Models

SCHOOL OF MATHEMATICS AND STATISTICS. Linear and Generalised Linear Models SCHOOL OF MATHEMATICS AND STATISTICS Linear and Generalised Linear Models Autumn Semester 2017 18 2 hours Attempt all the questions. The allocation of marks is shown in brackets. RESTRICTED OPEN BOOK EXAMINATION

More information

Correlation and Regression

Correlation and Regression Correlation and Regression October 25, 2017 STAT 151 Class 9 Slide 1 Outline of Topics 1 Associations 2 Scatter plot 3 Correlation 4 Regression 5 Testing and estimation 6 Goodness-of-fit STAT 151 Class

More information

Multivariate Statistics in Ecology and Quantitative Genetics Summary

Multivariate Statistics in Ecology and Quantitative Genetics Summary Multivariate Statistics in Ecology and Quantitative Genetics Summary Dirk Metzler & Martin Hutzenthaler http://evol.bio.lmu.de/_statgen 5. August 2011 Contents Linear Models Generalized Linear Models Mixed-effects

More information

Workshop 9.1: Mixed effects models

Workshop 9.1: Mixed effects models -1- Workshop 91: Mixed effects models Murray Logan October 10, 2016 Table of contents 1 Non-independence - part 2 1 1 Non-independence - part 2 11 Linear models Homogeneity of variance σ 2 0 0 y i = β

More information

STK4900/ Lecture 10. Program

STK4900/ Lecture 10. Program STK4900/9900 - Lecture 10 Program 1. Repeated measures and longitudinal data 2. Simple analysis approaches 3. Random effects models 4. Generalized estimating equations (GEE) 5. GEE for binary data (and

More information

On the Behavior of Marginal and Conditional Akaike Information Criteria in Linear Mixed Models

On the Behavior of Marginal and Conditional Akaike Information Criteria in Linear Mixed Models On the Behavior of Marginal and Conditional Akaike Information Criteria in Linear Mixed Models Thomas Kneib Department of Mathematics Carl von Ossietzky University Oldenburg Sonja Greven Department of

More information

36-720: Linear Mixed Models

36-720: Linear Mixed Models 36-720: Linear Mixed Models Brian Junker October 8, 2007 Review: Linear Mixed Models (LMM s) Bayesian Analogues Facilities in R Computational Notes Predictors and Residuals Examples [Related to Christensen

More information

Mixed effects models

Mixed effects models Mixed effects models The basic theory and application in R Mitchel van Loon Research Paper Business Analytics Mixed effects models The basic theory and application in R Author: Mitchel van Loon Research

More information

Ch 2: Simple Linear Regression

Ch 2: Simple Linear Regression Ch 2: Simple Linear Regression 1. Simple Linear Regression Model A simple regression model with a single regressor x is y = β 0 + β 1 x + ɛ, where we assume that the error ɛ is independent random component

More information

Introduction to Estimation Methods for Time Series models Lecture 2

Introduction to Estimation Methods for Time Series models Lecture 2 Introduction to Estimation Methods for Time Series models Lecture 2 Fulvio Corsi SNS Pisa Fulvio Corsi Introduction to Estimation () Methods for Time Series models Lecture 2 SNS Pisa 1 / 21 Estimators:

More information

2018 2019 1 9 sei@mistiu-tokyoacjp http://wwwstattu-tokyoacjp/~sei/lec-jhtml 11 552 3 0 1 2 3 4 5 6 7 13 14 33 4 1 4 4 2 1 1 2 2 1 1 12 13 R?boxplot boxplotstats which does the computation?boxplotstats

More information

Practical Econometrics. for. Finance and Economics. (Econometrics 2)

Practical Econometrics. for. Finance and Economics. (Econometrics 2) Practical Econometrics for Finance and Economics (Econometrics 2) Seppo Pynnönen and Bernd Pape Department of Mathematics and Statistics, University of Vaasa 1. Introduction 1.1 Econometrics Econometrics

More information

Mixed Model Theory, Part I

Mixed Model Theory, Part I enote 4 1 enote 4 Mixed Model Theory, Part I enote 4 INDHOLD 2 Indhold 4 Mixed Model Theory, Part I 1 4.1 Design matrix for a systematic linear model.................. 2 4.2 The mixed model.................................

More information

Ph.D. Qualifying Exam Friday Saturday, January 6 7, 2017

Ph.D. Qualifying Exam Friday Saturday, January 6 7, 2017 Ph.D. Qualifying Exam Friday Saturday, January 6 7, 2017 Put your solution to each problem on a separate sheet of paper. Problem 1. (5106) Let X 1, X 2,, X n be a sequence of i.i.d. observations from a

More information

Linear Mixed Models. One-way layout REML. Likelihood. Another perspective. Relationship to classical ideas. Drawbacks.

Linear Mixed Models. One-way layout REML. Likelihood. Another perspective. Relationship to classical ideas. Drawbacks. Linear Mixed Models One-way layout Y = Xβ + Zb + ɛ where X and Z are specified design matrices, β is a vector of fixed effect coefficients, b and ɛ are random, mean zero, Gaussian if needed. Usually think

More information

Random Intercept Models

Random Intercept Models Random Intercept Models Edps/Psych/Soc 589 Carolyn J. Anderson Department of Educational Psychology c Board of Trustees, University of Illinois Spring 2019 Outline A very simple case of a random intercept

More information

Topic 25 - One-Way Random Effects Models. Outline. Random Effects vs Fixed Effects. Data for One-way Random Effects Model. One-way Random effects

Topic 25 - One-Way Random Effects Models. Outline. Random Effects vs Fixed Effects. Data for One-way Random Effects Model. One-way Random effects Topic 5 - One-Way Random Effects Models One-way Random effects Outline Model Variance component estimation - Fall 013 Confidence intervals Topic 5 Random Effects vs Fixed Effects Consider factor with numerous

More information

Randomized Block Designs with Replicates

Randomized Block Designs with Replicates LMM 021 Randomized Block ANOVA with Replicates 1 ORIGIN := 0 Randomized Block Designs with Replicates prepared by Wm Stein Randomized Block Designs with Replicates extends the use of one or more random

More information

R Output for Linear Models using functions lm(), gls() & glm()

R Output for Linear Models using functions lm(), gls() & glm() LM 04 lm(), gls() &glm() 1 R Output for Linear Models using functions lm(), gls() & glm() Different kinds of output related to linear models can be obtained in R using function lm() {stats} in the base

More information

Econometrics of Panel Data

Econometrics of Panel Data Econometrics of Panel Data Jakub Mućk Meeting # 4 Jakub Mućk Econometrics of Panel Data Meeting # 4 1 / 30 Outline 1 Two-way Error Component Model Fixed effects model Random effects model 2 Non-spherical

More information

Peter Hoff Linear and multilinear models April 3, GLS for multivariate regression 5. 3 Covariance estimation for the GLM 8

Peter Hoff Linear and multilinear models April 3, GLS for multivariate regression 5. 3 Covariance estimation for the GLM 8 Contents 1 Linear model 1 2 GLS for multivariate regression 5 3 Covariance estimation for the GLM 8 4 Testing the GLH 11 A reference for some of this material can be found somewhere. 1 Linear model Recall

More information

Lecture 2. The Simple Linear Regression Model: Matrix Approach

Lecture 2. The Simple Linear Regression Model: Matrix Approach Lecture 2 The Simple Linear Regression Model: Matrix Approach Matrix algebra Matrix representation of simple linear regression model 1 Vectors and Matrices Where it is necessary to consider a distribution

More information

On the Behavior of Marginal and Conditional Akaike Information Criteria in Linear Mixed Models

On the Behavior of Marginal and Conditional Akaike Information Criteria in Linear Mixed Models On the Behavior of Marginal and Conditional Akaike Information Criteria in Linear Mixed Models Thomas Kneib Institute of Statistics and Econometrics Georg-August-University Göttingen Department of Statistics

More information

Outline. Mixed models in R using the lme4 package Part 3: Longitudinal data. Sleep deprivation data. Simple longitudinal data

Outline. Mixed models in R using the lme4 package Part 3: Longitudinal data. Sleep deprivation data. Simple longitudinal data Outline Mixed models in R using the lme4 package Part 3: Longitudinal data Douglas Bates Longitudinal data: sleepstudy A model with random effects for intercept and slope University of Wisconsin - Madison

More information

Topic 17 - Single Factor Analysis of Variance. Outline. One-way ANOVA. The Data / Notation. One way ANOVA Cell means model Factor effects model

Topic 17 - Single Factor Analysis of Variance. Outline. One-way ANOVA. The Data / Notation. One way ANOVA Cell means model Factor effects model Topic 17 - Single Factor Analysis of Variance - Fall 2013 One way ANOVA Cell means model Factor effects model Outline Topic 17 2 One-way ANOVA Response variable Y is continuous Explanatory variable is

More information

De-mystifying random effects models

De-mystifying random effects models De-mystifying random effects models Peter J Diggle Lecture 4, Leahurst, October 2012 Linear regression input variable x factor, covariate, explanatory variable,... output variable y response, end-point,

More information

Stat 579: Generalized Linear Models and Extensions

Stat 579: Generalized Linear Models and Extensions Stat 579: Generalized Linear Models and Extensions Mixed models Yan Lu March, 2018, week 8 1 / 32 Restricted Maximum Likelihood (REML) REML: uses a likelihood function calculated from the transformed set

More information

The consequences of misspecifying the random effects distribution when fitting generalized linear mixed models

The consequences of misspecifying the random effects distribution when fitting generalized linear mixed models The consequences of misspecifying the random effects distribution when fitting generalized linear mixed models John M. Neuhaus Charles E. McCulloch Division of Biostatistics University of California, San

More information

AMS-207: Bayesian Statistics

AMS-207: Bayesian Statistics Linear Regression How does a quantity y, vary as a function of another quantity, or vector of quantities x? We are interested in p(y θ, x) under a model in which n observations (x i, y i ) are exchangeable.

More information

Chapter 4: Constrained estimators and tests in the multiple linear regression model (Part III)

Chapter 4: Constrained estimators and tests in the multiple linear regression model (Part III) Chapter 4: Constrained estimators and tests in the multiple linear regression model (Part III) Florian Pelgrin HEC September-December 2010 Florian Pelgrin (HEC) Constrained estimators September-December

More information

STAT5044: Regression and Anova. Inyoung Kim

STAT5044: Regression and Anova. Inyoung Kim STAT5044: Regression and Anova Inyoung Kim 2 / 47 Outline 1 Regression 2 Simple Linear regression 3 Basic concepts in regression 4 How to estimate unknown parameters 5 Properties of Least Squares Estimators:

More information

Chapter 3: Maximum Likelihood Theory

Chapter 3: Maximum Likelihood Theory Chapter 3: Maximum Likelihood Theory Florian Pelgrin HEC September-December, 2010 Florian Pelgrin (HEC) Maximum Likelihood Theory September-December, 2010 1 / 40 1 Introduction Example 2 Maximum likelihood

More information

Introduction to Hierarchical Data Theory Real Example. NLME package in R. Jiang Qi. Department of Statistics Renmin University of China.

Introduction to Hierarchical Data Theory Real Example. NLME package in R. Jiang Qi. Department of Statistics Renmin University of China. Department of Statistics Renmin University of China June 7, 2010 The problem Grouped data, or Hierarchical data: correlations between subunits within subjects. The problem Grouped data, or Hierarchical

More information

For more information about how to cite these materials visit

For more information about how to cite these materials visit Author(s): Kerby Shedden, Ph.D., 2010 License: Unless otherwise noted, this material is made available under the terms of the Creative Commons Attribution Share Alike 3.0 License: http://creativecommons.org/licenses/by-sa/3.0/

More information

13.2 Example: W, LM and LR Tests

13.2 Example: W, LM and LR Tests 13.2 Example: W, LM and LR Tests Date file = cons99.txt (same data as before) Each column denotes year, nominal household expenditures ( 10 billion yen), household disposable income ( 10 billion yen) and

More information

Simple and Multiple Linear Regression

Simple and Multiple Linear Regression Sta. 113 Chapter 12 and 13 of Devore March 12, 2010 Table of contents 1 Simple Linear Regression 2 Model Simple Linear Regression A simple linear regression model is given by Y = β 0 + β 1 x + ɛ where

More information

22s:152 Applied Linear Regression. Returning to a continuous response variable Y...

22s:152 Applied Linear Regression. Returning to a continuous response variable Y... 22s:152 Applied Linear Regression Generalized Least Squares Returning to a continuous response variable Y... Ordinary Least Squares Estimation The classical models we have fit so far with a continuous

More information

Spatial inference. Spatial inference. Accounting for spatial correlation. Multivariate normal distributions

Spatial inference. Spatial inference. Accounting for spatial correlation. Multivariate normal distributions Spatial inference I will start with a simple model, using species diversity data Strong spatial dependence, Î = 0.79 what is the mean diversity? How precise is our estimate? Sampling discussion: The 64

More information

Advanced Quantitative Methods: maximum likelihood

Advanced Quantitative Methods: maximum likelihood Advanced Quantitative Methods: Maximum Likelihood University College Dublin March 23, 2011 1 Introduction 2 3 4 5 Outline Introduction 1 Introduction 2 3 4 5 Preliminaries Introduction Ordinary least squares

More information

22s:152 Applied Linear Regression. In matrix notation, we can write this model: Generalized Least Squares. Y = Xβ + ɛ with ɛ N n (0, Σ)

22s:152 Applied Linear Regression. In matrix notation, we can write this model: Generalized Least Squares. Y = Xβ + ɛ with ɛ N n (0, Σ) 22s:152 Applied Linear Regression Generalized Least Squares Returning to a continuous response variable Y Ordinary Least Squares Estimation The classical models we have fit so far with a continuous response

More information

Models for Clustered Data

Models for Clustered Data Models for Clustered Data Edps/Psych/Soc 589 Carolyn J Anderson Department of Educational Psychology c Board of Trustees, University of Illinois Spring 2019 Outline Notation NELS88 data Fixed Effects ANOVA

More information

MASM22/FMSN30: Linear and Logistic Regression, 7.5 hp FMSN40:... with Data Gathering, 9 hp

MASM22/FMSN30: Linear and Logistic Regression, 7.5 hp FMSN40:... with Data Gathering, 9 hp Selection criteria Example Methods MASM22/FMSN30: Linear and Logistic Regression, 7.5 hp FMSN40:... with Data Gathering, 9 hp Lecture 5, spring 2018 Model selection tools Mathematical Statistics / Centre

More information

Models for Clustered Data

Models for Clustered Data Models for Clustered Data Edps/Psych/Stat 587 Carolyn J Anderson Department of Educational Psychology c Board of Trustees, University of Illinois Fall 2017 Outline Notation NELS88 data Fixed Effects ANOVA

More information

36-463/663: Hierarchical Linear Models

36-463/663: Hierarchical Linear Models 36-463/663: Hierarchical Linear Models Lmer model selection and residuals Brian Junker 132E Baker Hall brian@stat.cmu.edu 1 Outline The London Schools Data (again!) A nice random-intercepts, random-slopes

More information

On the Behavior of Marginal and Conditional Akaike Information Criteria in Linear Mixed Models

On the Behavior of Marginal and Conditional Akaike Information Criteria in Linear Mixed Models On the Behavior of Marginal and Conditional Akaike Information Criteria in Linear Mixed Models Thomas Kneib Department of Mathematics Carl von Ossietzky University Oldenburg Sonja Greven Department of

More information

Introductory Econometrics

Introductory Econometrics Based on the textbook by Wooldridge: : A Modern Approach Robert M. Kunst robert.kunst@univie.ac.at University of Vienna and Institute for Advanced Studies Vienna November 23, 2013 Outline Introduction

More information

Random and Mixed Effects Models - Part II

Random and Mixed Effects Models - Part II Random and Mixed Effects Models - Part II Statistics 149 Spring 2006 Copyright 2006 by Mark E. Irwin Two-Factor Random Effects Model Example: Miles per Gallon (Neter, Kutner, Nachtsheim, & Wasserman, problem

More information

Linear Models and Estimation by Least Squares

Linear Models and Estimation by Least Squares Linear Models and Estimation by Least Squares Jin-Lung Lin 1 Introduction Causal relation investigation lies in the heart of economics. Effect (Dependent variable) cause (Independent variable) Example:

More information

ECON 4160, Autumn term Lecture 1

ECON 4160, Autumn term Lecture 1 ECON 4160, Autumn term 2017. Lecture 1 a) Maximum Likelihood based inference. b) The bivariate normal model Ragnar Nymoen University of Oslo 24 August 2017 1 / 54 Principles of inference I Ordinary least

More information

A Very Brief Summary of Statistical Inference, and Examples

A Very Brief Summary of Statistical Inference, and Examples A Very Brief Summary of Statistical Inference, and Examples Trinity Term 2009 Prof. Gesine Reinert Our standard situation is that we have data x = x 1, x 2,..., x n, which we view as realisations of random

More information

Confidence Intervals, Testing and ANOVA Summary

Confidence Intervals, Testing and ANOVA Summary Confidence Intervals, Testing and ANOVA Summary 1 One Sample Tests 1.1 One Sample z test: Mean (σ known) Let X 1,, X n a r.s. from N(µ, σ) or n > 30. Let The test statistic is H 0 : µ = µ 0. z = x µ 0

More information

One-stage dose-response meta-analysis

One-stage dose-response meta-analysis One-stage dose-response meta-analysis Nicola Orsini, Alessio Crippa Biostatistics Team Department of Public Health Sciences Karolinska Institutet http://ki.se/en/phs/biostatistics-team 2017 Nordic and

More information

Now consider the case where E(Y) = µ = Xβ and V (Y) = σ 2 G, where G is diagonal, but unknown.

Now consider the case where E(Y) = µ = Xβ and V (Y) = σ 2 G, where G is diagonal, but unknown. Weighting We have seen that if E(Y) = Xβ and V (Y) = σ 2 G, where G is known, the model can be rewritten as a linear model. This is known as generalized least squares or, if G is diagonal, with trace(g)

More information

y it = α i + β 0 ix it + ε it (0.1) The panel data estimators for the linear model are all standard, either the application of OLS or GLS.

y it = α i + β 0 ix it + ε it (0.1) The panel data estimators for the linear model are all standard, either the application of OLS or GLS. 0.1. Panel Data. Suppose we have a panel of data for groups (e.g. people, countries or regions) i =1, 2,..., N over time periods t =1, 2,..., T on a dependent variable y it and a kx1 vector of independent

More information

Statement: With my signature I confirm that the solutions are the product of my own work. Name: Signature:.

Statement: With my signature I confirm that the solutions are the product of my own work. Name: Signature:. MATHEMATICAL STATISTICS Homework assignment Instructions Please turn in the homework with this cover page. You do not need to edit the solutions. Just make sure the handwriting is legible. You may discuss

More information

Lecture 4. Random Effects in Completely Randomized Design

Lecture 4. Random Effects in Completely Randomized Design Lecture 4. Random Effects in Completely Randomized Design Montgomery: 3.9, 13.1 and 13.7 1 Lecture 4 Page 1 Random Effects vs Fixed Effects Consider factor with numerous possible levels Want to draw inference

More information

Generalized Linear Models

Generalized Linear Models Generalized Linear Models Lecture 3. Hypothesis testing. Goodness of Fit. Model diagnostics GLM (Spring, 2018) Lecture 3 1 / 34 Models Let M(X r ) be a model with design matrix X r (with r columns) r n

More information

8. Hypothesis Testing

8. Hypothesis Testing FE661 - Statistical Methods for Financial Engineering 8. Hypothesis Testing Jitkomut Songsiri introduction Wald test likelihood-based tests significance test for linear regression 8-1 Introduction elements

More information

Mixed-Models. version 30 October 2011

Mixed-Models. version 30 October 2011 Mixed-Models version 30 October 2011 Mixed models Mixed models estimate a vector! of fixed effects and one (or more) vectors u of random effects Both fixed and random effects models always include a vector

More information

Regression: Main Ideas Setting: Quantitative outcome with a quantitative explanatory variable. Example, cont.

Regression: Main Ideas Setting: Quantitative outcome with a quantitative explanatory variable. Example, cont. TCELL 9/4/205 36-309/749 Experimental Design for Behavioral and Social Sciences Simple Regression Example Male black wheatear birds carry stones to the nest as a form of sexual display. Soler et al. wanted

More information

Ma 3/103: Lecture 25 Linear Regression II: Hypothesis Testing and ANOVA

Ma 3/103: Lecture 25 Linear Regression II: Hypothesis Testing and ANOVA Ma 3/103: Lecture 25 Linear Regression II: Hypothesis Testing and ANOVA March 6, 2017 KC Border Linear Regression II March 6, 2017 1 / 44 1 OLS estimator 2 Restricted regression 3 Errors in variables 4

More information

Lecture 2: Linear Models. Bruce Walsh lecture notes Seattle SISG -Mixed Model Course version 23 June 2011

Lecture 2: Linear Models. Bruce Walsh lecture notes Seattle SISG -Mixed Model Course version 23 June 2011 Lecture 2: Linear Models Bruce Walsh lecture notes Seattle SISG -Mixed Model Course version 23 June 2011 1 Quick Review of the Major Points The general linear model can be written as y = X! + e y = vector

More information