Stat 579: Generalized Linear Models and Extensions

Stat 579: Generalized Linear Models and Extensions Mixed models Yan Lu Feb, 2018, week 7 1 / 17

Some commonly used experimental designs related to mixed models Two way or three way random/mixed effects Randomized complete block designs Nested design Split plot designs Repeated measures, longitudinal data 2 / 17

Two way random effects model, A and B both random y ij = µ + α i + β j + ɛ ij, i = 1,, a, j = 1,, b one observation per combination, n T = ab iid α i N(0, σ 2 a ) iid β j N(0, σ 2 b ) iid ɛ ij N(0, σ 2 ) 3 / 17

Two way mixed effects model, A fixed, B random A and B crossed, h observations each cell y ijk = µ + α i + β j + (αβ) ij + ɛ ijk i = 1,, a, j = 1,, b, k = 1,, n, n T = abn α i are constants subject to the restriction a i=1 α i = 0 β j iid N(0, σ 2 b ) (αβ) ij iid N(0, σ 2 ab ) ɛ ijk iid N(0, σ 2 ) 4 / 17

4/42 Example 1: small factorial (From Paul Darius s slides, slightly adapted.) A field experiment is to be conducted using all combinations of three varieties (of some cereal) and two methods of fertilization. Each combination will replicated four times, so that 24 plots of land (observational units) are needed. Factor Number Levels of levels Variety (A) 3 A1, A2, A3

10/42 From Hasse diagram to degrees of freedom to anova U A B A B E

10/42 From Hasse diagram to degrees of freedom to anova 1 U 3 A B 2 6 A B 24 E Show the number of levels of each factor.

10/42 From Hasse diagram to degrees of freedom to anova 1, 1 U 3 A B 2 6 A B 24 E Show the number of levels of each factor. Calculate degrees of freedom recursively by subtraction.

10/42 From Hasse diagram to degrees of freedom to anova 1, 1 U 3, 2 A B 2 6 A B 24 E Show the number of levels of each factor. Calculate degrees of freedom recursively by subtraction.

10/42 From Hasse diagram to degrees of freedom to anova 1, 1 U 3, 2 A B 2, 1 6 A B 24 E Show the number of levels of each factor. Calculate degrees of freedom recursively by subtraction.

10/42 From Hasse diagram to degrees of freedom to anova 1, 1 U 3, 2 A B 2, 1 6, 2 A B 24 E Show the number of levels of each factor. Calculate degrees of freedom recursively by subtraction.

10/42 From Hasse diagram to degrees of freedom to anova 1, 1 U 3, 2 A B 2, 1 6, 2 A B 24, 18 E Show the number of levels of each factor. Calculate degrees of freedom recursively by subtraction.

10/42 From Hasse diagram to degrees of freedom to anova 1, 1 U 3, 2 A B 2, 1 6, 2 A B Skeleton analysis of variance Stratum Source df U Mean 1 E A 2 B 1 A B 2 residual 18 24, 18 E Show the number of levels of each factor. Calculate degrees of freedom recursively by subtraction. Each gives a stratum (eigenspace of the covariance matrix).

From Hasse diagram to degrees of freedom to anova 1, 1 U 3, 2 A B 2, 1 6, 2 A B Skeleton analysis of variance Stratum Source df U Mean 1 E A 2 B 1 A B 2 residual 18 24, 18 E Show the number of levels of each factor. Calculate degrees of freedom recursively by subtraction. Each gives a stratum (eigenspace of the covariance matrix). Each gives a source for a fixed effect, contained in the stratum given by the highest below or equal to it. shows a stratum with no residual. 10/42

Example: Miles per gallon An automobile manufacturer wished to study the effects of factors A and B on gasoline consumption. Factor A: Drivers, two drivers are selected at random Factor B: Cars, two cars of the same model with manual transmission were randomly selected from the assembly line. Each driver drove each car twice over a 40-mile test course and the miles per gallon were recorded. y ijk = µ + α i + β j + (αβ) ij + ɛ ijk i = 1, 2, j = 1, 2, k = 1, 2, n T = 8 β = µ, X = 1 8 α = (α 1, α 2, β 1, β 2, (αβ) 11, (αβ) 12, (αβ) 21, (αβ) 22 ) 5 / 17

( α1 a = α 2 Z = ) ( β1, b = β 2 ), c = 1 0 1 0 1 0 0 0 1 0 1 0 1 0 0 0 1 0 0 1 0 1 0 0 1 0 0 1 0 1 0 0 0 1 1 0 0 0 1 0 0 1 1 0 0 0 1 0 0 1 0 1 0 0 0 1 0 1 0 1 0 0 0 1 (αβ) 11 (αβ) 12 (αβ) 21 (αβ) 22, Z = Z 1 a + Z 2 b + Z 3 c ( I2 1 = I 2 1 4 a + 2 I 2 1 2 ) b + I 4 1 2 c 6 / 17

Kronecker product: A m n, B p q a 11 B a 1n B A B =. a m1 B a mn B mp nq 7 / 17

R = I 8 σ 2 Var(y) = Var(Xβ + Zα + ɛ) Assume α i iid N(0, σ 2 a ), β j iid N(0, σ 2 b ), (αβ) ij iid N(0, σ 2 ab ), ɛ ij iid N(0, σ 2 ) G 8 8 = Var(α) = σ 2 a 0 0 0 0 0 0 0 0 σ 2 a 0 0 0 0 0 0 0 0 σ 2 b 0 0 0 0 0 0 0 0 σ 2 b 0 0 0 0 0 0 0 0 σ 2 ab 0 0 0 0 0 0 0 0 σ 2 ab 0 0 0 0 0 0 0 0 σ 2 ab 0 0 0 0 0 0 0 0 σ 2 ab = ZVar(α)Z + Var(ɛ) = ZGZ + R 8 / 17

Randomized complete block designs Example: boy s shoes. Goal: want to know effect of materials A and B Paired experiments boy 1 has A on both feet, boy 2 has B on both feet, huge variability from boy to boy Unpaired experiments removes boy-to-boy variability block: boys treatment: A and B Left foot Right foot boy 1 A B boy 2 B A. boy n A B 9 / 17

Randomized complete block designs Each treatment is included once in each block Randomization is done within each block, this is a restricted randomization design -complete: each treatment occurs within each block -block: usually random, not of interest - treatment: fixed, of interest Purpose of blocking: sort experimental units into groups within each of which the elements are homogeneous with respect to the response variable, differences between groups are as great as possible. 10 / 17

Model: y ij = µ + ρ i + τ j + ɛ ij, i = 1,, m, j = 1,, r i: block, ρ i block effect j: treatment, τ j treatment effect ρ i iid N(0, σ 2 ρ ), ɛ ij iid N(0, σ 2 ) τ j constants, r j=1 τ j = 0 α = ρ 1 ρ 2. ρ m, β = τ 1 τ 2. τ r 11 / 17

Example from Paul Darius s A field experiment is to be conducted using a block design the field is divided into 4 blocks within each block, the experiment was conducted randomly using all combinations of factor A: three varieties (of some cereal) factor B: two methods of fertilization. A total of 24 observations available for experiment. 12 / 17

19/42 Variant of Example 1: blocks The field is divided into 4 blocks of 6 observational units each, to take account of known or suspected differences in the soil.

19/42 Variant of Example 1: blocks The field is divided into 4 blocks of 6 observational units each, to take account of known or suspected differences in the soil. To be able extrapolate our results to other plots in other fields, we need to assume that there is no interaction between the factor Block and the factor Treatment (where Treatment = A B).

20/42 From Hasse diagram to degrees of freedom to anova Block 4, 3 1, 1 U 3, 2 A B 2, 1 6, 2 A B 24, 15 E

20/42 From Hasse diagram to degrees of freedom to anova Skeleton analysis of variance Block 4, 3 1, 1 U 3, 2 A B 2, 1 6, 2 A B 24, 15 E Stratum Source df U Mean 1 Block 3 E A 2 B 1 A B 2 residual 15

Split plot designs Split plot designs are originally developed for agriculture, and are frequently used in field, laboratory, industrial and social science experiments. Consider an investigation to study the effects of two irrigation methods (factor A) and two fertilizers (factor B) on yield of a crop, using four available fields as experimental units. 1. Completely randomized design A 1 B 1, A 1 B 2, A 2 B 1, A 2 B 2 would then be assigned at random to the four fields, no degrees of freedom 2. Consider split plot design: each of the two irrigation methods is randomly assigned to two of the four fields, called whole plots each whole plots are divided into two or more smaller areas called split plots two fertilizers are randomly assigned to the split plots within each whole plot 13 / 17

Model y ijk = µ + ρ i(j) + α j + β k + (αβ) jk + ɛ ijk i : whole plot, i = 1, 2, 3, 4 j: irrigation j = 1, 2 k = 1, 2, fertilizer α j : main effect of the jth irrigation method (jth whole plot treatment) β k : kth split plot treatment, main effect of the kth fertilizer ρ i(j) : effect of the ith whole plot, nested within the jth level of factor A (irrigation method) Two-stage of randomization 1. whole-plot treatments are randomly assigned to whole plots 2. split-plot treatments are randomly assigned to split plots 3. can be viewed as a type of incomplete block design, whole plots blocks, each whole plot being given only some of the full set of treatment 14 / 17

Example from Paul Darius s Consider an investigation to study the effects of two irrigation methods (factor I) and two fertilizers (factor F) on yield of a crop, using 8 available fields as experimental units. each of the two irrigation methods is randomly assigned to 4 of the 8 fields, called whole plots each whole plots are divided into two or more smaller areas called split plots two fertilizers are randomly assigned to the split plots within each whole plot 15 / 17

26/42 Example 3: Split plots (From Paul Darius s slides, slightly adapted.) A field trial is planned to study the effect of 2 irrigation methods (factor I) and 2 fertilizers (factor F). To irrigate a plot with a sprinkler without water spilling into the next plot, we need large plots, and there is room for only 8. Fertilizer can be applied to much smaller areas, so we can divide each plot into two subplots, one for each level of F.

27/42 Example 2: Hasse diagram and skeleton anova 1, 1 U 2, 1 I F 2, 1 Plot 8, 6 4, 1 I F 16, 6 E

27/42 Example 2: Hasse diagram and skeleton anova 1, 1 U 2, 1 I F 2, 1 Plot 8, 6 4, 1 I F 16, 6 E Skeleton analysis of variance Stratum Source df U Mean 1 Plots I 1 residual 6 E F 1 I F 1 residual 6

27/42 Example 2: Hasse diagram and skeleton anova 1, 1 U 2, 1 I F 2, 1 Plot 8, 6 4, 1 I F 16, 6 E Skeleton analysis of variance Stratum Source df U Mean 1 Plots I 1 residual 6 E F 1 I F 1 residual 6 Treatment = I F Plot Treatment = I

Example: repeated measures A: two types of incentives, on a persons ability to solve problem B: two types of problems, abstract and concrete problems each experimental subject could be asked to do each type of problem but could not be exposed to more than one type of incentive stimulus, because of potential interference effects 16 / 17

Incentive stimulus Subject Treatment order 1 2 A 1 1 A 1 B 1 A 1 B 2 2 A 1 B 2 A 1 B 1. n A 1 B 1 A 1 B 2 A 2 n+1 A 2 B 1 A 2 B 2 n+2 A 2 B 1 A 2 B 2. 2n A 2 B 2 A 2 B 1 Whole plot: subject, whole plot treatment: A 1 and A 2 Split plot: treatment orders 1 and 2, split plot treatment: B 1 and B 2 17 / 17