Linear Mixed Models: Methodology and Algorithms

Size: px

Start display at page:

Download "Linear Mixed Models: Methodology and Algorithms"

Isabella Wilson
5 years ago
Views:

1 Linear Mixed Models: Methodology and Algorithms David M. Allen University of Kentucky January 8, 2018

2 1 The Linear Mixed Model This Chapter introduces some terminology and definitions relating to the main topic of this monograph, linear mixed models. Examples and exercises are included.

3 Section 1.1 Definitions The models considered here assume there is an n-component normally distributed random vector Y that will be realized as collection of observations in an experimental situation. It is further assumed that the expected value of Y has the form Xβ where X is a known matrix and β is a vector of unknown parameters

4 The Traditional Model The traditional formula of the linear model is Y = Xβ + ε (1.1.1) where ε N(0, σ 2 ). This model is still important. Enough so that I give its variance structure a name: Definition 1.1. A vector with the variance of each element equal σ 2 and the covariance between each pair of elements equal 0 is said to have a basic variance structure

5 A Mixed Model In experimental designs, there are often blocking factors that are reasonably considered random. For the randomized complete block model, the block effects are usually considered random. With Z the indicator matrix for blocks and a vector of block effects, model becomes Y = Xβ + Z + ε. (1.1.2) Typically, it is assumed that N 0, σ 2 in which case Model (1.1.2) is called a variance component model. If the experimental design has multiple random factors, then the model would have additional Z terms

6 Origin of Terminology Formula (1.1.2) contains fixed effects Xβ and random effects Z in addition ε. This is the origin of the term mixed when referring to this type of model

7 Definition of the Linear Mixed Model The model expression (1.1.2) is useful for the statement of models in experimental design. However, this formulation is inadequate to describe more complicated variance structures. The definition of the linear mixed model used in this monograph is Definition 1.2. The linear mixed model is Y N(Xβ, V(θ)). (1.1.3) Here Y is an n-component vector of responses, X is a known matrix, and β is a vector of parameters. The variance matrix, V(θ), is a function of a vector of parameters, θ

8 An Advantage of the Linear Mixed Model The linear mixed model encompasses many models in statistics that, historically, were defined separately in most applied statistics books. The advantage of the linear model is that a methodology is developed and it applies to all the special cases

9 Generalized Linear Mixed Models Nowadays, there is an emphasis on expanding the framework to include non-normal distributions such as the binomial, Poisson, and exponential distributions. This expanded class of models is called generalized linear mixed models. The Stroup [1] takes this approach. My view is that there is so much that is specific to linear mixed models that it deserves a careful study before considering generalization to other distributions

10 Section 1.2 Examples This section contains examples of models cast in the form of Model (1.1.3). The basic variance structure is assumed for examples (1.1) (1.3)

11 One Population Example 1.1. Perhaps the simplest of all statistical models is that used to describe a simple random sample from a normally distributed population with mean μ and variance σ 2. With 5 observations, the model in terms of Equation (1.1.3) is Y = Y 1 Y 2 Y 3 Y 4 Y 5, X = , and β = μ

12 Two Populations Example 1.2. Suppose there are independent random samples from two normally distributed populations with respective means μ 1 and μ 2 and common variance σ 2. With 3 observations in each sample, the model in terms of Equation (1.1.3) is Y = Y 11 Y 12 Y 13 Y 21 Y 22 Y 23, X = , and β = μ1 μ 2. (1.2.1) This example extends to models for multi-sample situations in a straightforward way

13 Multiple Regression Example 1.3. The multiple regression model assumes the expected response is a constant plus a linear combination of some explanatory variables. In a study on infants, the response is systolic blood pressure and the explanatory variables are birth weight and age. With n = 6 the model in terms of Equation (1.1.3) is Y = Y 1 Y 2 Y 3 Y 4 Y 5 Y 6, X = , and β = β 0 β 1 β 2 The second column of X contains measurements of birth weight and the third column contains their ages

14 Section 1.3 Exercises The following exercises present models in scalar notation. In each case, give a vector Y, matrix X, and a vector β to conform with Definition (1.2)

15 The Basketball Problem Exercise 1.1. Label the University of Kentucky, University of Tennessee, and Vanderbilt University basketball teams as teams 1, 2, and 3 respectively. Let Y jk be the score of team minus the score of team j in the kth game between the two teams. Let γ be a parameter representing the power rating of the th team. Assume the model Y jk = γ γ j + ε jk where the e jk are distributed N(0, σ 2 ). Note that Y jk = Y j k so the choice to include Y jk or Y j k in the model is arbitrary. Games where (, j, k) = (1, 2, 1), (1, 2, 2), (1, 3, 1), (1, 3, 2), (2, 3, 1), (2, 3, 2) have been played

16 Four Populations Exercise 1.2. There are four conceptual populations having means μ 1, μ 2, μ 3, and μ 4. For an experiment to compare these means, assume the model Y j = μ + ε j where Y j is the response for the jth observation from the th population, = 1,, 4, j = 1, 2, 3, and the e j are distributed N(0, σ 2 )

17 References [1] Walter W. Stroup. Generalized Linear Mixed Models: Modern Concepts, Method Concepts, Methods and Applications. Boca Raton, FL: CRC Press, Taylor & Francis Group,

Generalized Linear. Mixed Models. Methods and Applications. Modern Concepts, Walter W. Stroup. Texts in Statistical Science.

Generalized Linear. Mixed Models. Methods and Applications. Modern Concepts, Walter W. Stroup. Texts in Statistical Science. Texts in Statistical Science Generalized Linear Mixed Models Modern Concepts, Methods and Applications Walter W. Stroup CRC Press Taylor & Francis Croup Boca Raton London New York CRC Press is an imprint