Linear Mixed Models: Methodology and Algorithms

Transcription

1 Linear Mixed Models: Methodology and Algorithms David M. Allen University of Kentucky March 6, 2017

2 C Topics from Calculus Maximum likelihood and REML estimation involve the minimization of a negative log likelihood function with respect to the parameters. Minimization is done using Newton s method which requires the derivatives of the negative log likelihood function. The likelihood function is a function of functions so the chain rule facilitates determining the derivatives. This Chapter gives the required calculus.

3 Section C.1 The Chain Rule The chain rule is a fundamental rule of differentiation. Some physicists claim that the chain rule is the most important theorem in all of mathematics (Hubbard and Hubbard [1]). C.1 29

4 Statement of the Chain Rule Let g be a m-vector of functions having n arguments and ƒ be a p-vector of functions having m arguments. If g is differentiable at and ƒ is differentiable at g(), then the composition ƒ (g()) is differentiable at, and its derivative is given by d d ƒ (g()) = = d d ƒ () =g() d d g() = C.1 30

5 Section C.2 Some Matrix Derivatives The likelihood function of the multivariate normal distribution involves both the determinant and inverse of the variance matrix. For application of Newton s algorithm, the first and second derivatives of the determinant and inverse of the variance matrix are required. Finding these derivatives is the subject of this section. C.2 31

6 Notation Any letter could be used to represent the matrix under discussion. I will use V since its use is in the context of a variance matrix. Assume the elements of V are functions of a vector parameters θ. This is emphasized by writing it as V(θ). C.2 32

7 Derivative of an Inverse Matrix The derivative of an inverse is the simpler of the two cases considered. The defining relationship between a matrix and its inverse is V(θ)V 1 (θ) = The derivative of both sides with respect to the kth element of θ is d d V(θ) V 1 (θ) + V(θ) V 1 (θ) = 0 θ k θ k Straightforward manipulation gives d d V 1 (θ) = V 1 (θ) V(θ) V 1 (θ) θ k θ k (C.2.1) C.2 33

8 Analogies There are two analogies to one variable calculus in the derivative above: derivative of a product and implicit differentiation. C.2 34

9 The Derivative of a Determinant For discussion of the derivative of a determinant, I temporarily suspend the dependence of V on θ and derive the derivative with respect it an element of V. The derivative with respect to an element of θ is brought in via the chain rule. C.2 35

10 The Cofactor of a Matrix For a square matrix V, the minor of its (, j) entry is defined to be the determinant of the submatrix obtained by removing from V its th row and jth column, and it is denoted by M j. Then C j = ( 1) +j M j is called the (, j) cofactor of V. C.2 36

11 The Determinant of a Matrix The determinant of V(n n) may be expressed as for any fixed j, or det(v) = det(v) = n j C j =1 n j C j for any fixed. These are called column and row expansions respectively. j=1 C.2 37

12 For a matrix Cofactor Matrix n n V = n1 n2 nn the cofactor matrix is C 11 C 12 C 1n C 21 C 22 C 2n C = C n1 C n2 C nn C.2 38

13 The Adjugate and Inverse Matrices The adjugate matrix is the transpose of the cofactor matrix dj(v) = C t. Provided det(v) = 0 the inverse of V is V 1 = 1 det(v) dj(v) C.2 39

14 The Derivative With Respect to an Element The derivative of the logarithm of the determinant of V with respect to an element is d d j log(det(v)) = 1 det(v) C j = V 1 j C.2 40

15 Derivative with Respect to θ Bring back the dependency of V on θ and apply the chain rule: d 1 log(det(v(θ)) = dθ k det(v) = n n =1 j=1 = tr V 1 n n d j (θ) C j dθ =1 j=1 k V 1 d V(θ) j dθ k d V t (θ) dθ k j (C.2.2) C.2 41

16 Exercises The following exercises depend on these quantities: Y =, A =, θ = [θ 1, θ 2 ] t, V(θ) = AA t θ 1 + θ 2. Y is a realization of N 4 (0, V(θ)). Exercise C.2.1. Let L(θ; Y) represent negative two times the log likelihood function of θ. Give the expression for L(θ; Y). You may ignore the constant term. Exercise C.2.2. Find the derivative of L(θ; Y) with respect to θ 1 evaluated at [θ 1, θ 2 ] = [3, 2]. C.2 42

17 Exercise C.2.3. Find the derivative of L(θ; Y) with respect to θ 2 evaluated at [θ 1, θ 2 ] = [3, 2]. C.2 43

18 References [1] John H. Hubbard and Barbara Burke Hubbard. Vector Calculus, Linear Algebra, and Differential Forms. Fifth edition. Ithaca, New York: Matrix Editions, C.2 44