An R Statistic for Fixed Effects in the Linear Mixed Model and Extension to the GLMM Lloyd J. Edwards, Ph.D. UNC-CH Department of Biostatistics email: Lloyd_Edwards@unc.edu Presented to the Department of Biostatistics, The University of Alabama at Birmingham August 17, 2016 1
1. Introduction 2. The Linear Mixed Model " 3. R : A n R Statistic for Fixed Effects in the Linear Mixed Model - Choice of Null Model - Model R " - Semi-partial R " 4. R : A n R " Statistic for Fixed Effects in the Generalized Linear Mixed Model 5. Software: SAS and R 6. Example Computations and Interpretations - Retrospective Longitudinal Study of Adults with Hypertension 7. Conclusions 2
1. Introduction ñ The linear mixed model (LMM) is the most widely used statistical tool for the analysis of longitudinal models with Gaussian errors. ñ The LMM explicitly specifies not only the mean structure, but also the covariance structure. ñ Hence three types of model comparisons can occur. 1. Compare mean models with the same covariance structure. Nested mean models are the most common. 2. Compare covariance models with the same mean structure. Two LMMs may be nested or nonnested in the covariance models. 3. Compare LMMs with different mean and different covariance structures. 3
1. Introduction ñ Here we describe an V only for model comparison 1, i.e., comparing nested mean models with the same covariance structure. ñ We use an approximate J statistic for a Wald test of fixed effects, the Kenward-Roger small sample J statistic, to define an V statistic for fixed effects in the linear mixed model. ñ The V statistic measures multivariate association between the repeated outcomes and the fixed effects in the linear mixed model and can also be interpreted as the proportion of variance explained by the fixed effects. ñ Defining V in terms of an J statistic for fixed effects allows computing it with results from fitting only a single model, i.e., there is no need to explicitly fit a null model. ñ The approach necessarily assumes the covariance structure holds for both the model of interest and the implied null model. 4
2. The Linear Mixed Model for Longitudinal Data ñ With R independ ent sampling units (often persons in practice), the linear mixed model for person 3 may be written ] œ \ " ^, /. (1) 3 3 3 3 3 Here, ] 3 is a : " 3 vector of observations on person 3; \ 3 is a : ; 3 known, constant design matrix for person 3, with full column rank ; " is a ; " vector of unknown, constant, population parameters. ^3 is a : 7 3 known, constant design matrix with rank 7for person 3, 3 7 " vector of unknown random effects is a : " vector of unknown random errors. / 3 3 Gaussian, 3 and / 3 are independent with mean! and, 3,3,! i œ / D 7! D 7. (2) 3 /3 / 5
2. The Linear Mixed Model for Longitudinal Data ñ Here ið Ñ is the covariance operator, while both D,3 œ D,3 7, and D/3 œ D/3 7/ are positive-definite, symmetric covariance matrices. Therefore ið] Ñmay be written ið] Ñ œ D œ ^ D ^ D. 3 3 3 3 w,3 3 /3 We assume that D 3 can be characterized by a finite set of parameters represented by an < " vector 7 which consists of the unique parameters in 7. and 7/. R Throughout 8œ 8. 3œ" 3 ñ We will also need to refer to a stacked data formulation of model (0) given by ] œ \" ^, /, (3) w w w w w with ] œ ] " â] R, \ œ \ " â\ R, ^ œ diag ^" ßâß^R, w w w w w w,œ, " â, R, and /œ / " â/ R. Here, µ ar;! ßD,3 7, ŒMR and / µ a 8! ßD/ for D/ œ diag D/" 7/ ßâßD/R 7/. As a result we have ] µ \ ", D with D œ ið] Ñ œ diag D ßâßD. a 8 " R w 6
" 3. R : A n R Model R " Statistic for Fixed Effects in the Linear Mixed Model ñ We consider a LMM that includes an intercept in \ 3, the fixed effects design, and may or may not include an intercept in (the random effects design). ñ The model J ("D s, s) statistic in the LMM corresponds to a comparison between two nested fixed effects models: the model of interest (full or maximum model) and a null model with no covariates except the intercept. ^3 ñ With \ 3 œ B âb and " œ " " â", 1 3 "ß3 ; "ß3! " ; " w 1. Model of Interest: ] œ \ 3 " ^, / 3 3 3 3 2. Null Model: ] œ 1 " ^, /. 3 3! 3 3 3 ñ The model J ("D s, s) statistic uses a single model fit to compare two models differing by the presence or absence of all fixed effect predictors except the intercept. 7
" 3. R : A n R Statistic for Fixed Effects in the Linear Mixed Model ñ The model J statistic corresponds to a test of the null hypothesis L À G " œ! for G œ! M! ; " " ; " or equivalently L! À "" œ " œ â œ "; " œ! where ; " is the numerator degrees of freedom (ndf) for full rank G. ñ The model J statistic is given by w " " 1 " s wd s w G G \ \ G G" s J( "D s, s) œ. (6) rank G ñ Approximations for denominator degrees of freedom (ddf) include Kenward- Roger, Satterthwaithe, Containment method, Residual method. 8
" 3. R : A n R Statistic for Fixed Effects in the Linear Mixed Model ñ From univariate linear model theory, we have J ( s, VÎ; " s ) " "D œ " V Î/, (7) where V" denotes our proposed V statistic and / is the (approximated) ddf. ñ Solving for V " yields " ; " J s s V" " / ("D, ) œ " ; " / " J( "D s, s). (8) ñ As stated previously, different choices exist for the approximate denominator degrees of freedom (d.f.), /, in the J ("s sd ) statistic for fixed effects. 9
" 3. R : A n R Statistic for Fixed Effects in the Linear Mixed Model ñ The natural extension of the univariate linear model uses the residual degrees of freedom, with / œ8 rank \, where 8is the total number of observations. ñ Under REML estimation, the Kenward-Roger J provides the most accurate inference in small samples, while the Satterthwaite method also does well. ñ The statistic reflects our belief that the covariance model for ] 3 should remain the same when the comparison centers on nested fixed effects. V " ñ The fact that J ( s, s "D)! insures!ÿv" ", with V" œ! indicating no multivariate association between ] and \. ñ On the other hand, as V nears 1, then the multivariate association between and \ becomes perfect. " ] 10
3. R : " An R Statistic for Fixed Effects in the Linear Mixed Model Semi-partial R " ñ R" is the only R statistic in its class that leads to defining a semi-partial V statistic. ñ Just as for overall regression, the partial J in the LMM defines a partial V " by way of equation, where the ndf, ; 1, is set equal to 1 (7). ñ Testing L! À " 4 œ! for 4 "ßáß; ", i.e., testing a particular fixed effect regression coefficient, gives a partial J statisticþ ñ It measures the marginal contribution of when all the other predictors have \ 5 already been included in the model. 11
4. R" : An R Statistic for Fixed Effects in the Generalized Model (GLMM) Linear Mixed ñ The generalized linear mixed model (GLMM) is one of the most commonly used random effects model for discrete outcomes and continuous outcomes with non-normal distributions. ñ We summarize the basic features of the GLMM here with a focus on longitudinal data. ñ With R independ ent sampling units (often persons in practice) and w c onditionally on the random effects, 3 œ, 3" á, 3; (; "), assume that the responses ] 34 of ] 3 ( 73 "), 4œ"ßáß73, 3œ"ßáßRß are independent with density function that is a member of the exponential family. 12
4. R" : An R Statistic for Fixed Effects in the Generalized Model (GLMM) ñ The conditional mean satisfies a linear regression model I] ( 34l, 3) œ. 34 œ2( ( 34), w w ( œ1i] [ ( l, )] œb " D, 34 34 3 34 34 3 @] ( l, ) œ+ @(. ) 34 3 34 34 Linear Mixed (9) where " (: ") is a vector of unknown fixed effect parameters, B34 (: ") and D34 (; ") are vectors of fixed and random effect explanatory variables (the first element of B 34 is a 1), + 34 is a known constant, and 9 is a dispersion parameter, 1œ2 " is referred to as the link function. 13
4. R" : An R Statistic for Fixed Effects in the Generalized Model (GLMM) Linear Mixed ñ The random effects, 3 are assumed to be sampled from a (multivariate) normal distribution with mean 0 (; ") and covariance matrix D, œ D,( ) (; ;) that depends on a vector ( < " ) of unknown variance components. ñ When estimating the parameters in the GLMM, the exact likelihood function involves an intractable high-dimensional integration that is difficult to compute. There are several approximations to the likelihood function and approximate maximum likelihood estimators (MLE) have been proposed in the literature. ñ Among the available estimation techniques, the penalized quasi-likelihood (PQL) by Breslow and Clayton (1993) is the most popular for the GLMM. ñ PQL approximates the high-dimensional integration using the well-known Laplace approximation and the approximated likelihood function has that of a Gaussian distribution. 14
4. R" : An R Statistic for Fixed Effects in the Generalized Model (GLMM) Linear Mixed ñ PQL estimation creates a transformed outcome for which the normal theory LMM with REML estimation applies. ñ PQL is attractive to practitioners due to its general availability, ease of use, and computational efficiency. ñ PQL is perhaps the most used estimation technique for the GLMM in commercial software. 15
4. R" : An R Statistic for Fixed Effects in the Generalized Model (GLMM) Linear Mixed ñ PQL Summary: Breslow and Clayton (1993) assumes that I( ] 3l, 3) œ. 3 can be derived from a first-order approximation to the hierarchical model that is valid in the limit as the components of dispersion approach zero. If we write the model as ] 3 œ. 3 % 3 with @Ð% 3Ñ œ + 3@ (. 3) and, µ a!,, a first-order Taylor series of 2 about "s and s, yields 3 ; D, 3 w w 2Ð( Ñ 2Ð( s Ñ 2 Ð( s Ñ\ (" " s) 2 Ð( s Ñ^ (, s, ) %, (10) 3 3 3 3 3 3 3 3 3 w where 2 Ðs Ñ œ `2Ðs w ( 3 ( 3ÑÎ`( 3 œ "Î1 Ð( s3ñ is a diagonal matrix of derivatives of the conditional mean evaluated at "s and ș 3. 16
4. R" : An R Statistic for Fixed Effects in the Generalized Model (GLMM) Linear Mixed Define ] 3 as the working vector of pseudo-responses 3 3 w ] œ1ð] Ñœ1Ð. s ÑÐ]. s ( s, (11) where. s œ2ð( s Ñ ( s w w and œ\ "s ^ s,. 3 3 3 3 3 3 When re-ordering terms in (10) we get 3 3 3 3 3 3) 3 ] œ \ " ^, 3 / 3, (12) where the working vector of pseudo-responses ] 3 can be specified as an approximate LMM with multivariate normal, 3 and / 3, independent with mean! and,! @ 3, œ / D -1! [. 3 3 17
4. R" : An R Statistic for Fixed Effects in the Generalized Model (GLMM) Linear Mixed w -1 The variance/covariance of ] 3 may be written D3 œ ^3 D, ^ 3 [ 3 where w œ diag Ö+ @ s 1 s (. )[ (. )]. [ -1 3 34 34 34 Model fitting is accomplished by iterating between updating the pseudoresponses ] 3 and fitting a LMM until convergence. ñ Recall from Section 3 that the choice of the null model plays a central role in defining a model V statistic. 18
4. R" : An R Statistic for Fixed Effects in the Generalized Model (GLMM) Linear Mixed ñ To define the model V statistic for fixed effects in the GLMM, the following two models we compare 1. Model of interest: ( œ \ " ^,, (13) 2. Null Model: or in the transformed LMM via PQL estimation 1. Model of Interest: ( 3 3 3 3 œ " 1 ^, 3! 3 3 3 3 3 3 3 3 3! 3 3 3 ] œ \ 3 " ^, / 2. Null Model: ] œ 1 " ^, /. ñ We define the model R ", an V statistic for fixed effects in the GLMM, as V " for fixed effects in the LMM for the transformed outcome vector ] 3 at convergence of PQL. 19
4. R" : An R Statistic for Fixed Effects in the Generalized Model (GLMM) Linear Mixed ñ R " for the GLMM here is defined for a single model and incorporates the null model given by (13). ñ In addition, since V gives rise to a natural definition of a (semi) partial V for the LMM, we now can compute partial V 's for the GLMM using R ". " 20
5. Software - https://github.com/bcjaeger/r2fixedeffectsglmm/ (1) SAS: %Glimmix_R2_V3.SAS This SAS macro computes model and semi-partial R statistics for each fixed effect in the model. Due to a lack of the non-central beta distribution in SAS, confidence limits are not currently supplied. The user may specify denominator degrees of freedom estimation. Kenward-Roger (DDF = KR) is recommended, and is fully compatible with the GLMM. (2) R: r2glmm Package This package computes model R and semi-partial R for the LMM and GLMM. Three methods of computation are provided: (1) The Kenward-Roger approach. (2) The method introduced by Nakagawa and Schielzeth (2013), which was later extended by Johnson (2014). (3) An approach using standardized generalized variance (SGV) that can be used for both mean model and covariance model selection. 21
Confidence limits for model R and semi-partial R are computed for each of the methods listed. 22
6. Example: Blood pressure in adults, continuous and binary outcomes ñ Retrospective longitudinal cohort study of adults with hypertension. ñ University of North Carolina Family Medicine Center ñ 459 hypertensive patients making at least 4 visits to the Center during a two year period, 1999-2001. Black=205, White/Asian/Other=254; Female=283, Male=176. Average age 59 years old. ñ Mean number of visits and std dev: 10.4 (5.66). Range: 4-45. ñ Outcomes: Longitudinal blood pressure level ( continuous SBP and DBP) and dichotomous blood pressure control (<140 systolic and < 90 systolic). Independent variables include age, race, gender, continuity of care, and linear time in years. ñ Both systolic and diastolic BP fell over the two years (systolic 2.2 mmhg/yr and diastolic 2.8 mmhg/yr). Lower systolic blood pressure was not associated with continuity of care or gender. Lower blood pressures were associated with Caucasian race (vs. African American race). 23
6. Example: Blood pressure in adults, continuous and binary outcomes ñ Fixed effects for the analysis include Age, Race, Gender, Continuity of Care (COC), and continuous, linear time in years. ñ Random effects included intercept and time with unstructured random effects covariance, D, (2 2). ñ Within-subject error variance D /3 œ 5 M : 3. 24
Outcome Effect Estimate (SE) NDF DDF F-value P-value Semi-Partial SBP Intercept 129.03 (3.03) 1 464 1809.65 <0.0001 -- Age 0.21 (0.04) 1 439 22.70 <0.0001 0.049 Race 3.97 (1.32) 1 443 9.01 0.0028 0.020 Gender 1.64 (1.35) 1 446 1.48 0.2239 0.003 COC -1.22 (2.31) 1 458 0.28 0.5988 0.001 Time -2.20 (0.60) 1 352 13.58 0.0003 0.037 R " 0.074 5 563 8.97 <0.0001 R " DBP Intercept 95.46 (1.64) 1 464 3373.29 <0.0001 -- Age -0.20 (0.02) 1 441 68.43 <0.0001 0.134 Race 1.85 (0.72) 1 447 6.62 0.0104 0.015 Gender -0.75 (0.73) 1 450 1.04 0.3076 0.002 COC -1.80 (1.26) 1 463 2.06 0.1522 0.004 Time -2.78 (0.30) 1 327 85.19 <0.0001 0.207 R " 0.247 5 556 36.53 <0.0001 BP Control Intercept -0.04 (0.31) 1 389 0.01 0.9054 -- (LMM from PQL) Age 0.01 (0.004) 1 369 5.75 0.0170 0.015 Race 0.36 (0.14) 1 380 6.88 0.0090 0.018 Gender 0.11 (0.14) 1 383 0.59 0.4431 0.002 COC -0.23 (0.24) 1 401 0.93 0.3351 0.002 Time -0.39 (0.07) 1 388 30.91 <0.0001 0.074 R " 0.082 5 486 8.63 <0.0001 OR (95% CI) BP Control Intercept -0.04 (0.30) 1 455 0.02 0.8841 -- (Final GLMM) Age 0.01 (0.004) 1 3852 5.60 0.0180 1.01 (1.00, 1.02) Race 0.35 (0.13) 1 3852 6.85 0.0089 1.41 (1.09, 1.84) Gender 0.12 (0.14) 1 3852 0.74 0.3910 1.12 (0.86, 1.46) COC -0.20 (0.23) 1 3852 0.76 0.3838 0.82 (0.52, 1.29) Time -0.37 (0.07) 1 459 8.37 <0.0001 0.69 (0.60, 0.79) 25
7. Conclusions ñ Due to the familiarity of the R statistic in the linear univariate model, there is naturally great interest in its extension to the linear mixed model. ñ Unfortunately, the development of R statistics for the linear mixed model has received comparatively little attention. ñ We propose an R statistic for assessing fixed effects in the linear mixed model. ñ We extended the proposed R statistic to the GLMM under PQL estimation for assessing fixed effects. ñ By way of a longitudinal data example, we demonstrated the utility of the proposed R statistic for both continuous and binary outcomes. 26
6. Conclusions ñ It should be noted, however, that is based on a marginal J statistic and cannot be used to determine subject-specific goodness-of-fit. R " ñ In addition, was developed for models with nested fixed effects and should R " not be used with non-nested fixed effects. 27
References 1. Edwards LJ, Muller KE, Wolfinger RD, Qaqish BF, Schabenberger O (2008). An R statistic for fixed effects in the linear mixed model. Statistics in Medicine, 27(29):6137-57. 2. Jaeger BC, Edwards LJ, Das K, Sen PK (2016). An R statistic for fixed effects in the generalized linear mixed model. In press, Journal of Applied Statistics. 3. Fisher M, Sloane P, Edwards L, Gamble G (2007). Continuity of care and hypertension control in a university-based practice. Ethnicity and Disease, 17(4):693-698. 4. Ballarini N, Jaeger BC (2015). Software in SAS and R programming languages to calculate model R and semi-partial R for fixed effects in the linear and generalized linear mixed model. https://github.com/bcjaeger/r2fixedeffectsglmm/ 28
5. B. Jaeger, r2glmm: Computes R squared for mixed (multilevel) models (LMMs and GLMMs), (2016). R package version 0.1.0. Available at https://github.com/bcjaeger/r2glmm. 6. Nakagawa S, Schielzeth H (2013). A general and simple method for obtaining R from generalized linear mixed-effects models. Methods Ecol. Evolut. 4: 133 142. 7. Johnson PC (2014). Extension of Nakagawa & Schielzeth s r2glmm to random slopes models. Methods Ecol. Evolut., 5: 944 946. 29