Ronald Heck Week 14 1 EDEP 768E: Seminar in Categorical Data Modeling (F2012) Nov. 17, 2012

Transcription

1 Ronald Heck Week 14 1 From Single Level to Multilevel Categorical Models This week we develop a two-level model to examine the event probability for an ordinal response variable with three categories (persist = 2, in school but behind = 1, left school early = 0). A two-level model is useful when we have a situation where there are individuals nested in higher social groupings. Such clustering violates the assumption of simple random sampling of individuals chosen from a population and, because, of this, can lead to biased estimates of parameters in a statistical model. Therefore, a type of model is needed that can adjust the estimates for the clustering of individuals within groups. We dealt with this issue previously when we had repeated measurements nested within individuals. Suppose we wish to develop a simple model to examine how student SES and the rigor of their school s curriculum might affect student persistence. Since this is an ordinal outcome, will use the reference group as persist. There are 12,916 individuals nested in 1,016 schools in this example. For this demonstration, I will use a level 1 predictor (student SES) and a level 2 predictor defined as mean perceptions of the rigor of the school s curriculum ranging from 1 to 3, with the sample mean = 1.90). One of the problems of defining a single level model is that the effect of the school s curriculum rigor on persistence will be incorrectly estimated based on the individuals in the sample rather than the number of schools (which obviously is much smaller). This will typically result in a smaller estimated standard error. In turn, because hypothesis tests are based on the ratio of the estimate of the parameter to its standard error (B/SE), having a standard error that is too small will result in more findings of statistical significance than would be expected. Here is a simple example. If the log odds is estimated as in a single level model, and the SE estimated as 0.14, the resulting t- ratio will be 2.0 (0.28/0.14). At p =.05, and a sufficient sample size, the estimate would be significant if the t-ratio is 1.96 or above. Now, suppose we define a two-level model where the estimate is still but the standard error is inflated to 0.18 (since there is a smaller sample at the school level). The new t-ratio would be calculated as 0.28/0.18 = 1.56, which would not be significant even at p =.10. Testing a Single Level Model Below is the model conducted as a single level analysis using Genlin Mixed in IBM SPSS. You can see that student SES is positively related to persisting versus the combined lower categories. For example, going up an SD in student SES increases the odds of persisting versus the combined lower categories by about 2.8 times. Going up a unit in attending a school with rigorous school curriculum (an ordinal indicator from 0 to 3) increases the odds of persisting versus the combined lower categories by about 7.9 times.

2 Ronald Heck Week 14 2 Table 1. Fixed Effects a a Probability distribution: Multinomial; Link function Cumulative logit Target: persist (without GED) Reference category: graduated Although both indicators are significantly related to the outcome, the model is likely misspecified since SES is a level 1 variable and perceptions of the school s curriculum is a level 2 variable. One of the advantages of a multilevel model is that we might look to see whether there are differences in the proportions of students who leave school early across the number of schools in our sample. Instead of treating the dropout rate as fixed, we can treat it as varying in size across the different schools in the sample. When we allow the size of an effect to vary, we refer to this as a random (or randomly varying) effect. It can then become interesting to find school variables that might explain the variability in the size of the school dropout rates. We can see in the following table that there are no random effects in this model, so it is referred to as a single-level model. Here is the information that shows you there are four fixed effects but no random effect; that is, the intercept effect does not vary across schools. Table 2. Covariance Parameters a a Target: persist (without GED) Table 3. Residual Effect a a Covariance Structure: Scaled Identity 1 This is the scale parameter

3 Ronald Heck Week 14 3 We can see in Table 2 that there are no random effects in this model, so it is a single-level model. That table provides the information that shows there are four fixed effects but no random effect; that is, the intercept for likelihood to persist does not vary across schools. Developing a Multilevel Model The extension of GLM techniques to multilevel data structures is referred to as generalized linear mixed modeling (GLMM). GLMM describes models for categorical data where the subjects are nested within groups or where repeated measures are nested within individuals (and perhaps within group structures). In these types of analyses, the focus is often on the variability in the effects of interest (i.e., intercepts, regression coefficients) at the group level (Hox, 2010). The result is a mixed-effect model, which includes both the usual fixed effects for the predictors and the variability in the specified random effects variability that occurs across higher level units or across time periods. Components for Generalized Linear Mixed Models { TC "Components for Generalized Linear Mixed Models" \f C \l "4" }The GLMM procedure has the same three GLM components, which include the random component that specifies the variance in terms of the mean ( ), or expected value, of Y ; the structural component that relates the transformed predictor of Y to a set of predictors, and the link function g(.) which connects the expected value ( ) of Y to the transformed [ f ( ) predicted values of ]; and the link function g() which converts the expected value Y (i.e., predictor. The link function therefore provides the relationship between the linear predictor and the mean of the distribution function. u = E[ Y ]) to the linear Specifying a Two-Level Model We start here by introducing the basic specification for a two-level model. For a two-level model, we include subscript i to represent an individual nested within a level-2 unit designated by subscript j. The level-1 model for individual i nested in group j is of the general form: x, (1) where x is a (p + 1) x 1 vector of predictors for the linear predictor of Y and is a vector of corresponding regression coefficients. An appropriate link function is then used to link the expected value of Y to. In this case, we will use the logistic link function. As with the single level model, there is no residual variance term included at level 1, since the underlying probability distribution associated with Y is not normally distributed. The odds of the cumulative probability are defined as being at or below the c th outcome category relative to being above the c th category, and the log odds for that ratio is as follows:

4 Ronald Heck Week 14: From Single-Level to Multilevel Categorical Models 4 ic P( c) c log log P( Y c) 1 c, (2) Because the proportional odds model with cumulative logit link function focuses on the likelihood of a variable falling into a particular category c or lower [P(Y c)] versus falling above it [P(Y c)], the model is often presented in the literature with the following structure: ic ic = log c qx q 1 ic, (3) where c are increasing model thresholds dividing the latent continuous ic of expected probabilities of Y. Each logit has its own threshold term, but the slope coefficients ( ) are the same across categories. i Therefore, the coefficients do not have c subscripts. Since confusion can result from the inverse relationship between the ordered outcome categories and the direction of the predictors in the linear model to predict η ic in Eq. 3, some software programs (e.g., IBM SPSS, Mplus) simply multiply the linear predictors β q X q by -1 to restore the direction of the regression coefficients such that positive coefficients increase the likelihood of being in the highest category and negative coefficients decrease it (Hox, 2010). q At level 2, the level-1 coefficients qj can become an outcome variable. Following Raudenbush et al. (2004), a generic structural model can be denoted as follows: W W W u, (4) qj q0 q1 1j q2 2 j qs Sqj qj where qs (q = 0,1,,S q ) are the level-2 coefficients, WSjare level-2 predictors, and u q qj are level-2 random effects. The specification of the level-2 covariance matrix depends on the number of randomlyvarying effects at level 2. For example, for a random intercept and single random slope we could specify 2 2 an unstructured covariance matrix, which has an intercept variance ( I ), a slope variance ( S ), and a covariance between them ( IS ): q 2 I IS IS 2 S, (5) Notice that a covariance matrix is a square matrix, which means that the variances are specified as diagonal elements of the matrix and the off-diagonal elements are the same above and below the diagonals. We could also specify a diagonal matrix, which assumes the covariance between the random intercept and slope is 0: 0 2 I 0 2 S. (6)

5 Ronald Heck Week 14: From Single-Level to Multilevel Categorical Models 5 Examining an Intercept-Only Model We often start with an intercept only (or unconditional) model (no predictors) first. For individual i in school j we have the following level 1 model: c c log 0j 2 C. (7) It is important to note that the intercept (first threshold) can vary across schools but the second threshold is treated as a fixed parameter (i.e., they have no j subscript) to maintain measurement invariance across the groups (Hox, 2010). At level 2, then, the random intercept will be as follows: u. (8) 0 j 00 0 j Using substitution, the single equation version of Eqs.7 and 8 will be the following, where the lowest threshold is redefined as the intercept (γ 00 ) so that it can be allowed to vary across schools: u. (9) c j IBM SPSS output provides the cutpoints in the continuous indicator ( ) and not the actual threshold value (which is the same for the second and subsequent cutpoints defining observed ordinal categories). Following is the intercept only model, which provides average thresholds for schools in the sample: Table 4. Fixed Effects a a Probability distribution: Multinomial; Link function: Cumulative logit Target: persist (without GED) Reference category: graduated We can use the odds ratios to determine the probability of being either in the lowest or second category versus persisting. For leaving school early, using the formula odds/(1+odds) we have 0.064/1.064 =

6 Ronald Heck Week 14: From Single-Level to Multilevel Categorical Models For the next threshold we will have 0.090/1.090 = 0.083, and = (suggesting on average about 2.2% still in school). Notice these estimates are slightly different from the ones in the table below, which reflect individual level percentages (N = 12,916) rather than school level (N = 1,016) proportions. Table 5. Single-Level Descriptive Statistics for Persist (without GED) Frequency Percent Valid Percent Cumulative Percent drop out Valid still in school Graduated Total How does the GENLIN MIXED routine determine which variables are the school level and the individual level? Basically, there are two approaches to organizing multilevel data sets. Programs like HLM provide separate data sets for the level 1 (individual) predictors and outcome and the level 2 (school) predictors. This results in building separate equations at each level. The advantage of this approach is that it is clear where each variable is being defined, since they are stored in separate data sets. The disadvantage is that the analyst cannot see into the combined data set to examine how the cross-level interactions are built, as well as other features regarding, for example, possible missing data. The second way, followed in SPSS, SAS, and Mplus (among others), is to include the level 1 and level 2 predictors in one data set. This has the advantage of allowing the analyst to look at the whole data set at once, but is more challenging in terms of combining the separate equations into one equation. In this latter approach, we depend on a subject variable (like a school identifier), which provides an indicator of which variables are specified at each level. When the school identifier is included, the program then knows that repeated values in the data set refer to level 2 variables, while unique values within each subject unit refer to level 1 predictors. An example should help illustrate the point. Below we see two schools in a sample data set. In the second column we can see unique values for SES. We can see repeated values for the rigor of the curriculum. If the computer knows the number of subject units and total number of individuals within those units, then it can calculate standard errors based on the number of individuals (as for SES) or schools (as for currigor_mean). In the sample data below, we can see there are two schools, one with 3 individuals and the other with two individuals. Keep in mind, however, that the default in SPSS is to eliminate cases that have missing values. The analyst might think she or he has 1000 individuals in 100 schools but find that through missing data the data set has been reduced considerably. There are approaches available for working with missing data, however.

7 Ronald Heck Week 14: From Single-Level to Multilevel Categorical Models 7 SchoolID SES Currigor_Mean In SPSS the level 1 and level 2 equations are combined through algebraic substitution, so there is only one equation. In HLM there appear to be two equations (one at each level), but the analyst cannot actually see how they are combined in that program (as well as how the cross-level interactions are actually created). One has to have faith in the program that the equations were combined correctly. Adding a Level 1 and Level 2 Predictor We can now add student SES and perceptions of the rigor of the school s curriculum to the model. The set of results will look a bit different. Table 6. Fixed Effects a Probability distribution: Multinomial; Link function: Cumulative logit Target: persist (without GED) Reference category: graduated In Table 6, both variables are still significant, but we can see that the size of the odds ratio for curriculum rigor is much smalller (about 4.5 versus 7.9). The accompaning standard error for the curriculum varible has been re-estimated as 0.113, which is actually a bit smaller than the previous single level model (0.127), but is more accurately estimated because clustering has been considered. Table 7 also shows that there is a level 2 random effect now (the randomly-varying intercept). We also can see in Table 8 that we have a scaled identity (1.0) estimate for the variance at level 1, since there is no level 1 residual error term. Notice also on the last line of the Random Effects table (Table 7), we have the number of level 2 units listed in the model. In this data set there are 1,016 schools.

8 Ronald Heck Week 14: From Single-Level to Multilevel Categorical Models 8 Table 7. Covariance Parameters Target: persist (without GED) Common subjects are based on the subject specifications for the residual and random effects and are used to chunk the data for better performance. a This is the number of columns per common subject. Table 8. Residual Effect a a Covariance Structure: Scaled Identity 1 This is the scale parameter GENLIN MIXED Model Syntax (Single Level Model) It is sometimes helpful to examine the model syntax for the single-level and the multilevel models. One can see below for the single level model that there is no Random subject designation, so there is no level 2 random intercept. This means the analysis is being treated as a single level analysis. Therefore the curricular effect which is defined at level 2 will be likely improperly estimated since the effects of nesting of individuals within schools is not taken into consideration. GENLINMIXED /DATA_STRUCTURE SUBJECTS=schid /FIELDS TARGET=persist2 TRIALS=NONE OFFSET=NONE /TARGET_OPTIONS REFERENCE=0 DISTRIBUTION=MULTINOMIAL LINK=LOGIT /FIXED EFFECTS=ses currigor_mean USE_INTERCEPT=TRUE /BUILD_OPTIONS TARGET_CATEGORY_ORDER=ASCENDING INPUTS_CATEGORY_ORDER=ASCENDING MAX_ITERATIONS=100 CONFIDENCE_LEVEL=95 DF_METHOD=RESIDUAL COVB=MODEL /EMMEANS_OPTIONS SCALE=ORIGINAL PADJUST=LSD.

9 Ronald Heck Week 14: From Single-Level to Multilevel Categorical Models 9 GENLIN MIXED Syntax (Two-Level Model) Below is the syntax for the same model estimated with the Random subject effect added to the syntax. You can see below that we have now added a Random effect (see line 6). The subject designation is where one places the identifier variable for the level 2 groups (in this case, we use the school identifier schid). Adding the random line will result in a two-level model and will produce accurate standard errors for the level 2 variable (i.e., based on the N of schools instead of the N individuals in the study). GENLINMIXED /DATA_STRUCTURE SUBJECTS=schid /FIELDS TARGET=persist2 TRIALS=NONE OFFSET=NONE /TARGET_OPTIONS REFERENCE=0 DISTRIBUTION=MULTINOMIAL LINK=LOGIT /FIXED EFFECTS=ses currigor_mean USE_INTERCEPT=TRUE /RANDOM USE_INTERCEPT=TRUE SUBJECTS=schid COVARIANCE_TYPE=VARIANCE_COMPONENTS /BUILD_OPTIONS TARGET_CATEGORY_ORDER=ASCENDING INPUTS_CATEGORY_ORDER=ASCENDING MAX_ITERATIONS=100 CONFIDENCE_LEVEL=95 DF_METHOD=RESIDUAL COVB=MODEL /EMMEANS_OPTIONS SCALE=ORIGINAL PADJUST=LSD. If you are using the MIXED model approach in SPSS, it is easier to see the variables at each level, since the degrees of freedom can be used to identify them. To illustrate, I will run the same model and pretend the outcome is a continuous variable (using MIXED) instead of ordinal (which was run with GENLIN MIXED). Notice with a random intercept, the degrees of freedom ( ) (which reflect the Satterthwaite adjustment for calculating the degrees of freedom for hypothesis tests) are close to the number of schools (N = 1016). This represents a type of estimate of the power (or effective sample size) for detecting the effect. For SES, the degrees of freedom are , which is consistent with the number of students at level 1 (N = 12,916). The degrees of freedom for curriculum rigor are , again reflecting the number of schools rather than students in the sample. Unfortunately, the categorical multilevel routine in SPSS does not provide this information in the output, although the adjustment is available. Table 9. Estimates of Fixed Effects a Parameter Estimate Std. Error df T Sig. 95% Confidence Interval Lower Upper Bound Bound Intercept Ses currigor_mean a. Dependent Variable: persist (without GED). Looking at the degrees of freedom is helpful in showing the analyst that the program has correctly located and estimated the level 1 and level 2 predictors consistent with what the analyst intends. Keep in mind the GENLIN MIXED two-level routine for categorical outcomes is still very new in SPSS and needs some further work. We have found that Version 20 works much better than Version 19 (when it

10 Ronald Heck Week 14: From Single-Level to Multilevel Categorical Models 10 was first introduced). I am not sure yet whether Version 21 (which just came out a month or so ago) actually has continued to improve the multilevel routine for categorical outcomes. This upcoming week we will work a bit with two-level models using these data and the data from last week where we estimated a single-level longitudinal model using the Generalized Estimating Equations (GEE) approach (which does not support random effects). Reference Hox, Joop J. (2010). Multilevel analysis: Techniques and applications (2nd ed.). New York: Routledge.