Ron Heck, Fall 2011 1 EDEP 768E: Seminar on Multilevel Modeling rev. 9/6/2011@11:27pm Week 3: Notes Building a Two-Level Model We will build a model to explain student math achievement using student-level (level 1) and school level (level 2) variables. It helps to have a type of theoretical model in mind. In this case, we will focus on the quality of schools academic organization and environment (school quality) and their average teaching effectiveness (teacheff) as contributing to outcomes beyond student composition (context) and size (enroll). Model 1: Unconditional Model (no predictors model) There are typically three equations that we worry about in a multilevel analysis: the level-1 model; the level-2 model for intercepts (and sometimes slopes), and the combined, singleequation model. For an unconditional model, at level 1 we have the following model for individual i in group j to represent math achievement: Y, Eq. 1 ij 0 j ij where 0 j is the intercept and ij are errors in predicting students level of achievement. At level 2, we have the model for schools, where we allow the intercept to vary at random across schools (shown by having the j subscript only):
2 u, Eq. 2 0 j 00 0 j where the average effect (or fixed effect) for the intercept is 00 and the random effect is u 0 j. Here is the key; we can substitute the level-2 model into the level-l model to end up with one combined equation. We replace 0 j in Eq. 1 with 00 u0 j from Eq. 2, which gives us the combined equation: Y u. Eq. 3 ij 00 0 j ij We always want to confirm with the model dimensions table that we are estimating the model we think were. The table shows we estimated 3 effects (i.e., fixed effect for the intercept, random effect for the intercept, and the residual). Fixed effects are the intercept level, and the effects of the predictors. Random effects are labeled u j (with a number before the j corresponding to the intercept ( u 0 j ) or predictor ( u1 j, u2 j,... u nj ) they go with. We can also examine the -2 log likelihood (as well as other fit indices) to compare subsequent models to the no predictors model. Subsequent models should reduce the size of the various fit indices.
3 The -2 log likelihood (125,532,076) can be used to formulate tests of fit of subsequent models against the baseline (no predictors) model. The intercept is 646.54. The t-test is usually not of much interest since it is a test that the intercept is 0. So here, we can reject the hypothesis that the intercept is zero, but that does not really help us much. Next we concentrate on the model s variance components (or covariance parameters).
4 From the variance component table, we can calculate the intraclass correlation (ρ) as ρ = 2 Between 2 2 Between Within 170 2302 The level-2 variance is also shown to vary significantly across schools (Wald Z = 6.912, p <.01), which suggests we might build a multilevel to explain variation in random slopes. Model 2: Within-School Predictors Typically, we will develop our within-groups model first. In this case, there are two variables that we wish to add (gender and socioeconomic status). At level 1 we have the following model. We will build on the level-1 model: Y female lowses. Eq. 4 ij 0 j 1j ij 2 j ij ij At level 2, the intercept model remains as in Eq. 2. u. 0 j 00 0 j However, we must account for the two new predictors at level 2. We usually add predictors as fixed first that is, we do not allow the slopes to vary across level 2 (by not adding random effects ( u 1 j or u 2 j ). When we have fixed level-1 predictors at level 2 we show them like this:, Eq. 5 1j 10. 2 j 20
5 We can again substitute the level-2 equation into the level-1 model to arrive at the combined single-equation model: Y female lowses u. Eq. 6 ij 00 10 ij 20 ij 0 j ij You can now count up the number of effects that are being estimated and compare them against the model estimation table. Notice that because we defined female and lowses as factors, they have two levels (since they are dummy coded). One level will be redundant. This is why we look at the number of parameters column primarily.
6 Regarding the fit indices, the -2 LL is now 125,223.422. The previous -2LL was 125,532,076. So the fit of the new model is better than the old model. We can construct a chi-square test comparing the fit of the two models, since the difference in -2LL has a chi-square distribution. (Baseline Model -2LL) (New Model -2LL) = chi square (df) = 125,532,076-125,223.422 = 308.654, 2df, p <.05 (required chi square, 2df = 5.99) Because the required chi square is only 5.99 (for 2 degrees of freedom), we can see that second model fits better than the null model (which we would expect). We can see that the average school level of achievement changes with the addition of the predictors. It is 639.87. The intercept can be interpreted as the level when the values of the predictors are 0 (female = 0; lowses = 0). Are the predictors significant? How would we interpret their effects? We can next turn our attention again to the variance components.
7 First, we see that there is still significant variance to be explained both within and between groups. This suggests we could continue to add within-group predictors. Second, we can see that the addition of the two within-school predictors has decreased the residual variance at level 1 (from 2131.79 to 2085.37). Adding the within-group predictors has also reduced the variance at the school level (from 169.98 to 117.11). This may seem strange, but it happens a lot in multilevel modeling. You can think of it as when individual-level controls are added, schools tend to become more homogeneous (i.e., their differences diminish somewhat). You can calculate the variance accounted for at each level as the following: 2 r ( - ) 2 2 Model 1 Model2 2 Model 1 Eq. 7 (169.98 117.11)/169.98 = level 2 variance accounted for (2131.79 2085.37)/2131.79 = level 1 variance accounted for Model 3: School-Level Predictors We will add four school-level variables. Two are context controls (school size and student composition, which is defined as the percentage of students of free/reduced lunch and percentage of students needing support services). The other two are process variables. The first is a composite of perceptions of the quality of the school s academic process, and the other is an aggregate of the effectiveness of the school s teachers from estimates of teacher classroom effectiveness.
8 We might have two hypotheses: 1) School with stronger perceived academic processes will have higher math outcomes. 2) School with stronger aggregate teacher effectiveness will have higher math outcomes. In this case, the level-1 model stays the same (as in Eq. 4). For the intercept model, we will add the four predictors as follows: zenroll _ y schcontext schqualcomp teacheff u Eq. 8 0j 00 01 02 03 04 0j If we substitute Eq. 8 and Eq. 5 into Eq. 4, we obtain the combined equation. Y zenroll _ y schcontext schqualcomp teacheff ij 00 01 j 02 j 03 j 04 j female lowses u 10 ij 20 ij 0 j ij Eq. 9 This will add 4 school-level parameters to explain the outcome (for a total of 9 parameters).
9 We can see that the -2 log likelihood is further reduced from the previous model 125,223.422 125,139.215 = chi square = 84.207 (4 df), p <.05. Here are the Fixed Effects for the Model. How would we explain them? Are the process indicators significant and in the direction we might expect? Do they support the hypotheses? Below are the Variance Components.
10 We can see the school level variance is further reduced (from 117.11 to 60.42). We can again calculate the variance reduction using Model 1 as the comparison. [M1 (169.98) M3 (60.42)]/168.98 = 109.56/169.98 =.645 This suggests we have accounted for 64.5% of the between-school variance with our proposed model. Note that the within-school variance does not change since the predictors were added at level 2. We might decide to investigate a random slope, but at this point we will not. Grand-Mean Centering Let s look at grand mean centering a variable. Let s suppose we want to add students reading level to the model.
11 Let s go back to Model 2 with only female and lowses. We will add the reading score. Now the math intercept is the level of math when the reading intercept, female and low SES are all 0. This does not make much sense because we know there is no student with a score of zero on the reading test. So we will grand mean center read. The mean is 637.45. So we use COMPUTE. We create a new reading variable we will call gmread (for grand-mean centered reading). Open TRANSFORM, Compute. In target write gmread. In the box, click in read. We will write an equation using the individual s score on read and the mean of the sample: read 637.45 This should create a new variable in the data set-- gmread. We can see the standard deviation is the same.
12 Now we will re-run Model 2 with gmread added. We see the results below. This model is much more consistent with our previous set of models in terms of the intercept. We can see the math intercept adjusts upward a little. Its meaning is now the average school math outcome for males (coded 0), average/high SES students (coded 0), and average reading ability (i.e., reading is not centered on the grand mean for the sample). We can see that a 1-point increase in reading scores is worth about a 0.95 increase in math scores.