MULTILEVEL MODELS. Multilevel-analysis in SPSS - step by step

Transcription

1 MULTILEVEL MODELS Multilevel-analysis in SPSS - step by step Dimitri Mortelmans Centre for Longitudinal and Life Course Studies (CLLS) University of Antwerp Overview of a strategy. Data preparation (centering and standardizing). Null random intercept model 3. Random intercept with Level- predictors 4. Adding Level- predictors to step 3 5. Testing random slopes level 6. Testing significant random slopes Level 7. Testing random slopes with level- predictors 8. Reducing parameters in var-covar 9. Testing cross-level interactions. Change estimation method and compare models. Standardizing

2 The basic formula General multilevel-model Y ij = + X ij + Z j + Z j X ij + u j X ij + j + r ij All fixed parameters are in this part = FIXED PART All random parameters are in this part = RANDOM PART 3 Overview of ML models Model RI Null random intercept / Intercept only / Unconditional means RI Random intercept FR Fully Random 4

3 PHASE : PREPARATION. Data preparation STEP Data preparation Actions: Look at the distribution of variables Check outliers Data centering and standardizing Check your centering and standardizing (see slides data management) 6 3

4 PHASE : VARIANCE COMPONENT MODELS. Null random intercept model 3. Random intercept with Level- predictors 4. Adding Level- predictors to step 3 PHASE. Null random intercept model PURPOSE: estimate a basic model to compare other models with 3. Random intercept with level- variables PURPOSE: how much variance can be explained with your level- predictors? 4. Random intercept with level and level variables PURPOSE: how much (additional) variance can be explained with your level- predictors? 8 4

5 STEP Null random intercept model THEORETICAL: Y ij = + j + r ij In the example: = Overall intercept: mean income over all countries j r ij = Non explained differences in income between the countries = Non explained differences in income between the individuals 9 STEP Null random intercept model *. STEP : Null random intercept model; * **********************************************; MIXED ink_centr /PRINT = SOLUTION TESTCOV /METHOD = REML /FIXED = INTERCEPT /RANDOM = INTERCEPT SUBJECT(cntry). 5

6 STEP Null random intercept model *. STEP : Null random intercept model; * **********************************************; MIXED ink_centr Put the Dependent variable after MIXED /PRINT = SOLUTION TESTCOV /METHOD = REML /FIXED = INTERCEPT /RANDOM = INTERCEPT SUBJECT(cntry). STEP Null random intercept model * *. STEP : Null random intercept model; * **********************************************; MIXED ink_centr /PRINT = SOLUTION TESTCOV /METHOD = REML PRINT /FIXED = INTERCEPT SOLUTION = Show the Fixed effects in the output /RANDOM = INTERCEPT SUBJECT(cntry). with the standard errors TESTCOV = Show significance tests for the variance components METHOD The estimation method Used method here = Restricted Maximum Likelihood 6

7 STEP Null random intercept model * *. STEP : Null random intercept model; * **********************************************; MIXED ink_centr /PRINT = SOLUTION TESTCOV /METHOD = REML FIXED /FIXED = INTERCEPT Add the fixed variables (independent variables) here. /RANDOM = INTERCEPT SUBJECT(cntry). RANDOM Add the random parts here. HERE we add only the intercept (for now). 3 STEP Null random intercept model OUTPUT FOR: Y ij = + j + r ij Grouping variable Number of parameters in the model 4 7

8 STEP Null random intercept model OUTPUT FOR: Y ij = + j + r ij with r ij ~ N, r j ~ N, : Estimated mean income over all countries is -.97 Unexplained variance (differences) on individuea level equals 4.58 Unexplained variance (differences) on Country level (level ) equals STEP Null random intercept model Summary table RI Intercept σ² r σ² σ² // 6 8

9 STEP Null random intercept model No explanation of the variance in income BUT a decomposition of the variance in income in parts σ² = Inter-group variance: differences between countries σ² r = Intra-group variance: differences between individuals QUESTION: Are there enough differences between countries to justify a multi-level analysis? 7 STEP Null random intercept model QUESTION: Are there enough differences between countries to justify a multi-level analysis? ANSWER: Intra-class correlation-coefficient (RHO) = σ² / ( σ² + σ² r ) 8 9

10 STEP Null random intercept model Intra-class correlation-coefficient (RHO) ( = σ² / ( σ² + σ² r ) ) =.484 / ( ) =.383 INTERPRETATION: 38.3 % of the total variance can be ascribed to level. PUT DIFFERENTLY: For 38.3 %, the differences in income between countries in the ESS are due to differences between countries (richer and poorer countries). 6.7 % is due to differences in individuals within these countries (richer and poorer individuals within a country). 9 STEP 3 Random intercept (level ) THEORETICAL: Y ij = + X ij + j + r ij In the example: = Overall intercept: mean income over all countries j r ij X ij = Non explained differences in income between the countries = Non explained differences in income between the individuals = General slope of the independent variable X (gender or education) QUESTION: How much of the original variance in j has been explained now?

11 STEP 3 Random intercept (level ) BASIC IDEA: Add all your level- predictors De variance components in the residuals should become smaller BECAUSE the level- predictors now take up a part of the explanation of the total variance. QUESTION: How big is the explained part? STEP 3 Random intercept (level ) * 3. STEP 3: Random intercept model with level predictors; * **********************************************************************; WITH: treat the independent as a continuous variable BY: treat the independent as a categorical variable MIXED ink_centr BY geslacht WITH opl_centr /PRINT = SOLUTION TESTCOV /METHOD = REML /FIXED = INTERCEPT geslacht opl_centr /RANDOM = INTERCEPT SUBJECT(cntry). Add your Level- independent variables

12 STEP 3 Random intercept (level ) OUTPUT FOR: Y ij = + X ij + j + r ij r ij ~ N, r j ~ N, : Overall intercept Coefficients at level- Error-variance on level Error-variance on level 3 STEP 3 Random intercept (level ) OUTPUT FOR: Y ij = + X ij + j + r ij r j ~ N, : ij ~ N, r The effect of gender and education on income is significant. Even after controlling for level- predictors there still are significant differences between individuals on level- and significant differences between the countries on level. 4

13 STEP 3 Random intercept (level ) Reduction of individual differences: From 4.58 to Reduction of differences between countries: From.4855 to.33 Is this enough? We need an R²-measure 5 STEP 3 Random intercept (level ) Regression-like R²-measure on level : r Basis -r Model / r Basis / 4.58 Regression-like R²-measure on level : Basis - Model / Basis /.4855 BUT: The estimation methods make this easy solution erroneous. 6 3

14 STEP 3 Random intercept (level ) No consensus yet about a good R²-measure. Correction of Bosker&Snijders (994) level : (r Basis + Basis )-(r Model + Model ) / (r Basis + Basis ) Correction of Bosker&Snijders (994) level : (r Basis +( Basis /n) )-(r Model +( Model /n) ) / (r Basis +( Basis /n) ) Whereby n = mean cluster size at level 7 STEP 3 Random intercept (level ) Correction of Bosker&Snijders (994) level : (r Basis + Basis )-(r Model + Model ) / (r Basis + Basis ) =.4 Correction of Bosker&Snijders (994) level : (r Basis +( Basis /n) )-(r Model +( Model /n) ) / (r Basis +( Basis /n) ) =.4 8 4

15 STEP 4 Random intercept (level +) THEORETICAL : Y ij = + X ij + Z j + j + r ij In the example: = Overall intercept: mean income over all countries j r ij = Non explained differences in income between the countries = Non explained differences in income between the individuals X ij = General slope of the independent variable X (gender or education) Z j = Direct effect of country variables on income (BNP or religion) QUESTION: How much variance in j from step 3 is now further explained? 9 STEP 4 Random intercept (level +) BASIC IDEA: Look how much the level predictors can explain (as a group) 3 5

16 STEP 4 Random intercept (level +) * 4. STEP 4: Random intercept model with level and level predictors; * *********************************************************; MIXED ink_centr BY geslacht WITH opl_centr bnp_centr bnppps_centr romkath_centr relig_centr /PRINT = SOLUTION TESTCOV /METHOD = REML /FIXED = INTERCEPT geslacht opl_centr bnp_centr bnppps_centr romkath_centr relig_centr /RANDOM = INTERCEPT SUBJECT(cntry). 3 STEP 4 Random intercept (level +) OUTPUT FOR: Y ij = + X ij + Z j + j + r ij r j ~ N, : ij ~ N, r Overall intercept Coefficients on level- Coefficients on level- Error-variance on level Error-variance on level 3 6

17 STEP 4 Random intercept (level +) OUTPUT FOR: Y ij = + X ij + Z j + j + r ij r j ~ N, : ij ~ N, r BNPPPS and % Roman Catholics are significant, BNP and Religiosity not. Even after controlling for level- predictors there still are significant differences between individuals on level- and significant differences between the countries on level. 33 STEP 4 Random intercept (level +) OUTPUT FOR: Y ij = + X ij + Z j + j + r ij r j ~ N, : ij ~ N, r Problem with BNP? 34 7

18 STEP 4 Random intercept (level +) Reduction of individual differences: 3 vs From to vs From 4.6 to Reduction of country differences: 3 vs From.3 to.55 3 vs From.486 to STEP 4 Random intercept (level +) Correction of Bosker&Snijders (994) level : Pseudo R² =.38 Correction of Bosker&Snijders (994) level : Pseudo R² =

19 STEP 4 Random intercept (level +) Preliminary conclusions:. The error-variance on level is only significant to the.6 level.. Two level- variables do not seem to work in this model: BNP and Religiosity. We remove them from the model when they are less important from a theoretical perspective. 3. Our model strongly explains differences on level. But also on the individual level the explanatory power of the independent variables is satisfactory. 37 PHASE 3: testing random slopes 5. Testing random slopes level 6. Testing significant random slopes Level 7. Testing random slopes with level- predictors 8. Reducing the parameters in var-covar 9

20 PHASE 3 5. Testing random slopes level PURPOSE: looking which level- predictor has significant slopes (without the level- predictors). 6. Testing significant random slopes Level PURPOSE: joining all significant random slopes from step 5 in one model 7. Testing random slopes with level- predictors PURPOSE: adding the level predictors back to model, including the random slopes from step 6 8. Reducing parameters in var-covar PURPOSE: simplifying the model (removing nonsignificant co-variances) in order to ease the estimation of the model. 39 STEP 5 Testing random slopes level THEORETICAL : Y ij = + X ij + j X ij + j + r ij In the example: = Overall intercept: mean income over all countries j r ij = Non explained differences in income between the countries = Non explained differences in income between the individuals X ij = General slope of the independent variable X (gender or education) j X ij = Differences in slopes (effect) of variable X (gender or education) between the countries REMARK: No longer country differences in this model! REMARK : Model is estimated for each independent variable on level separately QUESTION: Are there significant differences in the effect of X between the countries? 4

21 STEP 5 Testing random slopes level BASIC IDEA: Until now we did not allow for differences in slopes of the independent variables at level. FOR EACH INDEPENDENT, we will test if there are significant differences in slopes at level-. 4 STEP 5 Testing random slopes level * STEP 5: Testing the significance of the random slopes step-by-step. ***********************************************************************************; MIXED ink_centr BY geslacht WITH opl_centr /PRINT = SOLUTION TESTCOV COVTYPE should be Unstructured. /METHOD = REML /FIXED = INTERCEPT geslacht opl_centr /RANDOM = INTERCEPT opl_centr SUBJECT(cntry) COVTYPE(UN). We make eduction RANDOM here. 4

22 STEP 5 Testing random slopes level OUTPUT FOR: Y ij = + X ij + j X ij + j + r ij ij ~ N, r j j r ~N, : Overall intercept Coefficients on level- Error-variance on level Error-variance on level 43 STEP 5 Testing random slopes level Structure of the variance-covariance matrix Theoretical: j ~N, : j In SPSS: UN, UN, UN,

23 STEP 5 Testing random slopes level Structure of the variance-covariance matrix In SPSS: UN, UN, UN, The random slope is significant on the. level. MEANING: there is a significant different effect of education between the countries 45 STEP 5 Testing random slopes level * STEP 5: Testing the significance of the random slopes step-by-step. **********************************************************************************. MIXED ink_centr BY geslacht WITH opl_centr /PRINT = SOLUTION TESTCOV /METHOD = REML /FIXED = INTERCEPT geslacht opl_centr /RANDOM = INTERCEPT geslacht SUBJECT(cntry) COVTYPE(UN). Gender is made RANDOM here. 46 3

24 STEP 5 Testing random slopes level OUTPUT FOR: Y ij = + X ij + j X ij + j + r ij ij ~ N, r j j r ~N, : Overall intercept Coefficients on level- Error-variance on level Error-variance on level 47 STEP 5 Testing random slopes level Structure of the variance-covariance matrix Theoretical: j ~N, : j In SPSS: UN, UN, UN,

25 STEP 5 Testing random slopes level Evaluation of the random slope of gender In SPSS: UN, UN, UN, The random slope of the dichotomous variable gender can not be tested. If you include gender as a continuous variable in the model, this random slope turns out to be significant. 49 STEP 5 Testing random slopes level Conclusion: - Education and gender have significant slopes between the countries. - Gender is only estimable when you include the variable as a continuous measure. BUT: the effect of gender is unclear. In this example I will keep gender in the model with a random slope (only for educational purposes). 5 5

26 STEP 6 Testing random slopes level + THEORETICAL : Y ij = + X ij + j X ij + j + r ij j r ij = Overall intercept: mean income over all countries = Non explained differences in income between the countries = Non explained differences in income between the individuals X ij j X ij = General slope of the independent variable X (gender or education) = Differences in slopes (effect) of variable X (gender or education) between the countries Theoretically this is the same model as in step 5 but now we include ALL significant random slopes in the model. 5 STEP 6 Testing random slopes level + * STEP 6: Including all level- variables with significant random slopes. * ****************************************************************************************. MIXED ink_centr BY geslacht WITH opl_centr /PRINT = SOLUTION TESTCOV /METHOD = REML /FIXED = INTERCEPT geslacht opl_centr /RANDOM = INTERCEPT opl_centr geslacht SUBJECT(cntry) COVTYPE(UN). 5 6

27 STEP 6 Testing random slopes level + OUTPUT FOR: Y ij = + X ij + j X ij + j + r ij ij ~ N, r j j r ~N, : Error-variance on level Error-variance on level Equal digits are error-variances: UN(,) UN(,) UN (3,3) Intercept Error-variance of education Error-variance of gender 53 STEP 6 Testing random slopes level + OUTPUT FOR: Y ij = + X ij + j X ij + j + r ij ij ~ N, r j j r ~N, : Equal digits are error-variances: UN(,) Intercept UN(,) Error-variance of education Significant on the. level UN (3,3) Error-variance of gender NOT significant: SO: Gender NOT random in the model! 54 7

28 STEP 7 Testing random slopes with level- predictors THEORETICAL : Y ij = + X ij + Z j + j X ij + j + r ij j r ij = Overall intercept: mean income over all countries = Non explained differences in income between the countries = Non explained differences in income between the individuals X ij j X ij Z j = General slope of the independent variable X (gender or education) = Differences in slopes (effect) of variable X (gender or education) between the countries = Direct effect of country variables on income (BNP or religion) 55 STEP 7 Testing random slopes with level- predictors BASIC IDEA: We now check which influence the level- predictors have on income in a model with random slopes (= step 6). 56 8

29 STEP 7 Testing random slopes with level- predictors * STEP 7: Including all level- variables in the model of step 6 *. ******************************************************************************. MIXED ink_centr BY geslacht WITH opl_centr bnppps_centr romkath_centr /PRINT = SOLUTION TESTCOV /METHOD = REML /FIXED = INTERCEPT geslacht opl_centr bnppps_centr romkath_centr /RANDOM = INTERCEPT opl_centr SUBJECT(cntry) COVTYPE(UN). 57 STEP 7 Testing random slopes with level- predictors OUTPUT FOR: Y ij = + X ij + Z j + j X ij + j + r ij ij ~ N, r j j r ~N, : Overall intercept Coefficients on level- Coefficients on level- Error-variance on level Error-variance on level 58 9

30 STEP 7 Testing random slopes with level- predictors OUTPUT FOR: Y ij = + X ij + Z j + j X ij + j + r ij ij ~ N, r j j r ~N, : Both the predictors on level as on level are significant. Significant differences remain between the intercepts of the countries. Significant differences remain on level for the effect of education 59 STEP 8 Simplifying the model THEORETICAL : Y ij = + X ij + Z j + j X ij + j + r ij j r ij = Overall intercept: mean income over all countries = Non explained differences in income between the countries = Non explained differences in income between the individuals X ij j X ij Z j = General slope of the independent variable X (gender or education) = Differences in slopes (effect) of variable X (gender or education) between the countries = Direct effect of country variables on income (BNP or religion) We now look at the variance-covariance matrix 6 3

31 STEP 8 Simplifying the model Differences in intercept and slope between units of level. Y ij = + X ij + j X ij + j + r ij These are the conditions: rij ~ N(, σ²r) j ~N, : j This is the covariance between the random intercepts and the random slopes for variabele Xij. If this covariance is significant (= different from ), then it means that the changes in intercepts is systematically correlated to the changes in slopes. 6 STEP 8 Simplifying the model Model intercept Slopes Covariance FR many many Negative FR many many Positive FR many many Not significant () 6 3

32 3 63 STEP 8 Simplifying the model With independent X-variable on level With independent X-variables on level j j : ~N, Covariance between the random intercepts and the random slopes for variabele X ij. j j j : ~N, independents = 3 covariances 64 STEP 8 Simplifying the model With 3 independent X-variable on level With random slopes models, the number of parameters to be estimated increases very rapidly j j j j : ~N, 3 independents = 6 covariances

33 33 65 STEP 8 Simplifying the model With 3 independent X-variables on level j j j j : ~N, Non-significant covariances = The covariances is not significantly different from. SO: We might as well fix these to. CONSEQUENCE: Each fixed covariance is one parameter less to be estimated. 66 STEP 8 Simplifying the model With 3 independent X-variables on level j j j j : ~N, SPSS does not allow to fix each separate covariance to. BUT: there is an option to fix all co-variances to in one time.

34 34 67 STEP 8 Simplifying the model With 3 independent X-variables on level j j j j : ~N, COVTYPE(UN) 3 3j j j j : ~N, Without COVTYPE 68 STEP 8 Simplifying the model Look at the (significance of) the co-variances in the output of step 7 Covariance is NOT significant Consequence: Estimate the model in STEP 7 again without the COVTYPE-option.

35 STEP 8 Simplifying the model STEP 8: Reduction of parameters in var-covar matrix *********************************************************. MIXED ink_centr BY geslacht WITH opl_centr bnppps_centr romkath_centr /PRINT = SOLUTION TESTCOV /METHOD = REML /FIXED = INTERCEPT geslacht opl_centr bnppps_centr romkath_centr /RANDOM = INTERCEPT opl_centr SUBJECT(cntry). 69 STEP 8 Simplifying the model The co-variances are fixed at : there is no estimation of co-variances any more. Only the variances are given in the output. 7 35

36 PHASE 4: Refining and finishing 9. Testing cross-level interactions. Change the estimation method and compare your models. Standardizing PHASE 4 9. Testing cross-level interactions PURPOSE: checking if level- predictors correlate with level- predictors.changing the estimation method and comparing models PURPOSE: Compare models with Full Maximum Likelihood.Standardizing PURPOSE: Estimating the end model again with standardized variables to check the relative influence 7 36

37 STEP 9 Testing cross-level interactions THEORETICAL : Y ij = + X ij + Z j + Z j X ij + j X ij + j + r ij j r ij = Overall intercept: mean income over all countries = Non explained differences in income between the countries = Non explained differences in income between the individuals X ij j X ij Z j = General slope of the independent variable X (gender or education) = Differences in slopes (effect) of variable X (gender or education) between the countries = Direct effect of country variables on income (BNP or religion) Z j X ij = Interaction of country variables and individual variables on income 73 STEP 9 Testing cross-level interactions What is an interaction effect? A (significant) interaction effect shows how the effect of one independent variable changes within the levels of another independent variable

38 STEP 9 Testing cross-level interactions In regular OLS regression EG. Interaction between gender and education: How does the effect of education on income change between boys and girls? In multilevel regression EG. Interaction between education and GDP: How does the effect of education on income change for countries with a different GDP? 75 STEP 9 Testing cross-level interactions TWO BASIC PRINCIPLES.When an interaction effect is significant, the main effects NEED TO BE included in the model. EG: When the interaction effect between education and GDP_PPS is included THEN you also need the main effect of education and the main effect of GDP_PPS in the model

39 STEP 9 Testing cross-level interactions TWO BASIC PRINCIPLES. When an interaction effect is present in a model, the main effects have a different meaning. Meaning without interaction-effect - Education: the change in income when the educational level rises with one unit - BNP_PPS: the change in income when BNP_PPS rises with one unit 77 STEP 9 Testing cross-level interactions TWO BASIC PRINCIPLES. When an interaction effect is present in a model, the main effects have a different meaning. Meaning without interaction-effect - Education: the change in income when the educational level rises with one unit at a level of from BNP_PPS - BNP_PPS: the change in income when BNP_PPS rises with one unit at a level of from education 78 39

40 STEP 9 Testing cross-level interactions * * STEP 9: Testing cross-level interactions * **********************************************************************. MIXED ink_centr BY geslacht WITH opl_centr bnppps_centr romkath_centr /PRINT = SOLUTION TESTCOV /METHOD = REML /FIXED = INTERCEPT geslacht opl_centr bnppps_centr romkath_centr opl_centr*bnppps_centr /RANDOM = INTERCEPT opl_centr SUBJECT(cntry). Specifying an interaction effect = repeating both variables with * between both. 79 STEP 9 Testing cross-level interactions Main effects (both are significant) Interaction effect (not significant) 8 4

41 STEP 9 Testing cross-level interactions How to interpret the interaction effect? (if it had been significant). Interpret the main effects. Choose an interpretation perspective 3. Write the regression equation for one of the independent variables 8 STEP 9 Testing cross-level interactions Interpretation. Interpret the main effects Main effect education: When BNPPPS_centr is equal to, someone's income will increase with.59 scale points when his educational level increases with one unit. Main effect GDP: When education is equal to, someone's income will increase with.65 scale points when his countries BNP_PPS increases with one unit. 8 4

42 STEP 9 Testing cross-level interactions Interpretation. Interpret the main effects Because of the centering (mean = ) we can also say: Main effect education: At a mean BNPPPS_centr level, someone's income will increase with.59 scale points when his educational level increases with one unit. Main effect GDP: At a mean educational level, someone's income will increase with.65 scale points when his countries BNP_PPS increases with one unit. 83 STEP 9 Testing cross-level interactions Interpretation. Choose an interpretation perspective perspectives:. How does the effect of education on income changes for different levels of BNP_PPS?. How does the effect of BNP_PPS on income changes for different levels of education? 84 4

43 STEP 9 Testing cross-level interactions Interpretation. Choose an interpretation perspective How does the effect of education on income changes for different levels of BNP_PPS? is theoretically the only possibility here. BECAUSE: BNP_PPS will not change if people have a higher degree. But the educational effects can indeed change for countries with different BNP_PPS. BUT REMEMBER: Statistically, both perspectives are always possible. 85 STEP 9 Testing cross-level interactions Interpretation 3. Work out the regression equation PERSPECTIVE: How does the effect of education on income changes for different levels of BNP_PPS? - Write out the regression equation for education, when BNP_PPS takes on different levels. - WHICH levels of BNP_PPS? Usually: minimum, 5 th percentile, median, mean, 75 th percentile and maximum

44 STEP 9 Testing cross-level interactions Interpretation 3. Work out the regression equation STEP : Search in the EXAMINE output for the values of the minimum, 5 th percentile, median, mean, 75 th percentile and the maximum of BNPPPS_centr. EXAMINE VARIABLES=bnppps_centr /PERCENTILES(5,75) /STATISTICS DESCRIPTIVES. 87 STEP 9 Testing cross-level interactions Interpretation 3. Work out the regression equation STEP : Search in the EXAMINE output for the values of the minimum, 5 th percentile, median, mean, 75 th percentile and the maximum of BNPPPS_centr

45 STEP 9 Testing cross-level interactions Interpretation 3. Work out the regression equation STEP Minimum Q Mean Median.85 Q maximum 6.95 Remark: We assume that the other independent variable also take on the value of (and disappear from the equation). 89 STEP 9 Testing cross-level interactions Interpretation 3. Work out the regression equation Income = education +.65 BNPPPS +.8 EDUC*BNPPPS STEP : Vul in Minimum Q Median Mean Q3 maximum Income = opleiding +.65 (-65.88) +.8 OPL*(-65.88) Income = opleiding +.65 (-3.85) +.8 OPL*(-3.85) Income = opleiding +.65 () +.8 OPL*() Income = opleiding +.65 (.85) +.8 OPL*(.85) Income = opleiding +.65 (4.685) +.8 OPL*(4.685) Income = opleiding +.65 (6.9) +.8 OPL*(6.9) 9 45

46 STEP 9 Testing cross-level interactions Interpretation 3. Work out the regression equation Income = education+.65 BNPPPS +.8 EDUC*BNPPPS STEP : Vul in Minimum Income = -,999 +,4665 OPL Q Income = -,87 +,56 OPL Median Income = -,97 +,59 OPL Mean Income = -,53 +,59 OPL Q3 Income =,34 +,539 OPL maximum Income = 3, +,67 OPL Cfr main effect of education when BNP_PPS is equal to (see the main effect in the output). Interpretation interaction effect The effect of education rises (very slightly) as BNP_PPS increases. 9 STEP Comparing models with Full Maximum Likelihood Estimation in multilevel analysis - Maximum Likelihood - Generalized Least Squares (not in SPSS) - Generalized Estimation Equations (not in SPSS) - Markov Chain Monte Carlo (not in SPSS) - Bootstrapping (not in SPSS) 9 46

47 STEP Comparing models with Full Maximum Likelihood Maximum Likelihood - Maximizing the likelihood function - Population parameters are estimated in such a way that the probability of finding that particular model is maximized with the data at hand. - TWO versions: Full Maximum Likelihood (ML) Restricted Maximum Likelihood (REML) 93 STEP Comparing models with Full Maximum Likelihood Restricted Maximum Likelihood - Only the variance components are used in the estimation. - Standard in SAS (but not in SPSS) and in almost all multilevel programs - Best estimation for the variance components - CONSEQUENCE: Gives you the best estimation for the parameters and you should use this estimation method to obtain your parameters 94 47

48 STEP Comparing models with Full Maximum Likelihood Full Maximum Likelihood - Variance components and regression coefficients are involved in the estimation. - Disadvantage: Variance components are underestimated. USEFULL? - Easier estimation method - The difference between two models can be used in a chi²-difference test (is not possible with REML!). 95 STEP Comparing models with Full Maximum Likelihood CONSEQUENCE - Use REML for all models - Use ML at the end to compare estimated models with each other

49 STEP Comparing models with Full Maximum Likelihood * STEP : Using Full Maximum Likelihood to compare models. * ************************************************************************************. *Step. MIXED ink_centr /PRINT = SOLUTION TESTCOV Use Method=ML (or leave the METHOD line out). /METHOD = ML /FIXED = INTERCEPT /RANDOM = INTERCEPT SUBJECT(cntry). 97 STEP Comparing models with Full Maximum Likelihood Use the difference in DEVIANCE (- LOG LIKELIHOOD) and the difference in degrees of freedom in a Chi²-difference test 98 49

50 STEP Comparing models with Full Maximum Likelihood Calculating the degrees of freedom of your model Calculating the degrees of freedom by hand: Intercept Error variance level- Fixed slopes on level- p Error-variances for slopes on level- p Co-variances of slopes and intercept p Whereby: p = number of level- predictors q = number of level- predictors Co-variances between the slopes p(p-)/ Fixed slopes on level- Slopes for cross-level interactions q p*q 99 STEP Comparing models with Full Maximum Likelihood Make an overview table with tests DEVIANCE D(f) Prob. Chi² vs null model Prob. Chi² vs previous model STEP Null random intercept model 464,68 STEP 3 Random intercept with level ,937 9, STEP 4 Random intercept w level and 43,4 3,, STEP 6 Random slopes model 3647,95 9, STEP 7 Random slopes with level- 75,3,, STEP 8 Random slopes with level- (reduction) 77,357 7,,79 STEP 9 Cross level interactions 77,357,, 5

51 STEP Comparing models with Full Maximum Likelihood Make an overview table with tests DEVIANCE D(f) Prob. Chi² vs null model Prob. Chi² vs previous model STEP Null random intercept model 464,68 STEP 3 Random intercept with level ,937 9, STEP 4 Random intercept w level and 43,4 3,, This is the basic model. Chi²-difference test STEP 6 Random slopes model 3647,95 9, In the fourth column, you compare with this model - Test value = = Degrees of freedom = 9 = 7 STEP 7 Random slopes with level- 75,3,, - Result: Probability =. STEP 8 Random slopes with level- (reduction) 77,357 - Interpretation: 7 The model in, STEP3 is a significant,79 improvement of the Null random intercept model (STEP ) STEP 9 Cross level interactions 77,357,, STEP Comparing models with Full Maximum Likelihood Make an overview table with tests DEVIANCE D(f) Prob. Chi² vs null model Prob. Chi² vs previous model STEP Null random intercept model 464,68 STEP 3 Random intercept with level ,937 9, STEP 4 Random intercept w level and 43,4 3,, STEP 6 Comparison with Random the Null slopes random model intercept 3647,95 model (STEP ) 9, Interpretation: Is this model a significant improvement vs STEP? STEP 7 Random slopes met level- 75,3,, Comparison with the previous model (= STEP 3) STEP 8 Random slopes met level- (reductie) 77,357 Interpretation: 7 Is this model a significant, improvement,79 vs STEP 3? STEP 9 Cross level interacties 77,357,, 5

52 STEP Comparing models with Full Maximum Likelihood Make an overview table with tests DEVIANCE D(f) Prob. Chi² vs null model Prob. Chi² vs previous model STEP Null random intercept model 464,68 STEP 3 Random intercept met level ,937 9, STEP 4 Random intercept met level en 43,4 3,, STEP 6 Random slopes model 3647,95 9, STEP 7 Random slopes met level- 75,3,, BE CAREFULL here!! Random slopes met level- STEP 8Previous model is the model 77,357 7 (reductie) with co-variances. SO is the model without covariances,,79 a significant worsening vs. the previous model? STEP 9 Cross level interacties 77,357,, CONSEQUENCE: A probability higher than.5 points to a good model. 3 STEP The final model with standardized variables Standard output of multilevel models = not standardized. Interpretation in original (scale) units DISADVANTAGE: no comparison of the actual size of the parameters. 4 5

53 STEP The final model with standardized variables methods. Estimate the last model again with standardized variables (disadvantage = variance components also change). Standardize the parameter by hand 5 STEP The final model with standardized variables METHOD * STEP : Standardising in order to compare parameters. * ********************************************************************************************; MIXED ink_std BY gesl_std WITH opl_std bnppps_std romkath_std /PRINT = SOLUTION TESTCOV /METHOD = ML /FIXED = INTERCEPT geslacht opl_std bnppps_std romkath_std /RANDOM = INTERCEPT opl_std SUBJECT(cntry). 6 53

54 STEP The final model with standardized variables METHOD Stand. Coeff. = (non stand. Coeff) x (stan dev INdep.) (stan dev DEpendent) 7 STEP The final model with standardized variables METHOD DESCRIPTIVES VARIABLES=ink_centr geslacht opl_centr bnppps_centr romkath_centr /STATISTICS=STDDEV. 8 54

55 STEP The final model with standardized variables RESULT: Non standardized Standardized Non standardized Standardized Gender,8,6 BNP PPS,7,4 Education,59,3 % Rom Cath -,95 -,5 9 MULTILEVEL MODELS Multilevel-analysis in SPSS - step by step Dimitri Mortelmans Centre for Longitudinal and Life Course Studies (CLLS) University of Antwerp 55