Technical Appendix C: Methods

Transcription

1 Technical Appendix C: Methods As not all readers may be familiar with the multilevel analytical methods used in this study, a brief note helps to clarify the techniques. The general theory developed in chapter 1 predicts that on the demand side, individual characteristics (such as education, age, and access to the news media) will have a direct effect on individual level democratic orientations. In addition, the account predicts that on the supplyside, certain contextual or societal level factors in each nation state will also have an important direct effect on these orientations, including the role of human development, the historical experience of democratization in each society, the process and policy performance of democratic regimes, the powersharing structure of regimes, and levels of media freedom. To operationalize these factors, the key models in the book involve measurement at two distinct levels. A representative sample of individual respondents (level 1) is nested within national level contexts (level 2). The World Values Survey was conducted among a representative random sample of the adult population within each nation state. Many previous studies about political trust have employed Ordinary Least Squares (OLS) regression for analysis. The danger of using this method is that the standard errors of the regression coefficients can be inaccurate for contextual variables, by overestimating the degrees of freedom (number of cases), and therefore tests of significance can prove misleading. OLS models can seek to control for national variations by using a pooled model, including dummy variables for each country, but this becomes inefficient with the coverage of many nations. Alternatively OLS models can be run with no pooling, where separate models are run for each nation or type of media environment, but this is also clumsy. Given the use of multilevel data, hierarchical linear models (HLM) are most appropriate for analysis, including multilevel regression analysis. 1 The models in this study use restricted maximum likelihood techniques (REML) to estimate direct and cross level effects for hierarchical data. Individual respondents are thus grouped into nation states. Each nation state has a different set of parameters for the random factors, allowing intercepts and slopes to vary by nation. 2 Level 1 in our core models includes the following individual level measures: male gender (0/1), household income using a 10 point scale, age (in years), the education scale, and the media use 5 point scale (or each of the separate dummy variables for use of newspapers, radio/tv and the Internet). Level 2 includes many national level variables, exemplified by the standardized Freedom House index of democracy and the standardized level of economic development (per capita GDP (2006) in Purchasing Power Parity). In each case, unless lagged, the appropriate year of these indices was matched to the closest first year of fieldwork for the survey. All variables are described in Technical Appendix A and Table C.1. In SPSS 18.0 Mixed Models, the iterative restricted maximum likelihood (REML) algorithm was used for estimating parameters. In hierarchical linear models, as is customary, all independent variables were centered, by subtracting the grand mean (which becomes zero). The standardized independent variables all have a standard deviation of 1.0. This process also helps to guard against problems of collinearity in the independent variables in the OLS models. The dependent variables are all converted into 100 point scales for ease of comparison across different tables. The independent variables were treated as fixed components, reflecting the weighted average for the slope across all groups, while nation was treated as a random component, capturing the country variability in the slope. The strength of the beta coefficients (slopes) can be interpreted intuitively as how much change in the dependent variable is generated by a onepercent change in each independent variable. 1

2 The treatment of missing data is also important. Mean substitution replaced missing data for individuallevel income where this was omitted in the national surveys conducted in two countries. Models were tested with and without these treatments to check that they did not have a substantial effect on the interpretation of the results. The multilevel regression models used in this study usually generate small differences in the size of the slope coefficient (b) compared with the results of OLS models, but the average standard errors for level 2 variables often tend to be slightly larger. The process is thus more rigorous and conservative, avoiding Type I errors (false positives, concluding that a statistically significant difference exists when, in truth, there is no statistical difference). The goodness of fit statistic in OLS is the adjusted R 2, where models with a higher coefficient indicate that it accounts for more of the variance. In the REML model, by contrast, Schwarz s Bayesian Criterion (BIC) is used, where the model with the lower value is the best fitting. Table C.2 below compares the results of using both the OLS and the REML models, where the 100 point democratic satisfaction scale is used illustratively as the dependent variable. Comparison of the OLS and REML models in Table C.2 show that many (but not all) of the estimates of the slope and intercept are very similar. Nevertheless the OLS model can inflate the standard error and the appropriate degrees of freedom at national level, whereas the REML model is preferable by providing a more rigorous and conservative estimate of significance at national level. The estimates of covariance suggest that the national intercepts were significant and strong, capturing the variability in the democratic satisfaction scale among countries. Excluding the insignificant predictors generated the following equations. In the notation, 1 refers to level 1 (individual) and 2 to level 2 (national) variables. Model A: OLS Regression analysis Y DEMOSATISFACTION = x INCOME1 1.22x EDUCATION1.446x MEDIAUSE x HISTDEMOCRACY2. Model B: REML Multilevel Regression analysis Y DEMOSATISFACTION = x AGE x INCOME1.373 x EDUCATION x HISTDEMOCRACY2. 2

3 Table C.1: Description of the core independent variables, WVS Unstandardized Standardized (z scores) N Min Max Mean Min Max Mean INDIVIDUAL LEVEL Demographic characteristics Age (years) 78, Male Gender 78, Socio economic resources Household Income scale 78, Education scale 77, Media use Media use 100 point scale 65, Read newspaper (0/1) 70, Use radio/tv news(0/1) 70, Use internet/ (0/1) 69, NATIONAL LEVEL Historical index of democratization Development: GDP per capita in PPP 49 $155 $40,947 $10, Source: World Values Survey

4 Table C.2: Comparison of OLS and multilevel regression models explaining democratic satisfaction Model A: OLS regression Model B: REML Multilevel regression b SE Beta Sig. b SE Sig. INDIVIDUAL LEVEL Demographic characteristics Age (years) N/s *** Gender (male=1) N/s N/s Socio economic resources Household Income 10 pt scale *** *** Education 9 pt scale *** ** Media use Media use 100 point scale *** N/s NATIONAL LEVEL Historical index of *** * democratization Constant Goodness of fit: Adjusted R Goodness of fit: Schwarz s BIC 491,891 N. respondents 54,492 54,472 N. nations Note: All independent variables have been standardized between 0 1 using mean centering (z scores). Model A presents the results of OLS regression models while Model B presents the results of the REML multilevel regression models. The democratic satisfaction 100 point scale was the dependent variable. The 100 point media use scale combined use of newspapers, radio/tv, the internet, books, and magazines for information. Model A reports the unstandardized beta coefficients (b), the standard errors, the standardized Betas, and their significance. The OLS models were checked by tolerance tests to be free of any multi collinearity problems. See appendix A for details about the measurement, coding, and construction of all variables. Significant coefficients are highlighted in bold. Source: World Values Survey

5 1 Robert Bickel Multilevel Analysis for Applied Research: Its Just Regression! New York: The Guilford Press. 2 Stephen W. Raudenbush and Anthony S. Bryk Hierarchical Linear Models (2 nd ed). Thousand Oaks: Sage; Andrew Gelman and Jennifer Hill Data Analysis Using Regression and Multilevel/Hierarchical Models. New York: Cambridge University Press. 5