Factor analysis. George Balabanis

Transcription

1 Factor analysis George Balabanis

2 Key Concepts and Terms Deviation. A deviation is a value minus its mean: x - mean x Variance is a measure of how spread out a distribution is. It is computed as the average squared deviation of each number from its mean. Standard Deviation is the square root of the variance. Covariance is a measure of how much the deviations of two variables match. The equation is: cov(x,y) = SUM[(x - mean x )(y - mean y )]. When the match is best, high positive deviations in x will be matched with high positive deviations in y, high negatives with high negatives, and so on. Such a best-case match-up will result in the highest possible sum in the formula above. Standardization is the process of making variables comparable in magnitude and dispersion: one subtracts the mean from each variable and divides by its standard deviation, giving all variables a mean of 0 and a standard deviation of 1. Correlation is the covariance of standardized variables - that is, of variables after you make them comparable by subtracting the mean and dividing by the standard deviation.

3 Key Concepts and Terms Pearson's r: This is the usual measure of correlation, sometimes called product-moment correlation. Pearson's r is a measure of association which varies from -1 to +1, with 0 indicating no relationship (random pairing of values) and 1 indicating perfect relationship, taking the form, "The more the x, the more the y, and vice versa." A value of - 1 is a perfect negative relationship, taking the form "The more the x, the less the y, and vice versa Coefficient of determination, r 2 : The coefficient of determination is the square of the Pearsonian correlation coefficient. It represents the percent of the variance in the dependent variable explained by the independent.

4 Factor analysis Exploratory factor analysis (EFA) seeks to uncover the underlying structure of a relatively large set of variables. The researcher's à priori assumption is that any indicator may be associated with any factor. There is no prior theory and one uses factor loadings to feel the factor structure of the data. Confirmatory factor analysis (CFA) seeks to determine if the number of factors and the loadings of measured (indicator) variables on them conform to what is expected on the basis of pre-established theory.

5 Exploratory Factor Analysis A book manuscript by Ledyard Tucker and Robert MacCallum

6 Factor analysis Principal Components Analysis (PCA) is generally used when the research purpose is data reduction (to reduce the information in many measured variables into a smaller set of components). Factor analysis is generally used when the research purpose is to identify latent variables which contribute to the common variance of the set of measured variables, excluding variable-specific (unique) variance.

7 Uses of factor analysis To reduce a large number of variables to a smaller number of factors for modelling purposes, To select a subset of variables from a larger set, (based on which original variables have the highest correlations with the principal component factors). To create a set of factors to be treated as uncorrelated variables as one approach to handling multicollinearity in such procedures as multiple regression To validate a scale or index by demonstrating that its constituent items load on the same factor, and to drop proposed scale items which cross-load on more than one factor. To establish that multiple tests measure the same factor, thereby giving justification for administering fewer tests. To identify clusters of cases and/or outliers. To determine network groups by determining which sets of people cluster together (using Q-mode factor analysis)

8 sampling adequacy Measured by the Kaiser-Meyer-Olkin (KMO) statistics, sampling adequacy predicts if data are likely to factor well, based on correlation and partial correlation. In the old days of manual factor analysis, this was extremely useful. KMO can still be used, however, to assess which variables to drop from the model because they are too multicollinear. There is a KMO statistic for each individual variable, and their sum is the KMO overall statistic. KMO overall should be.60 or higher to proceed with factor analysis. If it is not, drop the indicator variables with the lowest individual KMO statistic values, until KMO overall rises above.60.

9 Bartlett's test of sphericity Is an indicator of the strength of the relationship among variables Bartlett's test of sphericity is used to test the null hypothesis that the variables in the population correlation matrix are uncorrelated. It needs to be statistically significant (<0.05)

10 loadings Factor loadings: The factor loadings, also called component loadings in PCA, are the correlation coefficients between the variables (rows) and factors (columns). the squared factor loading is the percent of variance in that variable explained by the factor.

11 Communality, h 2, is the squared multiple correlation for the variable using the factors as predictors. The communality measures the percent of variance in a given variable explained by all the factors jointly and may be interpreted as the reliability of the indicator When an indicator variable has a low communality, the factor model is not working well for that indicator and possibly it should be removed from the model If the communality exceeds 1.0, there is a spurious solution, which may reflect too small a sample or the researcher has too many or too few factors.

12 Eigenvalues: Also called characteristic roots A factor's eigenvalue is the sum of its squared factor loadings for all the variables. eigenvalues measure the amount of variation in the total sample accounted for by each factor. eigenvalues associated with the unrotated and rotated solution will differ, though their total will be the same

13 Criteria for determining the number of factors Kaiser criterion: drop all components with eigenvalues under 1.0. Scree plot: drop all further components after the one eigenvalues start to elbow. Variance explained criteria: keeping enough factors to account for 90% (sometimes 80%) of the variation. Joliffe criterion: crop all components with eigenvalues under.7. Comprehensibility.

14 Rotation Methods. Rotation serves to make the output more understandable and is usually necessary to facilitate the interpretation of factors

15 Rotations No rotation The original, unrotated principal components solution maximizes the sum of squared factor loadings, efficiently creating a set of factors which explain as much of the variance in the original variables as possible. Varimax rotation is an orthogonal rotation of the factor axes to maximize the variance of the squared loadings of a factor (column) on all the variables (rows) in a factor matrix, which has the effect of differentiating the original variables by extracted factor. Quartimax rotation is an orthogonal alternative which minimizes the number of factors needed to explain each variable. Equimax rotation is a compromise between Varimax and Quartimax criteria. Direct oblimin rotation is the standard method when one wishes a nonorthogonal solution -- that is, one in which the factors are allowed to be correlated.. Promax rotation is an alternative non-orthogonal rotation method which is computationally faster than the direct oblimin method and therefore is sometimes used for very large datasets.

16 Varimax (orthogonal) rotation

17 Oblique rotation

18 How high does a factor loading have to be to consider that variable as a defining part of that factor minimum cut-off of.3 or.35. Norman and Streiner (1994: 139) formula for minimum loadings when the sample size, N, is 100 or more: Min FL = 5.152/[SQRT(N-2)]. loadings are "weak" if less than.4, "strong" if more than.6, and otherwise "moderate." Norman, G. R., and D. L. Streiner (1994). Biostatistics: The bare essentials. St. Louis, MO: Mosby.

19 How many cases do I need to do factor analysis? no scientific answer to this question, onlyarbitrary "rules of thumb," Rule of 10. There should be at least 10 cases for each item in the instrument being used. STV ratio. The subjects-to-variables ratio should be no lower than 5 (Bryant and Yarnold, 1995) Rule of 100: The number of subjects should be the larger of 5 times the number of variables, or 100. Even more subjects are needed when communalities are low and/or few variables load on each factor. (Hatcher, 1994) Rule of 150: Hutcheson and Sofroniou (1999) recommends at least cases, more toward the 150 end when there are a few highly correlated variables, as would be the case when collapsing highly multicollinear variables. Rule of 200. There should be at least 200 cases, regardless of STV (Gorsuch, 1983) Significance rule. There should be 51 more cases than the number of variables, to support chi-square testing (Lawley and Maxwell, 1971)

20 Confirmatory factor analysis (CFA) CFA seeks to determine if the number of factors and the loadings of measured (indicator) variables on them conform to what is expected on the basis of preestablished theory.

21 terminology Indicators are observed variables, sometimes called manifest variables or reference variables, such as items (questions) in a survey instrument Latent variables are the unobserved variables or constructs or factors which are measured by their respective indicators

22

23 terminology The measurement model. The measurement model is that part (possibly all like the CFA) of a SEM model which deals with the latent variables and their indicators. The null model. The measurement model is frequently used as the "null model," differences from which must be significant if a proposed structural model (the one with straight arrows connecting some latent variables) is to be investigated further. In the null model, the covariances in the covariance matrix for the latent variables are all assumed to be zero.

24

25 terminology Metric: In SEM, each unobserved latent variable must be assigned explicitly a metric, which is a measurement range. This is normally done by constraining one of the paths from the latent variable to one of its indicator (reference) variables, as by assigning the value of 1.0 to this path or by setting the factor variances to 1

26 terminology Error and disturbance terms. An error term refers to the measurement error factor associated with a given indicator. Measurement error terms are not to be confused with residual error terms, also called disturbance terms, which reflect the unexplained variance in the latent endogenous variable(s) due to all unmeasured causes

27

28 terminology Correlated error terms refers to situations in which knowing the residual of one indicator helps in knowing the residual associated with another indicator. Loadings: The latent variables in SEM are similar to factors in factor analysis, and the indicator variables likewise have loadings on their respective latent variables

29 Loadings are used to assess the reliability of the latent variables Construct reliability,(at least.70) reliability = [(SUM(sl i )) 2 ]/[(SUM(sli)) 2 + SUM(e i ))]. where sl i = standardized loadings e i =error term or Error= 1 (square of the indicator's standardized loading). Variance extracted (at least.50). Variance extracted = [(SUM(sl i2 )]/[(SUM(sl i2 ) + SUM(e i ))].

30 Types of estimation of coefficients in CFA Maximum likelihood estimation (MLE) (the most common method). MLE makes estimates based on maximizing the probability (likelihood) that the observed covariances are drawn from a population assumed to be the same as that reflected in the coefficient estimates. That is, MLE picks estimates which have the greatest chance of reproducing the observed data. It needs large samples; indicator variables with multivariate normal distribution; valid specification of the model; and continuous indicator variables

31 Other estimation methods GLS (generalized least squares) [2nd most popular method]. It minimizes the sum of the differences between observed and predicted covariances rather than between estimates and scores It needs very large samples (n>2500) even for non-normal data. Ordinary least squares (OLS). It makes estimates based on minimizing the sum of squared deviations of the linear estimates from the observed scores Weighted least squares (WLS) requires very large sample sizes (>2,000 in one simulation study) for dependable results Unweighted least squares (ULS) also focuses on the difference between observed and predicted covariances, but does not adjust for differences in the metric (scale) used to measure different variables, whereas GLS is scale-invariant, and is usually preferred for this reason Asymptotically distribution-free (ADF) estimation does not assume multivariate normality

32 Goodness of fit tests Goodness of fit tests determine if the model being tested should be accepted or rejected. Kline (1998: 130) recommends at least four tests, such as chi-square; GFI, NFI, or CFI; NNFI (or TLI); and SRMR. Another list of which-to-publish lists chi-square, AGFI and RMSEA. Kline, Rex B. (1998). Principles and practice of structural equation modeling. NY: Guilford Press.

33 Goodness-of-fit tests The chi-square value should not be significant if there is a good model fit, Goodness-of-fit index, GFI (Jöreskog-Sörbom GFI): >.90 Adjusted goodness-of-fit index, AGFI: >.90 Standardized root mean square residual, standardized RMR (SRMR): The smaller the standardized RMR, the better the model fit. 0 indicates perfect fit Centrality index, CI: >.90 Comparative fit index, CFI >.90 Normed fit index, NFI, >.90 for some >.80 Non-normed fit index, NNFI or (Tucker Lewis Index, TLI) >.95 others >.90 or >.80 Root mean square error of approximation, RMSEA good if <.05 adequate if <.08

34 Increasing model fit- Modification indices (MI) MI is often used to alter models to achieve better fit, but this must be done carefully and with theoretical justification. In MI, improvement in fit is measured by a reduction in chi-square (recall a finding of chi-square significance corresponds to rejecting the model as one which fits the data).

35 Sample size In the literature, sample sizes commonly run for models with indicators. median sample size (in 72 SEM studies) was 198. Sample size should be at least 50 more than 8 times the number of variables in the model. Stevens (1996), is to have at least 15 cases per measured variable or indicator. Bentler and Chou (1987) recommend at least 5 cases per parameter estimate (including error terms as well as path coefficients). Bentler, P. M. and C. P. Chou (1987). Practical issues in structural modelling. Sociological Methods and Research. 16(1): Stevens, J. (1996). Applied multivariate statistics for the social sciences.