The European Commission s science and knowledge service. Joint Research Centre

Size: px

Start display at page:

Download "The European Commission s science and knowledge service. Joint Research Centre"

Ralph Buck Carpenter
5 years ago
Views:

1 The Euroean Commission s science and knowledge service Joint Research Centre

2 Ste 7: Statistical coherence (II) PCA, Exloratory Factor Analysis, Cronbach alha Hedvig Norlén COIN th JRC Annual Training on Comosite Indicators & Scoreboards 06-08/11/2017, Isra (IT)

Ste 7: Statistical coherence (II) The CI measures a multifaceted henomenon - a combination of different asects (SUB-PILLARS/PILLARS) (all related to the henomenon but different from each other)

3 Ste 7: Statistical coherence (II) The CI measures a multifaceted henomenon - a combination of different asects (SUB-PILLARS/PILLARS) (all related to the henomenon but different from each other) Each asect can be measured by a set of observable variables (INDICATORS) Framework of the Global Innovation Index 2017 CI PILLARS SUB-PILLARS INDICATORS 3 JRC-COIN Ste 7: Statistical Coherence (II)

4 Ste 7: Statistical coherence (II) CI PCA, FA are used to verify the internal consistency within each SUB-PILLAR (across INDICATORS) PILLARS SUB-PILLARS INDICATORS Framework of the Global Innovation Index JRC-COIN Ste 7: Statistical Coherence (II)

5 Ste 7: Statistical coherence (II) CI PCA, FA are used to verify the internal consistency within each SUB-PILLAR (across INDICATORS) within each PILLAR (across SUB-PILLARS) PILLARS SUB-PILLARS INDICATORS Framework of the Global Innovation Index JRC-COIN Ste 7: Statistical Coherence (II)

6 Ste 7: Statistical coherence (II) CI PCA, FA are used to verify the internal consistency within each SUB-PILLAR (across INDICATORS) within each PILLAR (across SUB-PILLARS) within the CI (across PILLARS) Search for 1-DIMENSIONALITY within each subset Framework of the Global Innovation Index 2017 PILLARS SUB-PILLARS INDICATORS 6 JRC-COIN Ste 7: Statistical Coherence (II)

7 Ste 7: Statistical coherence (II) Want to answer the following questions by using a multivariate method (PCA, FA, ) Is the hierarchical structure of our comosite indicator statistically well-defined? Is the set of available indicators sufficient/aroriate to describe the illars (dimensions) and the sub-illars (sub-dimensions) of the comosite indicator? 7 JRC-COIN Ste 7: Statistical Coherence (II)

8 Multivariate data The tye of analysis to be erformed deends on the data of the indicators Data Indicators Quantitative Qualitative Mixed/ structured PCA/FA PCA Princial Comonent Analysis FA Factor Analysis dispca/fa,(m)ca FAMD/MFA PCA for discrete data (based on olychoric correlations) (M)CA (Multile) Corresondence Analysis. FAMD Factor Analysis of Mixed Data MFA Multile Factor Analysis. 8 JRC-COIN Ste 7: Statistical Coherence (II)

9 Data matrix of indicators x11 x12... x1 x21 x22... x2 X =.. xn 1xn2... x n n = number of objects (countries, ) = number of variables (indicators) The revious stes in building our CI - Ste 3 Outlier detection and missing data estimation - Ste 4 Normalisation/Standardisation X correlated data set (without outliers and (without or a few) missing values) PCA, FA are based on correlations 9 JRC-COIN Ste 7: Statistical Coherence (II)

10 Exloratory FA Two main tyes of FA: Exloratory (EFA) and Confirmatory (CFA) EFA examining the relationshis among the variables/indicators and do not have an à riori fixed number of factors CFA a clear idea about the number of factors you will encounter, and about which variables will most likely load onto each factor 10 JRC-COIN Ste 7: Statistical Coherence (II)

11 Is PCA = Exloratory FA? PCA FA Similar in many ways but an imortant difference that has effects on how to use them 1) Both are data reduction techniques - they allow you to cature the variance in variables in a smaller set 2) Same (or similar) rocedures used to run in many software (same stes extraction, interretation, rotation, choosing the number of comonents/factors) 11 JRC-COIN Ste 7: Statistical Coherence (II)

12 Is PCA = Exloratory FA? NO! PCA FA Similar in many ways but an imortant difference that has effects on how to use them 1) Both are data reduction techniques - they allow you to cature the variance in variables in a smaller set 2) Same (or similar) rocedures used to run in many software (same stes extraction, interretation, choosing the number of comonents/factors) Difference: PCA is a linear combination of variables FA is a measurement model of a latent variable (statistical model) 12 JRC-COIN Ste 7: Statistical Coherence (II)

13 Is PCA = Exloratory FA? NO! PCA Observed indicators are reduced into comonents FA Latent factors drive the observed variables Item 1 X 1 w 1 λ 1 Item 1 X 1 ε 1 Item 2 X 2... w 2 Comonent Factor λ 2 Item 2 X 2... ε 2 ε i Error terms Item X w λ Item X ε PCA summarizes information of all indicators and reduces it into a fewer number of comonents Each PC i is a new variable comuted as a linear combination of the original variables PC = w x + w x w x Model of the measurement of a latent variable. This latent variable cannot be directly measured with a single variable. Seen through the relationshis it causes in a set of X variables. X X X 1 i = λ F + ε = λif + ε 1 i = λ F + ε 1 13 JRC-COIN Ste 7: Statistical Coherence (II)

Some historical notes PCA FA PCA robably the oldest of multivariate analysis methods.

calculations become cumbersome for four or more variables he suggests that they are still quite feasible

variances of the original variables, rincial comonents FA was develoed by Searman (1904) in the field of

14 Some historical notes PCA FA PCA robably the oldest of multivariate analysis methods. Pearson (1901) said that his methods can be easily alied to numerical roblems and although he says that the calculations become cumbersome for four or more variables he suggests that they are still quite feasible Hotelling (1933) chose his comonents so as to maximize their successive contributions to the total of the variances of the original variables, rincial comonents FA was develoed by Searman (1904) in the field of sychology (discovered that school children's scores on a wide variety of seemingly unrelated subjects were ositively correlated). Cattell (1952) exanded on Searman's idea of a two-factor theory of intelligence FA - Charles Edward Searman ( ) FA - Raymond Bernard Cattell ( ) PCA - Karl Pearson ( ) PCA - Harold Hotelling ( ) 14 JRC-COIN Ste 7: Statistical Coherence (II)

15 First stes in PCA FA Check the correlation structure of the data and do 1) and 2) 1) Bartlett s shericity test The Bartlett s test checks if the observed correlation matrix R diverges significantly from the identity matrix. H 0 : R = 1, H 1 : R 1 Test statistic 2 χ = n 1 6 log Under H 0, it follows a χ² distribution with a (-1)/2 degrees of freedom. R n = samle size = number of indicators Want to reject H 0 to be able to do PCA,FA! (check n/<5) (Bartlett (1937)) 15 JRC-COIN Ste 7: Statistical Coherence (II)

16 First stes in PCA FA Check the correlation structure of the data and do 1) and 2) 2) Kaiser-Meyer-Olkin (KMO) Measure for Samling Adequacy Indicators are more or less correlated, but the correlation between two indicators can be influenced by the others. Use artial correlation in order to measure the relation between two indicators by removing the effect of the remaining indicators (controlling for the confounding variable) KMO 2 rij 2 r + ij = 2 aij r ij Pearson correlation coefficient a ij artial correlation coefficient Want KMO close to 1 to be able to do PCA,FA! KMO>0.6 ok (Kaiser (1970), Kaiser-Meyer (1974)) 16 JRC-COIN Ste 7: Statistical Coherence (II)

17 First stes in PCA FA Check the correlation structure of the data and do 1) and 2) 2) Kaiser-Meyer-Olkin (KMO) Measure for Samling Adequacy Kaiser ut the following values on the results: 0.00 to 0.49 unaccetable 0.50 to 0.59 miserable 0.60 to 0.69 mediocre 0.70 to 0.79 middling 0.80 to 0.89 meritorious 0.90 to 1.00 marvelous 1) Bartlett s shericity test and 2) KMO test rovide info whether it is ossible to do PCA, FA but do not give info of the magic number - how comonents/factors are needed 17 JRC-COIN Ste 7: Statistical Coherence (II)

18 How to get the PCs Eigenvalue decomosition of a data correlation matrix (case of CI), (or covariance matrix) Singular Value Decomosition (SVD) of a data matrix after mean centering (normalizing) the data matrix 18 JRC-COIN Ste 7: Statistical Coherence (II)

19 Finding the magic number - determining how many comonents in PCA and factors in FA Many methods exist. The 3 most common are: 1) Kaiser Guttman Eigenvalues greater than one criterion (Guttman (1954), Kaiser (1960)). Select all comonents with eigenvalues over 1 (or 0.9) 2) Cattell s scree test (Cattell (1966)) Above the elbow aroach Scree 3) Certain ercentage of exlained variance e.g., >2/3, 75%, 80%, 19 JRC-COIN Ste 7: Statistical Coherence (II)

20 Examle PCA: The Global Talent Cometitiveness Index 2017 (GTCI 2017) 1. Enable Use PCA to exlore the dimensionality of the first sub-illar Regulatory Landscae The 5 variables should be correlated. Suorts the assumtion that they are all measuring the same concet! 20 JRC-COIN Ste 7: Statistical Coherence (II)

GTCI 2017 Sub-illar Regulatory Landscae 1) Bartlett s shericity test χ² - test statistic 588, df= (-1)/2=10 -value <<< 0.01 Reject H 0 2) KMO Measure for Samling Adequacy Overall MSA = 0.86 KMO>0.

21 GTCI 2017 Sub-illar Regulatory Landscae 1) Bartlett s shericity test χ² - test statistic 588, df= (-1)/2=10 -value <<< 0.01 Reject H 0 2) KMO Measure for Samling Adequacy Overall MSA = 0.86 KMO>0.6 Check the correlation structure MSA for each variable Government Effectiveness = 0.79 Business-Government Relations = 0.96 Political Stability = 0.94 Regulatory Quality = 0.87 Corrution = JRC-COIN Ste 7: Statistical Coherence (II)

22 GTCI 2017 Sub-illar Regulatory Landscae PCA results Total variance exlained Comonent Eigenvalue Variance Cumulative variance 3,85 = 0, , , , ,95 2 Pearson correlation coefficients between illar and rincial comonent 0,77 = 3,85/5 PC1 PC2 PC3 PC4 PC5 1 3,85 0,77 0,77 1. Government Effectiveness 0,96-0,09-0,17-0,06-0,2 2 0,65 0,13 0,90 2. Business-Government Relations 0,67 0,73 0,14 0,00-0,01 3 0,31 0,06 0,96 3. Political Stability 0,84-0,31 0,44 0,06-0,01 4 0,12 0,02 0,99 4. Regulatory Quality 0,94-0,05-0,23 0,24 0,08 5 0,06 0,01 1,00 5. Corrution 0,95-0,10-0,09-0,24 0,13 Sum 5 1 SS 3,85 0,65 0,31 0,12 0,06 Stoing criterion Eigenvalue>0,90 (or cumulative variance over 0,70) Comonent loadings = 0.96X + GE 0.67X + BGR 0.84X + 1 PS 0.94X + RQ PC 0. 95X C One dimension verified! 22 JRC-COIN Ste 7: Statistical Coherence (II)

23 Examle 2 PCA: The Global Talent Cometitiveness Index 2017 (GTCI 2017) 5. VT Skills Use PCA to exlore the dimensionality of the sub-illar Emloyability 4 variables correlated? 1. Ease of finding skilled emloyees 2. Rel of education system to the eco 3. Availability of scientists and engineers 4. Skills ga as major constraint 23 JRC-COIN Ste 7: Statistical Coherence (II)

GTCI 2017 Sub-illar Emloyability 1) Bartlett s shericity test χ² - test statistic 264, df= (-1)/2=6 -value <<< 0.01 Reject H 0 2) KMO Measure for Samling Adequacy Overall MSA = 0.73 KMO>0.

24 GTCI 2017 Sub-illar Emloyability 1) Bartlett s shericity test χ² - test statistic 264, df= (-1)/2=6 -value <<< 0.01 Reject H 0 2) KMO Measure for Samling Adequacy Overall MSA = 0.73 KMO>0.6 Check the correlation structure MSA for each variable 1. Ease of finding skilled emloyees = Relev of education system to the eco = Availability of scientists and engineers = Skills ga as major constraint = JRC-COIN Ste 7: Statistical Coherence (II)

25 GTCI 2017 Sub-illar Emloyability PCA results Total variance exlained Comonent Eigenvalue Variance Cumulative variance Pearson correlation coefficients between illar and rincial comonent PC1 PC2 PC3 PC4 1 2,66 0,66 0,66 Ind 1 0,91-0,28-0,08 0,30 2 0,91 0,23 0,89 Ind 2 0,90-0,09 0,41-0,12 3 0,28 0,07 0,96 Ind 3 0,92-0,04-0,33-0,21 4 0,15 0,04 1,00 Ind 4 0,42 0,91 0,00 0,07 SS 2,66 0,91 0,28 0,15 Stoing criterion Eigenvalue>0,90 (or cumulative variance over 0,70) PC PC 1 2 = 0,91X + 0,90X + 0,92X + 0,42X 2 PCs retrieved but Indicator 4 Skills 1 = 0.28X X 2 3 0,04X ,91X 4 ga as major constraint correlates considerably better with the PC2 25 JRC-COIN Ste 7: Statistical Coherence (II)

GTCI 2017 Sub-illar Emloyability PCA results Indicator 4 Skills ga as major constraint roblematic JRC Audit reort 2017: In the 2017 data set, the indicator Skills ga as major constraint has a very

26 GTCI 2017 Sub-illar Emloyability PCA results Indicator 4 Skills ga as major constraint roblematic JRC Audit reort 2017: In the 2017 data set, the indicator Skills ga as major constraint has a very low imact on the GTCI (Index) not found to be imortant at the overall index level. It is suggested that the GTCI develoment team reconsider the inclusion of this variable (or relace it with other variable/s) in next year s release. Correlation structure Indicators, Sub-illar, Pillar and Index GTCI 2018 reort (to be ublished, January 2018): Indicator Skills ga as a major constraint has been deleted entirely from the framework, as ointed out by the JRC last year. 26 JRC-COIN Ste 7: Statistical Coherence (II)

27 Cronbach s α Cronbach s α is measure of internal consistency and reliability Based uon classical test theory. Most common estimate of reliability, Cronbach (1951) Cronbach s α is defined as Ratio of the Total Covariance in the test to the Total Variance in the test. By test we mean the set of indicators constituting a sub-illar/ illar/ci. Let X ) be the variance of indicator i and Var X total variance in sub-illar/illar/ci Var ( ) ( i i = 1 Average covariance of an indicator with any other indicator is α = ( Var( X ) ) i= i Var( X i= i ) Var( X i )/ i= 1 / ( 1) = Var 1 i Var ( X i ) i= 1 i= 1 ( 1) ( X ) i= i Var( X i= i ) 1 1 ( ) Var X i= 1 i Var( X i ) = number of indicators 27 JRC-COIN Ste 7: Statistical Coherence (II)

28 Cronbach s α Cronbach s α is measure of internal consistency and reliability α = 1 1 i= 1 Var Var( X ) ( ) X i= 1 i i = number of indicators α coefficient ranges from 0 to 1. If all indicators are entirely indeendent from one another (i.e. are not correlated or share no covariance) α = 0. If all of the items have high covariances, then α will aroach 1. Caution! α increases when the number of indicators increases Cronbach (1951) 28 JRC-COIN Ste 7: Statistical Coherence (II)

29 Cronbach s α cut off values Cronbach s α is measure of internal consistency and reliability α = 1 1 i= 1 Var Var( X ) ( ) X i= 1 i i Cronbach's alha Internal consistency 0.9 α Excellent 0.8 α < 0.9 Good 0.7 α < 0.8 Accetable 0.6 α < 0.7 Questionable 0.5 α < 0.6 Poor α < 0.5 Unaccetable 29 JRC-COIN Ste 7: Statistical Coherence (II)

30 Cronbach s α GTCI 2017 Sub-illar Regulatory Landscae Cronbach s α is measure of internal consistency and reliability GTCI Sub-illar "Regulatory Landscae" Indicator Cronbach's Alha Corrected Item-Total Correlation 0,92 Cronbach s Alha if indicator is droed Ind 1. Government Effectiveness 0,93 0,88 Ind 2. Business- Government Relations 0,55 0,95 Ind 3. Political Stability 0,75 0,92 Ind 4. Regulatory Quality 0,89 0,89 Ind 5. Corrution 0,92 0,88 Cronbach's alha Internal consistency 0.9 α Excellent 0.8 α < 0.9 Good 0.7 α < 0.8 Accetable 0.6 α < 0.7 Questionable 0.5 α < 0.6 Poor α < 0.5 Unaccetable 30 JRC-COIN Ste 7: Statistical Coherence (II)

31 Cronbach s α GTCI 2017 Sub-illar Regulatory Landscae Cronbach s α is measure of internal consistency and reliability "Emloyability" GTCI Sub-illar Indicator Cronbach's Alha Corrected Item-Total Correlation 0,73 Cronbach s Alha if indicator is droed Ind 1. Ease of finding skilled emloyees 0,68 0,62 Ind 2. Relev of educ system to the economy 0,67 0,58 Ind 3. Availability of scientists and engineers 0,75 0,58 Ind 4. Skills ga as major constraint 0,23 0,90 Cronbach's alha Internal consistency 0.9 α Excellent 0.8 α < 0.9 Good 0.7 α < 0.8 Accetable 0.6 α < 0.7 Questionable 0.5 α < 0.6 Poor α < 0.5 Unaccetable Ind 4 roblematic and Cronbach s α may hence be used as another measure to include/exclude indicators! 31 JRC-COIN Ste 7: Statistical Coherence (II)

32 Last... PCA for discrete data A ractically imortant violation of the normality assumtion underlying the PCA occurs when the data are discrete! Several kinds of discrete data (binary, ordinal, count, nominal ) Different ways on how to roceed: 1) Simle aroach, use the discrete x s as if they were continuous in the PCA but instead of using the Pearson moment correlation coefficients use Searman or Kendall rank correlation coefficients. 2) PCA using olychoric correlations. Pearson (1901) founder of this aroach (as well!), develoed further by Olsson (1979) and Jöreskog (2004) 32 JRC-COIN Ste 7: Statistical Coherence (II)

33 Last... PCA using olychoric correlations The olychoric correlation (PCC) coefficient is a measure of association for ordinal variables which rests uon an assumtion of an underlying joint continuous normal distribution. (Later generalizations of different continuous distributions). PCC coefficient is MLE of the roduct-moment correlation between the underlying normal variables 33 JRC-COIN Ste 7: Statistical Coherence (II)

34 References Books Johnson, Alied Multivariate Statistical Analysis (6th Revised edition), Pearson Education Limited (2013) Jolliffe, Princial Comonent Analysis, Second Edition, Sringer (2002) Leandre, Fabrigar, Duane, Exloratory Factor Analysis, Oxford (2011) Rencher, Methods of multivariate analysis, Wiley (1995) Tinsley and Brown, Handbook of Alied Multivariate Statistics and Mathematical Modeling, Academic Press (2000) Tucker, MacCallum, Exloratory Factor Analysis (1997) 34 JRC-COIN Ste 7: Statistical Coherence (II)

THANK YOU Any questions? Welcome to email us at: jrc-coin@ec.euroa.eu You may contact us at @username & user@mail.

35 THANK YOU Any questions? Welcome to us at: You may contact us & The Euroean Commission s Cometence Centre on Comosite Indicators and Scoreboards COIN in the EU Science Hub htts://ec.euroa.eu/jrc/en/coin COIN tools are available at: htts://comosite-indicators.jrc.ec.euroa.eu/

The European Commission s science and knowledge service. Joint Research Centre

The European Commission s science and knowledge service Joint Research Centre Step 7: Statistical coherence (II) Principal Component Analysis and Reliability Analysis Hedvig Norlén COIN 2018-16th JRC Annual