Factor Analysis Edpsy/Soc 584 & Psych 594

Transcription

1 Factor Analysis Edpsy/Soc 584 & Psych 594 Carolyn J. Anderson University of Illinois, Urbana-Champaign April 29, / 52

2 Rotation Assessing Fit to Data (one common factor model) common factors Assessment of Fit to Data Rotation Confirmatory Factor Analysis 2 / 52

3 : Rotation Assessing Fit to Data Explore underlying structure data driven. Confirm underlying structure theory driven. method (Factor extraction) Eigen-decomposition based Eigen-decomposition of S which is the principal components solution. Iterative eigen-decompositions of S Ψ which is the Principal factor solution. Maximum likelihood estimation We now must assume that F and ɛ are multivariate normal. Tends to fit data better & yields scale invariance (ie., use either S or R). 3 / 52

4 Rotation Assessing Fit to Data Data from Espelage, D.L., Holt, M.K., & Henkel, R.R. (2003). Examination of Peer-Group contextual effects on aggression during early adolescence. Child Development, 74, Items have a 5 point response scale: Never, 1 to 2 times, 3 to 4 times, 5 to 6 times, 7 or more times Q36: I upset other students for the fun of it. Q37: In a group I teased other students. Q38: I fought students I could easily beat. Q39: Other students picked on me. Q40: Other students made fun of me. Q41: Other students called me names. Q42: I got hit and pushed by other students. Q43: I helped harass other students. 4 / 52

5 Rotation Assessing Fit to Data Items have a 5 point response scale: Never, 1 to 2 times,..., 7 or more times Q44: I teased other students. Q45: I got in a physical fight. Q46: I threatened to hurt or hit another student. Q47: I got into a physical fight because I was angry. Q48: I hit back when someone hit me first. Q49: I was mean to someone when I was angry. Q50: I spread rumors about other students. Q51: I started (instigated) arguments or conflicts. Q52: I encouraged people to fight. Q53: I excluded other students from my clique (group) of friends 5 / 52

6 Rotation Assessing Fit to Data What are the factors and which items load on which factors? Item F 1 F 2 F 3 Item F 1 F 2 F / 52

7 Rotation Assessing Fit to Data α here is Chronbach s alpha for measuring reliability. F 1 = Fighting (α =.88), F 2 = Bullying (α =.88), F 3 = Victimization (α =.87) Item F 1 F 2 F 3 Item F 1 F 2 F 3 36 X 45 X 37 X 46 X 38 X 47 X 39 X 48 X 40 X 49 X 41 X 50 X 42 X 51 X 43 X 52 X 44 X 53 X 7 / 52

8 Factor Analytic The One Common Factor Analytic Unique Variables Unique Variables & Common Variables Implications for Data Implications for Data (continued) The Data is Covariance Matrix Victim Statistics One Factor Solution Rotation 8 / 52

9 Factor Analytic Factor Analytic The One Common Factor Analytic Unique Variables Unique Variables & Common Variables Implications for Data Implications for Data (continued) The Data is Covariance Matrix Victim Statistics One Factor Solution Factor analysis (FA) is a latent variable model. ɛ 1 X 1 ɛ 2 X 2 ɛ 3 X 3 ɛ 4 X 4 l 21 l 11 l 31 l 41 F 1 The One Common Factor X 1 = µ 1 + l 11 F 1 + ɛ 1 X 2 = µ 2 + l 21 F 1 + ɛ 2 X 3 = µ 3 + l 31 F 1 + ɛ 3 X 4 = µ 4 + l 41 F 1 + ɛ 4 Rotation 9 / 52

10 The One Common Factor Analytic Factor Analytic The One Common Factor Analytic Unique Variables Unique Variables & Common Variables Implications for Data Implications for Data (continued) The Data is Covariance Matrix Victim Statistics One Factor Solution X i = µ i + l i1 F 1 + ɛ i The µ i s are the means of the X i s. Common Factor: F 1 is an unobserved random variable with mean 0 and variance φ. The l i1 s are the Factor Loadings. Specificities or Uniquenesses: ɛ i are independent over i, unobserved random variables with means equal to 0 and variances equal to ψ i. F 1 is independent of the ɛ i s. Rotation 10 / 52

11 Factor Analytic The One Common Factor Analytic Unique Variables Unique Variables & Common Variables Implications for Data Implications for Data (continued) The Data is Covariance Matrix Victim Statistics One Factor Solution Unique Variables A little Classical test theory: true score = observed score pure measurement error t i = X i e i If a factor model holds for the observed scores X i, then it should also hold for the true scores t i. The uniqueness in the factor model for the true scores contains specific errors due to the particular variables (items) selected; that is, Solve for X i : t i = l i1 F 1 + s i }{{} specif ic t i = l i1 F 1 + s i = X i e i = X i = l i1 F i + (s i + e i ) }{{} ɛ i Rotation 11 / 52

12 Unique Variables & Common Variables The unique variables ɛ i contain Pure measurement errors Specific errors Factor Analytic The One Common Factor Analytic Unique Variables Unique Variables & Common Variables Implications for Data Implications for Data (continued) The Data is Covariance Matrix Victim Statistics One Factor Solution The observed variables will be correlated because they all depend (in part) on F 1. The common factor model makes implications for covariance (and correlation) matrices; therefore, the data are the covariance (or correlation) matrix. Rotation 12 / 52

13 Factor Analytic The One Common Factor Analytic Unique Variables Unique Variables & Common Variables Implications for Data Implications for Data (continued) The Data is Covariance Matrix Victim Statistics One Factor Solution Implications for Data Let s turn it into a linear algebra problem: X 1 µ 1 X 2 µ 2 X =. µ =... Σ = X p L = l 11 l 21. l p1 Rotation 13 / 52 µ p ɛ = Ψ = So X µ = LF 1 + ɛ ɛ 1 ɛ 2.. ɛ p ψ ψ ψ p σ 11 σ σ 1p σ 12 σ σ 2p σ 1p σ 2p... σ pp var(f) = Φ = φ 11

14 Implications for Data (continued) Factor Analytic The One Common Factor Analytic Unique Variables Unique Variables & Common Variables Implications for Data Implications for Data (continued) The Data is Covariance Matrix Victim Statistics One Factor Solution X µ = LF 1 + ɛ Mean of X µ = LE(F 1 ) + E(ɛ) = 0 Covariance matrix for X or equivalently X µ: Σ = E[(X µ)(x µ) ] = E[(LF 1 + ɛ)(lf 1 ɛ) ] = E[(LF 1 + ɛ)(f 1 L ɛ )] = L E[F1 2 ] L + L E[F 1 ɛ ] +E[ɛF 1 ] L + E[ɛɛ ] }{{}}{{}}{{}}{{} φ 11 =1 0 0 Ψ = LL + Ψ Rotation 14 / 52

15 Factor Analytic The One Common Factor Analytic Unique Variables Unique Variables & Common Variables Implications for Data Implications for Data (continued) The Data is Covariance Matrix Victim Statistics One Factor Solution The Data is Covariance Matrix One common factor model implies : Σ = l 2 i1 = h2 i (l ψ 1) l 11 l l 11 l p1 l 11 l 21 (l ψ 2)... l 21 l p l 11 l I1 l 21 l I1... (l 2 I1 + ψ p) is the Communality of item i. Foreshadowing estimation methods: If Ψ were known, or Σ = (e 1 Σ Ψ = LL = (e 1 λ1 )( λ 1 e 1) λ 1 ) } {{ } L ( λ 1 e 1 ) } {{ } L Rotation 15 / 52 and Ψ = diag(σ L L )

16 Victim Statistics Factor Analytic The One Common Factor Analytic Unique Variables Unique Variables & Common Variables Implications for Data Implications for Data (continued) The Data is Covariance Matrix Victim Statistics One Factor Solution Descriptives Correlations Item Mean Std Dev q39 q40 q41 q42 q q q q Cronbach Coefficient Alpha with Deleted Variable Raw Variables Std Variables Correlation Correlation Deleted Variable with Total Alpha with Total Al q39: other students picked on me q40: students made fun of me q41: students called me names q42: got hit and pushed Rotation 16 / 52

17 One Factor Solution Factor Analytic The One Common Factor Analytic Unique Variables Unique Variables & Common Variables Implications for Data Implications for Data (continued) The Data is Covariance Matrix Victim Statistics One Factor Solution Factor Pattern Item Factor1 Communalities Specific l i1 h 2 i var: ψ i q39 other students picked on me q40 students made fun of me q41 students called me names q42 got hit and pushed (maximum likelihood estimation using the correlation matrix) Residual Correlation with Uniqueness ( ˆψ i ) on the diagonal: q39 q40 q41 q42 q q q q Root Mean Square Off-Diagonal Residuals: Overall = Rotation 17 / 52

18 A Closer Look at Implied Covariance Matrix Summary of Terminology and Components & Victim s Plot of Factor 18 / 52

19 Picture of this for p = 4 and m = 2. ɛ 1 X 1 ɛ 2 X 2 F 1 A Closer Look at Implied Covariance Matrix Summary of Terminology and Components & Victim s Plot of Factor ɛ 3 ɛ 4 X 3 X 4 F 2 19 / 52

20 A Closer Look at Implied Covariance Matrix Summary of Terminology and Components & Victim s Plot of Factor X 1 = µ 1 + l 11 F 1 + l 12 F l 1m F m + ɛ 1 X 2 = µ 2 + l 21 F 1 + l 22 F l 2m F m + ɛ 2.. X p = µ p + l p1 F 1 + l p2 F l pm F m + ɛ p X = µ + LF + ɛ E(F) = 0 and cov(f) = Φ = I... for now E(ɛ) = 0 and cov(ɛ) = Ψ = diag(ψ i ) cov(f,ɛ) = 0 L is matrix of Factor Loadings m q=1 l 2 iq = h2 i =Communality of item i. ψ i is Specific variance of item i. 20 / 52

21 A Closer Look at Implied Covariance Matrix Summary of Terminology and Components & Victim s Plot of Factor Using the model for i = 1,..., I observed variables and m common factors, the covariance matrix is X µ = LF + ɛ Mean of X µ= E(LF + ɛ) = LE(F) + E(ɛ) = 0 Covariance matrix for X or equivalently X µ: Σ = E[(X µ)(x µ) ] = E[(LF + ɛ)(lf ɛ) ] = E[(LF + ɛ)(f L ɛ )] = L E[FF ] }{{} I = LL + Ψ L + L E[F1ɛ ] +E[ɛF ] }{{}}{{} 0 I L + E[ɛɛ ] }{{} Ψ 21 / 52

22 The Common factor model implies : Σ = ( q l2 1q + ψ 1) q l 1ql 2q... q l 1ql pq q l 1ql 2q ( q l2 2q + ψ 2)... q l 2ql pq q l 1q l pq q l 2q l pq... ( q l 2 pq + ψ I ) A Closer Look at Implied Covariance Matrix Summary of Terminology and Components & Victim s Plot of Factor Variance for X i : σ ii = m q=1 l 2 im }{{} h 2 i +ψ i Covariance between X i and X k : m σ ik = l iq l kq q=1 22 / 52

23 A Closer Look at Implied Covariance Matrix A Closer Look at Implied Covariance Matrix Summary of Terminology and Components & Victim s Plot of Factor Covariance between X and F: cov(x, F) = cov((x µ), F) = E[(X µ)(f 0) ] = E[(LF + ɛ)(f) ] = LE(FF ) + E(ɛF ) }{{}}{{} I 0 = L = {l iq } p q The correlation between observed variables and the common factors equal cov(x i, F q ) l = iq h 2 i + ψ i 1 h 2 i + ψ i The correlations ρ(x i, F q ) are called Structure Coefficients. 23 / 52

24 Summary of Terminology and Components A Closer Look at Implied Covariance Matrix Summary of Terminology and Components & Victim s Plot of Factor F q are Common factors (latent variables). l iq are Factor Loadings. ɛ i are latent variables specific or Unique to item i. ψ i are the unique variance or Specifcities. h 2 i = m q=1 l iq are the common variance or Communalities. Correlation between X i and F q are Structure Coefficients: ρ(x i, F q ) = l iq mq l 2 iq + ψ i = l iq h 2 i + ψ i 24 / 52

25 A Closer Look at Implied Covariance Matrix Summary of Terminology and Components & Victim s Plot of Factor Basic Descriptive Statistics with a little item analysis: Corr Variable Mean Std Dev w/ Total Alph q36 upset students for fun q37 in group teased students q43 helped harass students q44 teased other students q49 mean to someone when angry q50 spread rumors q51 started arguments or conflicts q52 encouraged people to fight q53 excluded students from clique / 52

26 A Closer Look at Implied Covariance Matrix Summary of Terminology and Components & Victim s Plot of Factor Correlations: q36 q37 q43 q44 q49 q50 q51 q52 q53 q q q q q q q q q / 52

27 & Victim s A Closer Look at Implied Covariance Matrix Summary of Terminology and Components & Victim s Plot of Factor Correlations: q36 q37 q43 q44 q49 q50 q51 q52 q53 q39 q40 q What do you notice and what does this imply? 27 / 52

28 Plot of Factor Loadings A Closer Look at Implied Covariance Matrix Summary of Terminology and Components & Victim s Plot of Factor 28 / 52

29 A Closer Look at Implied Covariance Matrix Summary of Terminology and Components & Victim s Plot of Factor Factor Loadings Item Factor 1 Factor 2 Bully items q36 upset students for fun q37 in group teased students q43 helped harass students q44 teased other students q49 mean to someone when angry q50 spread rumors q51 started arguments or conflicts q52 encouraged people to fight q53 excluded students from clique Victim items q39 other students picked on me q40 students made fun of me q41 students called me names q42 got hit and pushed / 52

30 Residual Correlation Matrix w/ Uniqueness on Diagonal A Closer Look at Implied Covariance Matrix Summary of Terminology and Components & Victim s Plot of Factor q36 q37 q43 q44 q49 q50 q51 q52 q53 q q41 q42 q q Root Mean Square Off-Diagonal Residuals: Overall = / 52

31 Example Two: Fight A Closer Look at Implied Covariance Matrix Summary of Terminology and Components & Victim s Plot of Factor Deleted Correlation Variable with Total Alpha Label q fought students could beat q got in physical fight q threatened to hurt or hit student q physical fight because angry q hit back when hit first q38 q45 q46 q47 q38 fought students could beat q45 got in physical fight q46 threatened to hurt or hit student q47 physical fight because angry q48 hit back when hit first / 52

32 Correlations between Fight and Bully A Closer Look at Implied Covariance Matrix Summary of Terminology and Components & Victim s Plot of Factor Fight Bully q38 q45 q46 q47 q48 q q q q q q q q q Test of H o : Σ bully,fight = 0 versus H o : Σ bully,fight 0 F(45, 1367) = 8.72, p <.01, canonical correlation = / 52

33 Bully/Fight Factor Structure A Closer Look at Implied Covariance Matrix Summary of Terminology and Components & Victim s Plot of Factor 33 / 52

34 A Closer Look at Implied Covariance Matrix Summary of Terminology and Components & Victim s Plot of Factor Bully/Fight Factor Pattern Item Factor 1 Factor 2 Bully q Bully q Bully q Bully q Bully q Bully q Bully q Bully q Bully q Fight q Fight q Fight q Fight q Fight q Root Mean Square Off-Diagonal Residuals: Overall = / 52

35 Rotation Rotation Factor Pattern not Unique Desirable Structure Orthogonal Rotations Varimax for Victim & Bully Items Varimax Bully/Fight Factor Pattern Desirable (Oblique) Structure with Correlated Factors Oblique Rotation: Bully/Fight Factor Pattern Assessing Fit 35 / 52

36 Factor Pattern not Unique Two kinds: Rotation Factor Pattern not Unique Desirable Structure Orthogonal Rotations Varimax for Victim & Bully Items Varimax Bully/Fight Factor Pattern Desirable (Oblique) Structure with Correlated Factors Oblique Rotation: Bully/Fight Factor Pattern Assessing Fit Orthogonal Oblique Orthogonal: Let T be an orthogonal matrix such that Therefore, TT = T T = I Σ = LL + Ψ = LTT }{{} L + Ψ I = L L + Ψ Still keeping cov(f) = I and the model fit to data remains the same. 36 / 52

37 Desirable Structure for easier interpretation: ɛ 1 X 1 ɛ 2 X 2 F 1 Rotation Factor Pattern not Unique Desirable Structure Orthogonal Rotations Varimax for Victim & Bully Items Varimax Bully/Fight Factor Pattern Desirable (Oblique) Structure with Correlated Factors Oblique Rotation: Bully/Fight Factor Pattern ɛ 3 ɛ 4 X 3 X 4 F 2 Assessing Fit 37 / 52

38 Orthogonal Rotations Rotation Factor Pattern not Unique Desirable Structure Orthogonal Rotations Varimax for Victim & Bully Items Varimax Bully/Fight Factor Pattern Desirable (Oblique) Structure with Correlated Factors Oblique Rotation: Bully/Fight Factor Pattern Assessing Fit Change in coordinates corresponding to a rigid rotation of the axses. For interpretation, it s easier if there is a Simple Structure or Simple Solution such that each factor has either large or small (near zero) loadings. There are 10 methods for orthogonal rotation in SAS PROC FACTOR. The most commonly used one is varimax that is due to Henry Kaiser (1958). Maximize the sum of the variances of the vectors of loadings. Formally V = p (l 2 iq l 2 iq) 2 i=1 If you have a target matrix, then use procrustes. 38 / 52

39 Varimax for Victim & Bully Items Rotation Factor Pattern not Unique Desirable Structure Orthogonal Rotations Varimax for Victim & Bully Items Varimax Bully/Fight Factor Pattern Desirable (Oblique) Structure with Correlated Factors Oblique Rotation: Bully/Fight Factor Pattern Assessing Fit Initial Solution varimax Item Fac 1 Fac 2 Fac 1 Fac 2 Bully items q36 upset students for fun q37 in group teased students q43 helped harass students q44 teased other students q49 mean to someone when angry q50 spread rumors q51 started arguments or conflicts q52 encouraged people to fight q53 excluded students from clique Victim items q39 other students picked on me q40 students made fun of me q41 students called me names q42 got hit and pushed About the same. 39 / 52

40 Varimax Bully/Fight Factor Pattern Rotation Factor Pattern not Unique Desirable Structure Orthogonal Rotations Varimax for Victim & Bully Items Varimax Bully/Fight Factor Pattern Desirable (Oblique) Structure with Correlated Factors Oblique Rotation: Bully/Fight Factor Pattern Initial varimax Item Fac 1 Fac 2 Fac 1 Fac 2 Bully q Bully q Bully q Bully q Bully q Bully q Bully q Bully q Bully q Fight q Fight q Fight q Fight q Fight q Assessing Fit 40 / 52

41 Desirable (Oblique) Structure with Correlated Factors for easier interpretation: ɛ 1 X 1 ɛ 2 X 2 F 1 Rotation Factor Pattern not Unique Desirable Structure Orthogonal Rotations Varimax for Victim & Bully Items Varimax Bully/Fight Factor Pattern Desirable (Oblique) Structure with Correlated Factors Oblique Rotation: Bully/Fight Factor Pattern ɛ 3 ɛ 4 X 3 X 4 F 2 Assessing Fit 41 / 52

42 Oblique Rotation: Bully/Fight Factor Pattern Rotation Factor Pattern not Unique Desirable Structure Orthogonal Rotations Varimax for Victim & Bully Items Varimax Bully/Fight Factor Pattern Desirable (Oblique) Structure with Correlated Factors Oblique Rotation: Bully/Fight Factor Pattern Assessing Fit Initial varimax oblimin Item Fac 1 Fac 2 Fac 1 Fac 2 Fac 1 Fac 2 Bully q Bully q Bully q Bully q Bully q Bully q Bully q Bully q Bully q Fight q Fight q Fight q Fight q Fight q From oblimin, ˆρ(F 1, F 2 ) = / 52

43 Rotation Methods Principal Components Solution of Factor Principal Factor or Axis Solution A Modern Solution: Maximum Likelihood Maximum Likelihood MLE continued Rotation & Indeterminacy Assessing Fit 43 / 52

44 Methods Rotation Methods Principal Components Solution of Factor Principal Factor or Axis Solution A Modern Solution: Maximum Likelihood Maximum Likelihood MLE continued Rotation & Indeterminacy Assessing Fit method (Factor extraction) Eigen-decomposition based Eigen-decomposition of S or the principal components solution of the factor model. Iterative eigen-decompositions of S Ψ or the Principal factor or Principal Axis solution. Maximum likelihood estimation We now must assume that F and ɛ are multivariate normal. Tends to fit data better & yields scale invariance (ie., use either S or R). 44 / 52

45 Principal Components Solution of Factor Rotation Methods Principal Components Solution of Factor Principal Factor or Axis Solution A Modern Solution: Maximum Likelihood Maximum Likelihood MLE continued Rotation & Indeterminacy Assessing Fit Let ˆλ 1 ˆλ 2... ˆλ p be the eigenvalues of the sample covariance matrix S and ê 1, ê 2,...,ê p be corresponding eigenvectors. Let m < p (ie, the number of factors be less than the number of X variables), then the matrix of factor loadings equals ( ) L = ˆλ 1 ê 1, ˆλ 2 ê 2,..., ˆλ m ê m The estimate specific variances are provided by the diagonal elements of the matrix S L L, So Ψ = ψ ψ ψm where ψi = s ii m q=1 And estimates of communalities are h 2 i = m q=1 l2 iq Alternatively, use R instead of S (will get a different result). l 2 iq 45 / 52

46 Principal Factor or Axis Solution Rotation Methods Principal Components Solution of Factor Principal Factor or Axis Solution A Modern Solution: Maximum Likelihood Maximum Likelihood MLE continued Rotation & Indeterminacy Assessing Fit 1. Get an initial estimate of Ψ. Most common choice is the square of the multiple correlation (SMC) coefficient between X i and the other p 1 variables. The SMC is the diagonal element of R 1 and is our initial estimate of the ψ i s; that is, ψ i = the i th diagonal element of R 1 = SMC 2. Find the eigenvalues λ q and eigenvectors e i of R diag(ψ i ) and set L = ( λ 1e 1, λ 2e 2,..., λ me m). 3. New estimates of ψ ψ i = 1 m q=1 l 2 iq = 1 h 2 i. 4. Repeat steps 2 through 3 until convergence. 46 / 52

47 A Modern Solution: Maximum Likelihood Rotation Methods Principal Components Solution of Factor Principal Factor or Axis Solution A Modern Solution: Maximum Likelihood Maximum Likelihood MLE continued Rotation & Indeterminacy Assessing Fit With principal component solution, extraction of an additional factor does not effect the values already extracted; this is not the case with the MLE or the principal factor method. Both MLE and principal factor methods can run into problems (Heywood cases, improper solutions) where the ψs want to be negative. You get different results if you use S or R (ie, not scale invariant); however, this is not the case for MLE: If l iq is a loading from a (population) correlation matrix, then l iq σ ii is the corresponding loading from the (population) covariance matrix. You can get a better fit via MLE. Statistical tests become possible. With MLE it becomes possible to set values for l iq (and/or ψ i )...a confirmatory analysis / 52

48 Rotation Methods Principal Components Solution of Factor Principal Factor or Axis Solution A Modern Solution: Maximum Likelihood Maximum Likelihood MLE continued Rotation & Indeterminacy Assessing Fit Maximum Likelihood For MLE, we need to add the assumption that (or X µ). Note that X N(0,Σ) i.i.d where Σ = LL + Ψ. Σ = LΦL + Ψ Ψ} 1/2 ΣΨ {{ 1/2 } = Ψ} 1/2 {{ LΦ 1/2 } Φ} 1/2 L {{ Ψ 1/2 } +I Σ L L Σ Given Ψ, we can get the L s: = L L + I Σ I = Ψ 1/2 ΣΨ 1/2 I = PΛP = PΛ }{{ 1/2 } Λ} 1/2 {{ P } L L 48 / 52

49 MLE continued Starting with a initial guess for Ψ this gives us an initial estimate of L. Using an optimization algorithm, iteratively up-date estimates by maximizing L(L,Ψ) = n 2 (ln( Σ ) + Tr(SΣ 1 )) + constant Rotation Methods Principal Components Solution of Factor Principal Factor or Axis Solution A Modern Solution: Maximum Likelihood Maximum Likelihood MLE continued Rotation & Indeterminacy Assessing Fit or equivalently minimize F(L,Ψ) = ln( σ ) + Tr(SΣ 1 )) ln( S ) p 49 / 52

50 Rotation & Indeterminacy Rotation Methods Principal Components Solution of Factor Principal Factor or Axis Solution A Modern Solution: Maximum Likelihood Maximum Likelihood MLE continued Rotation & Indeterminacy Assessing Fit To improve interpretation, we can rotate factor loadings using either an orthogonal or oblique method. With oblique, Σ = LΦL The m m matrix Φ is the Factor Pattern Matrix that contains correlations between the factors. The p m matrix LΦ is the Factor Structure Matrix that contains the covariances (or correlations) between the X p 1 observed variables and the F m 1 latent variables. Due to the indeterminacy, computing factor scores is not a reasonable thing to do event though many computer programs will give them to you if asked. With MLE, you can fix parameters (e.g., set some loadings equal to 0) and this leads to confirmatory factor analysis. 50 / 52

51 Assessing Fit to Data Rotation Assessing Fit to Data Or How Many Factors Needed? 51 / 52

52 Or How Many Factors Needed? Rotation Assessing Fit to Data Or How Many Factors Needed? Data points: There are p variances and p(p 1)/2 covariances. Parameters to be estimated: There are pm Factor loadings and p unique variances. Residual correlation matrix Root mean squared errors Statistical tests for number of factors Proportion of total sample variance due to the q th factor Proportion of total sample variance accounted for by the m factors. Others. 52 / 52