AN INTRODUCTION TO STRUCTURAL EQUATION MODELING

Transcription

1 AN INTRODUCTION TO STRUCTURAL EQUATION MODELING Giorgio Russolillo CNAM, Paris

2 Structural Equation Modeling (SEM) Structural Equation Models (SEM) are complex models allowing us to study real world complexity by taking into account a whole number of causal relationships among latent concepts (i.e. the Latent Variables, LVs), each measured by several observed indicators usually defined as Manifest Variables (MVs). Factor analysis, path analysis and regression are special cases of SEM. SEM is a largely confirmatory, rather than exploratory, technique. It is used more to determine whether a model is valid than to find a suitable model. But some exploratory elements are allowed Key concepts: Latent variables (unobservable by a direct way): abstract psychological variables like «intelligence», «attitude toward the brand», «satisfaction», «social status», «ability», «trust». Manifest variables are used to measure latent concepts and they contain sizable measurement errors to be taken into account: multiple measures are allowed to be associated with a single construct. Measurement is recognized as difficult and error-prone: the measurement error is explicitly modeled seeking to derive unbiased estimates for the relationships between latent constructs. G. Russolillo slide 2

3 Structural Equation Modeling (SEM) Several fields played a role in developing Structural Equation Models : From Psychology, comes the belief that the measurement of a valid construct cannot rely on a single measure. From Economics comes the conviction that strong theoretical specification is necessary for the estimation of parameters. From Sociology comes the notion of ordering theoretical variables and decomposing types of effects. G. Russolillo slide 3

4 SEM: historical corner G. Russolillo slide 4

5 Sewall Wright and Path Analysis Sewall Wright (2 December Mars 988) American Geneticist, son of the economist Philip Wright Path Analysis has been developed in the 20s by S. Wright to investigate genetic problems and to help his father in economic studies. Path Analysis aims to study cause-effect relations among several variables by looking to the correlation matrix among them. The main newness is the introduction of a new tool to investigate cause-effect relations: the path diagram G. Russolillo slide 5

6 Factor Analysis and the idea of Latent Variable Charles Edward Spearman (0 September September 945) English psychologist C. Spearman proposed Factor Analysis (FA) at the begin of the 900s to measure intelligence in a objective way. The main idea is that intelligence is a MULTIDIMENSIONAL issue, thus the correlation observed among several variables should be explained by a unique underlying factor. The most important input from Factor Analysis is the introduction of the concept of factor, in other words of the concept of Latent Variable G. Russolillo slide 6

7 Thurstone and Multiple Factor Analysis Spearman approach has been modified in the following 40 years in order to considers more than one factor as cause of observed correlation among several set of manifest variables Louis Thurstone (29 May September 955) Psychologist, Psychometrician He is the father of the Multiple Factor Analysis. In the 50 s he meets Herman Wold. They decide to co-organize the Upspsala Symposium on Psycological Factor Analysis. Herman Wold (25 december february 992) Econometrician and Statistician Since their meeting, H. Wold started working on Latent Variables models. G. Russolillo slide 7

8 Causal models rediscovered Herbert Simon (June, 5 96 February 9, 200) Economist Nobel Prize for economic in 978 In 954 presents a paper proving that under certain assumptions correlation is an index of causality Hubert M. Blalock (23 Augut Febrary 99) Sociologist In 964 published the book Causal Inference in Nonexperimental Research, in which he defines methods able to make causal inference starting from the observed covariance matrix. He faces the problem of assessing relations among variables by means of the inferential method. They developed the SIMON-BLALOCK techinque G. Russolillo slide 8

9 Path analysis and Causal models Otis D. Duncan (December, 2 92 November, ) Sociologist He was one of the leading sociologists in the world. He introduces the Path Analysis of Wright's in Sociology. In the mid-60's comes to the conclusion that there is no difference between the Path Analysis of Wright and the Simon-Blalock model. With the economist Arthur Goldberger he comes to the conclusion that there is no difference between what was known in sociology as Path Analysis and simultaneous equations models commonly used in econometrics. Along with Goldberger he organizes a conference in 970 in Madison (USA) where he invited Karl Jöreskog. G. Russolillo slide 9

10 Covariance Structure Analysis and K. Jöreskog Karl Jöreskog Statistician, Professor at Uppsala University, Sweden In the late 50s, he started working with Herman Wold. He discussed a thesis on Factor Analysis. In the second half of the 60s, he started collaborating with O.D. Duncan and A. Goldberger. This collaboration represents a meeting between Factor Analysis (and the concept of latent variable) and Path Analysis (i.e. the idea behind causal models). In 970, at a conference organized by Duncan and Goldberger, Jöreskog presented the Covariance Structure Analysis (CSA) for estimating a linear structural equation system, later known as LISREL G. Russolillo slide 0

11 Soft Modeling and H. Wold Herman Wold (December 25, 908 February 6, 992) Econometrician and Statistician In 975, H. Wold extended the basic principles of an iterative algorithm aimed to the estimation of the PCs (NIPALS) to a more general procedure (PLS) for the estimation of relations among several blocks of variables linked by a network of relations specified by a path diagram. He proposed the PLS Approach to estimate Structural Equation Models (SEM) parameters, as a Soft Modeling alternative to Jöreskog's Covariance Structure Analysis G. Russolillo slide

12 Path Analysis with manifest variables G. Russolillo slide 2

13 Path analysis: drawing conventions x or x Manifest Variables (VM) or Unidirectional Path (cause-effect) or Bidirectional Path (correlation) Feedback relation or reciprocal causation ε or ε or ε Errors G. Russolillo slide 3

14 Drawing a regression model The multiple regression model (on centred variables) : y = γ x + γ 2 x 2 + ζ can be drawn by using a Path Diagram: ζ The error is an Unobserved / Latent variable Quality and Cost are Observed / Manifest Variables x x 2 γ γ 2 y Value, is an Observed / Manifest Variable Example: The Value for a brand in terms of Quality and Cost G. Russolillo slide 4

15 Path analysis: Wright s Tracing rules Tracing rules are a way to estimate the covariance between two variables by summing the appropriate connecting paths. They are:. Trace all paths between two variables (or a variable back to itself), multiplying all the coefficients along a given path 2. You can start by going backwards along a single-headed arrow, but once you start going forward along these arrows you can no longer go backwards 3. No loops. You cannot go through the same variable more that once for a given path 4. At maximum, there can be one double-headed arrow included in a path 5. After tracing all the paths for a given relationship, sum all the paths G. Russolillo slide 5

16 Tracing rule #2: an example You want to estimate cov(y,y) from this model, so you have to trace all the paths and add them: c x a ζ x 2 b y z ζ 2 z 2 You can add the paths acb, bca, z, but the path aca is not allowed G. Russolillo slide 6

17 Tracing rule #4: an example You want to estimate cov(y,x ) from this model: e d x x 2 b a y z ζ f c x 3 Permissible paths are a, ec, db, but the path adfc is not allowed G. Russolillo slide 7

18 Obtaining a numerical solution for model parameters The multiple regression model (standardized variables) : y = ax + bx 2 +cx 3 + ζ can be drawn by using a Path Diagram: e f d x x 2 b a c y z ζ d e DATA: correlation matrix x 3 f a, b and c are standardized partial correlation coefficients (path coefficients) to estimate z is the residual variance to estimate d, e and f are covariances (correlations) between the exogenous variables. G. Russolillo slide 8

19 Obtaining a numerical solution for path coefficients () Writing the set of paths for r(x,y) DATA: correlation matrix e d x x 2 a b y z ζ e f d x 3 x x 2 c a b y z ζ e f d x x 2 x 3 b a c y z ζ f x 3 c r Y = a + ec + db 0.70 = a c + 0.2b G. Russolillo slide 9

20 Obtaining a numerical solution for path coefficients (2) Writing the set of paths for r(x 2,y) DATA: correlation matrix e d x x 2 a b y z ζ e f d x 3 x x 2 c a b y z ζ e f d x x 2 x 3 b a c y z ζ f x 3 c r 2Y = b + da + fc 0.80 = b + 0.2a + 0.3c G. Russolillo slide 20

21 Obtaining a numerical solution for path coefficients (3) Writing the set of paths for r(x 3,y) DATA: correlation matrix e d x x 2 b a y z ζ f d e f x 3 x x 2 x 3 c e a ζ b c y z G. Russolillo slide 2 f d x x 2 x 3 a ζ b c r 3Y = c + fb + ea 0.30 = c + 0.3b a 0.70 = a c + 0.2b a = = b + 0.2a + 0.3c b = = c + 0.3b a c = 0.05 y z

22 Obtaining a numerical solution for the variance of the error e d x x 2 b a y z ζ f c r yy = = a 2 + b 2 + c 2 + 2( cea) + 2( cfb) + 2( bda) +z z = r yy a+b+c+ 2 cea = a 2 + b 2 + c cea = = = ( ) + 2( cfb) + 2( bda) ( ) + 2( cfb) + 2( bda) x 3 ( ) 2 + 2( 0.05) ( 0.24) ( 0.57) + 2( 0.05) ( 0.30) ( 0.70) + 2( 0.70) ( 0.20) ( 0.57) = G. Russolillo slide 22

23 Mediation Analysis G. Russolillo slide 23

24 Single mediator model MEDIATOR (INTERVENING) M INDEPENDENT VARIABLE DEPENDENT VARIABLE G. Russolillo slide 24

25 Mediator model: Total Effect. The independent variable causes the dependent variable: Y = i + ax + e MEDIATOR M INDEPENDENT VARIABLE X a DEPENDENT VARIABLE Y G. Russolillo slide 25

26 Mediator model: Direct effect of X on M 2. The independent variable causes the potential mediator: M = i 2 + bx + e 2 MEDIATOR b M INDEPENDENT VARIABLE X DEPENDENT VARIABLE Y G. Russolillo slide 26

27 Mediator model: Direct Effects of M and X on Y 3. The mediator causes the dependent variable controlling for the independent variable: Y = i 3 + a X + cm + e 3 MEDIATOR M c INDEPENDENT VARIABLE X a DEPENDENT VARIABLE Y G. Russolillo slide 27

28 Single mediator model Mediated (Indirect) effect : bc Direct effect : a Total effect : a = bc+a b MEDIATOR (INTERVENING) M c INDIRECT EFFECT INDEPENDENT VARIABLE X a DIRECT TOTAL EFFECT DEPENDENT VARIABLE Y G. Russolillo slide 28

29 Single mediator model: an example Structural Model ζ IQ Cognitive ability (IQ) z 2 ζ salary b c z RESULTS: estimates Years in school a Salary DATA: covariance matrix G. Russolillo slide 29

30 Path Models with Manifest Variables The multiple regression model can be generalized to paths where endogenous variables are on their turn causative of others endogenous variables. y = γ x + ζ ζ ζ 2 y 2 = β 2 y + ζ 2 x γ β 2 y y 2 y = # endo y y 2 Γ = γ 0 # endo # exo x = x [ ] # exo Β = 0 0 β 2 0 # endo # endo ζ = ζ ζ 2 # endo y = Γx + Βy +ζ G. Russolillo slide 30

31 Path Models with Manifest Variables y = γ x + γ 2 x 2 + ζ y 2 = β 2 y + ζ 2 ζ x γ β 2 y 2 x 2 γ 2 y ζ 2 y = # endo y y 2 Γ = γ γ # endo # exo x = x x 2 # exo Β = 0 0 β 2 0 # endo # endo ζ = ζ ζ 2 # endo y = Γx + Βy +ζ y = ( I B) ( Γx +ζ ) G. Russolillo slide 3

32 Analysing covariance structures of Path models ζ x γ β 2 y 2 x 2 γ 2 y ζ 2 Population Covariance matrix Σ = Σ xx Σ yx Σ yy Assuming that: i) the MVs are centered ii) The covariance between structural error and exogenous MVs is equal to zero We can write the covariance matrix among the MVs in terms of model parameters (implied covariance matrix): C = Σ( Ω) = Σ( Γ, B, Ψ) Path Coefficients Structural Error Covariance G. Russolillo slide 32

33 The sub-matrix Σ YY Covariance matrix of the endogenous MVs: y = ( I B) ( Γx +ζ ) Σ yy = E yy' = E I B = I B ( ) = E ( I B) ( Γx +ζ ) ( ) ( Γx +ζ ) ( x' Γ + ζ ) I B ( ) ( I B) Γx +ζ ( ) = ( ) E( Γxx' Γ +ζ x' Γ + Γx ζ +ζ ζ )( I B) = ( ) ΓE ( xx' ) = I B ( ) = Ψ E ζ ζ ( ) Γ + E ζ x' let Σ yy Ω Γ + ΓE x ζ ( ) + E( ζ ζ ) ( ) = ( I B) ΓΣ xx = ( I B) ( ) ( Γ + Ψ) I B G. Russolillo slide 33

34 The sub-matrix Σ XY y = ( I B) ( Γx +ζ ) xy' = ( x'γ'+ζ ')( I B) ' xy' = ( xx'γ'+ xζ ')( I B) ' E( xy' ) = Σ xx Γ' ( I B) ' Σ xy ( Ω) = Σ Γ ( I B) ' xx Cross-covariance matrix between endogenous and exogenous MVs expressed as a function of model parameters G. Russolillo slide 34

35 Path model implied covariance matrix Σ = Σ xx Σ yx Σ yy Population Covariance matrix S = s xx s yx s yy Empirical covariance matrix C S Σ C = Σ( Ω) = Σ xx Implied covariance matrix ( I B) ΓΣ xx ( I B) ΓΣ xx ( Γ + Ψ) I B ( ) G. Russolillo slide 35

36 An example Σ = Σ xx ( I B) ΓΣ xx ( I B) ( ΓΣ xx Γ + Ψ) ( I B) y = γ x + ζ ζ ζ 2 Γ = y 2 = β 2 y + ζ 2 γ 0 # endo # exo σ x,x σ y,x σ y,y σ y2,x σ y2,y σ y2,y 2 Β = 0 0 β 2 0 # endo # endo = σ x,x x γ β 2 y Σ xx = σ x,x # exo # exo γ σ x,x γ 2 σ x,x +ψ β 2 γ σ x,x β 2 γ 2 σ x,x +ψ y 2 Ψ = ψ 0 0 ψ 22 ( ) 2 β ( 2 γ 2 σ x,x +ψ ) +ψ 22 # endo # endo G. Russolillo slide 36

37 Model identification G. Russolillo slide 37

38 Some consideration on model identification A model is identifiable if its parameters are uniquely determined. Degrees of freedom (DF) DF = # equations (knowns) - # parameters to be estimated (unknowns) Necessary model identification condition: degree of freedom (DF) higher or different from zero, i.e. DF 0 This is a necessary (but not sufficient) condition G. Russolillo slide 38

39 Some consideration on model identification Perfect Identification: A statistical model is perfectly identified if the known information available implies that there is one best value for each parameter in the model whose value is not known. à ZERO degrees of freedom A perfectly identified model yields a trivially perfect fit, making the test of fit uninteresting. Overidentification: A model is overidentified if there are more knowns than unknowns. Overidentified models may not fit well and this is their interesting feature. à degrees of freedom > 0 G. Russolillo slide 39

40 Path Model identification T-rule: A Path Model (and more generally a SEM) is identified if the covariance matrix may be uniquely decomposed in function of the model parameters à if its DF 0, that is if the number of covariances is larger than the number of parameters to be estimated DF = 2 P + Q # of MVs in the model: P = # of exogenous MVs Q = # of endogenous MVs ( )( P + Q +) t Necessary (but not sufficient) condition # of parameters to be estimated 2 P + Q ( ) P + Q + ( ) Number of unique elements in the Σ matrix G. Russolillo slide 40

41 Model identification in Path Analysis. t-rule (necessary condition): the number of redundant elements in the covariance matrix of the observed variables must be greater of equal to the number of the unknown parameters 2. B=0 rule (sufficient condition) no endogenous variable affects any other endogenous variable 3. Recursive rule (sufficient condition), that is B is triangular and Ψ is diagonal 4. Order Condition (necessary condition): The number of variables excluded from each equation must be at least M (M is the number of the endogenous variables), all elements in Ψ are free (no constraints). 5. Rank condition:. Let s consider the matrix obtained by joining the matrixes (I-B) and -Γ. 2. Delete the non-zero columns of the ith row of this matrix. A necessary and sufficient condition for the identification of the ith equation is that the rank of the resulting matrix equals (M ) (no constraints in Ψ) G. Russolillo slide 4

42 Confirmatory Factor Analysis G. Russolillo slide 42

43 Factor Analysis: drawing conventions ξ or ξ Latent Variables (LV) x or x Manifest Variables (VM) or Bidirectional Path (correlation) δ or δ or δ Errors G. Russolillo slide 43

44 Confirmatory Single Factor Model x = manifest variable ξ = latent variable δ = measurement errors x = λ ξ + δ x 2 = λ 2 ξ + δ 2 Factor Loading ξ λ λ 2 λ 3 Common Factor x 3 = λ 3 ξ + δ 3 x x 2 x 3 Unique Factor δ δ 2 δ 3 x = x x 2 x 3 Λ x = λ λ 2 λ 3 ξ = [ ξ ] δ = δ δ 2 δ 3 x = Λ x ξ +δ G. Russolillo slide 44

45 Multiple Factor Model x = λ ξ + δ, x 2 = λ 2 ξ + λ 22 ξ 2 + δ 2 x 3 = λ 3 ξ + δ 3 ξ ξ 2 λ λ2 λ 3 λ 22 λ 42 λ 52 λ 62 x 4 = λ 42 ξ 2 + δ 4 x x 2 x 3 x 4 x 5 x 6 x 5 = λ 52 ξ 2 + δ 5 x 6 = λ 62 ξ 2 + δ 6 " $ $ $ $ Λ x = $ $ x 6 $ $ # $ " x % $ ' x = $ ' # $ &' λ 0 λ 2 λ 22 λ λ 42 0 λ 52 0 λ 62 % ' ' ' ' ' ' ' ' & ' # ξ = ξ & % ( $ ' ξ 2 δ δ 2 δ 3 δ 4 δ 5 δ 6 # δ & % ( δ = % ( $ % '( δ 6 x = Λ x ξ + δ G. Russolillo slide 45

46 Analysing covariance structures in CF models ϕ 2 ξ ξ 2 λ λ2 λ 3 λ 42 λ 52 λ 62 x x 2 x 3 x 4 x 5 x 6 Assuming that: δ δ 2 δ 3 δ 4 δ 5 δ 6 θ 2 i) the MVs, the LV and the errors are centered ii) Measurement errors and LVs do not covariate We can write the covariance matrix among the MVs in terms of model parameters (implied covariance matrix): C = Σ Ω ( ) = Σ( Λ,Φ,Θ) Loadings LV Covariance Measurement Error Covariance G. Russolillo slide 46

47 Analysing covariance structures in CF models Confirmatory factor model : Σ( Ω) = E(x x ) = E ( )( ) = E Λξ + δ ξ Λ + δ = x = Λ x ξ + δ The covariance matrix of the MVs can be rewritten in terms of model parameters: ( Λξ + δ ) Λξ + δ ( ) = = ΛE(ξ ξ ) Λ + Λ E(ξ δ ) + E(δ ξ ) Λ + E(δ δ ) let ( ) = Φ ( ) = Θ E ξξ ' E δδ ' Σ( Ω) = ΛΦΛ' + Θ G. Russolillo slide 47

48 Latent or Emergent Constructs? Latent Construct Emergent Construct x j x 2j x 3j x 4j x j x 2j x 3j x 4j Reflective (or Effects) Indicators Formative (or Causal) Indicators e.g. Consumer s attitudes, feelings Constructs give rise to observed variables (unique causeè unidimensional) Aim at accounting for observed variances or covariances These indicators should covary: changes in one indicator imply changes in the others. Internal consistency is measured (es. Cronbach s alpha) e.g. Social Status, Perceptions Constructs are combinations of observed variables(multidimensional) Not designed to account for observed variables These indicators need not covary: changes in one indicator do not imply changes in the others. Measures of internal consistency do not apply. G. Russolillo slide 48

49 An Example of CFA in Psychology A dataset by Long (J. Scott Long : Confirmatory Factor Analysis, SAGE Publications, 986) heads of families from the region of Hennepin, Illinois - Observed (manifest) variables: PSY67 = Psychological disorders 967 PHY67 = Psychophysiological disorders 967 PSY7 = Psychological disorders 97 PHY7 = Psychophysiological disorders 97 G. Russolillo slide 49

50 An Example of CFA in Psychology Data PSY67 PHY67 PSY7 PHY7 n corr PSY67... corr PHY corr PSY corr PHY stddev Data are covariances between manifest variables : Cov(X p, X p' ) = Cor(X p, X p' ) Var(X p ) Var(X p' ) G. Russolillo slide 50

51 An Example of CFA in Psychology The st model specified by Long theta3 theta24 D D2 D3 D4 Unique factors PSY67 L L2 PHY67 PSY7 L32 L42 PHY7 Common Factors XS XSI2 phi2 G. Russolillo slide 5

52 Long s model identification The st model specified by Long: Study of the specified model D theta3 D2 theta24 D3 D4 PSY67 L L2 PHY67 PSY7 L32 L42 PHY7 Model equations and model parameters XS phi2 XSI2 Model parameters λ, λ 2, λ 2, λ 22 ϕ = Var(ξ ), ϕ 22 = Var(ξ 2 ), ϕ 2 = Cov(ξ, ξ 2 ) θ = Var(δ ), θ 22 = Var(δ 2 ), θ 33 = Var(δ 3 ), θ 44 = Var(δ 4 ) θ 3 = Cov(δ, δ 3 ), θ 24 = Cov(δ 2, δ 4 ) Factor equations PSY67 = λ ξ + δ PHY67 = λ 2 ξ + δ 2 PSY7 = λ 2 ξ 2 + δ 3 PHY7 = λ 22 ξ 2 + δ 4 G. Russolillo slide 52

53 Long s model identification (2) The st model specified by Long: Study of the specified model Factor Equations! # # x = # # # "# x x 2 x 3 x 4 $! & λ 0 $! # & δ $ # & & # λ & = 2 0 &! ξ $ # # &# & δ & + # 2 & = Λξ +δ & # 0 λ 2 &# " ξ & 2 % # δ 3 & & # & # & %& "# 0 λ 22 %& δ ξ "# 4 %& Λ δ Covariance Decomposition Σ = E(xx') = E # $ (Λξ +δ)(λξ +δ)' % & = Λ E(ξξ ') Λ'+ E(δδ ') Φ Θ = ΛΦΛ'+ Θ Model parameters λ, λ 2, λ 2, λ 22 ϕ = Var(ξ ), ϕ 22 = Var(ξ 2 ), ϕ 2 = Cov(ξ, ξ 2 ) θ = Var(δ ), θ 22 = Var(δ 2 ), θ 33 = Var(δ 3 ), θ 44 = Var(δ 4 ) θ 3 = Cov(δ, δ 3 ), θ 24 = Cov(δ 2, δ 4 ) " Φ = E(ξξ ') = φ φ % $ 2 ' $ # φ 2 φ ' 22 & " $ θ 0 θ 3 0 $ 0 θ Θ = E(δδ ') = 22 0 θ $ 24 $ θ 3 0 θ 33 0 $ # $ 0 θ 24 0 θ 44 % ' ' ' ' ' &' G. Russolillo slide 53

54 Model identification in CFA: Scaling a LV In CFA, the scale of a LV is not determined. So we need give a scale to the LV imposing a contraint either on the LV variance or on one loading. By consequence, in a LV Path Model we need to scale each exogenous LV Unstandardized Estimates δ 2 δ x 2 x x 3 λ x 2 λ x 3 λ x = ξ δ 3 δ x λ x var(ξ ) = Standardized Estimates δ 2 x 2 x 3 λ x 2 λ x 3 ξ δ 3 G. Russolillo slide 54

55 Model identification in CFA: the t-rule The t-rule requires that: t 2 P ( P + ) # of parameters to be estimated # of MVs in a CFA model This is a necessary condition (but not sufficient): the number of free parameters (parameters to be estimates) must be less than or equal to the number of unique elements in the covariance matrix of X G. Russolillo slide 55

56 Model identification: the Three-Indicator Rule The Three-Indicator Rule is a sufficient condition a CFA model is identified if. every latent construct is associated with at least 3 manifest variables; 2. each manifest variable is associated with only one latent construct, i.e. each row of Λ has only one nonzero element 3. the unique factors are uncorrelated, i.e. Θ is a diagonal matrix G. Russolillo slide 56

57 Model identification in CFA: the Two-Indicator Rule The Two-Indicator Rule is a sufficient condition a CFA model is identified if. every latent construct is associated with at least 2 manifest variables; 2. each manifest variable is associated with only one latent construct, i.e. each row of Λ has only one nonzero element 3. the unique factors are uncorrelated, i.e. Θ is a diagonal matrix 4. There are at least two LVs with non-zero covariance G. Russolillo slide 57

58 Summing up identification rules in CFA (Assuming that all LVs are given a scale) Identification rule Necessary Condition Sufficient Condition T-rule Yes No Three-Indicator Rule No Yes Two-Indicator Rule No Yes G. Russolillo slide 58

59 Long s model identification (3) The st model specified by Long: Study of the specified model Factor Equations! # # x = # # # "# x x 2 x 3 x 4 $! & λ 0 $! # & δ $ # & & # λ & = 2 0 &! ξ $ # # &# & δ & + # 2 & = Λξ +δ & # 0 λ 2 &# " ξ & 2 % # δ 3 & & # & # & %& "# 0 λ 22 %& δ ξ "# 4 %& Λ δ Covariance Decomposition Σ = E(xx') = E # $ (Λξ +δ)(λξ +δ)' % & = Λ E(ξξ ') Λ'+ E(δδ ') Φ Θ = ΛΦΛ'+ Θ Model parameters λ, λ 2, λ 2, λ 22 ϕ = Var(ξ ), ϕ 22 = Var(ξ 2 ), ϕ 2 = Cov(ξ, ξ 2 ) θ = Var(δ ), θ 22 = Var(δ 2 ), θ 33 = Var(δ 3 ), θ 44 = Var(δ 4 ) θ 3 = Cov(δ, δ 3 ), θ 24 = Cov(δ 2, δ 4 ) " Φ = E(ξξ ') = φ φ % $ 2 ' $ # φ 2 φ ' 22 & " $ θ 0 θ 3 0 $ 0 θ Θ = E(δδ ') = 22 0 θ $ 24 $ θ 3 0 θ 33 0 $ # $ 0 θ 24 0 θ 44 % ' ' ' ' ' &' G. Russolillo slide 59

60 Long s model identification (4) The st model specified by Long: Study of the specified model D theta3 D2 theta24 D3 D4 Covariances P(P+)/2=0 Matrix Σ PSY67 L PHY67 L2 PSY7 L32 PHY7 L42 PSY67 PHY67 PSY7 PHY7 PSY67 PHY67 PSY7 PSY7 σ σ 2 σ 3 σ 4 σ 22 σ 23 σ 24 σ 33 σ 34 σ 44 XS phi2 XSI2 Model parameters λ, λ 2, λ 2, λ 22 ϕ = Var(ξ ), ϕ 22 = Var(ξ 2 ), ϕ 2 = Cov(ξ, ξ 2 ) θ = Var(δ ), θ 22 = Var(δ 2 ), θ 33 = Var(δ 3 ), θ 44 = Var(δ 4 ) 3 θ 3 = Cov(δ, δ 3 ), θ 24 = Cov(δ 2, δ 4 ) This model is not identifiable The parameters of the model may not be uniquely expressed as a function of the covariances between the manifest variables. N of parameters > N of covariances Some constraints are needed. G. Russolillo slide 60

61 Long s model identification (5) theta3 theta24 The st model specified by Long is not identifiable we need to add some constraints: D PSY67 L D2 PHY67 L2 D3 PSY7 L32 D4 PHY7 L42 XS XSI2 à Normalisation of the common factors Var(ξ ) = ϕ =, Var(ξ 2 ) = ϕ 22 = phi2 à Stability of the loadings along time: λ PSY67, = λ PSY7,2 λ PHY67, = λ PHY7,2 à Independence between unique factors: θ 3 = Cov(δ, δ 3 ) = 0 θ 24 = Cov(δ 2, δ 4 ) = 0 theta D theta22 D2 theta33 D3 theta44 D4 PSY67 PHY67 PSY7 PHY7 L L2 L L2 XSI XSI2 New model phi2 G. Russolillo slide 6

62 Long s model identification (6) The 2nd model (identifiable) specified by Long: Study of the specified model D theta D2 theta22 D3 theta33 D4 theta44 PSY67 L L2 PHY67 PSY7 L L2 PHY7 XSI XSI2 phi2 G. Russolillo slide 62

63 Long s model identification (7) The 2nd model specified by Long: Study of the specified model Factor Equations! # # x = # # # "# x x 2 x 3 x 4 $! & λ 0 $! # & δ $ # & & # λ & = 2 0 &! ξ $ # # &# & δ & + # 2 & = Λξ +δ & # 0 λ &# " ξ & 2 % # δ 3 & & # & # & %& "# 0 λ 2 %& δ ξ "# 4 %& Λ Model parameters λ, λ 2 ϕ 2 = Cov(ξ, ξ 2 ) θ = Var(δ ), θ 22 = Var(δ 2 ), θ 33 = Var(δ 3 ), θ 44 = Var(δ 4 ) δ Covariance Decomposition Σ = E(xx') = E # $ (Λξ +δ)(λξ +δ)' % & = Λ E(ξξ ') Λ'+ E(δδ ') Φ Θ = ΛΦΛ'+ Θ " φ Φ = E(ξξ ') = $ 2 $ # φ 2 θ θ Θ= E( δδ ') = 0 0 θ θ44 % ' ' & G. Russolillo slide 63

64 Long s model identification (8) The 2nd model specified by Long Study of the specified model Covariances PSY67 PHY67 PSY7 PHY7 P(P+)/2=0 PSY67 PHY67 PSY7 PSY7 σ σ 2 σ 3 σ 4 σ 22 σ 23 σ 24 σ 33 σ 34 σ 44 Matrix Σ theta theta22 D D2 PSY67 PHY67 L L2 XSI phi2 theta33 theta44 D3 D4 PSY7 PHY7 L L2 XSI2 Model parameters λ, λ 2 ϕ 2 = Cov(ξ, ξ 2 ) θ = Var(δ ), θ 22 = Var(δ 2 ), θ 33 = Var(δ 3 ), θ 44 = Var(δ 4 ) 7 This model is identifiable The number of parameters (7) is finally less than the number of covariances (0) Degrees of freedom (DF) = 0-7 = 3 G. Russolillo slide 64

65 Applying CFA to Long s second model () The 2nd model specified by Long Study of the specified model theta D PSY67 theta22 D2 PHY67 theta33 D3 PSY7 theta44 D4 PHY7 L L2 L L2 XSI XSI2 Σ = Λ ΦΛ + x x Θ phi2! # # # # # "# σ σ 2 σ 3 σ 4 σ 22 σ 23 σ 24 σ 33 σ 34 σ 44 $! & # & # & = # & # & # %& "# λ 2 2 λ λ 2 φ 2 λ φ 2 λ λ 2 λ 2 2 φ 2 λ λ 2 φ 2 λ 2 2 λ 2 λ λ 2 λ 2 2 $! & # & # & + # & # & # %& "# θ θ θ 3 0 θ 4 $ & & & & & %& G. Russolillo slide 65

66 Applying CFA to Long s second model (2) Visualising the estimates for model 2 by Long D.52 D2.22 D3.40 D4.6 PSY67.24 PHY PSY7.24 PHY XSI XSI2 C = 2 ˆλ ˆλ ˆλ2 2 ˆφ ˆλ 2 ˆφ ˆλ ˆλ2 2 2 ˆλ 2 ˆφ ˆλ ˆλ2 2 2 ˆφ ˆλ2 2 2 ˆλ ˆλ ˆλ2 2 ˆλ 2 + ˆθ ˆθ ˆθ 3 0 ˆθ *.30.67*.24.67*.24* *.24*.30.67* C = * G. Russolillo slide 66

67 Applying CFA to Long s second model (3) Observed (sample) and Implied Covariances and Correlations from model 2 by Long var( PHY 7) = var( λ PHY 7 φ XSI 2 + δ ) 2 PHY 7 = λ PHY 7 var( φ XSI 2 ) + var( δ PHY 7 ) Example: Var(PHY7) = Var(.30*XSI2 + D PHI7 ) =.30 2 Var(XSI2) + Var(D PHI7 ) = =.25 G. Russolillo slide 67

68 Path modeling with Latent Variables G. Russolillo slide 68

69 Two families of methods Covariance-based Methods The aim is to reproduce the sample covariance matrix of the manifest variables by means of the model parameters: the implied covariance matrix of the manifest variables is a function of the model parameters it is a confirmatory approach aimed at validating a model (theory building) SEM Component-based Methods The aim is to provide an estimate of the latent variable scores in such a way that they are the most correlated with one another (according to path diagram structure) and the most representative of each corresponding block of manifest variables. latent variable score estimation plays a main role it is more an exploratory approach, than a confirmatory one (operational model strategy) G. Russolillo slide 69

70 Path model with LVs: drawing conventions η or η Latent Variables (LV) x or x Manifest Variables (VM) or Unidirectional Path (cause-effect) or Bidirectional Path (correlation) Feedback relation or reciprocal causation ε or ε or ε Errors G. Russolillo slide 70

71 Path model with latent variables x = manifest variable associated to exogenous latent variables y = manifest variable associated to endogenous latent variables ξ = exogenous latent variable η = endogenous latent variable δ = measurement error associated to exogenous latent variables ε = measurement error associated to endogenous latent variables ζ = structural error δ 2 δ x 2 x λ x 2 λ x 3 λ x ξ γ 2 ζ 2 y 3 λ y 32 λ y 42 y 4 Loadings ε3 ε 4 δ 3 ε x 3 y γ Path Coefficients β 2 η 2 λ y 72 λ y 52 λ y 62 y 6 y 5 ε 6 ε 5 Loadings λ y η y 7 ε 7 λ y 2 ε 2 y 2 ζ Structural Model or inner model Measurement Model or outer model G. Russolillo slide 7

72 The measurement (outer) model For the exogenous MVs x = λ x ξ + δ δ 2 δ 3 δ x 2 x x 3 λ x 2 λ x 3 λ x ξ x 2 = λ x 2 ξ + δ 2 x 3 = λ x 3 ξ + δ 3 x = x x 2 x 3 Λ x = λ x x λ 2 x λ 3 ξ = [ ξ ] δ = δ δ 2 δ 3 x = Λ x ξ + δ G. Russolillo slide 72

73 The measurement (outer) model For the exogenous MVs (A reflective scheme must hold) x = Λ x ξ +δ Vector of the exogenous MVs Vector of the measurement errors associated to the exogenous MVs RECTANGULAR matrix with the loadings linking each exogenous MV to the corresponding exogenous LV Vector of the exogenous LVs G. Russolillo slide 73

74 The measurement (outer) model For the endogenous MVs y 3 ε 3 y = λ y η + ε λ y 32 λ y 42 y 4 ε 4 y 2 = λ y 2 η + ε 2 y 3 = λ y 32 η 2 + ε 3 y 4 = λ y 42 η 2 + ε 4 ε y λ y η 2 λ y 72 λ y 62 λ y 52 y 6 y 5 ε 6 ε 5 y 5 = λ y 52 η 2 + ε 5 y 6 = λ y 62 η 2 + ε 6 λ y 2 η y 7 ε 7 ε 2 y 2 y 7 = λ y 72 η 2 + ε 7 y = y y 2 y 3 y 4 y 5 y 6 y 7 Λ y = λ 0 λ λ 32 λ 42 λ 52 λ 62 λ 72 η = η η 2 ε = ε ε 2 ε 3 ε 4 ε 5 ε 6 ε 7 y = Λ y η +ε G. Russolillo slide 74

75 The structural (inner) model ξ γ 2 ζ 2 η = γ ξ + ζ η 2 η 2 = γ 2 ξ + β 2 η + ζ 2 γ β 2 η ζ η = η η 2 ξ = [ ξ ] Γ = γ γ 2 Β = 0 0 β 2 0 ζ = ζ ζ 2 η = Γξ + Βη +ζ G. Russolillo slide 75

76 The structural (inner) model Vector of the endogenous LVs Vector of the errors associated to the endogenous LVs η = Γξ + Βη + ζ Vector of the exogenous LVs Path-coefficients Matrix: it is a RECTANGULAR matrix with the path coefficients of the exogenous LVs. Path-coefficients Matrix: it is a SQUARED matrix with the path coefficients of the endogenous LVs. On the diagonal there are only zeros G. Russolillo slide 76

77 The Structural Equation Model δ 2 δ x y x 2 ξ 3 ε 3 ζ 2 y 4 ε 4 x3 y5 δ 3 η 2 ε 5 ε y y 6 ε 6 η y 7 ε 7 ε 2 y 2 ζ x = Λ x ξ + δ y = Λ y η + ε Measurement Models Structural Model η = Γξ + Βη + ζ η = ( I B) ( Γξ + ζ ) G. Russolillo slide 77

78 To summarize ζ ξ ξ 2 x y x x 2 x 3 x 4 x 5 x 6 x 2 Path Analysis with Manifest Variables δ δ 2 δ 3 δ 4 δ 5 δ 6 Confirmatory Factor Model x δ 2 x 2 ξ δ ε 3 y 3 ζ 2 y 4 ε 4 δ 3 x 3 η 2 y 5 ε 5 ε y y 6 ε 6 ε 2 y 2 η ζ y 7 ε 7 Path Analysis with Latent Variables (Structural Equation Model) G. Russolillo slide 78

79 Model assumptions δ 2 δ x y x ξ 3 ε 3 2 ζ 2 y 4 ε 4 x3 η y5 2 ε 5 y y 6 ε 6 y 7 ε 7 η ε 2 y 2 ζ δ 3 ε η = x = Λ x ξ + δ y = Λ y η + ε ( I B) ( Γξ + ζ ) Assuming that: i) the MVs, the LV and the errors (both in structural and measurement models) are centered ii) Two errors of different type (structural, exogenous measurement and endogenous measurement) do not covariate iii) Measurement errors and LVs do not covariate iv) The covariance between structural error and exogenous LVs is equal to zero G. Russolillo slide 79

80 Model assumptions δ 2 δ 3 ε δ x y x 2 ξ 3 ε 3 ζ 2 y 4 ε 4 x3 η y5 2 y y 6 ε 6 ε 5 η = x = Λ x ξ + δ y = Λ y η + ε ( I B) ( Γξ + ζ ) y 7 ε 7 η ε 2 y 2 ζ Hypotheses on Expectations E ( x) = 0, E ( y) = 0, E ( ξ ) = 0, E ( η) = 0, E ( δ) = 0, E ( ε) = 0, E ( ζ ) = 0 ( ) = 0, E ( ζ δ ) ( ε ) = 0 ( ) = 0, E ( δ η ) = 0, E ( ε ξ ) = 0, E ( ε η ) = 0 E δ ε E δ ξ E ( ζ ξ ) = 0 ( ) = 0, E ζ Hypotheses on Correlations G. Russolillo slide 80

81 Analysing the covariance in SEMs δ 2 δ 3 δ x y 3 ε x 2 ξ 3 ζ 2 y 4 ε 4 x3 η y5 2 ε 5 y y 6 ε 6 y 7 ε 7 η ε Population Covariance matrix Σ = Σ xx Σ yx Σ yy ε 2 y 2 ζ We can write the covariance matrix among the MVs in terms of model parameters (implied covariance matrix) C = Σ( Ω) = Σ( Γ,B,Λ x,λ y,φ,ψ,θ δ,θ ) ε Path Coefficients Loadings Exog. LV Covariance Structural Error Covariance Measurement Error Covariance G. Russolillo slide 8

82 Implied Covariance matrix Σ(Ω) Σ = Σ xx Σ yx Σ yy Covariance matrix of the population Σ( Ω) = Λ y It can be rewritten as a function of model parameters Λ x ΦΛ' x + Θ δ ( I B) Γ Φ Λ' x Λ y I B ( ) ( ΓΦ Γ + Ψ) ( I B) Λ' y + Θ ε Covariance matrix of the population implied by the model G. Russolillo slide 82

83 The sub-matrix Σ XX Outer model for exogenous blocks : Σ XX ( Ω) = E(x x ) = E ( )( ) = E Λ x ξ + δ ξ Λ x + δ = x = Λ x ξ + δ The covariance matrix of the exogenous MVs can be rewritten in terms of model parameters: ( Λ x ξ + δ ) Λ x ξ + δ ( ) = = Λ x E(ξ ξ ) Λ x + Λ x E(ξ δ ) + E(δ ξ ) Λ x + E(δ δ ) let ( ) = Φ ( ) = Θ δ E ξξ' E δδ' Σ XX ( Ω) = Λ ΦΛ' +Θ x x δ G. Russolillo slide 83

84 The sub-matrix Σ XX Covariance matrix of the exogenous MVs expressed as a function of model parameters Σ XX ( Ω) = Λ ΦΛ' +Θ x x δ G. Russolillo slide 84

85 The sub-matrix Σ YY Outer model for the endogenous blocks : The covariance matrix of the exogenous MVs can be rewritten in terms of model parameters: Σ YY ( Ω) = E(y = E Λ y η + ε y ) = E Λ y η + ε [( )( η Λ y + ε )] = ( )( Λ y η + ε) [ ] = = E Λ y η η Λ y + ε η Λ y + Λ y η ε + ε ε = = Λ y E(η η ) Λ y + Λ y E(ε η ) + E(η ε ) Λ y + E(ε ε ) let E ( ε ε ) = Θ ε y = Λ y η + ε Σ YY This matrix is a function of other parameters ( Ω) = Λ y E ( η η )Λ' y +Θ ε G. Russolillo slide 85

86 The sub-matrix Σ YY Covariance matrix of the endogenous LVs: η = ( I B) ( Γξ + ζ ) Σ ηη = E(η η ) = E [( I B) ( Γξ + ζ )] I B = E I B = I B [( ) ( Γξ + ζ )] ( ξ Γ + ζ )( I B) [( ) ( Γξ + ζ )] = ( ) E ( Γξ ξ Γ + ζ ξ Γ + Γξ ζ + ζ ζ )( I B) = ( ) ΓE ( ξ ξ ) = I B = [ Γ + E ( ζ ξ ) Γ + ΓE ( ξ ζ ) + E ( ζ ζ )] I B ( ) = Ψ let E ζ ζ Σ ηη Ω ( ) ( ) = ( I B) ( ΓΦ Γ + Ψ) ( I B) G. Russolillo slide 86

87 The sub-matrix Σ YY The covariance matrix of the endogenous MVs can now be obtained as a function of model parameters: Σ YY ( Ω) = Λ y E ( η η )Λ' y +Θ ε Σ YY ( Ω) = Λ y I B ( ) ( ΓΦ Γ + Ψ) ( I B) Λ' y+ Θ ε G. Russolillo slide 87

88 The sub-matrix Σ XY Measurement model: x = Λ x ξ + δ and y = Λ y η + ε The cross-covariance matrix between exogenous and endogenous MVs can be written as Σ XY ( Ω) = E ( XY' ) = E ( Λ x ξ +δ) Λ y η +ε ( ) = E Λ x ξ +δ ( η Λ y + ε ) = ( ) = = Λ x E(ξ η ) Λ y + Λ x E(ξ ε ) + E(δ η ) Λ y + E(δ ε ) The measurement errors are uncorrelated among them and with the exogenous LVs Σ XY ( ) ( Ω) = Λ x E ξ η Λ y G. Russolillo slide 88

89 The sub-matrix Σ XY η = ( I B) ( Γξ +ζ ) ξη' = ( ξ 'Γ'+ζ ')( I B) ' ξη' = ( ξξ 'Γ'+ ξζ ')( I B) ' E( ξη' ) = ΦΓ' ( I B) ' Σ XY ( Ω) = Λ Φ Γ ( I B) ' Λ' x y Cross-covariance matrix between endogenous and exogenous LVs expressed as a function of model parameters G. Russolillo slide 89

90 Analysing the covariance in SEMs Σ = Σ xx Σ yx Σ yy Population Covariance matrix S = s xx s yx s yy Empirical covariance matrix C S Σ C = Σ( Ω) = *,,, + Λ y Λ x ΦΛ' x + Θ δ * Φ Λ' x Λ y I B +, ( I B) Γ ( Implied covariance matrix ( ) ( ΓΦ Γ ( + Ψ) ( I B) (-./ Λ' y+ Θ ε - / / /. G. Russolillo slide 90

91 Model identification in SEM T-rule: A SEM is identified if the covariance matrix may be uniquely decomposed in function of the model parameters à if its DF 0, that is if the number of covariances is larger than the number of parameters to be estimated DF = 2 P + Q # of MVs in the model: P = # of exogenous MVs Q = # of endogenous MVs ( )( P + Q +) t Necessary (but not sufficient) condition # of parameters to be estimated 2 P + Q ( ) P + Q + ( ) Number of unique elements in the Σ matrix G. Russolillo slide 9

92 Model Identification in SEM Two-Step rule: à In the first step the analyst treats the model as a confirmatory factor analysis (both x and y variables are viewed as exogenous, relations between the LVs are expressed in terms of covariance) à The second step examines the latent variable equation of the original model and treat it as though it were a structural equation of observed variables N.B. Those are sufficient (but not necessary) conditions G. Russolillo slide 92

93 Model estimation G. Russolillo slide 93

94 Discrepancy function Estimation minimizes some discrepancy function between the implied covariance matrix and the observed one. F = f ( S Σ ( ˆΩ ) ) Σ ( ˆΩ ) = C = ˆΛ y ˆΛ x ˆΦ ˆΛ 'x + ˆΘ δ ˆΦ ˆΛ ' x ˆΛ y I ˆB ( I ˆB ) ˆΓ Estimated covariance matrix ( ) ( ˆΓ ˆΦ ˆΓ + ˆΨ )( I ˆB ) ' ˆΛ ' y + ˆΘ ε G. Russolillo slide 94

95 Main discrepancy function for estimation Assumptions: à data are multinormal à S follows a Wishart distribution à Both S and C are positive-definite Maximum Likelihood F ML = log C + tr( SC ) log S ( P + Q) Properties of the ML estimators: - Asymptotically unbiased - Consistent - Asymptotically efficient - The distribution of the ML estimators approximates a normal distribution as sample size increases G. Russolillo slide 95

96 Alternative discrepancy functions Unweighted Least Squares F ( ) 2 ULS = tr S-C 2 Generalised Least Squares F GLS = tr W S 2 F ADF / WLS ( - C) 2 Asymptotically Distribution Free ( ) T s c W ( s c) = G. Russolillo slide 96

97 Unweighted Least Squares - SEM Discrepancy function F ( ) 2 ULS = tr S-C 2 à When sample size is large, ULS and ML estimates are very similar. à However, ULS estimators are not the asymptotically most efficient estimators à Consistent estimators à Estimates differ when correlation instead of covariance matrices are analyzed à No assumption on the observed variables distribution However: ULS estimators implicitly weight all the residuals as if they had the same variance. A weight matrix is considered in the discrepancy in order to overcome the problem G. Russolillo slide 97

98 Generalised Least Squares - SEM Discrepancy function F GLS = 2 tr ( W ( S C) 2 ) à Consistent estimators à Asymptotic normality (Depending on the choice of W) à Common choice W=S (Scale-free least squares) or W=diag(S) (Asymptotically-free least squares), if W = I -> GLS = ULS In order to assure this property two assumptions must be done: à S elements are unbiased estimators of population covariances à S elements asymptotically follow a multinormal distribution with expectations equal to the population covariances and the generic covariance of s pp and s pp are equal to N ( σ ig σ ih +σ ih σ jg ) G. Russolillo slide 98

99 Estimation procedure Fitting functions are generally solved by using iterative numerical procedures. When the values for the parameters to be estimated in two consecutive steps differ by less than some present criterion the iterative process stops. Non convergence may happen Convergence is affected by: - the convergence criterion - the number of iteration allowed - the starting values - model specification Estimates from nonconvergent runs should not be used or given substantive interpretations. G. Russolillo slide 99

100 Model fit and validation G. Russolillo slide 00

101 Reliability The reliability rel(x i ) of a measure x i of a true score ξ modeled as x p = λ p ξ j + δ p is defined as: ( ) = λ 2 var( ξ ) p j var( x ) p rel x p = cor 2 ( x p,ξ ) j rel(x p ) can be interpreted as the variance of x p that is explained by ξ j G. Russolillo slide 0

102 Measuring the Reliability Question: How to measure the overall reliability of the measurement tool? In other words, how to measure the homogeneity level of a block X j of positively correlated variables? Answer: The composite reliability (internal consistency) of manifest variables can be checked using: à the Cronbach s Alpha à the Dillon Goldstein rho G. Russolillo slide 02

103 Composite reliability The measurement model assumes that each group of manifest variables is homogeneous and unidimensional (related to a single variable). The composite reliability (internal consistency or homogeneity of a block) of manifest variables is measured by either of the following indices: à the Cronbach s Alpha à the Dillon Goldstein rho α j = p p" P j + ρ j = " $ # P j p= cor( x pj,x p' j ) p p" λ pj cor( x pj,x p' j ) " $ # P j % ' & p= 2 λ pj P j % ' & λ pj p= ( ) P j P j The manifest variables are reliable if these indices are at least 0.7 G. Russolillo slide 03

104 Cronbach s α rationale : the model x q,, x pq,, x Pq q are supposed to be tau equivalent (or parallel) measures of a true score ξ q : Hp : x i = ξ + δ i, i =,..., P ( ) = 0 ( ) = 0 ( ) = 0 i i' E x i cov ξ,δ i cov δ i,δ i' λ i = (Tau-equivalency condition) var(δ i ) = var(δ i' ) i,i' (Parallelity condition) of tau- Cronbach s a measures the reliability of a simple sum equivalent or parallel measures H = P i x i Cronbach's α = rel ( H ) = cor 2 ( H,ξ ) = cor 2 x i ( ),ξ i G. Russolillo slide 04

105 Derivation of Cronbach s a formula ( ( ) ) cor ( 2 x i i,ξ) = cov2 ξ + δ i i,ξ var( x i )var( ξ). var x i i ( ) P var( ξ) = P P ( P )var x i ( ) = P i P = P 2 var ξ ( P ) P var = ( P ) = ( ) ( = cov2 Pξ + i δ i ),ξ var( x i )var( ξ) i P 2 var ξ ( P ) var ( ) + var( δ i ) i ( i x i ) i var x i var( x i ) i i ( ) P ( P ) var x i var i x i ( ) var ( ) = ( ) P var( ξ) ( ) i x i ( ) = var ( ( ξ +δ i )) = var ( Pξ + δ i ) = P 2 var ξ i 2. var x i i 3. var i x i ( ) = i i var ξ +δ i ( ) = var x i i ( ) = ( ) + i i' i i i = ( ) + var ( δ i ) = ( ) var ξ cov x i, x i' ( ) i x i i ( ) var δ i P ( P ) ( ) ( ) = P P 2 var 2 ξ var( x i )var ξ i ( ) + P var ξ 2 i ( ) + var δ i i ( ) + var ( δ i ) P var ξ i i' var x i ( ) + ( ) = P 2 var ξ i var cov x i, x j ( ) 2 var ξ ( ) i x i ( ) cov( x i, x j ) i i' α, and α = when all correlations between x i are equal to and all variances of x i are equal to each other. ( ) + var ( δ i ) i 3 G. Russolillo slide 05

106 Cronbach s α for standardized MVs If variables are standardised, var(h ) = var i x i Hence, we can write i i' cor(x P i, x i' ) α = P = P + cor(x i, x i' ) i i' cor x P i, x i' P P i i' P cor( x P + i, x i' ) P ( ) = P + cor(x i, x i' ) ( ) = P + i i' i i' ( ) ( ) cor x i, x i' P P cor( x i, x i' ) P Pr α= + ( P )r, where r = i i' P(P ) cor(x i,x i' ) if r = α = if r = 0 α = 0 A block of positively correlated variables, is essentially uni-dimensional if they show a high mean correlation G. Russolillo slide 06

107 Dillon Goldstein s ρ : the rationale Given the model x pq = λ pq ξ q +ε pq, var ( x p pq ) = var # λ ξ +ε pq q pq " p ( ) ( ) + var ε pq $ % = var λ ξ p pq q " ( ) = ( λ pq ) 2 var( ξ q ) #& $ %' + var ε pq ( ) Dillon Goldstein s ρ is defined as ρ q = ( λ ) 2 p pq var( ξ q ) ( λ ) 2 p pq var ξ q ( ) + var( ε pq ) p If the MVs and the LV are standardised, ρ DG can be estimated as ˆρ q = ( ˆλ ) 2 p pq ( ˆλ ) 2 2 p pq + ˆλ p pq ( ) NB: λ pq must be estimated! G. Russolillo slide 07

108 Assessing the fit of a SEM Overall model fit measures These tests help to evaluate if the model fit is good or not and if not it help to measure the departure of observed covariance matrix from the implied covariance matrix à They can not be applied if the model is exactly identified (because in this situation S always equals C) à Overall fit may be good but parameters estimates may not be statistically significant! G. Russolillo slide 08

109 Overall model fit measures The overall model fit measures are «functions» of the fitting function: f S,C ( ) = F We can show that: ( N ) 2 F χ DF N : number of observations F: Discrepancy function DF: Degrees of freedom of the model with DF = 2 P + Q ( )( P + Q +) t G. Russolillo slide 09

110 Overall model fit measures Chi-square Test - Global Validation Tests H 0 : Σ = C è Good fit H : Σ C Test Statistic 2 ( N ) F χ DF Decision Rule: The model is accepted if p-value 0.05 (We cannot reject the hypothesis H 0 ) or if Chi-square/DF 2 (or other thresholds such as 3 or 5) N.B.: For a fixed level of differences in covariance matrices, the estimate of the Chi-square increases with N The power (i.e. the probability of rejecting a false H 0 ) depends on the sample size. If the sample size is important, this test may lead to reject the model even if the data fit well the model! We cannot use this test to compare model estimated on different sample size G. Russolillo slide 0

111 Overall model fit measures: Test of Close-Fit RMSEA (Root mean square error of approximation) (Steiger and Lind) RMSEA = F 0 DF where F 0 = log C + tr( ΣC ) log Σ ( P + Q) The introduction of degrees of freedom in the index allows for parsimonious models. G. Russolillo slide

112 Overall model fit measures: Test of Close-Fit Use of RMSEA in practice RMSEA is estimated as: RMSEA estimated = F DF N where F = log C + tr( SC ) log S ( P + Q) à The model is accepted if RMSEA 0.05 or, at least, less than 0.08 à à A confidence interval is usually provided It depends very little on N G. Russolillo slide 2

113 An Example of CFA in Psychology The 2nd model specified by Long D.52 D2.22 D3.40 D4.6 Model 2 by LONG Chi-square = DF = 3 P-Value = Chi-square/DF = RMSEA = 0.03 P-Value = 0.0 PSY67.24 XSI PHY67 P-value smaller than α=0.05, H0: S = C REJECTED RMSEA is higher than PSY7.24 XSI PHY7 P-value smaller than α=0., H0: RMSEA < 0.05 REJECTED RMSEA LO 90 HI 90 PVALUE à Model 2 is not accepted. G. Russolillo slide 3

114 Indices based on a baseline model Model Comparison The SATURATED Model: This model contains as many parameter estimates as there are available degrees of freedom or inputs into the analysis. Therefore, this model shows 0 degrees of freedom. [This is the least restricted model possible] The INDEPENDENCE Model: This model contains estimates of the variance of the observed variables only. In other words, it assumes all relationships between the observed variables are zero (uncorrelated), no theoretical relationships. Therefore, this model shows the maximum number of degrees of freedom. [This is the most restrictive model possible and ANY TEST SHALL ALWAYS LEAD TO ITS REJECTION] G. Russolillo slide 4

115 Independence vs Saturated model INDEPENDENCE Model: FIT = min DF = max N_PAR = min N_CONSTR = max CHI2 = max P-value = min SATURATED Model: FIT = max DF = 0 N_PAR = max N_CONSTR = min CHI2 = 0 P-value = max G. Russolillo slide 5

116 Indices based on a baseline model Goodness-of-Fit Index (GFI) This index was initially devised by Joreskog and Sorbom (984) for ML and ULS estimation. It has then been generalised to other estimation criteria. GFI = F F IND Fit function that would results if all parameters were zero (fit function of the INDEPENDENCE Model) If a model is able to explain any true covariance between the observed variables, then F/F IND would be 0 à GFI= The model is accepted if GFI is at least 0.9 G. Russolillo slide 6

117 Indices based on a baseline model Adjusted Goodness-of-Fit Index (AGFI) This index take into account the df of the models by adjusting the GFI by a ratio of the DF used in the model and the max DF available AGFI DF ( ) IND = GFI DF Degree of freedom of the INDEPENDENCE Model If AGFI equals the unity, the model shows a perfect fit. N.B.: It is not bounded below by zero. The model is accepted if AGFI is at least 0.9 G. Russolillo slide 7

118 Indices based on a baseline model Bentler Comparative Fit Index (CFI) CFI = ( N ) F IND DF IND ( N ) F DF ( N ) F IND DF IND Fit function of the INDEPENDENCE Model Degree of freedom of the INDEPENDENCE Model It is bounded between 0 and, with higher value indicating better fit The model is accepted if CFI is at least 0.9 G. Russolillo slide 8

119 Indices based on a baseline model Bentler-Bonnet Non-Normed Fit Index (NNFI) (Also known as Tucker-Lewis index (TLI or rho 2 ) NNFI = F DF F DF IND IND IND IND F DF n It is usually (not necessarely) between 0 and, with higher value indicating better fit The model is accepted if TLI is at least 0.9 G. Russolillo slide 9

120 Root Mean Square Residual RMR = 2 P(P +) P p= p' p p'= (s pp' ˆσ pp' ) 2 Good fit if RMR < 0.05 Standardized RMR = 2 P(P +) P p= p' p p'= (r pp' ˆr pp' ) 2 Good fit if SRMR < 0. G. Russolillo slide 20

121 Comparing model fit measures Information-theory based indexes These indexes are used to compare the fit of nested models AIC = χ 2 + 2t ( ) ECVI = χ 2 n + 2 t n # of parameters to be estimated CAIC = χ 2 +! " + ln n ( ) # $ t ( ) Among two nested model we choose the one with the smallest index G. Russolillo slide 2

122 Model revision G. Russolillo slide 22

123 Testing path coefficients and loadings Path links may be removed if the associated regression coefficients are not statistically significant at a given significance level. SEM output traditionally consist of - the unstandardised estimate for each regression coefficient, - its standard error (S.E.), - the estimate divided by the standard error (C.R. for Critical Ratio) - the probability value (p-value) associated with the null hypothesis that the coefficient is 0. Decision Rule: Coeffficients with a absolute value for C.R. above 2 are considered as significantly different from 0 and the related link is kept. Coeffficients with a absolute value for C.R. below 2 are considered as not significantly different from 0 and the related link is not validated. G. Russolillo slide 23

124 Modification indices Adding a new link to the model.. The Modification Index (MI) represents we the expected drop in overall Chisquare value when a fixed parameter specified is freely estimated in a subsequent run. It is the difference between the Chi-square of the model in which the parameter under investigation is a fixed parameter and the Chi-square of the model in which this parameter is included in the set of parameter to be estimated (free parameters). The MI follows a Chi-square distribution with df. A fixed parameter is included in the model (and so estimated) if the value of the MI is higher than 4. Associated with each MI is an Expected Parameter Change (EPC) value that represents the predicted estimated change, in either a positive or negative direction G. Russolillo slide 24

125 Pay attention.. Any change in the model based on the decrease of chi-square measured by a modification index shall be carefully undertaken by evaluating the impact of the following consequences: Respecification of the Model (getting surprising results is always nice as long as they are not illogical...); Corresponding changes in the parameters (evaluation by means of EPC - Expected Parameter Change); Eventual impossible replication of empirically-based decision (capitalization on chance). G. Russolillo slide 25

126 Sentimental Engagement Developing a causality model with latent variables by means of the Maximum Likelihood Approach to Structural Equation Modeling (SEM-ML) - C. E. Rusbult : Commitment and satisfaction in romantic associations: A test of the investment model. Journal of Experimental Social Psychology, L. Hatcher : A step-by-step approach to using the SAS system for factor analysis and structural equation modeling. SAS Institute, blocks of manifest variables observed on 240 individuals: - Commitment - Satisfaction - Rewards - Costs - Size of investment - Alternative Value G. Russolillo slide 26

127 Sentimental Engagement Exemple of the investiment block indicators Please rate each of the following items to indicate the extent to which you agree or disagree with each statement. Use a response scale in which = Strongly Disagree and 7 = Strongly Agree.. I have invested a great deal of time in my current relationship. 2. I have invested a great deal of energy in my current relationship. 3. I have invested a lot of my personal resources (e.g., money) in developing my current relationship. 4. My partner and I have developed a lot of mutual friends which I might lose if we were to break up. G. Russolillo slide 27

128 Sentimental Engagement Exemple of the satisfaction block indicators. I am satisfied with my current relationship. 2. My current relationship comes close to my ideal relationship. 3. I am more satisfied with my relationship than is the average person. 4. I feel good about my current relationship. G. Russolillo slide 28

129 Sentimental Engagement Exemple of the alternarive value block indicators. There are plenty of other attractive people around for me to date if I were to break up with my current partner. 2. It would be attractive for me to break up with my current partner and date someone else. 3. It would be attractive for me to break up with my partner and play the field for a while. G. Russolillo slide 29

130 Sentimental Engagement F3 F4 F5 e8 v8 v e9 v9 Rewards e e2 v2 Costs e0 v0 e3 e4 e5 v4 v5 v3 e6 v6 Investments Sentimental Engagement SEM model e5 v5 F2 e6 v6 e7 v7 Satisfaction d2 e v e2 v2 e3 v3 Commitment d e4 v4 F e7 e8 e9 v7 v8 v9 F6 Alternatives G. Russolillo slide 30