SEM 2: Structural Equation Modeling

Transcription

1 SEM 2: Structural Equation Modeling Week 1 - Causal modeling and SEM Sacha Epskamp

2 Course Overview Mondays: Lecture Wednesdays: Unstructured practicals Three assignments First two 20% of final grade, last 10% of final grade Final project Presentations and report, 50% of final grade

3 The SEM 2 team Sacha Epskamp (sacha.epskamp@gmail.com) Gaby Lunansky (G.Lunansky@uva.nl) Eiko Fried (eiko.fried@gmail.com)

4 Schedule Week 1 Introduction to Structural Equation Modeling Monday May 8 Lecture Wednesday May 10 Practical Week 2 Causality and equivalent models Monday May 15 Lecture Wednesday May 17 - Practical Week 3 Latent variable models and network models Wednesday May 22 Lecture Wednesday May 24 - Practical Week 4 Presentations Monday May 29 Lecture Wednesday May 31 - Practical

5 Individual Assignments Each week, the assignment will be made available 11:00 on Wednesday, and will be due 11:00 the next Wednesday. Each assignment will contribute to 20% or 10% (last week) of your grade. Work on the assignments alone. Hand in a PDF file and an.r file (in case R was used). If you use Jasp, hand in the Jasp object as well as a screenshot of the options used. Make sure your PDF report is as standalone readable as possible. E.g., if you are asked to report a factor loading matrix, then report it in the PDF and not just say look at.r file. Assignments are due before 11:00. If you do not hand in an assignment before 11:00, you will get a 1. If you encounter any problems, or have any feedback, please let me know before the deadline, as then I can take it into account or help you.

6 Final Project Three options: 1. Perform a SEM analysis on your own data and write a report (individual) 2. Write a manual for semplot, Onyx, Jasp or Lavaan (individual or with a partner) 3. Research an area or a topic of SEM in more detail and teach fellow students about it See syllabus on blackboard Claim your project using the discussion board on blackboard as soon as possible! If you have another idea on a project not listed above, talk to me

7 Causal modeling This course will introduce structural equation modeling (SEM) In SEM, we will discuss modeling complex causal hypotheses Again, all variables are assumed normally distributed and all associations are assumed linear Causal hypotheses can be specified between observed and latent variables CFA is a special case of SEM

8 Causal models X Y X causes Y

9 Endogenous and exogenous Exogenous (independent) variables are variables of which the causal origin are not modeled Exogenous variables have a variance (sometimes not drawn) Exogenous variables, except residuals, are allowed to covary (sometimes not drawn) Latents: ξ (xi); observed: x (x is also used for indicators of latent exogenous variables) Residuals are exogenous Endogenous (dependent) variables are variables of which the causal origin are modeled Simply stated: endogenous variables have incoming arrows Endogenous variables do not have a variance by themselves Latents: η (eta); observed: y The causal equation for endogenous variables can be derived from the path diagram by summing all incoming edges

10 X Y X is exogenous, Y is endogenous

11 X Y y i = x i Causal effect goes from right hand side to left hand side. Experimentally changing x will change y, experimentally changing y will not change x

12 X β Y y i = βx i

13 X β Y ε y i = βx i + ε i

14 Exogenous variables have a variance (often not drawn) σ x 2 β X Y ε θ y i = βx i + ε i x N(µ x, σ x ) ε N(0, θ)

15 y i = βx i + ε i x N(µ x, σ x ) ε N(0, θ) Three observations (variance of x and y and covariance between x and y), three unknowns. Solvable! But why? Var(x) = σ 2 x Var(y) = β 2 σ 2 x + θ Cov(x, y) = βσ 2 x

16 Covariance Algebra Let Var(x) indicate the variance of x and Cov(x, y) indicate the covariance between x and y. The following rules can be derived: Var(x) = Cov(x, x) Cov(x, α) = 0 Cov(x, y) = Cov(y, x) Cov(αx, βy) = αβcov(x, y) Cov(x + y, z) = Cov(x, z) + Cov(y, z) Where α and β are constants (parameter) and x, y, and z are random variables.

17 Covariance Algebra Some consequences: Cov(αx + βy, z) = Cov(αx, z) + Cov(βy, z) = αcov(x, z) + βcov(y, z) Var(x + y) = Var(x) + Var(y) + 2Cov(x, y) Var(βx) = β 2 Var(x) Where α and β are constants (parameter) and x, y, and z are random variables.

18 Matrix Covariance Algebra Let Var(x) indicate the variance covariance matrix of vector x and Cov(x, y) indicate the covariance matrix between x and y. Then the following rules can be derived: Var(x) = Cov(x, x) Cov(Ax, By) = ACov(x, y)b Var(Bx) = BVar(x)B Cov(x + y, z) = Cov(x, z) + Cov(y, z) Where A and B are constant (parameter) matrices.

19 y i = βx i + ε i Var(x) = σ 2 x Var(y) = Var(βx + ε) = Cov(βx + ε, βx + ε) = Cov(βx, βx + ε) + Cov(ε, βx + ε) = Cov(βx, βx) + Cov(βx, ε) + Cov(ε, βx) + Cov(ε, ε) But since x is not correlated with the residuals, Cov(x, ε) = 0 and thus: Var(y) = β 2 Cov(x, x) + Cov(ε, ε) = β 2 Var(x) + Var(ε)

20 y i = βx i + ε i Cov(x, y) = Cov(x, βx i + ε i ) = Cov(x, βx i ) + Cov(x, ε i ) = βcov(x, x i ) = βvar(x)

21 Path analysis θ 1 θ 2 β 1 β 2 x y 1 y 2 x is exogenous, and both y 1 and y 2 are endogenous. θ 1 is the variance of ε 1. Causal model for y 2 : y i2 = β 2 y i1 + ε i2 y i2 = β 2 (β 1 x i + ε i1 ) + ε i2

22 θ 1 θ 2 β 1 β 2 x y 1 y 2 Number of parameters: 2 regressions +2 residual variances +1 exogenous variance (not drawn) = 5, number of observations: 3 variances and 3 covariances. 1 degree of freedom!

23 θ 1 θ 2 β 1 β 2 x y 1 y 2 Implied covariance between x and y 2 : Cov (x, y 2 ) = Cov (x, β 2 (β 1 x i + ε i1 ) + ε i2 ) = Cov (x, β 2 β 1 x + β 2 ε 1 + ε 2 ) = Cov (x, β 2 β 1 x) + Cov (x, β 2 ε 1 ) + Cov (x, ε 2 ) = β 1 β 2 Cov (x, x) = β 1 β 2 σ x

24 How many degrees of freedom?

25 How many degrees of freedom? 7 8/2 = 28 observed variances and covariances 7 regressions +7 variances +1 covariance = 15 parameters 13 degrees of freedom

26 Wright s path tracing Rules The correlation between any two variables can be expressed as the sum of the compound paths connecting them. To obtain a compound path: Trace backwards, change direction at a two-headed arrow, then trace forwards Do not go forward and then backward You can never pass out of one arrow head and into another arrowhead: heads-tails, or tails-heads, not heads-heads Must contain one, and only one, variance or covariance (bidirectional edge) Then to obtain the implied (co)variance: Compute the product of coefficients in each route between the variables of interest Sum over all distinct routes, where pathways are considered distinct if they contain different coefficients, or encounter those coefficients in a different order

27 Compound paths between A and B:

28 Compound paths between A and B: A B

29 Compound paths between A and B: A B Cov (A, B) = h

30 Compound paths between C and D:

31 Compound paths between C and D: C A A D and C A B D

32 Compound paths between C and D: C A A D and C A B D Cov (C, D) = a(var(a))b + ahc

33 Compound paths between C and itself:

34 Compound paths between C and itself: C W W C and C A A C

35 Compound paths between C and itself: C W W C and C A A C Var (C) = a 2 (var(a)) + Var(W )

36 Compound paths between F and G:

37 Compound paths between F and G: F D B B E G, F C A B E G, and F D A B E G.

38 Compound paths between F and G: F D B B E G, F C A B E G, and F D A B E G. Cov (C, D) = fc(var(b))dg + fbhdg + eahgd

39 Structural Equation Modeling Structural equation modeling (SEM) extends confirmatory factor analysis (CFA) by modeling the variance covariance matrix of latent variables with a path model Allows one to test causal hypotheses on the latent variables Includes path analysis for observed variables: Define one latent per observed variable Set factor loading to 1 Set residual variance to 0 In lavaan: use sem() function and define structural relationships using the ~ operator (same as used in regression analysis)

40

41

42

43

44

45

46

47

48 Exogenous measurement model (using full LISREL notation, it gets easier!): x i = Λ x ξ i + δ i x N(0, Σ x ) ξ N(0, Φ) δ N(0, Θ δ ), Allows you to derive the model-implied variance covariance matrix: Σ x = Var(x) = Var(Λ x ξ + δ) = Λ x Var(ξ)Λ x + Var(δ) = Λ x ΦΛ x + Θ δ

49 Endogenous model: y i = Λ y η i + ε i η i = Γξ i + Bη i + ζ i y N(0, Σ y ) ξ N(0, Φ) ζ N(0, Ψ), Note that Ψ is now diagonal! ε N(0, Θ ε ) Only different from CFA model in the added regression parameters Γ and B. Note that η i appears twice in the structural model, so let s first solve that: η i = Γξ i + Bη i + ζ i η i Bη i = Γξ i + ζ i (I B)η i = Γξ i + ζ i η i = (I B) 1 Γξ i + (I B) 1 ζ i

50 Now: Var(η) = Var ( (I B) 1 Γξ i + (I B) 1 ) ζ i = Var ( (I B) 1 ) ( Γξ i + Var (I B) 1 ) ζ i = (I B) 1 ΓΦ ( (I B) 1 Γ ) + (I B) 1 Ψ(I B) 1 Which can be used in: Var(y) = Λ y Var(η)Λ y + Θ ε And Cov(x, y) can similarly be derived. Way too complicated...

51 All-y notation Much easier, just treat exogenous variables as endogenous variables. All latents are then contained in η and all indicators in y. Only important to note is that Ψ then contains both exogenous variances and covariances (all freely estimated) as well as latent residual variances (usually without covariances).

52 All-y model: y i = Λη i + ε i η i = Bη i + ζ i = (I B) 1 ζ i y N(0, Σ) ζ N(0, Ψ) ε N(0, Θ) Results in: Σ = Var(y) = Var (Λη + ε) = Var (Λη) + Var (ε) = ΛVar (η) Λ + Θ = ΛVar ( (I B) 1 ζ ) Λ + Θ = Λ(I B) 1 Ψ(I B) 1 Λ + Θ

53 SEM model: Σ = Λ(I B) 1 Ψ(I B) 1 Λ + Θ Simply the CFA model with one extra matrix: B encoding regression parameters. Element β ij encodes the effect from variable j to variable i (note, this is opposite of how normally a directed network is encoded). The same identification rules as in CFA apply: Latent variables must be scaled by setting one factor loading or (residual) variance to 1 Model must have at least 0 degrees of freedom Next week we will discuss equivalent models.

54 θ 1 θ 2 β 1 β 2 x y 1 y 2

55 ψ 11 ψ 22 ψ 33 η 1 β 21 η 2 β 32 η y 1 y 2 y ψ Λ = 0 1 0, Ψ = 0 ψ 22 0, Θ = 0 0 0, B = β ψ β 32 0

56 ψ 11 ψ 22 ψ 33 η 1 β 21 η 2 β 32 η y 1 y 2 y 3 Lavaan automatically adds the latent dummy variables for you! The model is just: y2 ~ y1 y3 ~ y2

57 ψ 11 ψ 22 ψ 33 β 21 β 32 η 1 η 2 η 3 1 λ 21 1 λ 42 1 λ 63 y 1 y 2 y 3 y 4 y 5 y 6 θ 11 θ 22 θ 33 θ 44 θ 55 θ λ Λ =, Ψ = ψ ψ 0 λ , B = β ψ 33 0 β λ 63 Θ diagonal as usual.

58 ψ 11 ψ 22 ψ 33 β 21 β 32 η 1 η 2 η 3 1 λ 21 1 λ 42 1 λ 63 y 1 y 2 y 3 y 4 y 5 y 6 θ 11 θ 22 θ 33 θ 44 θ 55 θ 66 Lavaan model (using sem()): eta1 =~ y1 + y2 eta2 =~ y3 + y4 eta3 =~ y5 + y6 eta2 ~ eta1 eta3 ~ eta2

59 Structural Equation Modeling (SEM) SEM is a more general modeling framework than CFA Allows for modeling complex causal hypotheses Requires fitting CFA model (as it is nested in CFA model) Fitting SEM models using lavaan: Use sem() instead of cfa() Specify regressions using the ~ operator Same identification rules as in CFA apply Models can be assessed and compared in the same way as CFA models

60 This week: start looking for final project topic!