Structural Equa+on Models: The General Case. STA431: Spring 2015

Size: px

Start display at page:

Download "Structural Equa+on Models: The General Case. STA431: Spring 2015"

Clinton Payne
5 years ago
Views:

1 Structural Equa+on Models: The General Case STA431: Spring 2015

2 An Extension of Mul+ple Regression More than one regression- like equa+on Includes latent variables Variables can be explanatory in one equa+on and response in another Modest changes in nota+on Vocabulary Path diagrams No intercepts, all expected values zero Serious modeling (compared to ordinary sta+s+cal models) Parameter iden+fiability

3 Variables can be response in one equa+on and explanatory in another Variables (IQ = Intelligence Quo+ent): X 1 = Mother s adult IQ X 2 = Father s adult IQ Y 1 = Person s adult IQ Y 2 = Child s IQ in Grade 8 Y 1 = X X Y 2 = 2 + Y Of course all these variables are measured with error. We will lose the intercepts very soon.

4 Modest changes in nota+on Regression coefficients are now called gamma instead of beta Betas are used for links between Y variables Intercepts are alphas but they will soon disappear. We feel free to drop the subscript i; implicitly, everything is independent and iden+cally distributed for i = 1,, n. Y 1 = X X Y 2 = 2 + Y 1 + 2

5 Vocabulary Variables can be Latent or Manifest. Manifest means observable All error terms are latent Variables can be Exogenous or Endogenous Exogenous variables appear only on the right side of the = sign. Think X for explanatory variable. All error terms are exogenous Endogenous variables appear on the led of at least one = sign. Think end of an arrow poin+ng from exogenous to endogenous Betas link endogenous variables to other endogenous variables.

6 Path diagrams e 1 e 2 e 3 e 4 W 1 W 2 V 2 V 1 1 2

7 Path Diagram Rules Latent variables are enclosed by ovals. Observable (manifest) variables are enclosed by rectangles. Error terms are not enclosed Some+mes the arrows from the error terms seem to come from nowhere. The symbol for the error term does not appear in the path diagram. Some+mes there are no arrows for the error terms at all. It is just assumed that such an arrow points to each endogenous variable. Straight, single- headed arrows point from each variable on the right side of an equa+on to the endogenous variable on the led side. Some+mes the coefficient is wrigen on the arrow, but some+mes it is not. A curved, double- headed arrow between two variables (always exogenous variables) means they have a non- zero covariance. Some+mes the symbol for the covariance is wrigen on the curved arrow, but some+mes it is not.

8 Causal Modeling (cause and effect) The arrows deliberately imply that if A è B, we are saying A contributes to B, or partly causes it. There may be other contribu+ng variables. All the ones that are unknown are lumped together in the error term. Are these unknown variables are independent of the variables in the model? Probably not. Some+mes we can get around the problem with instrumental variables.

9 Serious Modeling Once you accept that model equa+ons are statements about what contributes to what, you realize that structural equa+on models represent a rough theory of the data, with some parts (the parameter values) unknown. They are somewhere between ordinary sta+s+cal models, which are like one- size- fits- all clothing, and true scien+fic models, which are like tailor made clothing. So they are very flexible and poten+ally valuable. It is good to combine what the data can tell you with what you already know. But structural equa+on models can require a lot of input and careful thought to construct. In this course, we will get by mostly on common sense. In general, the parameters of the most reasonable model need not be iden+fiable. It depends upon the form of the data as well as on the model. Iden+fiability needs to be checked. Frequently, this can be done by inspec+on.

10 Example: Halo Effects in Real Estate

11 Losing the intercepts and expected values Mostly, the intercepts and expected values are not iden+fiable anyway, as in mul+ple regression with measurement error. We have a chance to iden+fy a func6on of the parameter vector the parameters that appear in the covariance matrix Σ = V(D). Re- parameterize. The new parameter vector is the set of parameters in Σ, and also μ = E(D). Es+mate μ with x- bar, forget it, and concentrate on inference for the parameters in Σ. To make calcula+on of the covariance matrix easier, write the model equa+ons in centered form. The ligle legers c over the variables are invisible.

12 From this point on the models have no means and no intercepts. Now more examples

13 Mul+ple Regression X 1 Y X 2 Y = 1 X X 2 +

14 Regression with measurement error e 1 e 2 e 3 W 1 W 2 V Y = 1 X X 2 + W 1 = X 1 + e 1 W 2 = X 2 + e 2 V = Y + e 3

15 A Path Model with Measurement Error e 1 e 2 e 3 Y 1 = 1 X + 1 W V 1 V 2 Y 2 = Y X + 2 W = X + e 1 V 1 = Y 1 + e 2 1 V 2 = Y 2 + e 3 2

16 A Factor Analysis Model e 1 e 2 e 3 e 4 e 5 X 1 X 2 X 3 X 4 X 5 General Intelligence

17 A Longitudinal Model M 1 M 2 M 3 M 4 P 1 P 2 P 3 P 4

18 Es+ma+on and Tes+ng as Before X Y 1 Y All expected values equal zero. Y 1 = X + 1 Y 2 = Y V (X) =, V ( 1 ) = 1, V ( 2 ) = 2, X, 1, 2 are all independent. Everything is normal.

19 Distribu+on of the data X 1 Y 1,1 Y 1,2 1 A... X n Y n,1 Y n,2 1 A are independent normal with mean zero. = ( ) ( ) 2 ( )+ 2 1 A

20 A General Two- Stage Model Y i = Y i + X i + i F i = X i Y i D i = F i + e i D i (the data) are observable. All other variables are latent. Y i = Y i + X i + i is called the Latent Variable Model The latent vectors X i and Y i are collected into a factor F i. This is not a categorical independent variable, the usual meaning of factor in experimental design. D i = F i + e i is called the Measurement Model.

21 More Details Y i is a q 1 random vector. is a q q matrix of constants with zeros on the main diagonal. is a q p matrix of constants. X i is a p i is a q 1 random vector. 1 random vector. F i (F for Factor) is just X i stacked on top of Y i. It is a (p+q) vector. D i is a k 1 random vector. Sometimes, D i = W i V i 1 random is a k D i is a k e i is a k (p + q) matrix of constants. 1 random vector. 1 random vector. X i, i and e i are independent.

22 Y i = Y i + X i + i X F i = i Y i D i = F i + e i V (X i )= x V ( i )= V (F i )=V Xi Y i = V (Xi ) C(X i, Y i ) C(Y i, X i ) V (Y i ) = = V (e i )= V (D i )=

23 Recall the example e 1 e 2 e 3 Y 1 = 1 X + 1 W V 1 V 2 Y 2 = Y X + 2 W = X + e 1 V 1 = Y 1 + e 2 1 V 2 = Y 2 + e 3 2

24 Y = Y + X + D = F + e Y 1 Y 2 W V 1 V 2 = = Y1 Y X Y 1 Y X + 1 e 1 e 2 e 3 2 V (X) = x = V ( ) = = 0 V (e) = V (D) = ! ! ! 3 1 A

25 Observable variables in the latent variable model (fairly common) These present no problem Let P(e j =0) = 1, so Var(e j ) = 0 And Cov(e i,e j )=0 because if P(e j =0) = 1 Cov(e i, e j ) = E(e i e j ) E(e i )E(e j ) = E(e i 0) E(e i ) 0 = 0 0 = 0 So in the covariance matrix Ω=V(e), just set ω ij = ω ji = 0, i=1,,k

26 What should you be able to do? Given a path diagram, write the model equa+ons and say which exogenous variables are correlated with each other. Given the model equa+ons and informa+on about which exogenous variables are correlated with each other, draw the path diagram. Given either piece of informa+on, write the model in matrix form and say what all the matrices are. Calculate model covariance matrices Check iden+fiability

27 Recall the nota+on Y i = Y i + X i + i X F i = i Y i D i = F i + e i V (X i )= x V ( i )= V (F i )=V Xi Y i = V (Xi ) C(X i, Y i ) C(Y i, X i ) V (Y i ) = = V (e i )= V (D i )=

28 For the latent variable model, calculate Φ = V(F) So, Y = Y + X + Y Y = X + IY Y = X + (I )Y = X + (I ) 1 (I )Y = (I ) 1 ( X + ) Y = (I ) 1 X + (I ) 1 V (Y) = V (I ) 1 X +(I ) 1 = V (I ) 1 X + V (I ) 1 = (I ) 1 V (X) (I ) 1 > +(I ) 1 V ( )(I ) 1> = (I ) 1 11 > (I ) 1> +(I ) 1 (I ) 1>

29 For the measurement model, calculate Σ = V(D) D = F + e ) V (D) = V ( F + e) = V ( F)+V(e) = V (F) > + V (e) = > + =

30 Two- stage Proofs of Iden+fiability Show the parameters of the measurement model (Λ, Φ, Ω) can be recovered from Σ= V(D). Show the parameters of the latent variable model (β, Γ, Φ 11, Ψ) can be recovered from Φ = V(F). This means all the parameters can be recovered from Σ. Break a big problem into two smaller ones. Develop rules for checking iden+fiability at each stage.

31 Copyright Informa+on This slide show was prepared by Jerry Brunner, Department of Sta+s+cs, University of Toronto. It is licensed under a Crea+ve Commons Agribu+on - ShareAlike 3.0 Unported License. Use any part of it as you like and share the result freely. These Powerpoint slides are available from the course website: hgp://

Structural Equa+on Models: The General Case. STA431: Spring 2013

Structural Equa+on Models: The General Case STA431: Spring 2013 An Extension of Mul+ple Regression More than one regression- like equa+on Includes latent variables Variables can be explanatory in one equa+on