Psychology 830. Introduction to Covariance Structure Modeling

Transcription

1 Psychology 830 Introduction to Covariance Structure Modeling Class Notes Winter 2004 Michael W. Browne Ohio State University These notes have been developed collaboratively by Professors Michael Browne and Robert MacCallum of the Ohio State University. Please do not copy or distribute them without permission.

2 Index Module Page 1 Introduction to Covariance Structure Modeling 1 2 Path Diagrams 14 3 Mathematical Representation of the Data Model: RAM 32 4 Deriving the Covariance Matrix from the Data Model 47 5 Part I: Special Cases of the RAM Model 62 Part 2: Relationships Between the RAM and LISREL models 70 6 Identification and Falsifiability 86 7 Parameter Estimation: Discrepancy Functions and Maximum Likelihood Estimation Parameter Estimation: GLS, ADF and OLS Discrepancy Functions Using RAMONA in SYSTAT for Windows Illustrations of RAMONA Printouts Evaluation of Solutions: Parameter Estimates and Discrepancy of Approximation Sample Discrepancy, Discrepancy of Estimation and Overall Discrepancy Illustrations of Assessing Model Fit: Other Approaches and Indexes Power Analysis and Determination of Minimum Sample Size Equivalent Models Model Modification and Re-estimation Causal Indicators Analysis of Correlation vs. Covariance Matrices Missing Data 256 References 264

3 Greek Alphabet A! Alpha N / Nu B " Beta B 0 Xi > # Gamma O o Omicron? $ Delta C 1 Pi E % & Epsilon P 3 Rho Z ' Zeta D 5 Sigma H ( Eta T 7 ) * Theta E 8 Upsilon I + Iota F 9 : Phi K, Kappa X ; Chi A - Lambda G < Psi M. Mu H = Omega Seldom used X Expected value

4 MODULE 1 INTRODUCTION TO COVARIANCE STRUCTURE MODELING We begin with a general statement about covariance structure modeling. We shall gradually add more detail and definition to this statement. Covariance structure modeling (CSM) is a general technique that allows us to specify, estimate, and evaluate models of relationships among variables. A model in this context is an a priori hypothesis about a pattern of linear relationships among a set of variables. Specification involves the formal statement of the model in terms of mathematical equations or a path diagram. We shall study both methods of model specification, first considering path diagrams and then systems of mathematical equations. Estimation involves the determination of optimal estimates of parameters of the model, where the parameters are such things as weights indicating the effect of one variable on another. Evaluation involves the assessment of the degree of correspondence between the model and observed data as well as the degree to which the parameter estimates make sense in the light of substantive theory. The objective is to develop a parsimonious model that provides an explanation of observed relationships among variables that is interpretable and that fits the observed data adequately. Reference Material on CSM At the back of this note packet you will find a list of references. That list includes both journal articles that are cited herein with reference to specific topics, as well as general references on methodology. Some references of general interest are the following: Books: Bollen (1989). A comprehensive reference, but becoming a bit outdated. Kline (1998). A non-mathematical presentation of models and methods. Maruyama (1998). Organized around LISREL framework. Review of several introductory text books on structural equation modeling: Steiger, J. H. (2001). Recommended reading! A pdf file is available by from MWB. 1

5 Comprehensive technical review article: Browne and Arminger (1995). Other review articles: Bentler (1980). Early Annual Review chapter on latent variable models. Bentler (1986). Review of methodological developments published in Psychometrika. MacCallum and Austin (2000). Annual Review chapter reviewing use of CSM in applied research. Annotated bibliographies of CSM literature: Austin and Wolfle (1991). Austin and Calderón (1995). Cautionary articles: Cliff (1983). Breckler (1990). Also see MacCallum and Austin (2000). Articles on how to report results of CSM for publication: Raykov, Tomer, and Nesselroade (1991). Hoyle and Panter (1995). McDonald and Ho (2002) Also see MacCallum and Austin (2000). Introduction to CSM To begin to develop a detailed understanding of this methodology, we consider the type of data to which CSM is applicable. The technique is most commonly applied to correlational, or. observational data That is, the data are not experimental; no variables are manipulated or controlled. However, the method can be applied with great benefit to experimental data (Russell et al., 1998), especially for studying effects of 2

6 mediators and covariates as well as for incorporating the study of latent variables into analysis of experimental data. We observe a sample of R observations on a set of : manifest variables (MVs). The manifest variables are also called measured variables or observed variables. Manifest variables are usually not manipulated and values are not prespecified. Thus, we have a sample data matrix of order R :, with rows representing individuals and columns representing MVs. From this data matrix we can compute a covariance or correlation matrix for the MVs, of order : :Þ A correlation matrix has values of 1 on the diagonal and correlations among MVs off the diagonal. A covariance matrix has variances of MVs on the diagonal, and covariances off the diagonal. p p N p p Data Matrix p Correlation or Covariance Matrix The correlation or covariance matrix summarizes relationships among the MVs. A basic principle of SEM, just as in factor analysis, is that these relationships do not arise by chance, but rather arise as a result of some underlying processes or structure. An objective of SEM is to investigate the underlying processes that generate relationships amongst manifest variables. This is accomplished by formulating and testing one or more specific models of relationships among variables. Thus, SEM is a confirmatory rather than exploratory approach to data analysis. 3

7 Exploratory No explicit prior theory specified or tested. e.g. Multiple Regression Factor Analysis with rotation (Exploratory factor analysis) Confirmatory Researcher has an explicit prior theory that accounts for relationships and predicts patterns of correlations. This prior theory is stated as a formal model in either mathematical or graphical form. The model will contain some parameters, representing such things as linear influences of one variable on another. One objective of SEM is to estimate and interpret those parameters. The model will also specify a structure for the data. That is, if the model is true, then the MVs will exhibit a particular pattern of relationships. This allows us to ask the question: Are the data consistent with the theory? Answers: Yes À The theory is tenable No À Something is wrong with the theory or the data. It is important that an answer of "no" be at least possible, meaning that the model is FALSIFIABLE (i.e. that it can be shown to be incorrect in some situations.) In general, this confirmatory modeling approach involves a strategy wherein we have a particular theory to be evaluated. From that theory we construct a formal model, or competing alternative models. We then collect data on which to test the model(s). The models are fitted to the data and evaluated. If necessary, the theories/models may be modified and tested again on new data. 4

8 Covariance structure modeling, then, is a technique for evaluating models that are intended to explain the structure of covariances among MVs. We are modeling a covariance structure. Other more or less synonymous terms for this technique include: Analysis of Covariance Structures (general class of models) Structural Equation Modeling (less inclusive) LISREL (based on widely used computer software) Causal Modeling (discourage use of this term) Covariance Structures are a general class of models that include ARMA time Series models, multiplicative models for multi-faceted data, circumplex models as well as structural equation models. Structural Equation Models are primarily concerned with linear regression relationships between variables, some of which are manifest or observable and the rest latent or unobservable such as the common and unique factors in factor analysis. This course will be concerned primarily with Structural Equation Modeling (SEM). The Data Model The model of interest is first specified as a "data model." The data model involves both manifest variables (MVs) and latent variables (LVs). A latent variable is a hypothetical construct that can not be directly measured; e.g., depression, intelligence, attitude. For example, a common factor in factor analysis is a latent variable. In most models multiple MVs are used as indicators to represent each given LV. An indicator is an MV used as an approximate measure of an LV (e.g., an intelligence test as an indicator of the LV intelligence). The data model provides an hypothesized pattern of relationships among a set of manifest variables and latent variables. The data model is intended to account for individuals' observed scores on MVs. This model expresses the MVs as functions of other MVs and LVs. For reasons that will become clear subsequently, we do not estimate or evaluate the data model directly. 5

9 The Covariance Structure From the data model we derive a covariance structure model. This model expresses the covariances among the MVs as functions of model parameters. It is intended to account for covariances among MVs. Example: Parallel tests. Suppose we have a set of : tests of intelligence, B" ß B# ß ÞÞÞß B:. We hypothesize that they are parallel tests; i.e., that the true score for an individual is the same on each test, and that the error variance is the same for all tests. Data Model: B4 œ> / 4 4œ"ßáß: All tests have the same true score and error variance # VarÐ/ Ñ œ for all 4Þ 4 5 / This data model implies that the variance of each test is the same; let that value be!. Also, that the covariance between any two tests is the same; let that value be ". Thus, the data model implies a covariance structure: Ô! " " â "! " â Covariance structure: D œ Ö Ù " "! â Õ ã ã ã äø where! œ 5 5 " œ 5 # # > / # > i.e. In the population, all variances are equal and all covariances are equal. This provides an example of a data model and the implied covariance structure. Note the relationship between the data model and the covariance structure: Data model Ê Covariance Structure That is, the covariance structure is derived from the data model. but 6

10 Covariance structure /Ê Data Model Thus, a research finding that supports a particular covariance structure does not imply that the data model that generated that covariance structure must be true. Rather, the data model is tenable. Other data models may imply the same or very similar covariance structures. However, the contrapositive is valid: Covariance Structure false Ê Data Model false That is, a research finding that indicates that a particular covariance structure does not hold implies that the data model that generated that covariance structure must be false. We thus have a basis for testing the correspondence between the covariance structure and observed data. Obtain a sample of data on the MVs, and compute a sample covariance matrix, W. Compare the W to the covariance structure implied by the model. We wish to evaluate the departure of the sample covariance matrix W from the pattern œ could be due to random sampling error alone. is small enough to be acceptable. Consider again the example where we hypothesize that different tests are parallel tests. Suppose we have five tests and we administer them to a sample of subjects. From the sample data we compute the sample covariance matrix. If the hypothesis of parallel tests is correct, then this covariance matrix should exhibit approximately the structure shown above: diagonal elements (variances) equal, and off-diagonal elements (covariances) equal. If this structure holds in the sample covariance matrix, the data model of parallel tests is tenable. (We can not say that the data model is true because the covariance structure does not Ê the data model.) If not you reject the data model. i.e. You reject the hypothesis that all variables have the same true score. That is, if the covariance structure is rejected, then the data model can not be valid. 7

11 Historical Perspective: A brief review of the historical development of SEM. Bollen, pp. 4-9) Brief summary: LISREL (Jöreskog, 1973; also Keesling, Wiley) arose from the merging of methods from two different fields or traditions: path analysis and factor analysis. Path Analysis Sewall Wright (1934) (Biometrician) Testing models of causal relationships among manifest variables. Example: Suppose we wish to test an hypothesis about the pattern of relationships among seniority, job performance, salary, and job satisfaction. Our model is that seniority and performance influence salary, which in turn influences satisfaction. Also, seniority has a direct influence on satisfaction. Mathematically, we could write a set of equations representing these effects (oversimplified): Salary œ " (Seniority) " (Performance) " # Satisfaction œ " (Seniority) " (Salary) $ % We could also construct a path diagram, where each MV is represented by a rectangle and each linear effect is represented by a single-headed arrow: Seniority β 1 Salary β 2 β 4 β 3 Performance Job Satisfaction 8

12 This example incorporates the concept of sequential regression equations. A dependent variable in one regression equation can be an independent variable in another (e.g., Salary). Wright developed (ad hoc) methods for estimating parameters of such models. Approach heavily used in econometrics (Simultaneous equation models). Note that path analysis is a technique that is applied to manifest variables only. Factor Analysis In factor analysis we make use of the concept of a latent variable (LV) or common factor. An LV is a hypothetical construct. Not directly measured. An LV can be approximated, or represented with error, by a manifest variable (indicator). A path diagram representation of the relationship between an LV (math ability) and an indicator (math test), with error reflecting the fact that the indicator is not a perfect measure of the LV. This diagram follows the factor analytic concept that the MV is influenced linearly by the LV. The arrow representing the influence of the LV on the MV corresponds to a factor loading. The error term corresponds to a unique factor, and its variance is unique variance of the MV. Error Math Test Math Ability We need more than one indicator to represent an LV. This leads to the notion of multiple indicators and measurement models. A single indicator provides an inadequate representation of an LV. Furthermore, with only a single indicator it is not possible to estimate the error variance of the indicator in SEM. Multiple indicators provide a better representation of the LV, which is defined in effect as that which the multiple indicators have in common with each other. 9

13 When using multiple indicators, it is feasible to estimate the error variance associated with each indicator. Following is a path diagram showing multiple indicators for 2 LVs. As a whole, this model is a measurement model showing relationships between the LVs and their indicators (in confirmatory factor analysis). Math Verbal MT1 MT2 MT3 VT1 VT2 VT3 E1 E2 E3 E4 E5 E6 Structural equation modeling represents the merging of path analysis and factor analysis. From path analysis we draw the notion of modeling patterns of linear relationships among variables. From factor analysis we draw the notion of LVs. In combining these ideas, we gain the capability for modeling patterns of linear relationships among LVs. That is, we can represent in a single model the relationships of LVs to their indicators (measurement model) as well as hypothesized relationships of LVs to each other (sometimes called the structural model). 10

14 For example, following is a path diagram showing 3 LVs, each with 3 indicators, as well as an hypothesized pattern of linear effects among the LVs. A C B In a sense, such a model represents an extension of confirmatory factor analysis. CFA allows us to specify relationships of multiple LVs to their indicators, but allows only correlational (non-directional) influences of LVs to each other. In a structural equation model, we can test models of directional influences among LVs. Model Specification: Overview of Structural Equation Modeling Path Diagram (Pictorial Representation) Æ Mathematical Framework 11

15 The path diagram depicts the data model. As we shall see, we can specify the data model as well mathematically as a system of equations. From the data model we will derive a covariance structure model, which accounts for relationships amongst manifest variables in terms of relationships amongst latent variables. In this model, covariances among manifest variables are functions of the parameters of the model. Fitting the Model to Data: The covariance structure specifies the mathematical relationship of model parameters to MV variances and covariances. When we then observe a sample covariance matrix, one objective is to determine values of model parameters. However, the model will not fit the sample covariance matrix exactly. Given the sample covariance matrix, we will want to obtain "best" estimates of the parameters of the model, so that the model will explain our sample data as well as possible (minimum discrepancy). There are many computer programs that provide parameter estimates for structural equation models: LISREL (Jöreskog & Sörbom, 1996)) EQS (Bentler, 1995) CALIS (SAS) RAMONA (Browne & Mels, 1998, SYSTAT) SEPATH (Steiger, 1999, CSS STATISTICA) AMOS (Arbuckle, 1999, SPSS) MECOSA (Arminger & Schepers) MPlus (Muthen) Mx (Neale, 1997) Evaluating Solutions: Once a model is fitted to sample data and parameter estimates are obtained, the next step is to evaluate the solution. Several types of information are obtained and evaluated: Overall Fit, Fit Measures, Residuals. Parameter estimates, Confidence Intervals, Fit of Submodels. 12

16 General Comments SEM is a general methodology for specifying, fitting, and evaluating models of relationships among variables. Many commonly used models or common hypotheses can be recognized as special cases of SEM: Equality patterns for covariance/correlation matrices Regression Path Analysis Factor Analysis SEM can enhance the quality of research. It puts demands on the researcher to think carefully. The researcher must specify latent variables and indicators, as well as hypothesized directions of influence amongst latent variables. All this must be taken into consideration at the research design stage. The model is then fitted to the data, evaluated and possibly modified. 13

17 MODULE 2 PATH DIAGRAMS Introduction A path diagram is a graphical representation of a data model, specifying the hypothesized pattern of relationships among the MVs and LVs. Although there are some conventions for construction of path diagrams, there are also variations. In particular, some aspects of models may not be completely and explicitly represented in some path diagrams. Our approach will be to construct path diagrams so as to completely and explicitly represent all aspects of our model. This approach requires the investigator to be explicit about all aspects of the model, and also aids in the subsequent specification of the model for computer programs. In this module we shall begin by studying specific aspects of and rules for the construction of path diagrams. Eventually we shall combine these pieces into examples of fully specified diagrams. There exists a corresponding mathematical representation of models that captures the information in the path diagrams in a system of equations. We shall begin the study of the mathematical framework in Module 3. Representation of variables Manifest variables are represented by squares or rectangles: Manifest Variable Latent variables are represented by circles or ellipses: Latent Variable 14

18 Directional linear relationships are represented by single-headed arrows: Attitude Behavior The direction of such linear influence is determined by prior theory and becomes part of the testable model. Note that every circle or ellipse (latent variable) must emit at least one arrow. In our study of CSM, we shall confine our models to those involving only linear effects and relationships. Associated with such a directional effect is a numerical value analogous to a regression coefficient, representing the linear influence of the variable emitting the path on the variable receiving the path. Graphically, this coefficient would correspond to the slope of the regression line in a scatter plot. When the model is specified, these values are generally unknown. They are parameters to be estimated. Nondirectional (correlational) relationships are represented by doubleheaded arrows: Math Ability Verbal Ability Again, there is a numerical value associated with this two-headed arrow. This value represents the covariance between the two variables. If the two variables are standardized, then this value corresponds to the correlation between the two variables. When the model is specified, these values represent unknown parameters to be estimated. 15

19 The variance of a variable is represented by a double-headed arrow with both arrowheads touching the same variable: Attitude Exogenous Variables Every variable in the system that does not receive a one-headed arrow is exogenous, meaning its causes lie outside the system. Attitude Behavior Here, Attitude is exogenous. Every exogenous variable, whether it is an MV or LV, has a variance parameter. This variance parameter should be explicitly represented: Attitude Behavior Exogenous variables may be connected to each other by double-headed arrows, indicating covariance. See math ability and verbal ability, above. Endogenous Variables Every variable in the system that does receive at least one one-headed arrow is endogenous, meaning its causes lie at least partially within the system. In the model above, Behavior is endogenous. 16

20 Endogenous variables do not have variance parameters. The variance of an endogenous variable is implied by the model, and is a function of other parameters of the model. Endogenous variables may not be connected to each other or to exogenous variables by double-headed arrows. Such covariances are not legitimate parameters; rather, these values are implied by the model and are functions of other model parameters. For example, the two-headed arrow connecting Salary and Satisfaction in the following model is NOT a valid parameter. That is, the covariance between salary and satisfaction is implied by their joint dependence on seniority. Salary Seniority Satisfaction Error Terms for Endogenous Variables Usually, every endogenous variable has an error term, representing that portion of the variable not accounted for by the variables that are influencing it. z1 Attitude Behavior Here, z1 is the error term in the endogenous LV Behavior, representing that part of Behavior not accounted for by Attitude. Note that the error term is represented explicitly as a LV. It is not directly observable. Also, it is shown as having a variance parameter because it is exogenous. Note that 17

21 VarÐBehavior Ñ œ " # ÐVarÐAttitudeÑÑ VarÐD"Ñ In a sort of shorthand representation, error terms are often represented in path diagrams as follows: Attitude Behavior This representation is not explicit; it does not show the error term as an LV and does not show its variance. Error terms themselves are typically exogenous variables and thus have variance parameters. (It is possible, when analysing longitudinal data, to specify and fit models where error terms follow an autoregressive time series and are endogenous. These are not studied here.) Conventionally, the single-headed arrow emitted by an error term has a coefficient of and the error variance is a parameter to be estimated. For example: z1 Attitude Behavior Exogenous variables do not have error terms. Latent Variables with Multiple Indicators 18

22 LVs with multiple indicators exert directional influences from the LV to the indicators. Each indicator is then an endogenous variable and has a corresponding error term. Attitude Measure 1 e1 Attitude Attitude Measure 2 e2 Attitude Measure 3 e3 For MVs serving as indicators of LVs, the error terms correspond to the unique factors in factor analysis. For other endogenous variables (e.g., Behavior in previous model), the error terms correspond to residuals in regression analysis. Some literature refers to these two kinds of errors as "errors in variables" and "errors in equations." Errors in variables correspond to unique factors in factor analysis and represent lack of exact correspondence of an indicator to the corresponding LV. Errors in equations correspond to regression residuals in endogenous variables in a model (other than indicators of LVs.) Such errors reflect lack of perfect prediction of those endogenous variables. Setting the Scale of LVs An LV is a hypothetical construct, not directly measured. It has no empirical scale of measurement. However, in our model fitting we shall wish to estimate coefficients representing effects of LVs on other variables. The values of these coefficients depend on the scales of the LVs. Therefore, we must establish a scale for each LV. If we do not do this, it will be impossible to estimate model parameters. Setting the scale of the LVs involves imposing an " " identification condition in the specification of the model. Later we shall study the issue of identification. For present purposes, identification refers to the issue of whether it is possible to obtain unique estimates of the parameters of the 19

23 model. If we do not set the scale of each LV, then some parameters will not be identified. There are two alternative ways that are generally employed to establish a scale for each LV: (1) Fix at a value of the coefficient corresponding to one path emitted by the LV. Attitude Measure 1 e1 Attitude Attitude Measure 2 e2 Attitude Measure 3 e3 This approach equates the scale of the LV with the scale of the designated indicator true scores. Choice of indicator is arbitrary. (2) Fix at a value of the variance of the LV. Attitude Measure 1 e1 Attitude Attitude Measure 2 e2 Attitude Measure 3 e3 This approach defines the LV as having a variance of ; i.e., as being standardized. We encourage use of this approach because it simplifies interpretation of other parameters. If MVs and constructs are standardized, then paths 20

24 connecting them can be interpreted as standardized regression coefficients (one-directional paths) or correlations (two-directional paths). This approach is typically used in factor analysis (exploratory or confirmatory); factors are usually defined as having a variance of. If this approach is used for an exogenous LV, then the variance is a parameter defined as having a prespecified (fixed) value. If this approach is used for an endogenous LV, that variance is not a parameter but is rather a function of other parameters in the model. Therefore, the setting of the variance of an endogenous LV to constitutes the imposition of a constraint on other model parameters. In the path diagram, we represent this circumstance using a dotted line to correspond to the fixed variance of the endogenous LV. For example: Att Meas 1 e1 Income β Att Meas 2 Attitude e2 z1 Att Meas 3 e3 Note that in this model, the variance of the LV Attitude is not a parameter itself but is a function of model parameters: Z+<ÐAttitudeÑœ" # Z+<ÐIncomeÑ Z+<ÐD"Ñ Thus, fixing the variance of Attitude at implies a constraint on these parameters. The facility for setting variances of endogenous LVs to is not available in most CSM software, but is available in the software we shall use in this course (RAMONA). This is a very convenient and useful mechanism in that 21

25 it greatly simplifies interpretation of parameter estimates because the LVs in question can be considered as standardized. In most other CSM software, the scales of endogenous LVs must be set by fixing one of the associated factor loadings to, which does not yield simple interpretation of parameter estimates. Parameters of the Model Every single-headed and double-headed arrow in the path diagram corresponds to a parameter of the model (with the exception of any variances of endogenous LVs that are set to ). In specifying an entire model, each parameter is designated as being one of two types: 1) Free: Value unknown and to be estimated Ú Ý 2) ConstrainedÛ Ý Ü a) Fixed. Assigned a numerical value chosen by the investigator. (e.g. setting an LV variance to ) b) Required to be equal to other parameters. (e.g. requiring all unique variances in factor analysis to be equal.) Every single-headed arrow in the path diagram represents a regression coefficient. Those single-headed arrows connecting LVs to their indicators are factor loadings. Other single-headed arrows represent linear regression effects. Every double-headed arrow represents a covariance (when it connects two exogenous variables) or a variance (when it connects an exogenous variable to itself). In a typical model, the parameters include: 22

26 1) Regression coefficients and factor loadings corresponding to singleheaded arrows; 2) Variances for all exogenous variables, including MVs, LVs, and error terms; 3) Covariances for pairs of exogenous variables, as specified by the investigator. Example 1: Wheaton, Muthén, Alwin, Summers (1977) Study of Stability of Alienation On the following page is a fully specified path diagram representing a model of the stability of the attitude of alienation over two points in time. There are three LVs: Socioeconomic Status (SES) Alienation 1967 (Alntn67) Alienation 1971 (Alntn71) Each LV has two indicators. For SES: Education and SEI For Alntn: Anomia and Powerlessness (attitude scales) measured at each occasion. The model hypothesizes a pattern of effects among these variables. 23

27 E1 E2 E3 E4 ANOMIA67 POWERLS67 ANOMIA71 POWERLS71 Z1 ALNTN67 ALNTN71 Z2 SES EDUCTN SEI D1 D2 RAMONA Example 1 : Wheaton, Muthen, Alwin Summers 24

28 Special features to note regarding the specification of this model: SES is an exogenous LV. Its variance is a parameter. Alntn67 and Alntn71 are endogenous LVs. Their variances are not parameters (represented by dotted variance paths). The scales of the LVs are set by fixing their variances to. For SES, this fixed variance is a fixed parameter. For Alntn67 and Alntn71, these fixed variances imply constraints on other model parameters. Alntn67 and Alntn71 have associated residual terms, z1 and z2. Those residuals are exogenous variables and therefore have variance parameters. Each indicator has an error term; those error terms are exogenous and have variance parameters. These terms correspond to unique factors in factor analysis. The path coefficients from all error terms to associated variables are fixed at. The model hypothesizes that there are correlated errors between the two occasions of measurement for anomia and powerlessness. Note that paths that are that are fixed at zero. not present in the diagram correspond to parameters Note that this model can be viewed as a factor analysis model with directional influences among the factors. Let us consider how many free parameters there are in this model. For this purpose we define the "effective number of free parameters" as the number of parameters being estimated, minus the number of constraints imposed on those parameters. In the present model, there appear to be 19 parameters to be estimated. However, the constraints implied by fixing the variances of the two endogenous LVs at represent two constraints. 25

29 Thus, the effective number of free parameters is "* # œ "(Þ Note that we could have used a different method for fixing the scale of the two endogenous LVs. That is, we could have fixed one of the factor loadings at for each of them. If we had done this and then instead, and then counted parameters to be estimated, we would have found 17. Example 2: Duncan, Haller, Portes (1971) Study of Ambition Following is a path diagram specifying a model of relationships among measures of ambition for matched pairs of subjects; each pair consists of a respondent and best friend. There are two LVs: Respondent's Ambition Best Friend's Ambition Each LV has two indicators: Occupational Aspirations Educational Aspirations The model also includes predictors of Ambition. These predictors are exogenous MVs in the model; they do not serve as indicators of LVs: Respondent's Parental Aspirations Respondent's Intelligence Respondent's SES Best Friend's Parental Aspirations Best Friend's Intelligence Best Friend's SES The model hypothesizes a pattern of effects among these variables. 26

30 RAMONA Example 2: Duncan, Haller, Portes. REPARASP E1 REOCCASP Z1 REINTGCE E2 REEDASP REAMBITN RESOCIEC E3 BFEDASP BFAMBITN BFSOCIEC E4 BFOCCASP BFINTGCE Z2 BFPARASP REAMBITN Alternative Identification Conditions BFAMBITN 27

31 Special features to note regarding this model: The exogenous MVs are specified as having variance and covariance parameters. Estimation of these parameters is trivial. Given sample data, the estimates will simply be the sample variances and covariances. The Ambition variables are both endogenous and are hypothesized to have reciprocal directional relationships. This is not the same as a simple covariance relationship. The scales of the Ambition variables are set by fixing their variances to. These fixed variances imply constraints on other parameters of the model. An alternative way of fixing these scales is to set one of their associated factor loadings to. This alternative is portrayed in a small diagram below the primary diagram. When this is done, no variances are shown for the endogenous LVs; those variances are implied by the model. As an exercise, determine how many free parameters there are in this model. Generation of the Model Models of the kind described here are typically generated from a priori theory or sets of hypotheses about patterns of relationships among variables. In practice, a researcher must translate such theory and hypotheses into a formally specified model. That translation process involves (a) identification of the variables of interest; (b) identification of the hypothesized directional and nondirectional linear associations among those variable; (c) identification of the population of interest; a model is a hypothesis about the structure of the relationships among the variables of interest in a specified population. 28

32 In determining the variables to be included in the model, it is important that no major variables be omitted. Omission of important variables causes bias in resulting parameter estimates, as well as potentially misleading results and interpretations. For example, suppose variables A, B, and C all influence variable D. But suppose in our design and model specification, we omit variable A from consideration. Our subsequent estimates of the influences of B and C on D will be biased, possibly highly so. That is, our estimates of those influences may be much different than they would have been had A been included. This is called the "omitted variable" problem. Due to this problem, it is important that we do not omit major variables from our model and design. Selection of Indicators of LVs In the social sciences the variables of interest are very often LVs. Thus, the researcher has the task of selecting indicators to represent each LV. Although there is no simple guideline for the "best" number of indicators to use, 3-5 indicators per LV seems desirable. Fewer indicators results in inadequate representation of the construct. More indicators may result in difficulty in achieving adequate fit of the model to the data. The use of large numbers of indicators for the LVs is not encouraged. It is most desirable that indicators have substantial reliability and validity. A LV is defined in effect as that which its multiple indicators have in common. If indicators are not highly valid, then the LV is not represented with validity in the model. If indicators are not very reliable, then they will have high error variance, again resulting in poor representation of the LV. Issues involving properties of desirable indicators are discussed by Little, Lindenberger, and Nesselroade (1999). A variety of kinds of indicators are used in practice. Indicators may be scales, subscales, or single items. The use of single items may be problematic because single items tend to be much less reliable than are scales made up of multiple items. 29

33 A common situation arises wherein we have only a single multi-item scale to represent a variable of interest. For example, our model may include a LV corresponding to job satisfaction, and we have a single measure of job satisfaction which is obtained from a 20-item job satisfaction scale. In such instances, we have several options as to how to proceed. One option is to use the scale as a single indicator of the LV. In that case, the LV is not a true LV, but is represented as a single MV in our model. A second option is to use the items as multiple indicators. In the example, this approach would result in the use of 20 indicators for the LV. This approach is problematic in that the number of indicators is too large and the indicators themselves are likely to have low reliability since they are single items. A third approach is to use parcels. A parcel is a composite of several items. For example, we could construct 4 parcels, each defined as a sum of 5 items. We would thus have 4 indicators that would be more reliable than single items and which would exhibit higher correlations with each other than would single items. We would avoid the problem of having too many indicators (using single items). And we would be representing job satisfaction as a true LV with multiple indicators, rather than as an MV. Kishton and Widaman (Educational and Psychological Measurement, 1994, pp ) review the use of parcels and provide guidance as to how to form them. LV vs. MV Models A LV model contains LVs with multiple indicators. A MV model contains no LVs. A MV model is a hypothesis about a pattern of directional and nondirectional relationships among a set of MVs. LV models have advantages over MV models. In MV models, it is not possible to estimate the error in the MVs; in such models (e.g., path analysis), MVs are treated as error free. The presence of error biases the resulting parameter estimates representing relationships among MVs. In LV models, error in the indicators is explicitly represented and estimated. This reduces the bias in parameter estimates representing relationships among LVs. In addition, LV models are often more appropriate than MV models in psychological research because psychological theories typically involve hypothetical constructs (attitude, intelligence, personality, depression, etc.). 30

34 It is more appropriate to model these constructs as LVs rather than as errorfree MVs. A useful strategy in model testing and development is to specify and evaluate alternative a priori models. These alternative models may arise from competing theories, or may be based on interesting logical alternatives. Comparison of alternative models often provides more interesting and informative results than evaluation of a single model. Ideally, all aspects of variable selection and model specification are completed as part of the research design process, prior to the collection of data. 31

35 MODULE 3 MATHEMATICAL REPRESENTATION OF THE DATA MODEL: RAM In Module 2 we studied the specification of structural equation models using path diagrams. In the current Module we consider a mathematical representation of such models. In the literature there are several different mathematical frameworks that have been proposed; e.g., the LISREL framework (Jöreskog, 1973; Jöreskog & Sörbom, 1996); the Bentler-Weeks framework (Bentler & Weeks, 1980), and the RAM framework (McArdle & McDonald, 1984). These systems vary with respect to the number of matrices and number of equations employed. But the systems are very closely related and essentially any structural equation model can be represented equivalently using any of these systems. In this module we present a slightly modified version of the RAM (reticular action model) framework. This modified version of RAM is implemented in the computer program RAMONA (RAM Or Near Approximation; Browne & Mels, 1998). The RAM/RAMONA representation is simpler than alternative representations since the number of matrices and equations employed is smaller. To begin to develop the basic notions of a mathematical representation of a structural equation model, consider a path diagram of the Wheaton- Muthen-Alwin-Summers (WMAS) model defined in Module 2. On the following page is a modified version of the path diagram, with a mathematical symbol introduced for each non-fixed path. The basis for this notation will be developed later. For now, consider these symbols as representing the true but unknown values of the corresponding parameters. Notice that the diagram has two types of arrow: single headed (regression weights) and double headed (variances/covariances). It also has two types of geometrical symbol: rectangles (manifest variables) and ellipses/circles (latent variables). As will be seen subsequently, these two types of arrow and two types of geometrical symbol form the basis of the RAM representation. 32

36 φ 10 φ 11 φ 4 E1 φ E2 E3 φ E4 5 6 φ 7 ANOMIA67 POWERLS67 ANOMIA71 POWERLS71 φ 2 β 1 β 2 β 3 β 4 φ 3 Z1 ALNTN67 β 9 ALNTN71 Z2 β 7 β 8 φ 1 = SES β 5 β 6 EDUCTN SEI φ 8 D1 φ 9 D2 Wheaton, Muthen, Alwin, Summers Example 33

37 In the diagram, the parameter associated with each arrow is represented by either a specified numerical value (for fixed parameters) or a mathematical symbol (for free parameters). A basic principle of defining a mathematical representation of the data model is that we may write one linear equation for each variable in the model that receives at least one single-headed arrow. That is, we write one equation for each endogenous variable. Each such equation expresses a given endogenous variable as a linear function of the variables in the model that exert linear influences on that endogenous variable. For example, Anomia67 is endogenous. It is influenced by Alienation67 and the error term E1. We write the following equation to represent these influences: ANOMIA67 œ " 1(ALNTN67) "Þ!ÐE1) œ " (ALNTN67) E1 1 This is essentially a linear regression equation expressing Anomia67 as a linear function of Alienation67, plus an error term. The form of this equation clarifies why we fix the coefficients associated with error term paths to. In such linear equations, residual terms are not weighted; they are simply residuals. For another example, consider Alienation71. According to the model: ALNTN71 œ " (ALNTN67) " (SES) Z2 * ) By following this procedure we could write a system of linear equations that would capture all of the directional influences in the model. In this example there would be 8 such equations. In general, there will be as many equations as there are endogenous variables. In order to work with such models mathematically and in a general form, it is necessary to develop a mathematical framework for representing such a system of equations. 34

38 For this mathematical framework, we define one matrix, two vectors and one matrix equation. Begin by defining the following: Column Contains all variables in system, including MVs and LVs, where LVs include both common factors and all error terms. We can conceive as containing scores on all variables for a typical individual in the population. The sequence of variables is arbitrary, but there is a convenient ordering: Endogenous MVs Exogenous MVs (if any) Endogenous LVs Exogenous LVs Error terms For the WMAS example, would be defined as follows: Ô AN67 POW67 AN71 POW71 ED SEI AL67 AL71 SES Z1 Z2 E1 E2 E3 E4 Ö D1 Ù ÕD2 Ø 35

39 Coefficient matrix F (upper case beta): The columns of F correspond to the variables in the same order. The rows of F also correspond to the variables in the same order. An element of F corresponds to a path coefficient representing the linear influence of one variable on another; specifically, the influence of the variable (emitting the arrow) defined by the column on the variable (receiving the arrow) defined by the row. There is one non-zero element of F corresponding to each single-headed arrow in the path diagram. Zero elements in F correspond to nonexistent paths in the path diagram. For rows of F corresponding to exogenous variables, all entries in F are zero because an exogenous variable receives no single-headed arrows from other variables. Some non-zero elements in F have a fixed numerical value; e.g., values of representing paths from error terms to MVs or LVs. Other non-zero elements in F have unknown values represented by symbols, ", with subscripts corresponding to particular paths in the path diagram. These are free parameters to be estimated. 36

40 For the WMAS example, the matrix F would have the following form: 1. AN67 2. POW67 3. AN71 4. POW71 5. ED 6. SEI 7. AL67 8. AL71 9. SES 10. Z1 11. Z2 12. E1 13. E2 14. E3 15. E4 16. D1 17. D Ô! ""! "! "#! "!! " $ "!! "% "! "& "! "' " Ù! "( " "*! ") "! â! â! ã! ã!!!!! Ö ã! ã Ù Õ! â â! Ø The blank entries in F are actually zeros. Every nonzero element in this matrix corresponds to a single-headed path in the WMAS path diagram. Elements represented by " 45 represent paths going from the variable represented by column 5 to the variable represented by row 4. Each such path has either a fixed numerical value or an unknown value to be estimated. Fixed numerical values in this example correspond to the paths from error terms to corresponding variables, and are set to. Coefficients corresponding to other paths are free parameters. Endogenous variables have at least one nonzero entry in their corresponding row of F, meaning they receive at least one path in the path diagram. (Rows 1-8) Exogenous variables have no nonzero entries in their corresponding rows of F, meaning they receive no paths in the path diagram. (Rows 9-17) 37

41 Column B : Column B is equivalent to column except that all endogenous variables are replaced by zero. Exogenous variables remain intact. In the WMAS example, column B has the following œ B Ô Some nonzero elements in row of F 0 Endogenous variables SES Z1 Z2 E1 All elements zero in row of F E2 Exogenous variables E3 E4 Ö Ù D1 Õ D2 Ø 38

42 DATA MODEL: Given these matrices, the RAMONA data model can be represented B This equation defines the endogenous variables as linear combinations of all variables in the system. For the WMAS example, this equation has the following B Ô AN67 Ô!!!!!! ""!!!! "!!!!! Ô AN67 Ô POW67! "#! " POW67 AN71!! " $ " AN71 POW71!! "% " POW71 ED! "& " ED SEI! "' " SEI AL67! "( " AL67 AL71!!!!!! "*! ")! "!!!!!! AL71 Ù SES œ! â! â! SES Z1 ã! ã Z1 Z2! Z2 E1! E1 E2! E2 E3! E3 E4! E4 Ö D1 Ù Ö ã! ã Ù Ö D1 Ù Ö ÕD2 Ø Õ! â â! Ø ÕD2 Ø Õ Following rules of matrix multiplication, a given variable (element) on the left side of this equation is equal to the corresponding row of F multiplied by column plus the corresponding entry in. For example, consider the variable Anomia67. The following equation results: AN67 œð "" AL67Ñ Ð" E1Ñ! œ " ÐAL67) E1 B!!!!!!!! SES Z1 Z2 E1 E2 E3 E4 D1 D2 Ù Ù Ø 39

43 This equation corresponds to the linear equation we obtained earlier from the path diagram. Consider the equation for ALNTN71: AL71 œð "* AL67Ñ Ð ") SESÑ Ð 1 Z2) 0 œ " (AL67) " (SES) Z2 * ) Again, this equation matches the one we obtained earlier from the path diagram. Now consider an equation for an exogenous variable, such as SES: SES œð! AN67) â Ð! D2) SES The model simply equates exogenous variables to themselves. This is the reason that the vector is needed in the B In general, this single matrix equation represents a system of linear equations, one for each endogenous variable in the model. Covariance matrix Q: We will also be interested in the covariance matrix Q, which is the covariance matrix of the variables in, the exogenous B Q œ CovÐ@ ß@ The rows and columns of Q will correspond to the variables B. Because all endogenous variables have been replaced by zero B ß matrix Q will contain zeros for all covariances involving endogenous variables with each other or with exogenous variables. Matrix Q will contain nonzero variances for all exogenous variables, and zero or nonzero covariances for all pairs of exogenous variables. B B w Ñ There will be a nonzero element of path diagram. Q for every double-headed arrow in the This covariance matrix is not an explicit part of the data model, but will be relevant when we formulate the covariance structure model. 40

44 In the WMAS example, this matrix Q would have the following form:! Ô 0 â 0 â â 0! ã ã!!!!!! 0 â â 0 â â 0 SES 0 â â 0 Z1 ã ã 9# 0 Z2 0 9$ E1 9% 0 9"! 0 E2 0 9& 0 9"" E3 9"! 0 9' 0 E4 0 9"" 0 9( D1 Ö ã ã 9 0 Ù ) D2 Õ0 â â Ø Q is a symmetric matrix. * Q has a null row/column for every endogenous variable. (Vector corresponding null B has a Every double-headed arrow in the WMAS path diagram corresponds to a nonzero element in this matrix, excluding (implied) variances of endogenous LVs represented by dotted double-headed arrows. Each nonzero element in Q represents a parameter of the model. Some of these elements may be fixed, typically at to set a scale for exogenous LVs. Other parameters are typically free, representing variances and covariances to be estimated. If an off-diagonal element of Q is not fixed at zero, then neither corresponding diagonal element can be fixed at zero. That is, one can not define a nonzero covariance involving a variable with zero variance. 41