How well do Fit Indices Distinguish Between the Two?

Transcription

1 MODELS OF VARIABILITY VS. MODELS OF TRAIT CHANGE How well do Fit Indices Distinguish Between the Two? M Conference University of Connecticut, May 2-22, 2 bkeller2@asu.edu INTRODUCTION More and more researchers in social sciences are interested in studying the longitudinal course of constructs. It important for us to know which type of change process best characterizes the construct. 2 M Conference May 2-22, 2

2 INTRODUCTION Substantive researchers are often interested in whether the longitudinal course of a construct is best described by.... Stability 2. Variability. Trait change. A combination of different processes (Eid, Courvoisier, & Lischetzke, 2) RESEARCH GOALS To determine how well can we distinguish between variability and trait change processes based on common fit statistics. To determine what conditions can make it difficult to distinguish between the two types of processes. M Conference May 2-22, 2

3 TRAIT CHANGE VS. VARIABILITY MAKING A DISTINCTION Following Nesselroade (99) and Eid (2) definitions we make the following distinction:. Processes of variability involves a short-term, potentially reversible fluctuation around a fixed construct (i.e., trait) that is invariant across the span of measurement. 2. Processes of trait change involves a long-term, potentially irreversible modification of a construct that allows for change across the span of measurement. M Conference May 2-22, 2

4 MAKING A DISTINCTION Processes of variability oscillates around a set point. Processes of trait change follows a long-term trajectory. They answer two distinctly different conceptual questions. MAKING A DISTINCTION Trait Change Variability 8 M Conference May 2-22, 2

5 MAKING A DISTINCTION An example of variability process: Positive affect (or other mood states) can be viewed as a process of variability. I have some "trait" value of positive affect and I fluctuate around that value depending on the day (e.g., I dented my car, I would have lower positive affect as compared to my average, trait, level). 9 MAKING A DISTINCTION An example of trait change process: Math ability of children over time can be seen as a trait change process. I start with some baseline math ability and over time it (hopefully) grows and my math ability increases. M Conference May 2-22, 2

6 MAKING A DISTINCTION In our simulation models we used... Multiple-indicator latent growth curve model as prototypical model for measuring trait changes. Latent state-trait models as prototypical models for measuring state-variability processes. LATENT STATE-TRAIT (LST) MODELS 2 M Conference May 2-22, 2

7 LATENT STATE-TRAIT (LST) THEORY Latent models are based on latent state-trait (LST) theory developed in Steyer, Ferring, & Schmitt (992). In classical test theory we decompose a score into two parts: Observed score = True score + Error In LST theory we extend this by decomposing the true score further: True score = Trait + State residual SINGLE-TRAIT MULTI-STATE Y SR Y2 Y Y2 SR2 trait Y2 Y SR Y2 Y M Conference May 2-22, 2

8 COVARIANCE STRUCTURE SR γ = γ2 Y Y2 λ = γ Y λ2 λ SR2 γ = γ2 γ Y2 Y2 λ = λ2 λ trait SR γ = γ2 γ Y Y2 Y λ = λ2 λ MEAN STRUCTURE SR Y Y2 α = α2 α SR2 Y Y2 α = α2 trait E(trait) Y2 α SR Y Y2 Y α = α2 α M Conference May 2-22, 2

9 LATENT STATE-TRAIT (LST) THEORY We have three main coefficients of interest in LST theory:. Consistency (CO) 2. Occasion-specificity (OS). Reliability (Rel) CONSISTENCY Consistency is a measurement of the individual differences explained by the person's intrinsic trait. CO(Y it ) = λ 2 i Var(trait) Var(Y it ) 8 M Conference May 2-22, 2

10 CONSISTENCY SR Y Y2 λ = Y λ2 SR2 Y2 λ λ = λ2 trait Y2 λ SR Y Y2 λ = λ2 λ Y 9 OCCASION-SPECIFICITY Occasion-specificity is a measurement of the individual differences explained by the situation and/or person by situation interaction (note that these are inseparable by definition). OS(Y it ) = γ 2 i Var(SR t ) Var(Y it ) 2 M Conference May 2-22, 2

11 OCCASION-SPECIFICITY Y SR Y2 Y SR2 γ = γ2 γ γ = γ2 Y2 trait γ Y2 γ = γ2 Y SR Y2 γ Y 2 RELIABILITY Reliability is a measurement of the individual differences that are not explained by measurement error. Rel(Y it ) = Var(ε it ) Var(Y it ) = OS(Y it ) + CO(Y it ) 22 M Conference May 2-22, 2

12 RELIABILITY ε Y SR ε2 Y2 ε Y SR2 ε2 ε22 Y2 trait ε2 Y2 ε Y SR ε2 Y2 ε Y 2 MODELING TRAIT CHANGE 2 M Conference May 2-22, 2

13 MULTIPLE-INDICATOR LGC MODEL Y Y2 Y intercept Y2 Y2 Y Y2 slope Y 2 MULTIPLE-INDICATOR LGC MODEL Y Y2 trait Y Y2 Y2 Y Y2 Y trait2 trait 2 M Conference May 2-22, 2

14 COVARIANCE STRUCTURE Y Y2 trait Y Y2 Y2 Y Y2 trait2 trait Y 2 TRAIT (INTERCEPT) Y Y2 Y λ2 λ trait Y2 λ2 λ Y2 λ2 Y λ Y2 Y trait2 trait 28 M Conference May 2-22, 2

15 CHANGE SCORE (SLOPE) Y Y2 trait Y Y2 λ2 Y2 λ Y 2 Y2 Y 2λ2 2λ trait2 trait 29 MEAN STRUCTURE Y Y2 α = α2 trait Y Y2 α α = α2 α E(trait) Y2 α = E(trait2 trait) Y Y2 Y α2 α trait2 trait M Conference May 2-22, 2

16 MODELING TRAIT CHANGE A Previous study (Geiser, Keller, Lockhart, Eid, Cole, & Koch) had shown analytically that non-invariant LST models can fit trait change data quite well under certain conditions (i.e., LST models can mask a trait change process). NON-INVARIANT STMS Y SR Y2 Y Y2 SR2 trait Y2 Y SR Y2 Y 2 M Conference May 2-22, 2

17 COVARIANCE STRUCTURE SR γ = γ2 Y Y2 λ = γ Y λ2 SR2 γ2 = γ22 γ2 Y2 Y2 λ λ2 λ22 λ2 trait SR γ = γ2 γ Y Y2 Y λ λ2 λ MEAN STRUCTURE SR Y Y2 α = α2 α SR2 Y Y2 α2 α22 trait E(trait) Y2 α2 SR Y Y2 Y α α2 α M Conference May 2-22, 2

18 TRAIT CHANGE VS. VARIABILITY Properly specified LST models with invariant parameters should not fit trait change data in theory. Under some conditions these models were hard to distinguish from LGCs based on fit. Therefore, we wanted to know more about the conditions under which LGC and LST models (with or without invariant parameters) are difficult to distinguish. SIMULATION STUDY M Conference May 2-22, 2

19 HYPOTHESES LST models can fit growth data closely if:. They are specified with non-invariant parameters. 2. Sample size is small.. The number of time points is small.. The growth factor variance is small. SIMULATION STUDY Four-step procedure:. Generate data using LST-LGC hybrid model (, replications). 2. Analyze the data with population model.. Analyze the data with STMS model with non-invariant loadings and intercepts model (configural model).. Analyze the data with STMS model with time-invariant loadings and intercepts (strong invariance). 8 M Conference May 2-22, 2

20 LST-LGC POPULATION MODEL Y SR Y2 Y trait SR2 Y2 Y2 SR Y Y2 trait2 trait Y 9 COVARIANCE STRUCTURE γ = γ2 Y SR γ Y2 Y λ2 λ trait γ = γ2 Y2 λ2 SR2 λ γ Y2 SR γ = γ2 γ Y Y2 Y λ2 λ 2 2λ2 2λ λ2 λ trait2 trait M Conference May 2-22, 2

21 MEAN STRUCTURE SR SR2 Y Y2 Y Y2 α = α2 α α = α2 α trait E(trait) Y2 α = E(tr2 tr) SR Y Y2 α2 α trait2 trait Y TRAIT CHANGE SR Y Y2 Y trait SR2 Y2 Y2 Y SR Y2 Y trait2 trait 2 M Conference May 2-22, 2

22 VARIABILITY SR Y Y2 trait SR2 Y Y2 Y2 SR Y Y2 trait2 trait Y SIMULATION STUDY We varied conditions:. Number of measurement occasions ( to ). 2. Amount of slope factor variance (%, %, %, %, 2% of the intercept factor variance).. Amount of mean change across time (zero, small, medium, large).. Amount of occasion-specific variability (%, %, %, 2%, %, %, % of the variance explained by SR).. Sample size (N =,, 2, 2,,,,, ). M Conference May 2-22, 2

23 SIMULATION STUDY Special cases to note:. Perfect stability: No trait change, no state variability 2. Perfect variability: No trait change, but state variability. Perfect trait change: Trait change, but no state variability SIMULATION STUDY We looked at two measures of fit:. Chi-square test of model fit. 2. Approximate fit indices (requires all for good fit ): RMSEA. SRMR. CFI.9 M Conference May 2-22, 2

24 RESULTS SMTS Non-Invariant Model CHI-SQUARE TEST OF MODEL FIT 8 M Conference May 2-22, 2

25 Non-Invariant chi statistics All mean changes Var(SR) = % % % 2% % % % Var(slope) = 2% % Percentage of correct choice % Number of Measurement Occasions 2 % % Sample Size APPROXIMATE FIT INDICES M Conference May 2-22, 2

26 Non-Invariant fit statistics All mean changes Var(SR) = % % % 2% % % % Var(slope) = 2% % Percentage of correct choice % Number of Measurement Occasions 2 % % Sample Size 2 2 RESULTS SMTS Time-Invariant Model 2 M Conference May 2-22, 2

27 CHI-SQUARE TEST OF MODEL FIT Time Invariant chi statistics Zero mean change Var(SR) = % % % 2% % % % Var(slope) = 2% % Percentage of correct choice % Number of Measurement Occasions 2 % % Sample Size M Conference May 2-22, 2

28 Time Invariant chi statistics Small mean change Var(SR) = % % % 2% % % % Number of Measurement Occasions Var(slope) = 2% % % % % Percentage of correct choice Sample Size Time Invariant chi statistics Medium mean change Var(SR) = % % % 2% % % % Number of Measurement Occasions Sample Size Var(slope) = 2% % % % % Percentage of correct choice 2 M Conference May 2-22, 2

29 Time Invariant chi statistics Large mean change Var(SR) = % % % 2% % % % Var(slope) = 2% % Percentage of correct choice % Number of Measurement Occasions 2 % % Sample Size 2 2 APPROXIMATE FIT INDICES 8 M Conference May 2-22, 2

30 Time Invariant fit statistics Zero mean change Var(SR) = % % % 2% % % % Number of Measurement Occasions Var(slope) = 2% % % % % Percentage of correct choice Sample Size 9 Time Invariant fit statistics Small mean change Var(SR) = % % % 2% % % % Number of Measurement Occasions Sample Size Var(slope) = 2% % % % % Percentage of correct choice 2 M Conference May 2-22, 2

31 Time Invariant fit statistics Medium mean change Var(SR) = % % % 2% % % % Number of Measurement Occasions Var(slope) = 2% % % % % Percentage of correct choice Sample Size Time Invariant fit statistics Large mean change Var(SR) = % % % 2% % % % Number of Measurement Occasions Sample Size Var(slope) = 2% % % % % Percentage of correct choice 2 2 M Conference May 2-22, 2

32 CONCLUSION HYPOTHESES LST models can fit growth data closely if:. They are specified with non-invariant parameters. 2. Sample size is small. (For chi-square test of model fit). The number of time points is small.. The growth factor variance is small. M Conference May 2-22, 2

33 CONCLUSION LST models are harder to distinguish from growth models when:. The number of waves is three or less. 2. Sample size is small. (How small is small? It is very dependent on the number of time points). Non-invariant loadings and intercepts are allowed in the LST model.. Slope factor variance is around % of the intercept factor variance.. SR variance is % or more of the explained variance.. Trait mean change is small. RECOMMENDATIONS With sample sizes 2 or more use the chi-square test of model fit. With sample sizes 2 or less use the approximate fit indices. Have as much as time points (more is better). NOTE: Many LST applications used 2 time points, this makes it impossible to explicitly test if your model is a variability process or a trait change process (see Geiser & Lockhart 22). Test for measurement invariance. M Conference May 2-22, 2

34 LIMITATIONS Studied only normally distributed data. Did not vary reliability. Assumed correlation between intercept and slope factors. Only looked at indicators per time point. Studied only linear growth models. Future research: Study non-linear growth models. SPECIAL THANKS Developers of MplusAutomation (without it this simulation couldn t have been done!). Leona Aiken for travel funding. Craig Enders for presentation advice. 8 M Conference May 2-22, 2

35 REFERENCES: SOFTWARE H. Wickham (29). ggplot2: elegant graphics for data analysis. Springer New York, 29. Michael Hallquist and Joshua Wiley (2). MplusAutomation: Automating Mplus Model Estimation and Interpretation. R package version.-. package=mplusautomation Muthén, L. K., & Muthén, B. O. (998-22). Mplus User's Guide. Seventh Edition. Los Angeles, CA: Muthén & Muthén. R Core Team (2). R: A language and environment for statistical computing. R Foundation fo Statistical Computing, Vienna, Austria. URL 9 REFERENCES Eid, M., Courvoisier, D. S., & Lischetzke, T. (2). Structural equation modeling of ambulatory assessment data. In M. R. Mehl & T. S. Connor (Eds.), Handbook of research methods for studying daily life (pp. 8-). New York: Guilford. Geiser, C., & Lockhart, G. (22). A comparison of four approaches to account for method effects in latent state-trait analyses. Psychological Methods,, Nesselroade, J. R. (99). Interindividual differences in intraindividual change. In L. M. Collins & J. L. Horn (Eds.), Best methods for the analysis of change. Recent advances, unanswered questions, future directions (pp. 92-). Washington, DC: American Psychological Association. McArdle, J. J. (988). Dynamic but structural equation modeling of repeated measures data. In J. Nesselroade & R. Cattell (Eds.), Handbook of multivariate experimental psychology (2nd ed., pp. ). New York, NY: Plenum Press. Steyer, R., Ferring, D., & Schmitt, M. J. (992). States and traits in psychological assessment. European Journal of Psychological Assessment, 8, M Conference May 2-22, 2

36