The Relationship between Factor Analytic and Item Response Models

Transcription

1 The Relatonshp between Factor Analytc and Item Response Models Akhto Kamata Department of Educaton Polcy and Leadershp Department of Psychology Center on Research and Evaluaton Southern Methodst Unversty November 7, 014 TARDIS at Unversty of North Texas Ths presentaton s based on: Kamata, A. & Bauer, D. J. (008). A note on the relatonshp between factor analytc and tem response theory models. Structural Equaton Modelng. 15,

2 Overvew Background Famous Formula to Convert FA to IRT Parameters Four Possble Parameterzatons for Bnary FA model General Transformaton Formula Numercal Demonstraton Summary

3 Background Latent varable models wth categorcal data have been often studed n the framework of tem response theory (IRT). Factor analyss and SEM have modeled equvalent models. Ther modelng wth categorcal varables are equvalent to IRT, but may be dfferent n parameterzatons. The boundary between categorcal FA/SEM and IRT s becomng less obvous. 3

4 IRT, FA, SEM, or ML? Background Wthn (Student Level) y 1 1 e 1 y 1 y 1 Between (School Level) e y y y ~ N(0, ) e 3 B ~ N(0, B) y 3 3 y 3 y 3 B e 17 S 1 y y 17 y 17 Study S 1 S N S ~ (0, ) 1 1 Teacher Experence 4

5 Background Three Modelng Frameworks Item response theory modelng Kamata & Bauer (008) Takane & De Leeuw (1987) Kamata, Bauer, & Myazak (008) Kamata, (001) Fox & Glas (001) Rjmen et al. (003) Muthen (00) Skrondal & Rabe-Hesketh (004) Structural equaton modelng / Factor analyss modelng Multlevel modelng Bauer (003) Curran (003) 5

6 One-Factor Bnary Factor Analytc Model y e y 1 f y 0 f y. where y : y : : : e : Observed tem response (1 = correct, 0 = ncorrect) Latent contnuous response varable Factor loadng parameter Threshold parameter Error 6

7 Graphcal representaton of Bnary FA model y Dstrbuton of y Dstrbuton of y 1 p y 1 p y 1 Dstrbuton of y 0 0 p y 1 Ey ( 1) 1 y e E(y ) Ey ( 0) V( y ) 1 0 V( y ) V( y ) 0 1 7

8 -parameter IRT model 1 p y f where f (.) : : : : cumulatve normal or logstc dstrbuton tem dscrmnaton parameter threshold parameter ablty parameter Item dffculty parameter s obtaned by: b. 8

9 Famous Formula to Convert from Bnary FA to IRT Parameters e.g., Takane & de Leeuw (1987); McDonald (1999) 1 and 1 where : : : : Item dscrmnaton parameter n -parameter IRT Threshold parameter n -parameter IRT Factor loadng parameter n Bnary FA Threshold parameter n Bnary FA 9

10 Mplus syntax for fttng bnary one-factor FA model for LSAT6 Data DATA: FILE IS LSAT6.dat; VARIABLE: NAMES ARE tem1-tem5; categorcal are tem1-tem5; MODEL: ks BY tem1-tem5; 10

11 Mplus Results for LSAT6 Data MODEL RESULTS Two-Taled Estmate S.E. Est./S.E. P-Value KSI BY ITEM ITEM ITEM ITEM ITEM Thresholds ITEM1$ ITEM$ ITEM3$ ITEM4$ ITEM5$ Varances KSI

12 Do the famous formulas work for these results? 1 and 1 Estmate S.E. Est./S.E. P-Value KSI BY ITEM ITEM ITEM ITEM ITEM No! has to be smaller than 1.0 for the formula to be functonal. 1

13 These famous formulas are qute restrcted, because they assume that are fully standardzed, whch means; Latent factor (ablty) s standardzed. Underlnng propensty for response=1 (latent response varable) s standardzed. Is ths the only way to scale bnary FA parameters? No. For our example, underlnng propensty for response=1 was standardzed, but not the latent factor. 13

14 Four Possble Parameterzatons for Bnary FA model Reference Indcator Standardzed Factor Margnal 1 =1, 1 =0 E) = 0, V()=1 V(y )= 1 V(y )= 1 Condtonal 1 =1, 1 =0 V(e)=1 Note: Other parameterzatons are also possble. E) = 0, V()=1 V(e)=1 What to do wth parameter estmates from these parameterzatons? More general formulas are needed. 14

15 The General Transformaton Formulas See Kamata and Bauer (008) for dervatons. V ( ) V ( e ) E( ) V ( e ) 15

16 Transformaton Formulas for the 4 parameterzatons Margnal Reference Indcator V ( ) 1 V ( ) [ E( )] 1 V ( ) Standardzed Factor 1 1 Condtonal V ( ) [ ( )] E Reference Indcator Standardzed Factor Margnal 1 =1, 1 =0 E) = 0, V()=1 V(y )= 1 V(y )= 1 Condtonal 1 =1, 1 =0 V(e)=1 E) = 0, V()=1 V(e)=1 16

17 Numercal Demonstraton LSAT 6 data 5 dchotomously score tems 1,000 examnees 17

18 One factor FA model was ftted by 4 dfferent parameterzatons. Logstc dstrbuton of e was assumed. Parameters were estmated by SAS NLMIXED: maxmum lkelhood wth numercal ntegraton. Parameters were transformed nto IRT parameters by usng the formulas n the last table. Parameters were also estmated by fttng a -parameter logstc IRT model by BILOG-MG (maxmum lkelhood wth numercal ntegraton). 18

19 Condtonal Margnal Reference Standardzed Reference Standardzed (.581) (.1184).8754 (.367) (.4351) (.3673) (.3806).77 (.1867).8909 (.38).6884 (.1851).6569 (.099) (.057) (.8740) (1.0769) (.8790) (.8978).773 (.057) var() (.46) (.0900) (.0763) (.0990) (.1354).900 (.367) (.443).8906 (.480).863 (.678).5858 (.0994).665 (.0969).5670 (.1035).5490 (.16) (.1749) (.6410).048 (.6618).846 (.670).179 (.7409) (.1749) (.1507) (.0703) (.0563) (.0796) (.107) Note. Values n parentheses are standard errors

20 b. Transformed IRT model parameter estmates and drect estmates of IRT parameters Factor Analyss Drect Condtonal Margnal IRT Reference Standardzed Reference Standardzed

21 Summary Whch parameterzatons should one use? Ultmately, t s arbtrary. There has been long runnng dscusson on the relatonshp between bnary FA and IRT. Ths study clarfes there are many ways to parameterze parameters. One practcal mplcaton s that f results are syntheszed across dfferent studes, we have to ensure that parameters are n the same scale. 1

22 Agan, IRT, FA, SEM, or ML? Wthn (Student Level) y 1 1 e 1 y 1 y 1 Between (School Level) e y y y ~ N(0, ) e 3 B ~ N(0, B) y 3 3 y 3 y 3 B e 17 S 1 y y 17 y 17 Study S 1 S N S ~ (0, ) 1 1 Teacher Experence

23 3