Local response dependence and the Rasch factor model

Size: px

Start display at page:

Download "Local response dependence and the Rasch factor model"

Pauline Natalie Freeman
5 years ago
Views:

1 Local response dependence and the Rasch factor model Dept. of Biostatistics, Univ. of Copenhagen Rasch6 Cape Town

2 Uni-dimensional latent variable model X 1 TREATMENT δ T X 2 AGE δ A Θ X 3 X 4 Latent variable Θ, X 1,..., X 4 items. Monotone relationships. [Holland and Rosenbaum, 1986]

3 Uni-dimensional latent variable model - Rasch model X 1 TREATMENT δ T X 2 AGE δ A Θ X 3 X 4 Latent variable Θ, X 1,..., X 4 items. Monotone relationships. Rasch: invariance, sufficiency

4 Rasch model for item response X i Item parameters β i = (β i0, β i1, β i2,...) P(X i = x Θ = θ) = exp(xθ + β ix )K i (θ) 1 (1) for x = 0, 1,..., m i. β i0 = 0 for convenience and K i (θ) = m i h=0 exp(hθ + β ih)

5 Assumption of local independence Response vector X = (X i ) i I P( X = x Θ = θ) = i I P(X i = x Θ = θ) yielding P( X = x Θ = θ) = exp ( Rθ + i I β ixi ) K(θ) 1 (2) where K(θ) = i I K i(θ) R = i I X i sufficient.

6 Marginal probability P( X = x) = P( X = x Θ = θ)ϕ(θ)dθ ( ) = exp β ixi exp(rθ)k(θ) 1 ϕ σ (θ)dθ i I

7 Conditional probability P( X = x R = r) = exp ( i I β ix i ) γ R ( β) (3) β = ( β i ) i I, (3) independent of θ

8 Likelihood functions, inference MML l MML ( β, σ) = v i I β ixi + log exp(rθ)k(θ) 1 ϕ σ (θ)dθ (4) CML (complete cases) l CML ( β) = v Pairwise conditional l PW ( β) = v β ixvi log γ Rv ( β) (5) i I β ixvi log γ R (i,i ) i i v ( β) (6) Implementation: ConQuest [Wu et al., 2007], DIGRAM [Kreiner, 2003], RUMM [Andrich et al., 2010]. Standard software [Christensen, 2006].

9 Uni-dimensional latent variable model X 1 TREATMENT δ T X 2 AGE δ A Θ X 3 X 4 Latent variable Θ, X 1,..., X 4 items. Monotone relationships. [Holland and Rosenbaum, 1986]

10 Uni-dimensional latent variable model, local dependence X 1 TREATMENT δ T X 2 AGE δ A Θ X 3 X 4 Latent variable Θ, X 1,..., X 4 items. Monotone relationships. [Andrich, 1991]

11 Uni-dimensional latent variable model, local dependence X 1 TREATMENT δ T X 2 AGE δ A Θ X 3 X 4 Latent variable Θ, X 1,..., X 4 items. Monotone relationships. Rasch: invariance, sufficiency. [Kreiner and Christensen, 2007]

12 Testing LD 1 Generalized Tjur test [Tjur, 1982] 2 Include interaction term in (9), i.e. loglinear Rasch model [Kelderman, 1984] 3 Item splitting [for dichotomous items] 4 Correlation structure in residuals

13 Testing LD 1 Generalized Tjur test Result for three dichotomous Rasch items [Tjur, 1982]: X 1 X 2 X 1 + X 3 generalization [Kreiner and Christensen, 2004]: X 1 X 2 R (1) and X 1 X 2 R (2) where R (1) = i 1 X i and R (2) = i 2 X i are rest scores. (Note: two p-values, need to control type I error rate). 1 Implementation: Standard software or the item screening in DIGRAM

14 Testing LD 2 Loglinear Rasch model Rasch model (9): ( P( X = x Θ = θ) = exp Rθ + i I β ixi ) K(θ) 1 include interaction term [Kelderman, 1984]: ( ) P( X = x Θ = θ) = exp Rθ + β ixi + δ(x 1, x 2 ) i I K(θ) 1 (7) compare (9) and (7) using likelihood ratio test. 2 Implementation: DIGRAM

15 Testing LD 3 Item splitting Splitting dependent item X 2 into Xi2 [Andrich and Kreiner, 2010] Illustration (dichotomous items) (0) and X i2 (1). ID X 1 X ID X 1 X2 (0) X 2 (1) Fit model for items X 2 (0), X 2 (1), X 3,... compare item parameters ( DIF test). Need asymptotic (co)variance for formal test. 3 Implementation: RUMM, DIGRAM

16 Item splitting X 1 TREATMENT δ T X 2 AGE δ A Θ X 3 X 4

17 Item splitting X 1 TREATMENT δ T X 2 AGE δ A Θ X 3 X 4

18 Testing LD 4 Residuals Local dependence for (X 1, X 2 ). [Yen, 1984, Q3] Observe (X 1v, X 2v ) v=1,...,n Estimate (θ v ) v=1,...,n Compute standardized residuals R iv = X iv E(X i ˆθ v ) V (X i ˆθ v ) and ρ OBS = corr(r 1, R 2 ) Test LD: Is observed value of ρ OBS due to random variation 4 Implementation: RUMM

19 Testing LD using residuals Local dependence for (X 1, X 2 ). [Yen, 1984, Q3] Observe (X 1v, X 2v ) v=1,...,n Estimate (θ v ) v=1,...,n Compute residuals R iv = X iv E(X i ˆθ v ) and ρ OBS = corr(r 1, R 2 ) Test LD: Is observed value of ρ OBS due to random variation But we don t know the null distribution of ρ...

20 Testing LD using residuals Local dependence for (X 1, X 2 ). [Yen, 1984, Q3] Observe (X 1v, X 2v ) v=1,...,n Estimate (θ v ) v=1,...,n Compute residuals R iv = X iv E(X i ˆθ v ) and ρ OBS = corr(r 1, R 2 ) Test LD: Is observed value of ρ OBS due to random variation But we don t know the null distribution of ρ... No formal test, rule-of-thumb ρ OBS > ρ 0, ρ 0 = 0.2, 0.3, 0.6,...

21 The problem with residuals we don t know the distribution of ˆθ ˆθ is a biased estimate of θ R iv can only take m i + 1 different values under the Rasch model corr(r 1, R 2 ) < [Kreiner and Christensen, 2011a]

22 Data example (9 items from the SF36, acute leukemia) How much of the time during the past 4 weeks Did you feel full of pep? Have you been a very nervous person? Have you felt so down in the dumps that nothing could cheer you up? Have you felt calm and peaceful? Did you have a lot of energy? Have you felt downhearted and blue? Did you feel worn out? Have you been a happy person? Did you feel tired? Response options: All of the Time, Most of the Time, A Good Bit of the Time, Some of the Time, A Little of the Time, None of the Time

23 Generalized Tjur test X 9 X 8 X 7 X 6 Θ X 5 X 4 X 1 X 2 X 3 Latent variable Θ, X 1,..., X 9 items. Results from item screening [Kreiner and Christensen, 2011b]

24 Log linear Rasch model X 9 X 8 X 7 X 6 Θ X 5 X 4 X 1 X 2 X 3 Latent variable Θ, X 1,..., X 9 items. Same as Tjur + five additional (neg. LD also found, not shown)

25 Correlation matrix of observed residuals

26 Correlation matrix of observed residuals

27 LD summary Generalized Tjur tests and residuals disagree x x x x x x x x x x x All models incorrect. Agreement about the most highly significant pair type I error known for Tjur tests. need to know the (multivariate) distribution of the correlation matrix (or the distribution of residuals) under the Rasch model.

28 Rasch measurement X 1 TREATMENT δ T X 2 AGE δ A Θ X 3 X 4 Latent variable Θ, X 1,..., X 4 items.

29 Rasch measurement, multidimensionality X 1 TREATMENT δ 1,A δ 1,T Θ 1 X 2 AGE δ 2,T δ 2,A Θ 2 X 3 X 4 Latent variable Θ, X 1,..., X 4 items.

30 Two-dimensional model Set of items split into d disjoint sets. For d = 2: θ = [ θ1 θ 2 I = I 1 I 2, (8) with items in I 1 and I 2 measuring latent variables θ 1 and θ 2 respectively. P(X = x Θ = θ) = exp ( R 1 θ 1 + R 2 θ 2 + i I η ) ix i K( θ) = exp ( d R dθ d + i I η ) ix i where R d = i I d x i and K( θ) = d K( θ) i I d K i (θ d ) ]

31 Marginal probability ( ) ( ) P( X = x) = exp β ixi exp R d θ d K(θ) 1 ϕ Σ ( θ)d θ i I can write d dim. marginal likelihood function. Unidim. likelihood function (4) nested within, but singular. can t compare nested models using LRT [Christensen et al., 2002, Harrell-Williams and Wolfe, 2014] d

32 Conditional probability P( X = x R = r) = exp ( i I β ix i ) d γ(d) r d (( β i ) i Id ) d dim. conditional likelihood function is the sum l CML ( β (d) ). Unidim. likelihood function (4) nested within? d

33 Rasch measurement, multidimensionality Patterns in residuals t-test observed vs. expected sub score correlationn Rasch factor model Summary of tests and diagnostics by Horton et al [Horton et al., 2013]

34 Patterns in residuals

35 Patterns in residuals

36 Tests currently avaliable Use patterns in residuals or Tjur tests to provide hypothetises beware of circular logic I 1 I 2 t-test, observed vs. expected sub score correlation and other tests are significant for many hypothetical I = I 1 I 2 Lack of overview, many tests, risk of type I error.. (still) need to know the (multivariate) distribution of the matrix of residuals under the Rasch model.

37 Rasch factor model Use P( X = x θ) = P( X = x R = r) P( R = r θ) }{{} ν to write extended likelihood l EML ( β, ν) = d l CML ( β (d) ) + v log ν v with the probabilities of the marginal sub score vector distribution as unrestricted parameters ν. Compare nested models using LRT [Christensen et al., 2002]

38 Factor models (1, 5, 8) + (2, 6) + (3, 4) + (7, 9) (1, 2, 5, 6, 8) + (3, 4) + (7, 9) (1, 3, 4, 5, 8) + (2, 6) + (7, 9) (1, 5, 8) + (2, 6) + (3, 4, 7, 9) (1, 2, 5, 6, 8) + (3, 4, 7, 9) (1, 3, 4, 5, 8) + (2, 6, 7, 9) (1, 2,..., 9) Compare nested models using LRT (or use BIC [Schwarz, 1978])

39 Factor models l =..., NPAR = 205 l =..., NPAR = 184 l =..., NPAR = 194 l =..., NPAR = 190 l =..., NPAR = 141 l =..., NPAR = 159 l =..., NPAR = (45 1) + (45 1) = 88 Compare nested models using LRT (or use BIC [Schwarz, 1978])

40 Andrich, D. (1991). Book review: Langeheine and Rost Latent Trait and Latent Class Models. Psychometrika, 56(1): Andrich, D. and Kreiner, S. (2010). Quantifying response dependence between two dichotomous items using the rasch model. Applied Psychological Measurement, 34(3): Andrich, D., Sheridan, B., and Luo, G. (2010). RUMM2030 [Computer software and manual]. RUMM Laboratory, Perth, Australia. Christensen, K. B. (2006). Fitting polytomous Rasch models in SAS. Journal of applied measurement, 7(4):

41 Christensen, K. B., Bjorner, J. B., S., K., and Petersen, J. H. (2002). Testing unidimensionality in polytomous Rasch models. Psychometrika, 67: Harrell-Williams, L. and Wolfe, E. (2014). Performance of the likelihood ratio difference (g2 diff) test for detecting unidimensionality in applications of the multidimensional rasch model. Journal of applied measurement, 15(3): Holland, P. W. and Rosenbaum, P. R. (1986). Conditional association and unidimensionality in monotone latent variable models. The Annals of Statistics, 14(4): Horton, M., Marais, I., and Christensen, K. B. (2013). Dimensionality, pages

42 John Wiley & Sons, Inc. Kelderman, H. (1984). Loglinear Rasch model tests. Psychometrika, 49: Kreiner, S. (2003). Introduction to digram. Research Report 10, Department of Statistics, University of Copenhagen. Kreiner, S. and Christensen, K. B. (2004). Analysis of local dependence and multidimensionality in graphical loglinear rasch models. Communications in Statistics - Theory and Methods, 33: Kreiner, S. and Christensen, K. B. (2007). Validity and objectivity in health-related scales: Analysis by graphical loglinear rasch models.

43 In Multivariate and Mixture Distribution Rasch Models, Statistics for Social and Behavioral Sciences, pages Springer New York. Kreiner, S. and Christensen, K. B. (2011a). Exact evaluation of bias in Rasch model residuals. In Baswell, editor, Advances in Mathematics Research vol.12, pages Nova publishers. Kreiner, S. and Christensen, K. B. (2011b). Item screening in graphical loglinear rasch models. Psychometrika, 76(2): Schwarz, G. E. (1978). Estimating the dimension of a model. Annals of Statistics, 6: Tjur, T. (1982). A connection between Rasch s item analysis model and a multiplicative poisson model.

44 Scandinavian Journal of Statistics, 9: Wu, M. L., Adams, R. J., Wilson, M. R., and Haldane, S. A. (2007). ACER ConQuest Version 2: Generalised item response modelling software. Australian Council for Educational Research, Camberwell. Yen, W. M. (1984). Effects of local item dependence on the fit and equating performance of the three-parameter logistic model. Applied Psychological Measurement, 8(2):

Conditional maximum likelihood estimation in polytomous Rasch models using SAS

Conditional maximum likelihood estimation in polytomous Rasch models using SAS Karl Bang Christensen kach@sund.ku.dk Department of Biostatistics, University of Copenhagen November 29, 2012 Abstract IRT