Local response dependence and the Rasch factor model

Local response dependence and the Rasch factor model Dept. of Biostatistics, Univ. of Copenhagen Rasch6 Cape Town

Uni-dimensional latent variable model X 1 TREATMENT δ T X 2 AGE δ A Θ X 3 X 4 Latent variable Θ, X 1,..., X 4 items. Monotone relationships. [Holland and Rosenbaum, 1986]

Uni-dimensional latent variable model - Rasch model X 1 TREATMENT δ T X 2 AGE δ A Θ X 3 X 4 Latent variable Θ, X 1,..., X 4 items. Monotone relationships. Rasch: invariance, sufficiency

Rasch model for item response X i Item parameters β i = (β i0, β i1, β i2,...) P(X i = x Θ = θ) = exp(xθ + β ix )K i (θ) 1 (1) for x = 0, 1,..., m i. β i0 = 0 for convenience and K i (θ) = m i h=0 exp(hθ + β ih)

Assumption of local independence Response vector X = (X i ) i I P( X = x Θ = θ) = i I P(X i = x Θ = θ) yielding P( X = x Θ = θ) = exp ( Rθ + i I β ixi ) K(θ) 1 (2) where K(θ) = i I K i(θ) R = i I X i sufficient.

Marginal probability P( X = x) = P( X = x Θ = θ)ϕ(θ)dθ ( ) = exp β ixi exp(rθ)k(θ) 1 ϕ σ (θ)dθ i I

Conditional probability P( X = x R = r) = exp ( i I β ix i ) γ R ( β) (3) β = ( β i ) i I, (3) independent of θ

Likelihood functions, inference MML l MML ( β, σ) = v i I β ixi + log exp(rθ)k(θ) 1 ϕ σ (θ)dθ (4) CML (complete cases) l CML ( β) = v Pairwise conditional l PW ( β) = v β ixvi log γ Rv ( β) (5) i I β ixvi log γ R (i,i ) i i v ( β) (6) Implementation: ConQuest [Wu et al., 2007], DIGRAM [Kreiner, 2003], RUMM [Andrich et al., 2010]. Standard software [Christensen, 2006].

Uni-dimensional latent variable model X 1 TREATMENT δ T X 2 AGE δ A Θ X 3 X 4 Latent variable Θ, X 1,..., X 4 items. Monotone relationships. [Holland and Rosenbaum, 1986]

Uni-dimensional latent variable model, local dependence X 1 TREATMENT δ T X 2 AGE δ A Θ X 3 X 4 Latent variable Θ, X 1,..., X 4 items. Monotone relationships. [Andrich, 1991]

Uni-dimensional latent variable model, local dependence X 1 TREATMENT δ T X 2 AGE δ A Θ X 3 X 4 Latent variable Θ, X 1,..., X 4 items. Monotone relationships. Rasch: invariance, sufficiency. [Kreiner and Christensen, 2007]

Testing LD 1 Generalized Tjur test [Tjur, 1982] 2 Include interaction term in (9), i.e. loglinear Rasch model [Kelderman, 1984] 3 Item splitting [for dichotomous items] 4 Correlation structure in residuals

Testing LD 1 Generalized Tjur test Result for three dichotomous Rasch items [Tjur, 1982]: X 1 X 2 X 1 + X 3 generalization [Kreiner and Christensen, 2004]: X 1 X 2 R (1) and X 1 X 2 R (2) where R (1) = i 1 X i and R (2) = i 2 X i are rest scores. (Note: two p-values, need to control type I error rate). 1 Implementation: Standard software or the item screening in DIGRAM

Testing LD 2 Loglinear Rasch model Rasch model (9): ( P( X = x Θ = θ) = exp Rθ + i I β ixi ) K(θ) 1 include interaction term [Kelderman, 1984]: ( ) P( X = x Θ = θ) = exp Rθ + β ixi + δ(x 1, x 2 ) i I K(θ) 1 (7) compare (9) and (7) using likelihood ratio test. 2 Implementation: DIGRAM

Testing LD 3 Item splitting Splitting dependent item X 2 into Xi2 [Andrich and Kreiner, 2010] Illustration (dichotomous items) (0) and X i2 (1). ID X 1 X 2 1 1 1 2 0 1 3 1 0 4 1 1 5 0 0... ID X 1 X2 (0) X 2 (1) 1 1. 1 2 0 1. 3 1. 0 4 1. 1 5 0 0..... Fit model for items X 2 (0), X 2 (1), X 3,... compare item parameters ( DIF test). Need asymptotic (co)variance for formal test. 3 Implementation: RUMM, DIGRAM

Item splitting X 1 TREATMENT δ T X 2 AGE δ A Θ X 3 X 4

Item splitting X 1 TREATMENT δ T X 2 AGE δ A Θ X 3 X 4

Testing LD 4 Residuals Local dependence for (X 1, X 2 ). [Yen, 1984, Q3] Observe (X 1v, X 2v ) v=1,...,n Estimate (θ v ) v=1,...,n Compute standardized residuals R iv = X iv E(X i ˆθ v ) V (X i ˆθ v ) and ρ OBS = corr(r 1, R 2 ) Test LD: Is observed value of ρ OBS due to random variation 4 Implementation: RUMM

Testing LD using residuals Local dependence for (X 1, X 2 ). [Yen, 1984, Q3] Observe (X 1v, X 2v ) v=1,...,n Estimate (θ v ) v=1,...,n Compute residuals R iv = X iv E(X i ˆθ v ) and ρ OBS = corr(r 1, R 2 ) Test LD: Is observed value of ρ OBS due to random variation But we don t know the null distribution of ρ...

Testing LD using residuals Local dependence for (X 1, X 2 ). [Yen, 1984, Q3] Observe (X 1v, X 2v ) v=1,...,n Estimate (θ v ) v=1,...,n Compute residuals R iv = X iv E(X i ˆθ v ) and ρ OBS = corr(r 1, R 2 ) Test LD: Is observed value of ρ OBS due to random variation But we don t know the null distribution of ρ... No formal test, rule-of-thumb ρ OBS > ρ 0, ρ 0 = 0.2, 0.3, 0.6,...

The problem with residuals we don t know the distribution of ˆθ ˆθ is a biased estimate of θ R iv can only take m i + 1 different values under the Rasch model corr(r 1, R 2 ) < 0 0 1 0. 1. [Kreiner and Christensen, 2011a]

Data example (9 items from the SF36, acute leukemia) How much of the time during the past 4 weeks Did you feel full of pep? Have you been a very nervous person? Have you felt so down in the dumps that nothing could cheer you up? Have you felt calm and peaceful? Did you have a lot of energy? Have you felt downhearted and blue? Did you feel worn out? Have you been a happy person? Did you feel tired? Response options: All of the Time, Most of the Time, A Good Bit of the Time, Some of the Time, A Little of the Time, None of the Time

Generalized Tjur test X 9 X 8 X 7 X 6 Θ X 5 X 4 X 1 X 2 X 3 Latent variable Θ, X 1,..., X 9 items. Results from item screening [Kreiner and Christensen, 2011b]

Log linear Rasch model X 9 X 8 X 7 X 6 Θ X 5 X 4 X 1 X 2 X 3 Latent variable Θ, X 1,..., X 9 items. Same as Tjur + five additional (neg. LD also found, not shown)

Correlation matrix of observed residuals 1.00 0.32 0.06 0.12 0.16 0.21 0.16 0.14 0.13 1.00 0.05 0.01 0.31 0.39 0.04 0.12 0.13 1.00 0.19 0.10 0.02 0.02 0.01 0.02 1.00 0.11 0.08 0.22 0.12 0.31 1.00 0.25 0.03 0.05 0.12 1.00 0.00 0.09 0.19 1.00 0.39 0.16 1.00 0.25 1.00

Correlation matrix of observed residuals 1.00 0.32 0.06 0.12 0.16 0.21 0.16 0.14 0.13 1.00 0.05 0.01 0.31 0.39 0.04 0.12 0.13 1.00 0.19 0.10 0.02 0.02 0.01 0.02 1.00 0.11 0.08 0.22 0.12 0.31 1.00 0.25 0.03 0.05 0.12 1.00 0.00 0.09 0.19 1.00 0.39 0.16 1.00 0.25 1.00

LD summary Generalized Tjur tests and residuals disagree x x x x x x x x x x x All models incorrect. Agreement about the most highly significant pair type I error known for Tjur tests. need to know the (multivariate) distribution of the correlation matrix (or the distribution of residuals) under the Rasch model.

Rasch measurement X 1 TREATMENT δ T X 2 AGE δ A Θ X 3 X 4 Latent variable Θ, X 1,..., X 4 items.

Rasch measurement, multidimensionality X 1 TREATMENT δ 1,A δ 1,T Θ 1 X 2 AGE δ 2,T δ 2,A Θ 2 X 3 X 4 Latent variable Θ, X 1,..., X 4 items.

Two-dimensional model Set of items split into d disjoint sets. For d = 2: θ = [ θ1 θ 2 I = I 1 I 2, (8) with items in I 1 and I 2 measuring latent variables θ 1 and θ 2 respectively. P(X = x Θ = θ) = exp ( R 1 θ 1 + R 2 θ 2 + i I η ) ix i K( θ) = exp ( d R dθ d + i I η ) ix i where R d = i I d x i and K( θ) = d K( θ) i I d K i (θ d ) ]

Marginal probability ( ) ( ) P( X = x) = exp β ixi exp R d θ d K(θ) 1 ϕ Σ ( θ)d θ i I can write d dim. marginal likelihood function. Unidim. likelihood function (4) nested within, but singular. can t compare nested models using LRT [Christensen et al., 2002, Harrell-Williams and Wolfe, 2014] d

Conditional probability P( X = x R = r) = exp ( i I β ix i ) d γ(d) r d (( β i ) i Id ) d dim. conditional likelihood function is the sum l CML ( β (d) ). Unidim. likelihood function (4) nested within? d

Rasch measurement, multidimensionality Patterns in residuals t-test observed vs. expected sub score correlationn Rasch factor model Summary of tests and diagnostics by Horton et al [Horton et al., 2013]

Patterns in residuals 1-0.61 0.12-0.21 2 0.76 0.14 0.05 3-0.05 0.14 0.80 4 0.02 0.59 0.52 5-0.56-0.27 0.24 6 0.67 0.17-0.15 7 0.27-0.64 0.12 8-0.32 0.63-0.26 9-0.11-0.69 0.11

Patterns in residuals 2 0.76 0.14 0.05 6 0.67 0.17-0.15 7 0.27-0.64 0.12 8-0.32 0.63-0.26 4 0.02 0.59 0.52 1-0.61 0.12-0.21 5-0.56-0.27 0.24 9-0.11-0.69 0.11 3-0.05 0.14 0.80

Tests currently avaliable Use patterns in residuals or Tjur tests to provide hypothetises beware of circular logic I 1 I 2 t-test, observed vs. expected sub score correlation and other tests are significant for many hypothetical I = I 1 I 2 Lack of overview, many tests, risk of type I error.. (still) need to know the (multivariate) distribution of the matrix of residuals under the Rasch model.

Rasch factor model Use P( X = x θ) = P( X = x R = r) P( R = r θ) }{{} ν to write extended likelihood l EML ( β, ν) = d l CML ( β (d) ) + v log ν v with the probabilities of the marginal sub score vector distribution as unrestricted parameters ν. Compare nested models using LRT [Christensen et al., 2002]

Factor models (1, 5, 8) + (2, 6) + (3, 4) + (7, 9) (1, 2, 5, 6, 8) + (3, 4) + (7, 9) (1, 3, 4, 5, 8) + (2, 6) + (7, 9) (1, 5, 8) + (2, 6) + (3, 4, 7, 9) (1, 2, 5, 6, 8) + (3, 4, 7, 9) (1, 3, 4, 5, 8) + (2, 6, 7, 9) (1, 2,..., 9) Compare nested models using LRT (or use BIC [Schwarz, 1978])

Factor models l =..., NPAR = 205 l =..., NPAR = 184 l =..., NPAR = 194 l =..., NPAR = 190 l =..., NPAR = 141 l =..., NPAR = 159 l =..., NPAR = (45 1) + (45 1) = 88 Compare nested models using LRT (or use BIC [Schwarz, 1978])

Andrich, D. (1991). Book review: Langeheine and Rost Latent Trait and Latent Class Models. Psychometrika, 56(1):155 168. Andrich, D. and Kreiner, S. (2010). Quantifying response dependence between two dichotomous items using the rasch model. Applied Psychological Measurement, 34(3):181 192. Andrich, D., Sheridan, B., and Luo, G. (2010). RUMM2030 [Computer software and manual]. RUMM Laboratory, Perth, Australia. Christensen, K. B. (2006). Fitting polytomous Rasch models in SAS. Journal of applied measurement, 7(4):407 17.

Christensen, K. B., Bjorner, J. B., S., K., and Petersen, J. H. (2002). Testing unidimensionality in polytomous Rasch models. Psychometrika, 67:563 574. Harrell-Williams, L. and Wolfe, E. (2014). Performance of the likelihood ratio difference (g2 diff) test for detecting unidimensionality in applications of the multidimensional rasch model. Journal of applied measurement, 15(3):267 275. Holland, P. W. and Rosenbaum, P. R. (1986). Conditional association and unidimensionality in monotone latent variable models. The Annals of Statistics, 14(4):1523 1543. Horton, M., Marais, I., and Christensen, K. B. (2013). Dimensionality, pages 137 158.

John Wiley & Sons, Inc. Kelderman, H. (1984). Loglinear Rasch model tests. Psychometrika, 49:223 245. Kreiner, S. (2003). Introduction to digram. Research Report 10, Department of Statistics, University of Copenhagen. Kreiner, S. and Christensen, K. B. (2004). Analysis of local dependence and multidimensionality in graphical loglinear rasch models. Communications in Statistics - Theory and Methods, 33:1239 1276. Kreiner, S. and Christensen, K. B. (2007). Validity and objectivity in health-related scales: Analysis by graphical loglinear rasch models.

In Multivariate and Mixture Distribution Rasch Models, Statistics for Social and Behavioral Sciences, pages 329 346. Springer New York. Kreiner, S. and Christensen, K. B. (2011a). Exact evaluation of bias in Rasch model residuals. In Baswell, editor, Advances in Mathematics Research vol.12, pages 19 40. Nova publishers. Kreiner, S. and Christensen, K. B. (2011b). Item screening in graphical loglinear rasch models. Psychometrika, 76(2):228 256. Schwarz, G. E. (1978). Estimating the dimension of a model. Annals of Statistics, 6:461 464. Tjur, T. (1982). A connection between Rasch s item analysis model and a multiplicative poisson model.

Scandinavian Journal of Statistics, 9:23 30. Wu, M. L., Adams, R. J., Wilson, M. R., and Haldane, S. A. (2007). ACER ConQuest Version 2: Generalised item response modelling software. Australian Council for Educational Research, Camberwell. Yen, W. M. (1984). Effects of local item dependence on the fit and equating performance of the three-parameter logistic model. Applied Psychological Measurement, 8(2):125 145.