Whats beyond Concerto: An introduction to the R package catr. Session 4: Overview of polytomous IRT models

Transcription

1 Whats beyond Concerto: An introduction to the R package catr Session 4: Overview of polytomous IRT models The Psychometrics Centre, Cambridge, June 10th, 2014

2 2 Outline: 1. Introduction 2. General notations 3. Difference models 4. Divide-by-total models 5. Summary

3 3 1. Introduction Current version of catr is version 3.0 (available online) Up to version 2.6, only dichotomous IRT models were embedded into catr Version 3.0 holds now most known polytomous IRT models Goal of this session: 1. To briefly describe the general framework for polytomous IRT models 2. To set up the notations and IRT models that are embedded in catr

4 4 2. General notations With dichotomous items, only two responses are possible With polytomous items, more than two responses are allowed For item j, set g j + 1 as the number of possible responses Responses are coded as k {0, 1,..., g j } Settiong g j = 1 yields a dichotomous item Item responses can be ordinal: e.g., ( never, sometimes, often, always ) nominal: e.g., color, political affiliation, etc. p j is the set of item parameters (depending on the model) P jk (θ) = P r(x j = k θ, p j ) is the probability of answering response k to item j, given proficiency θ and item parameters

5 5 2. General notations Two main categories of polytomous IRT models (Thissen & Steinberg, 1986): 1. Difference models: probabilities P jk (θ) are set as differences between cumulative probabilities P jk (θ) = P r(x j k θ, p j ) 2. Divide-by-total models: probabilities P jk (θ) are set as ratios of values divided by the sum of these values: t jk P jk (θ) = g j l=0 t jl Notations to be used come from Embretson and Reise (2000)

6 6 2. General notations Most-known difference models: Graded response model (GRM; Samejima, 1969) Modified graded response model (MGRM; Muraki, 1990) Most-known difference models: Partial credit model (PCM; Masters, 1982) Generalized partial credit model (GPCM; Muraki, 1992) Rating scale model (RSM; Andrich, 1978) Nominal response model (NRM; Bock, 1972) All these models are available in catr

7 7 3. Difference models With difference models, probabilities P jk (θ) are set as differences between cumulative probabilities P jk (θ) = P r(x j k θ, p j ) that is, the probability of selecting response in (k, k + 1,..., g j ) Graded response model (GRM; Samejima, 1969): P jk (θ) = exp [α j (θ β jk )] 1 + exp [α j (θ β jk )] with P j0 (θ) = 1 and P j,g j (θ) = P j,gj (θ) α j discrimination (slope) parameter and β jk threshold (intercept) parameters of cumulative probabilities Only g j thresholds β j1,..., β j,gj are necessary to distinguish the g j + 1 response categories

8 8 3. Difference models Response category probabilities P jk (θ) are found back as follows (with 0 < k < g j ): Illustration with P j,gj (θ) = P j,g j (θ) P jk (θ) = P jk (θ) P j,k+1 (θ) P j0 (θ) = 1 P j1 (θ) 4 response categories (i.e. g j = 3), α j = 1.5, β j1 = 0.5, β j2 = 0.5, and β j3 = 1.2

9 9 3. Difference models P * jk (θ) Cumulative probability k = 1 k = 2 k = θ

10 10 3. Difference models P jk (θ) Probability k = 0 k = 1 k = 2 k = θ

11 11 3. Difference models With GRM, threshold parameters β jk may vary across items items may not have the same number of response categories g j Modified graded response model (MGRM; Muraki, 1990): modification of GRM to allow for common thresholds across all items With MGRM, thresholds β jk are split in two components: b j (general intercept parameter for item j) and c k (threshold parameter between categories k and k + 1, common to all items) Cumulative probability under MGRM: P jk (θ) = exp [α j (θ b j + c k )] 1 + exp [α j (θ b j + c k )]

12 12 3. Difference models With GRM, all items have the same number of categories (since all c k parameters are equal across items) MGRM applicable for Likert scales with same response categories across items, such as e.g. Never - Rarely - Sometimes - Often - Always

13 13 4. Divide-by-total models With divide-by-total models, probabilities P jk (θ) are set directly as ratios of values divided by the sum of these values (across response categories) Partial credit model (PCM; Masters, 1982): with P jk (θ) = exp k t=0 (θ δ jt ) g j r=0 exp r t=0 (θ δ jt ) 0 (θ δ jt ) = 0 or exp t=0 0 (θ δ jt ) = 1 t=0

14 14 4. Divide-by-total models Detailed equations with three response categories (i.e. g j = 2 and k {0, 1, 2}): g j r=0 exp r (θ δ jt ) = t=0 2 r=0 exp r (θ δ jt ) t=0 = 1 + exp (θ δ j1 ) + exp (θ δ j1 + θ δ j2 ) Denominator is then equal to g j r exp (θ δ jt ) = 1 + exp (θ δ j1 ) + exp (2 θ δ j1 δ j2 ) r=0 t=0

15 15 4. Divide-by-total models Response category probabilities are then 1 P j0 (θ) = 1 + exp (θ δ j1 ) + exp (2 θ δ j1 δ j2 ), and P j1 (θ) = P j2 (θ) = exp (θ δ j1 ) 1 + exp (θ δ j1 ) + exp (2 θ δ j1 δ j2 ), exp (2 θ δ j1 δ j2 ) 1 + exp (θ δ j1 ) + exp (2 θ δ j1 δ j2 ), Idea: to allow for partial credit in the item response (e.g., false - incomplete - almost correct - correct responses)

16 16 4. Divide-by-total models Only g j thresholds (δ j1,..., δ j,gj ) are necessary Illustration with g j = 3 (i.e. four response categories) (δ j1, δ j2, δ j3 ) = (1, 1, 0.5)

17 17 4. Divide-by-total models P jk (θ) Probability k = 0 k = 1 k = 2 k = 3 β j1 = 1 β j2 = 1 β j3 = θ

18 18 4. Divide-by-total models Generalized partial credit model (GPCM; Muraki, 1992) extends the PCM by introducing a discrimination parameter α j for the item: exp k P jk (θ) = t=0 α j (θ δ jt ) g j r=0 exp r t=0 α j (θ δ jt ) with 0 α j (θ δ jt ) = 0 t=0 Illustration with same parameters as PCM, i.e. (δ j1, δ j2, δ j3 ) = (1, 1, 0.5), and with α j = 1.5

19 19 4. Divide-by-total models PCM Probability k = 0 k = 1 k = 2 k = θ

20 20 4. Divide-by-total models GPCM Probability k = 0 k = 1 k = 2 k = θ

21 21 4. Divide-by-total models PCM allows for different thresholds δ jt across the items, and also different response categories Rating scale model (RSM; Andrich, 1978) is a modification of PCM to allow equal thresholds across items With RSM, thresholds δ jt are split in two components, λ j (general intercept parameter) and δ t (threshold parameter common to all items): with P jk (θ) = exp k t=0 [θ (λ j + δ t )] g j r=0 exp r t=0 [θ (λ j + δ t )] 0 [θ (λ j + δ t )] = 0 t=0

22 22 4. Divide-by-total models Finally, Nominal response model (NRM; Bock, 1972) specifies response category probabilities as follows: with P jk (θ) = exp (α jk θ + c jk ) g j r=0 exp (α jr θ + c jr ) α j0 θ + c j0 = 0 Illustration with four categories (i.e.g j = 3) and (α j1, c j1 ) = (1, 1), (α j2, c j2 ) = (1.2, 0.5), (α j3, c j3 ) = (0.8, 0)

23 23 4. Divide-by-total models P jk (θ) Probability k = 0 k = 1 k = 2 k = θ

24 24 5. Summary Polytomous IRT models extend dichotomous IRT models by allowing more than two possible responses Response categories can be the same across all items or they may vary Polytomous IRT models: GRM, MGRM, PCM, GPCM, RSM, NRM,... MGRM and RSM are convenient only if the same response categories (as they share common threshold parameters for all items) Extensions of algorithms for dichotomous IRT model calibration to the polytomous framework exist

25 25 5. Summary Each model can be fully described with specific sets of parameters for each item j: (α j, β j1,..., β j,gj ) for GRM (α j, b j, c 1,..., c g ) for MGRM (δ j1,..., δ j,gj ) for PCM (α j, δ j1,..., δ j,gj ) for GPCM (λ j, δ 1,..., δ g ) for RSM (α j1, c j1,..., α j,gj, c j,gj ) for NRM These orders of parameters are used in catr...

26 26 References Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43, Bock, R.D. (1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika, 37, Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum Associates. Masters, G.N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, Muraki, E. (1990). Fitting a polytomous item response model to Likert-type data. Applied Psychological Measurement, 14,

27 27 References Muraki, E. (1992). A generalized partial credit model: application of an EM algorithm. Applied Psychological Measurement, 16, Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika monograph (Vol. 17). Thissen, D., & Steinberg, L. (1986). A taxonomy of item response models. Psychometrika, 51,