A model of skew item response theory

Transcription

1 1 A model of skew item response theory Jorge Luis Bazán, Heleno Bolfarine, Marcia D Ellia Branco Department of Statistics University of So Paulo Brazil ISBA 2004 May 23-27, Via del Mar, Chile

2 2 Motivation On educational evaluation research is typical to obtain differences in scholar performance due to social-economic status. As an example, in Peru, Bazán et al. (2001) report differences observed in a mathematical test for sixth grade, students in favor of students with high social-economic status. Test scores show a negative asymmetric distribution since most of the students with high social-economic status tend to obtain high scores in the test. Purpose Modelling a dichotomic responses matrix corresponding to a test with I items applied in n subjects. y 11 y y 1I y 21 y y 2I y n1 y n2... y ni

3 3 Item response theory For i = 1..., n examinees that respond j = 1,..., I items of a test, a probit-normal item response model (Albert, 1992), is : Y ij U i, η j Bernouilli(p ij ), (1) p ij = P(y ij = 1 U i, η j ) = Φ ij (m ij ), (2) with Φ ij a cdf standard normal, a symmetrical item characteristic curve (ICC), is it a nondecreasing function of U i. m ij = η T j U i = a j U i b j, (3) is linear in U i, with a j discrimination parameter and b j difficulty parameter. Supposing conditional independence in tens, and examinees response independents, a multivariate response conjoint density is: p(y = y U = u, η) = An additional supposition is n I i=i j=1 F y ij ij (1 F ij) 1 y ij. (4) U i N(0,1). (5)

4 4 An appropriate formulation of probit-normal model For i = 1..., n examinees that responding j = 1,..., I tens of a test, a probit-normal model is: Z ij = m ij + e ij, (6) e ij N(0,1), (7) 1, Z ij > 0; y ij =, (8) 0, Z ij 0. Notice that p ij = P(Y ij = 1) = P(Z ij > 0) = Φ(m ij ). It shows that a lineal normal structure of a auxuliar latent variable produce a equivalent model with normal-probit model. A augmented likelihood, with D = (Z, y) augmented data and Z auxiliar latent variable is: L(u, η D) = n I φ(z ij ; m ij, 1)I(Z ij, y ij ), i=1 j=1 em que I(Z ij, y ij ) = I(Z ij > 0)I(y ij = 1) + I(Z ij 0)I(y ij = 0).

5 5 A Skew-Normal IRT For i = 1..., n examinees that responding j = 1,..., I itens of a test, a skew-normal item response model (SN-IRT) is: Z ij = m ij + e ij, (9) e ij SN(0,1, δ j ) (10) 1, Z ij > 0 y ij = 0, Z ij 0, (11) e ij = δ j X ij (1 δ 2 j )1/2 W ij ; X ij HN(0, 1), W ij N(0,1). (12) Notice that p ij = P(Y ij = 1) = P(Z ij > 0) = Φ SN (m ij, λ j ) is standard skew-normal cdf (Azzalini, 1985). A new item characteristic curve determines a new link function skew-probit that is more general that probit link function. A new link is different of a skew link as give in Chen et al. (1999). Notice that Z ij Z ij X ij = x ij N(m ij δ j x ij, 1 δ 2 j )

6 6 probabilidade de acerto l= 2 l= 1 l= 0 l= 1 l= theta Figure 1: Item characteristic curve for values different of λ parameter

7 7 Likelihood A likelihood of (u T, η T, δ T ) T for SN-IRT model are: original likelihood: Give D obs = y observed data, or L(u, η, λ D obs ) = L(u, η, λ D obs ) = n n i=i j=1 I i=i j=1 I Φ SN (m ij ; λ j ) y ij (1 Φ SN (m ij ; λ j )) 1 y ij, [2Φ 2 [ (mij, 0) ; δ j ] ] y ij [ 1 2Φ 2 [ (mij, 0) ; δ j ] ] 1 y ij. augmented likelihood: Give D = (Z, X, y), augmented data L(u, η, λ D) = n I φ(z ij ; δ jx ij + m ij, 1 δ 2 j )I(Z ij, y ij)φ(x i,j ; 0, 1)I(X ij > 0), i=1 j=1 com m ij = a j u i b j e I(Z ij, y ij ) = I(Z ij > 0)I(y ij = 1) + I(Z ij 0)I(y ij = 0).

8 8 Priors π(u, η, λ) = n i=1 g 1i (u i ) I j=1 g 2j (η j )g 3j (λ j ), (13) with g 1i = φ(0, 1), g 2j (η j ) = g 21j (a j )g 22j (b j ) and g 21j e g 22j should be proper to guarantee a proper distribution (see Albert & Gosh, 2000, Ghosh et al., 2001). We considered (Sahu, 2002), g 21 φ(µ a, s 2 a) and g 22 φ(0, s 2 b ). It is, ( s 2 g 2 φ 2 (µ η, Σ η ) with µ η = (µ a, 0) a 1 ) e Σ η =. 1 s 2 b Additionally is considered g 3 = φ SN (λ; w) with µ a, s 2 a, s 2 b, κ, ω known values. We consider the SN-IRT model in terms of δ j = λ j /(1 + λ 2 j )1/2, which takes values in the interval ( 1, 1), so that we can consider a uniform prior in ( 1,1) (t-student (µ = 0, σ 2 = 1/2 and v = 2) for λ j.

9 9 Sensitivity analysis using different priors for a and b Bayesian estimation procedures based in MCMC was implemented in WINBUGS by using Gibbs sampling method (Spiegelhalter et al., 1996). Chains with iterations were generated considering thin=1, 5, 10 and discarding the 500 first iterations, so that effective sample sizes were 49500, 9900 and 4950, respectively. When using MCMC, the sampled values for initial iterations of the chain are discarded because of their dependence on starting state and to guarantee the convergence. Also, in this SN-IRT model, presence of autocorrelation between chain values is expected when latent variables are introduced (Chen et al. 2000). Due to it, thin values up to 10 are recommended. A data set from mathematics test (Bazan et al., 2004a) is used for the analysis. In this application, 14 item of the Mathematical Test available for download in were applied to 131 students of high socio-economical status. Item response vectors are available from authors upon request.

10 10 In Table 1 shown some priors considered in the literature for the item parameter in the probit-normal model. N(0,1)I(0,) is notation for a normal distribution with mean 0 and variance 1 trimmed for negative values. Table 1: Prior specifications for item parameter in the the probit-normal model prior autor a prior b prior A Spiegelhalter et al (1996) N(0,1)I(0,) N(0,1000) B Albert e Ghosh (2000) N(0,1) N(0,1) C Congdon (2001) N(1,1) N(0,1) D Sahu (2002), Patz e Junker (1999) N(1,0.5)I(0,) N(0,2) E Sahu (2002), Albert e Ghosh (2000) N(0,1) N(0,1000) F Jhonson e Albert (2000) N(2,1) N(0,1)

11 The model is more sensitive to the prior specification for the difficulty parameter what for discrimination parameter. Priors B, C, D and F produce less dispersed estimates which estimates obtained with more diffuse priors as A and E. It is the case of the items 4, 7 and specially in the case of the item 11. This result is not observed for the probit-normal model, which is insensible to the specification of different priors as it has been observed for the same data by (Bazn, et al. 2004a). The estimates using priors B,C,D and F are positive and significantly correlated among them, but they have negative and low correlations with the estimates using the priors A and B. On the other hand, the estimates using the priors A and E has positive correlation among them that are not significant. The correlations can be explained by the fact that the priors B, C, D and F have priors precise for the b parameter, and the priors A and B has vague priors in this parameter. This result indicates that the specification of vague priors for the parameter of difficulty is not pertinent in the skew-probit-normal model. 11

12 12 References Albert, J. H. (1992). Bayesian Estimation of Normal Ogive Item Response Curves Using Gibbs Sampling. Journal of Educational Statistics, 17, Albert, J.H. and Ghosh, M. (2000). Item response modeling. Generalized Linear Models: A Bayesian Perspective(D. Dey, S. Ghosh and Mallick, eds.), Marcel-Dekker, New York, Azzalini A. (1985). A class of distributions which includes the normal ones. Scand. J. Statistical, 12, Bazán, J., Espinosa G.,& Farro Ch. (2002). Rendimiento y actitudes hacia la matemática en el sistema escolar peruano. In Rodriguez, J., Vargas, S. (eds.). Análisis de los Resultados y Metodología de las Pruebas Crecer Documento de trabajo 13. Lima: MECEP-Ministerio de Educación. Pp Bazán, J., Bolfarine, H., & Aparecida, R. (2004a). Estimação Bayesiana considerando MCMC para o modelo probit-normal da Teoria da Resposta ao Item. 7 0 Encontro Brasileiro de Estadística Bayesiana. UFSCar-São Carlos. Brazil. Fevereiro de 2004.

13 13 Chen, M-H, Shao, Q. M, & Ibrahim, J. G (2000). Monte Carlo Methods in Bayesian Computation. New York: Springer Verlag. Chen, M-H, Shao, Q. M, & Ibrahim, J. G (2000). Monte Carlo Methods in Bayesian Computation. New York: Springer Verlag. Ghosh, M., Ghosh, A., Chen, Ming-Hui & Agresti, A. (2000). Noninformative priors for one parameter item response models.journal of Statistical Planning and Inference.88, Patz, R. J., & Junker, B. W. (1999). A straighforward approach to Markov Chain Monte Carlo methods for item response models. Journal of Educactional and Behavioral Statistics,24, Sahu, S. K. (2002). Bayesian Estimation and Model Choice in Item Response Models. Journal of Statistical Computation and Simulation, 72, Spiegelhalter, D. J., Thomas, A., Best, N. G., & Gilks, W.R.(1996). BUGS 0.5 examples (Vol. 1 Version i). Cambrigde, UK: University of Cambride.