Pairwise Parameter Estimation in Rasch Models

Transcription

1 Pairwise Parameter Estimation in Rasch Models Aeilko H. Zwinderman University of Leiden Rasch model item parameters can be estimated consistently with a pseudo-likelihood method based on comparing responses to pairs of items irrespective of other items. The pseudo-likelihood method is comparable to Fischer s (1974) Minchi method. A simulation study found that the pseudo-likelihood estimates and their (estimated) standard errors were comparable to conditional and marginal maximum likelihood estimates. The method is extended to estimate parameters of the linear logistic test model allowing the design matrix to vary between persons. Index terms: item parameter estimation, linear logistic test model, Minchi estimation, pseudo-likelihood, Rasch model. For Rasch model item parameter estimation, conditional maximum likelihood (CML) and marginal maximum likelihood (MML) methods are available in many computer programs (for an overview see Glas, 1989). The computational problems encountered in the past can now be handled (e.g., large item sets can be analyzed on personal computers). To avoid lengthy computation times, Fischer (1974; Fischer & Scheiblechner, 1970) investigated alternative estimation methods in the early 1970s. One method compared pairs of item responses irrespective of all other item responses. Because the method yielded x2 a Pearson optimization criterion, the method was called Minchi estimation (Fischer, 1974). Van der Linden & Eggen (1986) proposed a similar method based on maximizing the corresponding pseudo-likelihood. This approach also was studied by Arnold & Strauss (1988), Andrich (1988), and Choppin (1968). However, the van der Linden & Eggen (1986) method was rejected because no statistical properties of the estimators could be established, despite the fact that the method was simple, fast, and seemingly accurate. Properties of the pseudo-likelihood method, such as consistency and the efficiency of the estimators, were investigated here. A simulation study was conducted to compare the pseudo-likelihood estimates to the CML and MML estimates. Also, the pseudo-likelihood method is extended to the linear logistic test model (LLTM; Fischer, 1983) allowing the design matrix to vary between persons. Pseudo-Likelihood Estimation Consider a set of k dichotomous items to which a sample of N persons responded. All persons need not respond to all items, but it is assumed here that they have. The probability that person v with trait level 0, (v = 1,..., N) responded correctly (or to the keyed alternative) to item i with easiness parameter a, (i = 1,..., k) satisfies the Rasch model: * ―f~. ).-―,/ J One item parameter is fixed to 0 for identifiability reasons; thus, only k - 1 item parameters are estimated. Lets be the probability that person v responded correctly to item i (YVI = 1) and incorrectly to item j (YV) = 0) given that v responded correctly to only one of them (Fischer, 1974) APPLIED PSYCHOLOGICAL MEASUREMENT Vol. 19, No. 4, December 1995, pp. Copyright 1995 Applied Psychological Measurement Inc /g5/ $

2 370 The as can be estimated by maximizing the pseudo-loglikelihood of all pairs of item responses given 0: Although Equation 3 appears to be a loglikelihood equation, it is not because the (i,j) combinations are dependent. Therefore, Equation 3 is a pseudo-loglikelihood equation. Standard maximum likelihood theory will not apply to the estimator d that maximizes Equation 3. Therefore, whethe a is consistent must be evaluated. Also, a suitable expression for the covariance matrix of â must be identified. Other properties are also important (such as bias and small sample behavior), but are not considered here. Consistency of the Estimates The consistency of a can be proved by noting that the expectation of l(a~9) equals where the superscript 0 denotes that true values are considered. The last step is possible due to local stochastic independence between Yv, and F given Av. Equating the derivatives of Equation 4 to 0 with respect to a yields which is true when for each term Hence, the expectation of 1(«0) is maximal when the differences between the estimated parameters (&, - are equal to the differences between the true item parameters (a, - a ). 0 cancels when equating 8E (1(a )0 )] /8a to 0; thus, no assumptions concerning 0 or its distribution are necessary. Therefore, the conditioning argument is not further considered here. By fixing the mean of the estimates to 0, â is a consistent estimator of a. This result also applies to CML and MML: only differences between item parameters are estimated consistently by their CML or MML estimators (Glas, 1989). The above conclusion about the consistency of 6c is valid only when a exists; it does not ensure uniqueness. As yet, no sufficient and uniqueness conditions for the existence of & are known, &)

3 371 but a necessary condition is that there is at least one person responding correctly to item i and incorrectly to item j for all combinations (i, j). The pseudo-score function or first-order derivative of l(a) for a, is where n,~ is the number of persons responding correctly to item i and incorrectly to item j. Estimates of the Covariance Matrix It can be shown (Connolly & Liang, 1988, p. 504) that under mild regularity conditions (a - a ) is asymptotically normally distributed with mean 0 and covariance matrix Because the contributions of the N persons to the score functions are independent of each other and have expectation 0, an estimate of the covariances of the score functions is given by (see Connolly & Liang, 1988, p. 504) Accordingly, the covariance matrix of a can be estimated by substituting â in Equations 5 and 6. This results in simple counting and arithmetic. The negative matrix of second-order derivatives of Equation 1 is of order k, but has rank k - 1, and has diagonal entries E~(n~~ + n~,)f,~(1 - f~) and off-diagonal entries -(n,~ + n)f, C 1 - f,). The covariance matrix of the score functions (Equation 9) is also of order k, has rank k -1, and has entries Evg,g~, where (h = 1,..., k; h # i) An Illustrative Simulation The pseudo-likelihood estimator was compared with the CML and MML estimators in a simulation study using 13 items and N = 500. Item responses were simulated according to a method described elsewhere (Zwinderman, 1991). Item parameters were fixed (see Table 1) and Os were sampled from the standard normal distribution. For each person-item combination, the response probability was computed according to Equation 1. The simulated observation was set to 1 if this probability was greater than a random number generated from a [0,1 uniform distribution and 0 otherwise. CML and MML estimates were computed using the RSP program (Glas & Ellis, 1994). Results are reported in Table 1. The pseudo-likelihood estimates were not very different from the CML and MML estimates, nor were the corresponding standard errors (SES). In general, CML and MML estimates were very similar to each other, but the pseudo-likelihood estimates were slightly different. However, none of the three sets of estimates was in general best (i.e., none was the closest to the true item parameters). The estimated SEs of the pseudo-likelihood estimates were on average slightly larger, which is probably due to the fact that the covariance matrix in Equation 5 tends to be unstable and underestimates the true covariance matrix.

4 372 Table 1 Pseudo-Likelihood, CML, and MML Estimates of the Simulated Item Parameters Extension of the Pairwise Estimation Method to the LLTM In the LLTM (Fischer, 1974, 1983), the item parameters in Equation 1 are modeled as a function of a design matrix X and basic weights p, where X is the same for all persons. The model is extended here by allowing the design matrix X to vary between persons. Inserting into Equation 1 yields where Xv, is a vector of length m with known entries and 0 is a vector of length m with unknown parameters. X is a vector with the observed values of m covariates, and these vary both between and within persons (from item to item). Thus, Equation 11 is the logistic regression model with random effects, and fit may be interpreted as a regression weight. p can be estimated using CML or MML estimation, but it is difficult. When using CML, the basic symmetric functions and their first- and second-order derivatives (Liou, 1994; Verhelst, Glas, & van der Sluis, 1984) must be evaluated for all contributions to the loglikelihood. These cannot be grouped together in sum score groups as in the ordinary Rasch model. With 10,000 persons responding to 50 items, the CML loglikelihood of the ordinary Rasch model consists of 49 terms, one for each sum score group; however, in the LLTM it consists of 50 x 10,000 terms because the design matrix can vary. When using MML, a similar numerical explosion occurs, regarding the evaluation of the integral involved in the MML method. In contrast, the numerical work of the pseudo-likelihood approach remains relatively limited. The pseudo-loglikelihood is still given by Equation 3 with P substituted for a, buts is now defined as: Equating the derivatives of E[1(0)] to 0 with respect to P yields

5 373 for each term of Equation 4, where po refers to the true value. Therefore, p, which maximizes l(~), is a consistent estimator of true P, if it exists. Again no uniqueness and sufficient conditions for the existence of 0 are known, but in addition to the conditions mentioned before, it is necessary that each covariate X has nonzero positive variance. The pseudo-score equation for the regression parameter of the pth covariate, ~ip, is: where U(.) is defined as in Equation 7. Because the contributions of the N persons to the score functions are independent and have expectation 0, the covariance matrix of 0 can be estimated with the estimator in Equation 8. The negative matrix of second-order derivatives of the pseudo-loglikelihood is of order and rank m with elements ( p, q = 1,..., m) The covariance matrix of the score functions is also of order and rank m and has elements which unfortunately do not reduce to a simple equation. An Example Method. The binary item responses of 209 asthmatic patients to an 11-item questionnaire that measures functional independence were analyzed (Maille, Koning, Zwinderman, & Kaptein, 1994). The ages of the respondents varied from 41 to 87. The mean age was 64, and the standard deviation was 14 years. According to Glas R, test.in the RSP program (Glas & Ellis, 1994), the Rasch model fit the data (R, 57, = df 40, = p.04), = except when the sample was split into two groups (persons 65 years or younger and persons older than 65; R1 62, df = 10, = p <.0001). Therefore, differential item functioning was investigated using Equation 11. Two sets of covariates (X and Z) were constructed: (1) xv~ = 1 for item j + 1 (j = 1,..., 10) and 0 otherwise; and (2) zvl the interaction between xv, and v s age. The first set of covariates (X) was simply a transformation of the usual item parameters, but normalized so that a, 0. The = second set indicated whether the item easiness parameters varied over persons of different ages. Age could also be incorporated into Equation 11, but its regression parameter could not be estimated with the pseudolikelihood method due to the conditioning argument in Equation 12 (nor with the CML method). Results. Table 2 shows the CML and pseudo-likelihood estimates of the regression parameters. As in the simulation, the CML and pseudo-likelihood estimates were similar, as were their SEs. Item 3 (&dquo;have you difficulty walking stairs?&dquo;) appeared to be more difficult for older persons than for younger [Wald test (Rao, 1973, p. 417): p <.001]. For example, the conditional probability of a correct response to Item 3 given a 0 value of the latent functional independence scale (0 0) = was.27 for a 50-year old and.86 for a 75-year old. It was concluded that difficulties walking stairs were not particularly indicative of the functional limitations experienced by asthmatic patients, but that they reflected the aging process. Conclusions Using the pseudo-likelihood method to estimate Rasch model item parameters provides estimators that are consistent and similar in efficiency to CML and MML estimators, at least for the small datasets examined here. The pseudo-likelihood method was developed to solve the estimation problems of the Rasch model, but many have already been solved. Nevertheless, pseudo-likelihood estimation is a powerful tool that may be used for situations in item response theory in which estimation is not yet straightforward, such as in linear logistic models or longitudinal or dynamic logistic test models.

6 374 Table 2 Pseudo-Likelihood and CML Estimates of the Item Parameters, and the Interaction Effect of Age and the Item Parameters (ax Age) *p <.05 according to the Wald test. **p <.001 according to the Wald test. The present pairwise estimation method also has limitations. The estimator of the covariance matrix of the estimators (Equation 8) tends to be unstable, because it depends on both the parameter estimates and the data. In LLTMS, the regression parameters of covariates that vary between persons, but are constant within persons, cannot be estimated. Finally, the method does not generalize to other item response theory models. It only generalizes partially to polytomous item responses. For example, the item parameters of the unidimensional polytomous Rasch model with equidistant scoring of the categories (Andersen, 1973) may be estimated consistently and efficiently by comparing all pairs of items i and j with a response to category c on item i and a response not in c to item j. However, the category parameters of this model cannot be estimated by such pairwise comparisons. References Andersen, E. B. (1973). Conditional inference for multiple choice questionnaires. British Journal of Mathematical and Statistical Psychology, 26, Andrich, D. (1988). Rasch models for measurement. Beverly Hills: Sage. Arnold, B. C., & Strauss, D. (1988). Pseudolikelihood estimation (Technical Rep. 164). Riverside: University of California, Department of Statistics. Choppin, B. (1968). An item bank using sample-free calibration. Nature, 219, Connolly, M. A., &, Liang, K. Y. (1988). Conditional logistic regression models for correlated binary data. Biometrika, 75, Glas, C. A. W. (1989). Contributions to estimating and testing Rasch models. Unpublished doctoral dissertation, University of Twente, Enschede, The Netherlands. Glas, C. A. W., & Ellis, J. L. (1994). RSP: The Rasch scaling package [Computer program]. Groningen: Institute Progamma. Fischer, G. H. (1974). Einfuehrung in die theorie psychologischer tests [Introduction to the theory of psychological tests]. Bern: Huber. Fischer, G. H. (1983). Logistic latent trait models with linear constraints. Psychometrika, 48, Fischer, G. H., & Scheiblechner, H. (1970). Algorithmen und programme für das probabilistischen testmodell von Rasch [Algorithms and programs for the probabilistic test model of Rasch]. Psychologische Beiträge, 12, Liou, M. (1994). More on the computation of higherorder derivatives of the elementary symmetric functions in the Rasch model. Applied Psychological Measurement, 18, Maille, A. R., Koning, C. J. M., Zwinderman, A. H., & Kaptein, A. A. (1994). De ontwikkeling van een kwaliteit van leven vragenlijst voor CARA-patienten [The development of a quality of life questionnaire for

7 375 patients with respiratory illness]. Gedrag en Gezondheid, 22, Rao, C. R. (1973). Linear statistical inference and its applications. New York: Wiley. van der Linden, W. J., &, Eggen, T. J. H. M. (1986). An empirical Bayesian approach to item banking. Applied Psychological Measurement, 10, Verhelst, N. D., Glas, C. A. W., & van der Sluis, A. (1984). Estimation problems in the Rasch model: The basic symmetric functions. Computational Statistics Quarterly, 1, Zwinderman, A. H. (1991). A generalized Rasch model for manifest predictors. Psychometrika, 56, Acknowledgments This research was supported by a grant from the Netherlands Organization for Scientific research (NWO). Author s Address Send requests for reprints or further information to Aeilko H. Zwinderman, Department of Medical Statistics, University of Leiden, P.O. Box 9604, 2300 RC Leiden, The Netherlands. a.h.zwinderman@medstat.medfac. leidenuniv.nl.