33. 12, or its reciprocal. or its negative.

Transcription

1 Page 6 The Point is Measuement In spite of most of what has been said up to this point, we did not undetake this poject with the intent of building bette themometes. The point is to measue the peson. Because of the complete symmety of the model, eveything we have done fo items, we can do again fo people just by evesing the subscpts. Fo any two people who took some of the same items, count the numbe N1 that peson answeed coectly and peson 1 missed; also the numbe N1 that peson 1 passed and peson missed. The elative abilities of the people will paallel expessions 3 and 5: 33. B N 1, o its ecipocal. B 1 N b b lnn N o its negative. 1 1 ln 1 It is not necessay that the people take all the same items, just so thee is some ovelap fo the pai. This can be genealized to any numbe of people by building the bigge matces (see Tables III. and III.3) and solving the simultaneous equations. But, thee ae some easons we don t want to do this. It would be awkwad (and not pue Raschian) to assign diffeent Scale Scoes to people with the same aw scoe. People who missed vey few items (o passed vey few) give us almost no useful data to wok with 1. The matces will have one ow and one column fo each peson tested and quickly become unwieldy fo even a modest-sized assessment. Howeve, we know that the numbe coect scoe is the sufficient statistic fo ability; we don t need a scoe fo evey peson, just fo evey numbe coect scoe. The issues can be cicumvented o at least mitigated by indexing the people by thei aw scoes and tabulating the esults as though eveyone with the same scoe is the same peson. On the othe hand, if we know the item difficulties well enough, we do what we have done since Panchapakesan (1969). Counts to Measues A moe computationally efficient pocess that has been the wokhose of US-based Rasch analysis (Wght and Panchapakesan, 1969) assumes the difficulties ae known (i.e., good enough estimates of the difficulty paametes ae available); no additional data ae needed. The ability estimate b associated with the aw scoe is the value that satisfies the basic equation: b di B e 35. P( x 1)., b di i1 i1 B Di i1 1 e whee is a aw scoe fom 1 to -1; is the total numbe of items; P(x) is the pobability of a coect esponse on item i fo a peson with the ability b associated with 1 No appoach to estimation has a paticulaly satisfying answe to the question of what to do with the people with zeo o pefect scoes. The same issue exists fo items but it is easie to ignoe. We will etun to the topic shotly with a couple of contved suggestions.

2 Page 7. Because total aw scoe is the sufficient statistic fo estimating ability, eveyone who took the same items and got the same aw scoe gets the same estimated ability b. Hence the pobability and estimate can be indexed by the aw scoe, instead of the peson. Equation 35 simply says the expected total scoe p is equal to the obseved total scoe ; if they aen t equal enough, the ability estimate needs adjusting. If the expected scoe is low, the estimated ability is inceased; if the expected scoe is too high, the estimate is deceased. The di ae taken to be known. We only need to fiddle with the b until the equation is tue. That s all thee is to computing abilities; the est is details fo doing the fiddling. The ability estimate is adjusted by you favote numec method until equation 35 is satisfied. Wght & Panchapakesan (1969) applied Newton s method to do the iteating: 36. b t 1 b t i1 p i1. p (1 p ) An effective stating value fo this pocess is: b ln d which is often zeo. whee di i d 1 is the cente of the item difficulties, Table III.7 shows the athmetic fo a small test with 10 dichotomous items. It is typical in this situation fo the pocess to stabilize in two o thee iteations. This pocess fo estimating ability can be deved with standad maximum likelihood methods: define a likelihood function fo the data; take the fist devative; set equal to zeo; solve and but fist check that you ve got a maximum not a minimum. That s all the detail I m going to give, but I will note that the symmety of the model means you can tun the notation aound and do exactly the same thing fo items. Table III.7: Calculations of ogit Abilities fo a Test with 10 Dichotomous Items Item ogit Raw Round Round Std Scoe Initial One Two Eo

3 Page 8 Item ogit Raw Scoe Initial Round One Sum of p Round Two Sum of p(1-p) Std Eo All Right o All Wong Equation 37 fo the stating value makes it obvious, but it also follows moe subtly and moe pofoundly fom the estimation equation 35, that pefect scoes, both =0 and =, ae poblems. Thee is no ability low enough to eve satisfy equation 35 when is 0, no high enough when is. In the eal wold, it is geneally necessay to manufactue something to epot fo examinees with these scoes, although it would be much pefeed to avoid giving tests so fa off taget. One tactic is to solve the equations fo non-intege scoes abitaly close to the pefect scoes, say, within 0.5. Whethe the taget should be off by 0.5, o 0.1, o 0.33, o some othe value is completely abitay; the smalle the value, the moe exteme the solutions will be. It is moe a policy decision than psychometc issue about how much punishment o ewad should be attached to those scoes. Anothe stategy, with slightly moe psychometc undepinning and avoids the abitay choice of taget, poduces almost the same esults by assigning to a scoe of zeo the logit ability fo a aw scoe of one minus its squaed standad eo of measuement: 38. b 0 b1 s1. Analogously fo a pefect scoe of, the logit ability estimate is the estimate fo a scoe of -1 plus its squaed standad eo. The simple ationale fo this tactic is that the

4 Page 9 diffeence between logit ability estimates fo any adjacent scoes is vey nealy equal to the squaed standad eo of measuement. The moe eudite explanation is that, because the squaed standad eo is the invese of the denominato of expession 36, this tactic is equivalent to using expession 36 to estimate the ability fo zeo (o ) using the stating value b1 (o b-1) and stopping afte the fist iteation. This is the method used in Table III.7, although thee decimals implies moe pecision than I feel about this step. 39. b0 = (1.071) = b10 = (1.073) = Fo tests with dichotomous items, the standad eos fo 1 and -1 will typically be a little moe than one. Squang that gives about 1.15 o 1.. Using eithe of these values in place of s1 o s-1 gives almost the same esult as eithe of the othe methods. We e making the numbes up anyway but that s too simple to be given seous scholaly consideation. That s the touble with Rasch. Standad Eos of Measuement A statistician is a peson with a bag of standad eos and who can poduce the appopate one fo any situation. Theodoe Bancoft The Pai algothm has been cticized fo the lack of an asymptotic standad eo estimato and who wouldn t want one of those. That doesn t mean that we don t have a suggestion. A easonable possibility fo the standad eo fo each element of the R matx is: nij n ji 41. sij. 4 nij n ji 4nijn ji It would be aggegated to the ow aveage as: sij ji 4. si * whee * is the numbe of defined elements in ow i of R and the facto 1/4 ases because of a lack of independence in R. Because evey item esponse can influence seveal item pais, the counts ae not independent and hence best case values. But at least the estimates won t be walking aound naked and unchapeoned. Fo the so-called maginal maximum likelihood estimation pocess we used to estimate abilities, we do have an estimate of the asymptotic standad eo fo the logit ability at each aw scoe. Fo the simple case of dichotomous items, the standad eo fo the ability estimate at a aw scoe is: The numeato will be plus o minus one because we ae using the ability estimate fo the scoe that is one off fom whee we want to be.

5 Page s 1 p(1 p). i 1 These apply to the estimates poduced by expession 35 and ae sometimes efeed to as conditional standad eos to distinguish them fom The Standad Eo of tue scoe theoy but at this point I pefe to think of them as functions of athe than conditioned on and foget all about the thing they have supplanted. Standad Eo of Measuement; Not Standad Eo of Scoe The standad eo function s 1 p (1 p ) defines a bowl-shaped cuve, i 1 meaning we ae moe confident of ou measuements nea the cente of the test than at the extemes. Some find this upsetting because it seems to un counte to what they wee taught in thei fomative yeas. Tue scoe theoy tells us to have moe confidence in scoes at the extemes than in the cente of the scoe ange, i.e., a dome-shaped function of the fom i1 p i ( 1 p i ). Thee eally is no inconsistency; we ae talking about two diffeent standad eos. One is the standad eo fo a measue and the othe is the standad eo fo a scoe. If we give a test that is much too easy, we have a vey good idea what a peson s scoe will be: pefect o vey nea to it and hence a small standad eo fo the scoe. But a pefect scoe is consistent with a vey lage ange of abilities, fom hee to infinity; hence, a huge standad eo fo the measue. At the extemes of the scoe anges, we know what the tue scoes must be but have vey little idea what the abilities ae. Convesely, nea the cente of a test, we have the least confidence in the numbe coect scoe and the most confidence in the logit measue. In the moden wold, p ( 1 p ) should be thought of as the infomation i1 function, which is maximized at the cente of the test. No one seously thought giving an off-taget test was a good idea.