RB-66-4D ~ E S [ B A U R L C L Ii E TI KR- 21 FOR FORMULA SCORED TESTS WITH OMITS SCORED AS WRONG Robet L. Linn, Robet F. Boldt, Ronald L. Flaughe, and Donald A. Rock N This Bulletin is a daft fo inteoffice ciculation. Coections and suggestions fo evision ae solicited. The Bulletin should not be cited as a efeence without the specific pemission of the authos. It is automatically supeseded upon fomal publication of the mateial. Educational Testing Sevice Pinceton, New Jesey August 1966
KR-2l fo Fomula Scoed Tests with Omits Scoed as Wong Abstact An epession was deived fo computing KR-2l based on fomula Bcoe Stati6tics fo the special case when the numbe of items on the test equals the numbe ight plus the numbe wong (i.e" eithe thee ae no omits o omits ae scoed wong). The effect of using means and vaiances based on fomula scoed tests in the KR-2l fomula usually given in tetbooks was also discussed.
KR-21 fo Fomula Scoed Tests with Omits Scoed as Wong One of the most attactive featues of Kude-Richadson fomula 21 (KR-21) is the ease with which it can be computed fom a minimal amount of data (Kude &Richadson, 1937). Given only the test mean, vaiance, and the numbe of items, the KR-21 povides an estimated lowe bound fo the test eliability based on item homogeneity (Gulliksen, 1950). A limitation of the fomula as usually pesented, though, is that the test scoes must be based on "ights only" scoing. If the test is scoed by a fomula such as the numbe ight minus a faction of the numbe wong, then the esulting test mean and vaiance ae inappopiate fo use in the KR-21. The pupose of this pape is twofold: fist, to deive an epession fo computing KR-2l based on fomula scoe statistics fo the special case when the numbe of items on the test equals the numbe ight plus the numbe wong (i.e., eithe thee ae no omits o omits ae scoed wong); and second, to indicate the effect of using means and vaiances based on fomula scoed tests in the KR-2l fomula usually given in tetbooks. With the stated assumption (that numbe of items is equal to ights plus wongs) the choice of scoing weight will have no effect on eliability. Fo puposes of eposition a fomula fo estimating the "ights only" eliability using fomula scoe deived statistics will be developed. The assumptions that the aveage covaiance between nonpaallel items is equal to the aveage covaiance betyeen paallel items, and that all items ae of the same difficulty, which ae equied by the KR-2l (Gulliksen, 1950), yill be assumed thoughout this pape.
-2- A convenient fom of the KR-21 is: (1) whee is the KR-21, n is the numbe of items, M is the test mean based on the sum of the numbe ight, and 8 2 is the test vaiance based on the sum of the numbe ight. If it is assumed that fo each peson n=+w' (2 ) whee n is the total numbe of items, is the numbe ight and w is the numbe wong, then the usual fomula scoe Xi fo individual i : whee K is the constant used fo fomula scoing, can be witten as X. =. - (n -.)/K 111 (4) The mean based on fomula scoes, M,is given by whee N 16 the numbe of people. Substituting (4) into (5): 1 N n 1 N M = - E - - + - E. N 1=1 i K NK 11:1 1 (6) 1 =M K _K+I M - -en - M ) n - K - - if (8)
-3- which yields (9) The vaiance of X, 8 2 is given by S2 =S2 _ g cov (, n _ ) + ~ S2 K ~ n- (10) 'Whee S2 is the vaiance of the numbe ight, is the vaiance of the numbe wong, and cov (, n - ) is the covaiance of the numbe ight with the numbewong. Since n is a constant, cov (, n - ) = - S2, theefoe (11) 2 2 K + 1 = S( K ), (12) and Substituting equations (8), (9), and (13) into (1): )~ (14)
-4- An altenative deivation of equation (15) is pesented in Appendi A. It could also be deived fom the geneal KR-20 computing equation given by Dessel (1940) fo tests scoed as a linea c~blnation of ights and wongs. Dessel' s fomula (6) which is a modified KR-20 fomula is also based on the assumption that n = + w. Equation (15) povides a simple means of calculating the KR-21 diectly fom fomula scoe mean, vaiance, numbe of items and the coection constant, K, whee fomula scoes ae calculated accoding to equation (3) and n = + w To detemine the effect of using means and vaiances based on fomula scoed tests in the KR-21, M,and (1) fo M and 8 2 espectively. 8 2 can be substituted into equation If this is done and equation (15) is subtacted f'om the esult, the diff'eence, 6, is found to be (16) If M and 8 2 and K then in equation (16) ae epessed in tems of M s2, ' n, n n - 1 Since M is always less than n unless all test scoes ae pefect and (K + 1)S2 is positive, 6 must always be positive and will be lage fo a difficult test than fo an easy test. As K inceases, 6, the amount of bias due to using fomula scoe statistics in KR-21, will decease
-5- monotonically appoaching zeo as K appoaches infinity, i.e., the bias is zeo ~hen wongs ae scoed zeo. Since equation (15) is based on the assumption that thee ae no 0~it6, o that omits ae scoed wong, it should be used ~th caution. In pactice, thee ae almost always some omits and omits ae not scoed wong in fomula scoes of the type indicated by equation (3). The KR-21 based on numbe ight is also sensitive to the numbe of o~its. Ho~eve, the effect of omits may be moe seious with fomula scoing than with "ights only" scoing since wongs and omits ae given diffeent weights in scoing. The effect of omits on the esults of equation (15) ~ill depend on the vaiances and covaiances of ights, wong~ and omits. would povide an indication of this effect. An empiical investigation Cetainly, neithe the KR-21 no equation (15) should be used fo highly speeded tests. If omits ae not scoed wong, and the numbe of ights, wongs, and omits fo each individual ae known then Kaitz's (1945) fomula (11) 1s the fomula that should be used fo the KR-2l with fomula scoing.
Refeences Dessel, P. L. Some emaks on the Kude-Richadson eliability coefficient. Psychometika J 1940, 2, 305-310. Gulliksen J H. Theoy of mental tests. New Yok: Wiley, 1950. Kaitz, H. B. A note on eliability. Psychometika, 1945, 10, 127-131. Kude, G. F., &Richadson, M. W. The theoy of the estimation of test eliability. Psychometika, 1937, ~, 151-160.
-7- Appendi A Fom equation (7) the mean difficulty p can be obtained: A.I The epectation fo an item g is given by E(g) ::: p - ~l - p) A.2 M :::- n A.4 Let 5. ::: 1 if peson i gets item g coect and 5. ::: 0 if peson l.g 19 i gets item g wong, then the item vaiance 8 2 is: g E~ + ]8. -:Y 2 1 (K + 1L. E(5. )2 K 19 K i2 19 \K n K ig K n _ 2 (!o + M ) K + 1 E(8 ) (1 M~2 ::: + 1- +- \ '. ' A 5 A.6 A 7 The Kude-Richadson fomula 20 can be witten as _i~l s~] [1 = ( n ) n - 1 02 o A.8
-8- SUbstituting equation A.7 into A.8 and assuming that all items ae of equal di~ficulty and theefoe 8 2 i6 the same fo all g, esults in s ;: which educes to ;: (n n 1) 1 - n t: [(K + 1) K + ~) s2 [ K - 1 _!.~+!'.] (n n ) 1 _ K ~ n K - 1 8 2 - (M: + ~ 2Jl.1 A 9 A.lO Equation A.lO is identical to equation (15).