Uivrsity f Psylvaia ShlarlyCmms partmtal Paprs (ASC) Abrg Shl fr Cmmuiati -5-0 Cmputig Krippdrff 's Alpha-Rliability Klaus Krippdrff Uivrsity f Psylvaia, kkrippdrff@as.up.du Fllw this ad additial wrks at: http://rpsitry.up.du/as_paprs Part f th Cmmuiati Cmms Rmmdd Citati Krippdrff, K. (0). Cmputig Krippdrff's Alpha-Rliability. Rtrivd frm http://rpsitry.up.du/as_paprs/43 Pstprit vrsi. This papr is pstd at ShlarlyCmms. http://rpsitry.up.du/as_paprs/43 Fr mr ifrmati, plas tat libraryrpsitry@pbx.up.du.
Cmputig Krippdrff 's Alpha-Rliability Abstrat Krippdrff s alpha (α) is a rliability ffiit dvlpd t masur th agrmt amg bsrvrs, drs, judgs, ratrs, r masurig istrumts drawig distitis amg typially ustruturd phma r assig mputabl valus t thm. α mrgd i tt aalysis but is widly appliabl whrvr tw r mr mthds f gratig data ar applid t th sam st f bjts, uits f aalysis, r itms ad th qusti is hw muh th rsultig data a b trustd t rprst smthig ral. isiplis Cmmuiati Sial ad Bhaviral Sis Cmmts Pstprit vrsi. This wrkig papr is availabl at ShlarlyCmms: http://rpsitry.up.du/as_paprs/43
Cmputig Krippdrff s Alpha-Rliability Klaus Krippdrff kkrippdrff@as.up.du 0..5 Krippdrff s alpha () is a rliability ffiit dvlpd t masur th agrmt amg bsrvrs, drs, judgs, ratrs, r masurig istrumts drawig distitis amg typially ustruturd phma r assig mputabl valus t thm. mrgd i tt aalysis but is widly appliabl whrvr tw r mr mthds f gratig data ar applid t th sam st f bjts, uits f aalysis, r itms ad th qusti is hw muh th rsultig data a b trustd t rprst smthig ral. s gral frm is: whr is th bsrvd disagrmt amg valus assigd t uits f aalysis: k mtrik k ad is th disagrmt wuld xpt wh th dig f uits is attributabl t ha rathr tha t th prprtis f ths uits: k mtrik ( ) k Th argumts i th tw disagrmt masurs, k,, k ad, rfr t th frquis f valus i iid matris, t b dfid blw. Algbraially, wh bsrvrs agr prftly, bsrvd disagrmt =0 ad =, whih idiats prft rliability. Wh bsrvrs agr as if ha had prdud th rsults, = ad =0, whih idiats th abs f rliability. =0 urs wh bsrvrs ar uabl t distiguish amg uits r assig valus t thm draw radmly frm a lltiv stimat f th ppulati f data. T rly data gratd by ay mthd, ds t b far frm ths tw xtrm ditis, idally =. Fr rliability sidratis, s rag is: 0 Systmati disagrmt Samplig rrrs Ulik thr spializd ffiits, is a gralizati f svral kw rliability idis. It abls rsarhrs t judg a varity f data with th sam rliability stadard. applis t: Ay umbr f bsrvrs, t just tw Ay umbr f atgris, sal valus, r masurs Ay mtri r lvl f masurmt (mial, rdial, itrval, rati, ad mr) Implt r missig data Larg ad small sampl sizs alik, t rquirig a miimum valuats rliability variabl at a tim. It ffrs thr aalytial pssibilitis t prstd hr.
Rliability data dupliat th prss f gratig data whs rliability is i qusti. Giv suh data, -rliability a b mputd i fur mputatial stps, graphd blw. Rliability ata Rliability ata Matrix Obsrvd iffr -Agrmt O Variabl Ciids Futi u iu k k kju mu Ths fur mputatial stps will b dfid ad illustratd with fur kids f data f irasig mplxity: A. Biary r dihtmus data, tw bsrvrs, missig data Pag B. Nmial data, tw bsrvrs, missig data Pag 3 C. Nmial data, ay umbr f bsrvrs, missig data Pag 4. All mtris, ay umbr f bsrvrs, missig data Pag 5 Fially, E. A gral mputatial frm is prstd, bypassig iid matris Pag 9 A. Biary r dihtmus data, tw bsrvrs, missig data Cstrut a rliability data matrix; hr, a bsrvrs-by-n uits matrix, taiig N valus, ad k: Uits: u N Obsrvrs: i: i i iu in j: j j ju jn Fr xampl, wh tw bsrvrs judg N=0 uits, th -by-0 data matrix tais 0 valus: Itms judgd: 3 4 5 6 7 8 9 0 Mg: 0 0 0 0 0 0 0 0 Ow: 0 0 0 0 0 0 Tabulat iids withi uits. Ciid matris aut fr all valus taid i a rliability data matrix. Thy diffr frm th familiar tigy matris, whih aut fr uits i tw dimsis, t valus. Th imprta f this diffr bms appart i C. It a -by- iid matrix, uits ar trd twi, as -k pairs ad as k- pairs. I th xampl, uit is trd as a 0- pair f valus ad as a -0 pair f valus. Uit is trd as tw - pairs f valus, t.: Valus: 0 0 0 00 0 0 0 0 4 4 0 4 6 Numbr f Valus: 0 =N 4 6 0
Ardigly, 00 rprsts th t 0-0 pairs withi uits 4, 5, 7, 8, ad 0. 0 rprsts th fur 0- pairs i uits, 3, 6, ad 9, ad 0 rprsts th fur -0 pairs i th fur sam uits. rprsts th tw - pairs fud ly i uit. 0 =4 is th umbr f 0s i th rliability data matrix, =6 is th umbr f s, ad =N=0 is th ttal umbr f valus paird. Fr ths biary data, mismathig iids ur i tw lls 0, 0 f qual frquy, 4. skip Cmput -rliability (mst simpl frm): biary ( 0 ) 0 4 I th xampl: biary (0 ) 0. 095 4 6 B. Nmial data, tw bsrvrs, missig data Cstrut a rliability data matrix just as i A abv. Fr a -by- xampl: Itms judgd: 3 4 5 6 7 8 9 0 B: a a b b d d d a Grry: b a b b b d d d Tabulat iids withi uits. Th gral frm f a iid matrix ad with frquis frm th abv xampl trd ar: Valus:. k.. a b d. k.. a.. 4.... b 4.. 6...... 6.. 6. k.. = k k d. 4. 6......... k.. = k k 4 6 6 6 4 Whr k = u Numbr f -k pairs i uit u spifially: ab = a-b pair i uit ba = b-a pair i uit aa = a-a pairs i uit bb = 4 = b-b pairs i uit 3 + b-b pairs i uit 4 ad s frth. a =4 is th umbr f as b =6 is th umbr f bs ad s frth. =4 is th ttal umbr f valus fr tw bsrvrs: = N skip Cmput -rliability (mst simpl frm): mial A A A ( ) ( ) ( ) ( ) 3
Whri A is th prt f bsrvd maths i uits u ad A is th prt f maths btaiabl by ha. Th mputatial frm furthr simplifis th dd mputatis. I th xampl: mial (4 )( 4 6 4 ) [4(4 ) 6(6 ) 6(6 ) 6(6 ) ( )] 0.69 4(4 ) [4(4 ) 6(6 ) 6(6 ) 6(6 ) ( )] C. Nmial data, ay umbr f bsrvrs, missig data Cstrut a rliability data matrix just as i A ad i B abv, but fr m bsrvrs: Uits u:... u..... N Obsrvrs:... u..... N i i i... iu..... in j j j... ju..... jn..... m m m... mu..... mn Numbr f bsrvrs valuig u: m m... m u..... m N Wh data ar missig, this matrix tais lss tha mn tris ad m u is variabl. Fr xampl, a 4 bsrvrs-by- uits rliability data matrix: Uits u: 3 4 5 6 7 8 9 0 Obsrvr A: 3 3 4... Obsrvr B: 3 3 4 5. 3 Obsrvr C:. 3 3 3 3 4 5. Obsrvr : 3 3 4 4 5. Numbr m u f valus i uit u: 3 4 4 4 4 4 4 4 4 3 4 Nt that 7 ut f th 48 pssibl valus i this matrix ar missig. m u varis frm t 4. Tabulat iids withi uits. Th iid matrix appars as i B: Valus:. k.. 3 4 5. k.. 7 4/3 /3 /3. 9.... 4/3 0 4/3 /3. 3.... 3 /3 4/3 8 /3. 0. k.. = k k 4 /3 /3 /3 4. 5.... 5.... 3 3. k.. = k k 9 3 0 5 3 40 Numbr f - k pairsi uit u Ulik i th tw-bsrvr as i B: k u mu Nt that ah uit tais m u (m u ) iids. A iid matrix auts fr all pairs f valus fud i u. Uit tais 3(3 )=6 pairs f mathig s. It tributs 4
6/(3-)=3 t th ll, fr ah valu. Uit tais 4(4-)= pairs, 6 mathig - pairs, 3 mismathig -3 pairs, ad 3 mismathig 3- pairs. It adds 6/(4-)= t, 3/(4- )= t 3, t 3, ad 4 t th ttal, thus fully autig fr its 4 valus. Uit 6 tais 4(4-)= pairs f mismathig valus, ah adds /(4-)=/3 t a diffrt ll. Th l valu 3 i uit affrds mpariss, (-)=0 pairs ad ds t add t this aut. Thus, th margis f iid matris d t rprst all valus that ur i a rliability data matrix, ly ths that a b paird withi uits, hr =40 pairabl valus vr all uits. Skip Cmput -rliability just as i B mial A A A ( ) ( ) ( ) ( ) I th xampl: (40 )(7 0 8 4 3) [9(9 ) 3(3 ) 0(0 ) 5(5 ) 3(3 )] mial 0.743 40(40 ) [9(9 ) 3(3 ) 0(0 ) 5(5 ) 3(3 )]. Ay mtri, ay umbr f bsrvrs, missig data Cstrut a rliability data matrix just as i C Tabulat iids withi uits just as i C Isrt th diffr futi mtri k that is apprpriat t th mtri f th giv data it th tw disagrmts ad dfid i th bgiig f this dumt. Nt that auts fr diffrt mtris r lvls f masurmt by wighig th bsrvd ad xptd iids by th squard diffr btw th iidig valus. iffrs a b xprssd as mathmatial futis ad i th frm f a tabl. Th lattr maks thir rlativ magituds traspart. Itrval ad rati mtri diffrs ar futis f th valus big paird. Ordial diffrs dpd thir frquis f usig valus. Ad mial diffrs ar addd hr t graliz stp. Nmial mtri diffrs Tw valus ithr math, r thy d t: Nmial atgris, ams: a b d f a 0 b 0 0 iff = k mial k 0 iff k d 0 0 f 0 5
Ordial mtri diffrs Valus hav th maig f raks ad diffrs btw raks dpd hw may raks thy ar apart frm ah thr. Fr xampl, with frquis frm data i C (ad uusd rak addd t shw that it ds t mattr): rdial k g k g g Raks: st d 3 rd 4 th 5 th 6 th k st 0.5 30 3.5 34 st = 9 d 0.5 9.5 3 d = 3 3 rd 506 3 0 7.5 0.5 3rd = 0 4 th 900 36 56 0.5 4 4th = 5 5 th 99 46 00 6.3 0.5 5th = 0 6 th 56 59 3 6.3 0 6th = 3 st,3 rd=.5 4 th,6 th=4 Ordial mtri diffrs may b stadardizd: 0 rdial k by: rdial k g k g g max mi whr max is th largst ad mi th smallst rak amg all raks usd. Stadardizati ds t afft, hwvr. k Itrval mtri diffrs Valus diffr algbraially: itrval k Itrval valus: - 0 3 4-0 3 4 5 k 0 0 3 4 4 0 3 9 4 0 3 6 9 4 0 4 5 6 9 4 0 Itrval mtri diffrs may b stadardizd as wll: 0 itrval k by: itrvall k k max whr max is th largst ad mi is th smallst valu urrig i th data. This stadardizati ds t afft ithr. mi 6
Rati mtri diffrs Algbrai diffrs btw tw valus ar xprssd rlativ t a abslut zr pit. Thy ar prprtial t th magitud f thir valus: rati k Rati valus: 0 3 4 5 k k 3 4 5 0 0 () () () 3 () 4 () 5 0 () () 3 () 4 () 3 4 5 6. 0 () () 3 () 5 6 7 3.5.04 0 () () 7 8 4.36..0 0 () 5.44.8.06.0 0 9 Cirular mtri diffrs Shrtst diffrs btw ay tw valus a irular sal with arbitrary dpits but a fixd irumfr U = th umbr f qual itrvals a irl. Cirular valus: 0 3 4 5 With th si futi xprssd i dgrs: 0 0.5.75.75.5 k irular k si 80 U.5 0.5.75.75.75.5 0.5.75 With th si futi xprssd i radia: 3.75.5 0.5.75 irular k k si U 4.75.75.5 0.5 5.5.75.75.5 0 Biplar mtri diffrs Algbrai diffrs ar xprssd rlativ t th tw dpits, mi ad max, f th sal. Nar th tr, a biplar mtri bhavs lik a itrval mtri ad ar th pls it bhavs lik a rati mtri. plar k ( k ) ( k )( mi Biplar valus: - - 0 max k ) - 0 3 4 7 6 35 44 -.43 0 3 35 44 53 0.333.067 0 53 6.600.50.067 0 7.00.600.333.43 0 7
rdial Cmput -rliability (th mputatially mst ffiit frm): k mtri k k mtri k k mtri Nt that th sums i this frm umrat ly f th tw symmtrial ff-diagal triagls f a iid matrix. Its tris, k as wll as th prduts k ar wightd by a apprpriatly hs diffr futi mtri k. Cmputatis ar illustratd with th umrial data i C abv, itrprtd as rdial, itrval ad rati data rsptivly. Zr frquis appar as 0 i th list f multipliatis: k With data i C as rdial data: rdial k k rdial k k rdialk k I th Exampl: 4 4.5 30 0.5 9 0 7.5 0 0 40 3 3 3 3 3 3 0.85 93 90.5 9530 9334 30.5 359 333 057.5 03.5 534 itrval k With data i C as itrval data: itrval k itrval k itrvalk k I th xampl: 4 4 3 0 0 0 0 (40 ) 3 3 3 3 3 3 9 3 9 0 9 5 3 9 3 4 30 3 5 3 3 3 0 5 0 3 5 3 k 0.849 rati k With data i C as rati data: rati I th xampl: 40 9 3 3 9 0 4 k rati k k rati k k 4 3 4 0 0 0 0 3 3 3 4 3 5 3 5 3 6 3 7 3 4 3 9 5 9 3 30 35 33 0 5 5 6 5 6 7 7 0 3 8 53 9 0.797 8
E. A gral mputatial frm, bypassig iid matris: Start frm a rliability data matrix as i C abv: Uits u:... u..... N Obsrvrs:... u..... N i i i... iu..... in j j j... ju..... jn..... m m m... mu..... mn Wh data ar missig, this matrix will tai fwr tha mn valus. Fr th 4 bsrvrs-by- uits xampl f rliability data usd i C ad abv: Uits u: 3 4 5 6 7 8 9 0 Obsrvrs: A: 3 3 4... B: 3 3 4 5. 3 C:. 3 3 3 3 4 5. : 3 3 4 4 5. Nt that Out f th mn=4=48 pssibl valus i this matrix, 7 ar missig. Th valu 3, assigd by bsrvr B t th th uit at b paird with xistig valus i that uit, at tribut t bsrvd agrmts r disagrmts, drps ut wh strutig a iid matrix, ad has t b igrd. Thus, this matrix tais a ttal f..= 40 pairabl valus. Istad f ad, umrat th valus fud i uits ad rat a valus-by-uits matrix: Uits:... u...... Valus:... u............... u....... k k k... uk....... k..... Ttals:..... u......... Whr u = th umbr f valus assigd t uit u. u m bsrvrs. uk by aalgy u. = u = th umbr f valus assigd t uit u. = u u. u = th umbr f pairabl valus urrig i th rliability data (mittig all uits with l r valus: u.).. = u u. u. = th ttal umbr f all pairabl valus i th rliability data (mittig all uits with l r valus: u.);.. mn 9
it rval Fr th abv xampl Uits u : 3 4 5 6 7 8 9 0 Valus, k : 3 0 0 0 0 0 3 0 0 0 9 =. 0 3 0 0 4 0 4 0 0 0 3 =. 3 0 4 4 0 0 0 0 0 0 0 =. 3 4 0 0 0 0 0 4 0 0 0 0 0 5 =. 4 5 0 0 0 0 0 0 0 0 0 3 0 0 3 =. 5 Ttals u. : 3 4 4 4 4 4 4 4 4 4 40 =.. Nt that th margial sum. 3 f pairabl valus 3 mits th ly l valu i uit,.=. Cmput with th apprpriat mtri diffr futi as dfid i abv: mtri (.. ). u k u k.. u k mtri uk mtri If th abv xampl sists f mial data (prft agrmt s as 0 disagrmt): mi al 0 3 0 0 0 6 0 40 4 4 4 0. 743 93 90 95 9330 35 3305 03 53 k 3 0 0 0 If th abv xampl sists f itrval data (prft agrmt s as 0 disagrmt): 3 0 0 0 0 4 9 3 9 0 9 5 3 9 3 4 3... 4 3 0 3 5 40 0. 849 k 3 0 0 0 0 4 3 3 3... 5 3 Mst mputatis f a b prfrmd with SPSS r SAS mars writt by Adrw Hays. Availabl at http://www.afhays.m: G t SPSS ad SAS Mars th t KALPHA. Rfrs: Klaus Krippdrff (004). Ctt Aalysis, a Itrduti t Its Mthdlgy, d Editi. Thusad Oaks, CA: Sag Publiatis spially Chaptr, pags -56. Klaus Krippdrff (004). Rliability i Ctt Aalysis: Sm mm Misptis ad Rmmdatis. Huma Cmmuiati Rsarh 30,3: 4-433. http://rpsitry.up.du/as_paprs/4 Last assd 0..5 Adrw F. Hays, ad Klaus Krippdrff (007). Aswrig th all fr a stadard rliability masur fr dig data. Cmmuiati Mthds ad Masurs : 77-89. http://www.u.du/urss/007fall/jm/80/00/haysadkrippdrff.pdf Last assd 0..5 0