MANY FACET, COMMON LATENT TRAIT POLYTOMOUS IRT MODEL AND EM ALGORITHM. Dimitar Atanasov

Transcription

1

2 Pliska Sud. Mah. Bulgar. 20 (2011), 5 12 STUDIA MATHEMATICA BULGARICA MANY FACET, COMMON LATENT TRAIT POLYTOMOUS IRT MODEL AND EM ALGORITHM Dimiar Aanasov There are many areas of assessmen where he level of performance can no be deermined using ordinal, objecive es. For example, here are aspecs of wriing language skills which should be subjecively qualified by judges based on some crieria. Usually such daa is sudied using he Many Face Rasch Model. To address he problem wih Iem Response Theory (IRT) approach a Many Face IRT model for polyomous iem response in he case of common laen rai is presened. For esimaion of he parameers of he model he EM-algorihm is derived. 1. Inroducion The mehods of analysis of a classical es iems deermine he resul of he exam as a ineracion beween he abiliies of he individuals and he properies of he es iems. A common problem arise when he he answer of he individual is judged by a referee or raer. Such is he case of essay evaluaion, where number or raers rae he essay according o some crieria on a fixed scale. This adds o he process of evaluaion addiional faces of raer severiy and he difficuly of he paricular crieria. In a similar way, if an individual can choose differen hemes for he essay, here is a face, represening he difficuly of he chosen heme. In order o have a fair assessmen of he suden abiliy all hese faces should be included in he process of evaluaion Mahemaics Subjec Classificaion: 91E45. Key words: iem response heory, many faces, EM algorihm.

3 6 Dimiar Aanasov From differen poin of view i is needed o have coordinaion and agreemen beween he raers, so heir severiy should be compared as well he way of applicaion of differen crieria. One of he classical approaches o he problem is he so called Many Face Rasch Model (Linacre, 1994). This model expands he Rasch-based models for achievemen evaluaion (Andrich, 1985) and includes he influence of he differen faces o he final resul. In his work we presen an approach o his problem based on he Iem Response Theory (IRT) properies (parameers) of he faces. An EM algorihm for esimaion of hese parameers is derived. We assume ha he abiliies of he sudens can be esimaed using he unidirecional scale. Therefore one can suppose ha here is a common laen rai, which is responsible for he performance of he suden on he raing crieria. 2. The Model Le us consider he classical essay scoring, where a number of raers evaluaes he essays according o he se of crieria. For any of he crieria, he raer gives a score using a linear non-decreasing scale. The esimaion of he parameers of he model is based on he following represenaion, common in he Many Face approach (Linacre, 1994) ( ) Pnijk log B n D i C j F k, P nij(k 1) where P nijk is he probabiliy an individual n o be raed on crieria i from raer j wih rae k. The parameer B n is he suden abiliy, D i is he crieria difficuly, C j is he raer severiy and F k is he level, needed o achieve grade k from grade k 1. The model can be expanded wih addiional faces as well as wih an assumpion ha he sep F k can differ for differen raers. According o he IRT model (Crocker & Algila, 1986), he probabiliy for correc performance on he given es iem, defined by 2-parameric logisic model is P(θ) e ad(θ b), where θ is he level of he measured abiliies of he individual, he parameers of he model b and a represen he difficuly and he level of discriminaion of

4 Many Face, Common Laen Trai Polyomous IRT Model... 7 he iem respecively. The value D 1.7 is a scaling parameer, giving an approximaion o he Normal ogive curve. To represen he model in he erms of he IRT concep le us consider he following model. Le θ be he laen abiliy of he individual. Le he essay of he suden be raed on he se of J crieria using a scale from 0 o K 1. A specific abiliy Θ j,j 1,...,J is relaed o any of hese crieria. For he relaion beween specific abiliies and he generic laen abiliy of he individual he following facor model holds Θ j α j θ + ε j, where α j is a facor loading and ε j is a N(0,σ 2 ) random error. Le us noe wih r (r 1,...,r J ) he raes of he essay over all crieria. Then for j 1,... J we have P(r j k θ) P(Θ j δ k θ) P(α j θ + ε j δ k ) P(ε j δ k α j θ) 1 Φ ε (δ k α j θ) Φ( α jθ δ k ), σ 2 where δ k is he is he level of he specific abiliy θ j for an examinee o achieve grade of k or higher, Φ ε is he disribuion funcion of ε and Φ is he disribuion funcion of he sandard normal disribuion. Le he level δ k be a resul of influence of h differen faces of he assessmen procedure. In he case of a equiable assessmen hese faces should no depend on he assessmen scale. Then δ k can be represened by he facor model δ k γ i i + k + ǫ, where i are he assessmen faces and γ i are heir facor loadings, k is a specific value, needed for rae k, ǫ is a N(0,σδ 2 ) random error, independen from ε j,j 1,...,J. Then P(α j θ + ε j δ k ) P(ε j ǫ γ i i + k α j θ) Φ(α j θ γ i i k ),

5 8 Dimiar Aanasov for properly normalized α j, γ i and k. For he probabiliy of o receive grade caegory k from a person wih laen rai θ we have P(r j k θ) P(δ k θ < δ k+1 θ) Φ(α j θ γ i i k ) Φ(α j θ γ i i k+1 ). If we use he approximaion of he normal ogive curve wih a logisic funcion we have 1 (1) P(r j k θ) È 1 + e È 1 D(α h jθ γ i i k ) 1 + e D(α h jθ γ i i k+1 ) Remark In he general facor model, used for he relaion beween he specific abiliies and he laen rai of he individual he se of crieria can be replaced by differen face of he assessmen if i is inheri in he individual. A model, where all he facor loading α j,j 1,...,J are equal, can be used as well. In ha case he assessmen crieria should be considered as one of he faces in he facor model for δ k. In similar way an ineracions beween he laen rai and oher faces can be included in he model. Hence all he configuraion of faces given in Linacre (1994) can be presened. 3. The EM Algorihm For he esimaion of he parameers of he model an EM algorihm is derived. A similar algorihm in he case of IRT model wihou influence of differen faces is given in Woodruff & Hanson (1997). A complee descripion of he concep of he EM algorihm is given in McLachlan & Krishnan (2008). The idea is o maximize he complee likelihood funcion on he se of unknown parameers of he model as well as on he value of he unknown laen rai θ. Le θ ake discree values θ [1],...,θ [P] wih corresponding probabiliies π 1,...,π P. Le R {r ij } be a marix of he sudens raes, where r ij is he grade of he individual i,i 1,...,N on he crieria j, j 1,...,J. Le H H ij, i, i 1,...,N; j 1,...,h be he marix, which defines he levels of he assessmen faces, where H ij is he level of he face j, which influences he grade of he individual i. Wih r i and H i we noe he i-h row of he marices R and H respecively.

6 Many Face, Common Laen Trai Polyomous IRT Model... 9 is Then he probabiliy of a given grade r i over he all populaion of individual f(r i,π,h i ) f(r i θ [],,π,h i )π, 1 where is he se of unknown parameers in he model (1) and π is he vecor π (π 1,...,π P ). If he raes of he differen crieria are independen we have (2) f(r i θ [],,H i ) j1 k1 P(r ij k θ [],,H i ) I(r ijk), wherep(r ij k θ [],,H i ) is he probabiliy, defined by (1) and I( ) is he indicaor funcion. Therefore he likelihood funcion can be represened as L(R θ,,π,h) N 1 π j1 k1 P(r ij k θ [],,H i ) I(r ijk). The probabiliy ha a randomly chosen individual wih a given level of abiliies θ [] has a se of raes r i is Then he complee likelihood is L(R,θ,π,H) f(r i,θ [],π,h) f(r i θ [],,H i ).π. N P 1 j1 k1 P(r ij k θ [],,H i ) I(r ijk) π P 1 j1 k1 N P(r ij k θ [],,H i )π (3) P 1 j1 k1 H ℵ P(r.j k θ [],,H i ) ηh jkπ ν, where ℵ denoes he se of differen values of he observed faces H i, P(r.j k θ [],,H i ) is he probabiliy of he rae k on he crierion j given he value of

7 10 Dimiar Aanasov he oher parameers kep fixed. Noe wih ν he number of individual wih he value of he laen rai θ [] and ηjk H is he number of individual wih laen rai θ [], raed on crierion j wih grade caegory k, under he values of he faces in H. The (ηjk H,ν ), 1,...,P, j 1,...,J, k 0,...,K 1, H ℵ is a sufficien saisics. The aim of he algorihm is o find he expeced value of he ηjk H and ν given se of values for and π (E-sep). The resul is subsiued in (3) and afer ha (M-sep) he complee likelihood funcion (3) is maximized on and π. The obained ML esimaors are subsiued in he E-sep for calculaing new values for he expecaion of ηjk H and ν and so on. This procedure coninues unil some opimal crieria is reached, for example if he change of he likelihood funcion is relaively small. Le us firs find he disribuion of he laen rai θ given he se of raes r i and he parameers of he model and π. The values (s) and π (s) are found on he previous sep (s 1) of he algorihm. f(θ [] r i, (s),π (s),h i ) f(θ [],r i (s),π (s),h i ) f(r i (s),π (s),h i ) f(θ [],r i (s),π (s),h i ) f(r i θ [p] (s),π (s),h i )P(θ [p] (s),π (s),h i ) p1 f(ri θ [], (s),π (s),h i )P(θ [] (s),π (s),h i ) f(r i θ [p] (s),π (s),h i )π p (s) p1 f(ri θ [], (s),π (s),h i )π (s) f(r i θ [p] (s),π (s),h i )π p (s) p1

8 Many Face, Common Laen Trai Polyomous IRT Model (4) j1 k1 p1 j1 k1 P(r ij k θ [],,H i ) I(r ijk) π (s) P(r ij k θ [p], (s),h i ) I(r ijk) π (s) p, where in he las equaion (2) is subsiued and P(θ [] (s),π (s),h i ) π (s) E-sep. Then ˆν E(ν R, (s),π (s),h) N f(θ [] r i, (s),π (s),h i ) N j1 k1 p1 j1 k1 P(r ij k θ [], (s),h i ) I(r ijk) π (s) P(r ij k θ [p], (s),h i ) I(r ijk) π (s) p ˆη H jk E(ηH jk R, (s),π (s),h) N P(r ij k θ [], (s) ), where P(r ij k θ [], (s) is calculaed from (1). M-sep. The values ˆν and ˆη jk H are subsiued in he complee likelihood funcion (3), afer ha he new values (s+1) and π (s+1) are obained by is maximizaion. Taking he logarihm of (3) we have log L(R,θ,π,H) (5) J 1 j1 k1 H ℵ ˆη jk H P(r.j k θ [],,H i ) + ˆν log(π ) 1 The maximizaion of (5) is equivalen o he maximizaion of boh erms separaely. The second erm is he logarihm of he he likelihood funcion of he mulinomial disribuion. Therefore is maximum is in he poin π (s+1) ˆν N.

9 12 Dimiar Aanasov This is he value, used in he nex E-sep. The value of is calculaed by maximizing he firs erm in he equaion (5). REFERENCES [1] D. Andrich.. An Elaboraion of Guman Scaling wih Rasch Models for Measuremen, Sociological Mehodology. Vol , [2] L. Crocker, J. Algila. Inroducion o Classical and Modern Tes Theory. Warsworh, [3] M. Linacre. Many Faceed Rasch Measuremen, Univ of Chicago Social Research, [4] G. McLachlan, T. Krishnan. The EM Algorihm and Exenions. Second Ediion. John Weley, [5] D. Woodruff, B. Hanson. Esimaion fo Iem Response Models using EM Algorihm for Finie Mixures. Presened a he Annual Meeing of he Psychomeric Sociey, Galinburg, Tennessee, Dimiar Aanasov New Bulgarian Universiy 21, Monevideo Sr. Sofia, Bulgaria daanasov@nbu.bg