EDMS Modern Measurement Theories. Multidimensional IRT Models. (Session 6)

Transcription

1 EDMS 74 - Modern Measurement Theores Multdmensonal IRT Models (Sesson 6) Sprng Semester 8 Department of Measurement, Statstcs, and Evaluaton (EDMS) Unversty of Maryland Dr. André A. Rupp, (3) , ruppandr@umd.edu

2 Types of Multdmensonal Models

3 Types of Multdmensonalty

4 Compensatory vs. Non-compensatory Models Compensatory Model P ( ) γ K ( k ( γ ) ( K α k k α k k δ ) k δ ) Non-compensatory Model P ( ) γ ( γ ) K k ( α k ( α ( k ( β k k k β )) k ))

5 Multdmensonal Item Indces

6 Graphcal Illustraton a.8, a.3, d.3, D -.73 see Ackerman (994)

7 Multdmensonal Indces Dscrmnaton Index (Eucldan length of dscrmnaton vector under orthogonalty) MDISC K k α k Dffculty Index (sgned dstance of equprobablty lne from orgn) MDIFF δ MDISC

8 Contour Plots

9 Vector Plots

10 Vector Plots a.4, a.4, MDISC.46 d -.5, D. a., a.8, MDISC.8 d, D -.43

11 Multdmensonal Indces (contnued) Angle n the Drecton of Maxmum Slope ϕ k arccos α k MDISC Multdmensonal Informaton Functon I, ϕ ( ) P [ P ( ) ] ϕ ( )( P ( ))

12 Crteron-referenced Interpretatons

13 The Multdmensonal Random Coeffcents Multnomal Logt Model (MRCMLM)

14 The MRCMLM Model Formulaton P r ( ) M r M ( T b a ) ( T b a ) rk k k k b rkk K r K r b r k k r r K K a rk a rk r r where b c s a vector of the assgned scores n category c of tem across all dmensons, s a vector of tem parameters, and a c s a vector wth weghts for these parameters. Adams, Wlson, & Wu (997)

15 Example ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ) ( ) ( ) ( ) ( ) ( ) ( ) ( X P X P X P X P X P X P X P,,, 3 3 A B

16 Example

17 Example (contnued) ,,, A B

18 Example (contnued) ,,, A B

19 Compensaton Revsted P ( ) K k ( α ) k ( k β k ) ( α ( β )) k k k K k K k k K K α α k ( ) ( α ( β )) k β k k k k ( α ( β )) k k k K k α k k k β k k

20 Non-parametrc Dmensonalty Assessment

21 DIMTEST The purpose of DIMTEST s to assess whether a test s essentally undmensonal.. Select a group of M tems that s known or beleved to be essentally undmensonal and dmensonally dstnct from the remanng tems on the test. Ths s called the Assessment Subtest (AT) and should consst of at least tems, but should not exceed about a thrd or a fourth of the tems on the test.. Select a second group of M tems from the remanng ones whose dffculty dstrbuton s smlar to those of AT. Ths set of tems s called Assessment Subtest (AT) and needs to be of equal sze to AT. 3. The remanng (J M) tems are the Parttonng Subtest (PT), whch s used to group examnees nto B bns va ther total PT score. Compute the DIMTEST statstc for these groups by usng AT and AT. The dea s to nvestgate whether AT s dmensonally dstnct from PT by focusng on a dfference n varance components; AT s used to elmnate a statstcal basng effect. see Jang & Roussos (7)

22 DIMTEST (contnued)

23 DIMTEST (contnued) DIMTEST Statstc T B T AT b b B T AT T ~ N(,) H H : d : d AT PT AT PT >

24 DETECT Whle DIMTEST s a statstcal hypothess test DETECT s a statstcal algorthm that produces an ndex that quantfes the spread of the tem vectors around a sngle domnant dmenson (.e., quantfes the amount of multdmensonalty) n the case of approxmate smple structure. The dea behnd DETECT s to perform a non-parametrc agglomeratve herarchcal cluster analyss usng a proxmty measure CCPROX that s based on the condtonal covarance between tems. To merge tems nto clusters, CCPROX s used. Then, the smlarty of clusters s often measured by the unweghted pargroup method of averages (.e., the average of all proxmtes between all tems from two dfferent clusters). The DETECT ndex can be computed for each possble parttonng of tems nto subsets and s maxmzed for the correct number of domnant dmensons.

25 DETECT (contnued) DETECT Index DETECT all all pars pars δ [ ] ( X, X ) E COV ' COV ˆ X, X ' ' D COV where COV ˆ X, X ' n k n k COV ˆ ( X J k n, k, k, X ', k ) constant δ ' same cluster dfferent clusters whch s the CCPROX measure.

26 DETECT (contnued)

27 Example Test Desgn HCA/CCPROX Results

28 Example (contnued)

29 Example (contnued)