IRT Potpourri. Gerald van Belle University of Washington Seattle, WA

Transcription

1 IRT Potpourri Gerald van Belle University of Washington Seattle, WA

2 Outline. Geometry of information 2. Some simple results 3. IRT and link to sensitivity and specificity 4. Linear model vs IRT model cautions 5. Measuring change a simple model 6. Change in ability in ACT cohort 2

3 P( θ ) /[ + exp{ a( θ b)}] θ 3

4 P( θ ) /[+ exp{ a(θ b)}] s.d. s.d. P(-P) binomial s.d Argument: Reflect s.d. in y direction through curve in x direction ? in the θ direction θ 4

5 P( θ ) /[+ exp{ a(θ b)}] s.d s.d. in θ direction θ 5

6 P( θ ) /[+ exp{ a(θ b)}] s.d This is close to what we do; work from tangent at P(θ). This is known as the delta method, or propagation of error s.d. in θ direction θ 6

7 P( θ ) /[+ exp{ a(θ b)}] s.d..75 s. d.(y axis, i.e. P scale) Slope dp ( θ ) / d( θ ) s.d (x axis, i.e. θ scale) s.d. in θ direction θ 7

8 Coup de Grace s.d.(y axis) Slope dp( θ ) / d( θ ) s.d. ( θ scale) or s.d. ( θ scale) ( y axis) s.d. dp( θ ) / d( θ ) P( P). dp( θ ) / d( θ ) For the 2PL model this becomes, s.d.( θ scale). a P( P) Where "a" is the discrimination of the item. 8

9 Some Simple Results Result : I( θ ) For the s.d. ( θ ) logistic 2. model this becomes, I( θ ) a 2 P ( P ). () Does not depend on " b, " the difficulty of the item. (2) Since max P ( P ) / 4, and I( θ ) a 4 2, s.d.( θ ) 2 a. 9

10 Some Simple Results (cont d) Result 2 Total Result 3 var( θ ) test information I( θ ) I( θ ) k k I( θ ) [s.d. ( θ )] [s.d. ( θ )] I ( θ ) k k for k items. a 2 (equality at P 0.5) 2 [Harmonic mean of s.d.( θ ) ] k 0

11 3. IRT and link to sensitivity and specificity ydem ynorm SensitivityA Likelihood ratio A/B 0 -SpecificityB -3-2 Second - FHL Psychometric Workshop August theta 2005

12 4. Linear model and IRT model cautions Logistic Formulation: P( Y θ ) + exp[ { β 0 + βθ }] 2PL Formulation: P( Y θ ) + exp[ { a( θ b)}] + exp[ { ab + aθ}] β o -ab β a 2

13 4. Linear model and IRT model cautions Becomes even trickier with, say, Uniform DIF: Logistic regression with uniform DIF: P( Y θ ) + exp[ { β + G δ + β 0 θ }] 2PL with uniform DIF: P( Y θ ) + exp[ a( θ b + G δ )] + exp[ { ab + a G δ + aθ}] 3

14 5. Measuring change a simple model Importance of cognitive change:. Clinical interest 2. Research interest 3. Clinical trials of new agents 4. Normal aging Questions:. How to model change in IRT environment? 2. What items are important for detecting change? 4

15 5 The model ) ( ) ( ) ( ) ( ),,, and ( i i i i D D D D i F i B i e e e e X X P β θ α β θ α β θ α β θ α β α θ Assumes two measurements at baseline and final. 2PL formulation with D.7 included in model. Same shift of for every one; as in clinical trial model. Assume conditional independence given θ and.

16 The result e Dα P( P( X X B i B i 0 and and X X F i F i ) 0) Odds ratio for off - diagonal elements n n 0 0 Pˆ( X Pˆ( X B i B i 0 and and X X F i F i ) 0) estimate of e Dα by item among discordant subects. 6

17 Estimation of ˆ Dαˆ ln n n 0 0 can be estimated from each item. Note that precision depends only on the discrimination. Or does it? v var( ˆ ) +. ˆ Dα n 0 n 0 It turns out that the off - diagonal frequencies are determined by the difficulty of 2 the item. 7

18 Estimation of continued. Estimates can be combined; weighted average 2. We confirmed formulae by simulations and numerical Integration (Run by Doug Tommet) 3. Surprisingly little effect of discrimination and difficulty 4. A picture shows why this is the case 5. The picture also shows the three most important aspects for assessing change 8

19 9

20 Change in ability in ACT cohort ACT Adult Changes in Thought Inception cohort of normal elders started 994 N2579 at start Followed every two years Demented subects followed every year CASI primary instrument for assessing cognition 20

21 Analytic strategy. Assess cognitive status using PARSCALE 2. Arrange all subects into one matrix for all times to estimate θ 3. After obtaining θ s we used hierarchical linear model in STATA to analyze change over time 4. In this presentation we look at change over time for particular subgroups 5. Primary purpose is to show Doug Tommet s computing prowess 2

22 22

23 23

24 24

25 25

26 26

27 27

28 28

29 29

30 30