Application of Item Response Theory Models for Intensive Longitudinal Data

Transcription

1 Application of Item Response Theory Models for Intensive Longitudinal Data Don Hedeker, Robin Mermelstein, & Brian Flay University of Illinois at Chicago Models for Intensive Longitudinal Data, 2006, Walls & Schafer (Eds), Oxford. Supported by National Cancer Institute grant 5PO1 CA98262 (Mermelstein, PI) 1

2 Rasch model Probability of a correct response to item j (Y j = 1) conditional on ability of subject i (θ i ): 1 P (Y ij = 1 θ i ) = 1 + exp[ a(θ i b j )] θ i = level of latent trait for subject i, usually normally distributed b j = item difficulty; determines position of the logistic curve along the ability scale (further curve is to the right, more difficult the item) a = slope or discriminating parameter; represents degree to which the item response varies with ability θ 2

3 Rasch model with three items (a = 1) b j = -1, 0, and 1 for a relatively easy, moderate, and difficult item, 3

4 Two-parameter logistic model 1 P (Y ij = 1 θ i ) = 1 + exp[ a j (θ i b j )] a j = slope (discrimination) parameter and b j is the difficulty parameter for item j values of a j =.75, 1, and

5 As noted by Bock & Aitkin (1981), convenient to represent the twoparameter model as P (Y ij = 1 θ i ) = exp[ (c j + a j θ i )] where c j = a j b j = item-intercept parameter, or log P (Y ij = 1 θ i ) = c 1 P (Y ij = 1 θ i ) j + a j θ i mixed-effects logistic regression c j = fixed effects; a j = random-effect variance parameters 5

6 IRT in mixed model form Let λ i = n i 1 vector of logits for subject i. The Rasch model can then be written as λ i = X i β + 1 i σ υ θ i X i = n i n item indicator matrix obtained from I n β = n 1 vector of item difficulty parameters (i.e., b j parameters in IRT notation) 1 i = n i 1 vector of ones θ i = latent trait (i.e., random effect) of subject i, N (0, 1) σ υ = common slope parameter (i.e., a parameter in the IRT notation); indicates heterogeneity of random subject effects 6

7 Two-parameter model can be represented in matrix form as λ i = X iβ + X it θ i where T is a vector of standard deviations, T = [ ] σ υ1 σ υ2... σ υn correspond to the IRT discrimination parameters 7

8 Parameterization In two-parameter IRT model: n b j = 0 and j=1 n a j = 1 j=1 item difficulty centered around zero item discrimination (multiplicatively) centered around one mixed model estimates need to be rescaled and/or standardized to be consistent with IRT results 8

9 Rasch model estimates for the LSAT-6 data MIXED item IRT (ˆb j ) raw ( ˆβ j ) transformed (ˆb j ) â = ˆσ = â = IRT estimates from Thissen, 1982 b j = β j 1 n n j =1 β j ensures that the mean of these transformed estimates equals zero 9

10 Two-parameter probit model estimates for the LSAT-6 data IRT MIXED raw MIXED transformed difficult discrim difficult discrim difficult discrim item ˆbj â j ˆβj ˆσ j ˆbj â j IRT estimates from Bock & Aitkin, Transform σ j estimates so product equals 1 a j = exp log σ j 1 n n j =1 log σ j 2. Reverse sign of β j estimates, and transform so that Σ( β j /a j ) = 0, using the standardized a j estimates from previous step b j = (β j /a j ) 1 n n j =1 ( β j /a j ) 10

11 Ecological Momentary Assessment (EMA) data aka experience sampling and diary methods Subjects provide frequent reports on events and experiences of their daily lives (e.g., responses per subject collected over the course of a week or so) electronic diaries: palm pilots or personal digital assistants (PDAs) Capture particulars of experience in a way not possible with more traditional designs e.g., allow investigation of phenomena as they happen over time Reports could be time-based, following a fixed-schedule, randomly triggered, event-triggered 11

12 Illustration: Adolescent Smoking Study (Mermelstein) 8th or 10th graders - EMA data from baseline Either had never smoked, but indicated a probability of future smoking, or had smoked in the past 90 days, but had not smoked more than 100 cigarettes in lifetime Adolescents carried hand held computers at all times during one-week period; trained both to respond to random prompts and to event record (initiate data collection) smoking episodes Each entry was date and time-stamped (by computer) For inclusion in analyses, adolescent must have smoked at least one cigarette during seven-day baseline data collection period; 152 (of 562) adolescents met this inclusion criterion 12

13 Proportion (n) smoking reports by day of week and time of day Day of Week 3am to 9am to 2pm to 6pm to 10pm to 8:59am 1:59pm 5:59pm 9:59pm 2:59am Monday 1.3 ( 2) 4.6 ( 7) 4.0 ( 6) 9.2 ( 14) 2.6 ( 4) Tuesday 1.3 ( 2) 6.6 ( 10) 15.8 ( 24) 14.5 ( 22) 3.3 ( 5) Wednesday 5.9 ( 9) 9.2 ( 14) 21.7 ( 33) 11.8 ( 18) 3.3 ( 5) Thursday 5.3 ( 8) 9.2 ( 14) 19.7 ( 30) 17.1 ( 26) 1.3 ( 2) Friday 9.2 ( 14) 9.9 ( 15) 21.7 ( 33) 19.1 ( 29) 6.6 ( 10) Saturday 0.0 ( 0) 10.5 ( 16) 16.5 ( 25) 11.8 ( 18) 13.2 ( 20) Sunday 0.7 ( 1) 4.0 ( 6) 4.0 ( 6) 9.9 ( 15) 2.6 ( 4) items are these 35 time periods dichotomous response is whether or not the subjects recorded smoking in these periods due to data sparseness, only considered main effects of Day and Time (not interaction) 13

14 subject s latent ability can be construed as underlying degree of smoking behavior interest is to see how this behavior relates to these day-of-week and time-of-day periods item s difficulty indicates the relative frequency of smoking reports during the time period item s discrimination refers to the degree to which the time period distinguishes levels of the latent smoking behavior variable Our hypothesis is that time- and day-related characteristics of smoking among these beginning smokers may serve as early behavioral markers of dependence development 14

15 IRT model fit statistics Common Time varying Day varying Day and Time slopes slopes slopes varying slopes 2 log L AIC BIC parameters q Likelihood ratio tests comparisons to common slopes model χ df p < comparisons to time varying slopes model χ df 2 6 p < comparison to day varying slopes model χ df 4 p <

16 Difficulty estimates (Day and Time varying slopes model) 16

17 Discrimination estimates (Day and Time varying slopes model) 17

18 Empirical Bayes ability estimates from two-parameter model 18

19 Number of smoking reports versus empirical Bayes estimates (r =.96) 19

20 Minimum and maximum EAP estimate stratified by number of smoking reports: Day of week and time of day of smoking reports (1=report, 0=no report) Number of EAP 3am to 9am to 2pm to 6pm to 10pm to reports estimate Day 8:59am 1:59pm 5:59pm 9:59pm 2:59am Sat Mon Wed Fri Sat Tue Wed Fri Fri Sat Tue Wed Thu

21 Minimum and maximum EAP estimate stratified by number of smoking reports: Day of week and time of day of smoking reports (1=report, 0=no report) Number of EAP 3am to 9am to 2pm to 6pm to 10pm to reports estimate Day 8:59am 1:59pm 5:59pm 9:59pm 2:59am Wed Fri Sat Thu Fri Sat

22 Summary Mixed model software can be used to estimate basic IRT models some translation of the parameter estimates is necessary to properly express the mixed model results in IRT form IRT modeling of EMA data from adolescent smoking study whether or not a smoking report had been made in each of 35 time periods (crossing of seven days and five time intervals within each day) weekend and evening hours yielded the most frequent smoking reports, however morning and, to some extent, mid-week reports were most discriminating in separating smoking levels Here, focus on dichotomous data, but IRT models for ordinal and nominal outcomes can also be estimated 22

23 Dataset construction For example, the data are as follows for an individual with id 1001 who did not get any of the five items correct (id, lsat6, item1, item2, item3, item4, item5): Because the mixed model does not need to assume an equal number of observations per individual, individuals missing a particular item would have less than five lines of data (or have a missing value code for the missed item response) 23

24 / Rasch logistic model in mixed regression form / PROC NLMIXED; PARMS c1=0 c2=0 c3=0 c4=0 c5=0 a=1; z = c1 item1 + c2 item2 + c3 item3 + c4 item4 + c5 item5 + a theta; IF (lsat6=0) THEN p = 1 - (1 / (1 + EXP(-z))); ELSE p = 1 / (1 + EXP(-z)); IF (p > 1e-8) THEN ll = LOG(p); ELSE ll = -1e20; MODEL lsat6 GENERAL(ll); RANDOM theta NORMAL(0,1) SUBJECT=id OUT=ebest1; RUN; 24

25 / 2 parameter probit model in mixed regression form / PROC NLMIXED ; PARMS a1=1 a2=1 a3=1 a4=1 a5=1 c1=0 c2=0 c3=0 c4=0 c5=0; BOUNDS a1>0, a2>0, a3>0, a4>0, a5>0; z = (c1 item1 + c2 item2 + c3 item3 + c4 item4 + c5 item5) + (a1 item1 + a2 item2 + a3 item3 + a4 item4 + a5 item5)*theta; IF (lsat6=0) THEN p = PROBNORM(-z); ELSE p = PROBNORM(z) ; IF (p > 1e-8) THEN ll = LOG(p); ELSE ll = -1e20; MODEL lsat6 GENERAL(ll); RANDOM theta NORMAL(0,1) SUBJECT=id OUT=ebest2; RUN; 25