Homework 9 STAT 530/J530 November 22 nd, 2005

Transcription

1 Homework 9 STAT 530/J530 November 22 nd, 2005 Instructor: Bran Habng 1) Dstrbuton Q-Q plot Boxplot Heavy Taled Lght Taled Normal Skewed Rght Department of Statstcs LeConte 203 ch-square dstrbuton, Telephone: normal dstrbuton, E-mal: habng@stat.sc.edu t-dstrbuton, & unform dstrbuton 1 2 Homework 9 Homework 9 Q-Q plots Box Plots 3 4

2 Homework 9 2) Assume we are tryng to verfy that a data set s multvarate normal. Brefly explan why can we stop checkng f we fnd just one q-q plot, bvarate box-plot, or chsquare plot s extremely far from where t should be? For MANOVA why should we contnue the checkng f we fnd one that s only slghtly off? Homework 9 3) For the county data on the exam the total county ncome was dvded by the populaton. Smlarly on the homework the crab measurements were dvded by the total crab sze. Brefly explan why scalng n ths manner s desrable. 5 6 Homework 9 4) Whch of the followng of the followng are NOT reasons to standardze the data: standardzaton reduces dfferences between the ndvdual observatons t allevates the problem of dfferent varables usng possbly dfferent unts of measurement t adjusts for varables havng dfferent varances t causes dfferent groups of observatons to separate more on each varable t makes t so that one varable doesn t domnate all of the others smply because of a larger range of values. 7 Homework 9 5) Usng Factor Scores Usng a Representatve Queston Usng a Sum Score a. There s only one score for each observaton b. Each observaton receves fve scores c. The results for each ndvdual observaton wll heavly depend on the other observatons used to ft the model d. It can only be performed f the dfferent questons have smlar sorts of scales e. It keeps less nformaton than the other two STAT methods 530/J530 B.Habng Unv. of S.C. 8

3 Next Thursday 24 th : Thanksgvng No Class Tuesday 29 th : Structural Equaton Modelng Thursday 1 st : Homework 10 s due, ce cream feld trp as penance for Homework 6 grade beng late! Wth tme for questons whle we eat. 5:30pm Tuesday, December 6 th Fnal Exam s Due Item Response Theory IRT s a class of methods for modelng the relatonshp between the responses to the tems or questons on a test, scale, or questonnare and the underlyng latent trat(s) that the test s desgned to measure The Data Items S u b j e to tem. c t s U j s the response of examnee j Sample Data Sets The GRE A test to measure clncal depresson A survey to measure aborton atttudes A survey to measure llegal behavors and socal support An nstrument to measure pan 11 12

4 The Goals Estmatng the propertes of the tems Choosng the most effectve tems Estmatng subjects ablty levels Determnng the dmensonal structure of the latent construct. Detectng bas n the questons Equatng/Lnkng dfferent forms Three Alternatves Classcal Test Theory Classcal Item Analyss Factor Analyss Classcal Test Theory The bass of classcal test theory s the formula X = T + E observed = true + error score score Where E(X)=T and Cor(E,T)=0 Usng CTT Relablty of the test s 2 XT ρ TT ' ρ = The standard error of measurement s ˆ σ E = ˆ σ X 1 ρ 2 ˆ XT Can get an estmate of T usng regresson 2 T = ρ ( X μ ) + μ XT X T 15 16

5 Weaknesses n CTT Entrely test and populaton dependent Does not descrbe ndvdual tems Classcal Item Analyss Item Dffculty measured by Item p-value (percentage correct) Item Dscrmnaton measured by the Bseral or Pont-Bseral Correlaton between the tem score and total observed score Has a constant standard error of measurement across all true score levels Both depend on the partcular examnees, and dscrmnaton depends on the remanng test tems Factor Analyss X1 = a11f1+ La1 mfm + e1 X = a F + La mfm + e M X = a F + La F + e p p1 1 pm m p The X are the observed varables, the F j are the m common factors, the e are the specfc errors, and the a j are the factor pxm factor loadngs. Weaknesses n Factor Analyss Desgned for contnuous observed responses Hypothess testng assumes multvarate normalty of the underlyng latent trats 19 20

6 Item Response Theory U=(U 1, U, U n ) s the vector of tem responses u s a partcular possble response to tem Θ are the latent trats measured by the test θ s a partcular level of the latent trats The goal s to model: P[U = u Θ = θ] Monotone Homogenety Model Three commonly made assumptons are: Local Independence = Item responses are condtonally ndependent gven θ Undmensonalty = θ s scalar Monotoncty = P[U k Θ=θ] s ncreasng n θ Item Response Functons Brnbaum s 3PL Model The P[U k Θ=θ] are the tem response functons (IRFs) for the test. They are sometmes called tem characterstc curves (ICCs). For dchotomous tems these are P (θ)= P[U =1 Θ=θ] One of the common models for dchotomous tem tests s Brnbaum s (1968) three parameter logstc model. P ( θ ) = c 1 c + 1+ exp( 1.7a ( θ b )) 23 24

7 3PL Model a =dscrmnaton, b =dffculty, c =guessng Invarance Because the models are based on an underlyng latent trat they have an nvarance property. When the IRT model fts the data the same IRFs are obtaned regardless of the ablty dstrbuton of the sample of examnees or whch other tems are used on the test PL or Rasch Model Benefts of Rasch Rasch s (1960) model s the 3PL model wth the guessng set to 0 and the dscrmnaton constant across all tems. The tem response functons do not cross, so that tems can be ranked on dffculty regardless of examnee ablty level. P ( θ ) = 1 1+ exp( a( θ b )) The margnal sums are suffcent statstcs for the examnee ablty and tem dffculty respectvely

8 Objectve Measurement Problem wth Rasch It often just doesn t ft the data! vs Other Models Polytomous/Lkert Scale tems (cumulatve probablty, adjacent category, or contnuaton rato) Wdely used and studed. Multdmensonal Abltes Dffcult to ft and not wdely used. Can be re-parameterzed as non-lnear factor analyss. Other Models Non-Monotone Response Functons (unfoldng models) Very recently developed Locally Dependent Items (testlet models or conjunctve IRT) There are several strong proponents of testlet models. Conjunctve IRT model has receved lttle attenton

9 Estmaton The tem parameters and q s are commonly estmated by margnal maxmum lkelhood usng the EM algorthm. (Frst tems, then examnees.) MCMC s used for some more complcated models. For most procedures, 20 dchotomous tems and 400 examnees s consdered a small sample sze. Goodness of Ft There are no wdely accepted test statstcs for testng goodness of model ft (although most IRT packages wll produce some) Resdual plots are often used to determne whether the IRF has the correct parametrc form Standardzed Resdual Plots Checkng LI and d=1 vs. The twn assumptons of local ndependence and undmensonalty are often tested by fttng a undmensonal model and then testng for lack of local ndependence (for example observng correlatons between the resduals)

10 Condtonal Covarances Zhang and Stout (1999) demonstrated the relatonshp between the covarance of an tem par condtoned on the best undmensonal sub-trat, Cov(U,U j Θ TT =θ) s drectly related to the underlyng dmensonal structure. Geometrc Representaton Items on opposte sdes of Θ TT have negatve CCOVs those on the same have postve CCOV Based Procedures The CCOVs can be estmated usng d=1 parametrc models, or by usng the observed test score as a proxy for Θ TT. Selectng Items If a parametrc model s ft, then each tem produces an tem nformaton curve. They can then be used to construct hypothess tests, or be converted nto a dstance for clusterng or scalng

11 Impact An tem demonstrates mpact f t has dfferent statstcal propertes for members n dfferent groups. Test Equty 41 Dfferental Item Functonng DIF s when the tem has dfferent propertes for members of dfferent groups after controllng for the ablty t s supposed to measure. Test Equty 42 Test Equty An tem s based f t has dfferent statstcal propertes for members of dfferent groups that are due to factors n the test beyond what the construct s desgned to measure. 43 Equatng / Lnkng If tests share at least a few tems n common then the estmated scores can be placed on a common metrc by fttng the two exams smultaneously. If the tests don t share tems n common t s necessary to make assumptons about the underlyng populatons. 44