DIBELS as a Predictor of Proficiency on High Stakes Outcome Assessments for At- Risk Readers

Size: px

Start display at page:

Download "DIBELS as a Predictor of Proficiency on High Stakes Outcome Assessments for At- Risk Readers"

Kathryn Heath
5 years ago
Views:

1 DIBELS as a Predictor of Proficiency on High Stakes Outcome Assessments for At- Risk Readers John L. Hosp Janice A. Dole Michelle K. Hosp University of Utah

2 Research Question How well do 2 nd grade benchmark scores on Dynamic Indicators of Basic Early Literacy Skills (DIBELS) predict 2 nd and 3 rd grade scores on the State Criterion Referenced Tests (CRTs)?

3 Background Theoretical and Empirical Research Base L. Fuchs, Fuchs, M. Hosp, & Jenkins (2001) Good, Simmons, & Kame enui (2001) Stage and Jacobsen (2001) Buck and Torgesen (2003) Florida (Reading First) Carlisle, Schilling, Scott, & Zeng (2004) Michigan (Reading First) McGlinchey & Hickson (2004) Michigan (at-risk students)

4 Background This study s theoretical and empirical interest: Predictive validity of DIBELS (NWF and ORF) on state CRTs. Comparison and contrast to other reports of validity. This study s practical interest: DIBELS as a useful tool for identifying students most at-risk for not achieving proficiency on high-stakes outcome measures.

5 INSTRUMENTS DIBELS Method Nonsense Word Fluency (NWF) administered Fall of 2 nd grade Oral Reading Fluency (ORF) administered Fall, Winter, Spring of 2 nd grade State Criterion Referenced Test (CRT) Total Language Arts Score administered Spring of 2 nd and 3 rd grade Iowa Test of Basic Skills (ITBS) Total Reading Score administered Fall of 3 rd grade

6 Participants 667 students 46.5% male, 53.5% female 58.4% white, 32.8% Hispanic, 3.0% Pacific Islander, 2.0% American Indian, 1.7% African-American, 1.6% Asian, 0.5% other 66.8% Free/Reduced Lunch 14.1% English Learners 9.9% Students with Disabilities

7 Correlations of 2 nd Grade DIBELS Measures to 3 rd Grade ITBS & CRTs CRTs Spring CRTs Spring ITBS Fall 2 nd 3 rd 3rd NWF Fall 2 nd ORF Fall 2 nd ORF Winter 2 nd ORF Spring 2 nd ORF Growth (Fall to Spring)

8 2 nd grade Fall NWF as a Predictor of 2 nd Grade Spring State CRTs State Criterion Referenced Tests Nonsense Word Fluency Nonproficient Proficient Total < 50 NWF n = 152(a) n = 158 (b) n = 310 > 50 NWF n = 53 (c) n = 293(d) n = 346 Total n = 205 n = 293 n = 656 Sensitivity = a/(a+c) = 74%; Specificity = d/(b+d) = 65%; Positive Predictive Power = a/(a+b) = 49%; Negative Predictive Power = d/(c+d) = 85%; Overall Correct Classification = (a+d)/n = 68%.

9 2 nd grade Fall ORF as a Predictor of 2nd Grade Spring State CRTs State Criterion Referenced Test Oral Reading Fluency Nonproficient Proficient Total < 44 ORF n = 183 (a) n = 153 (b) n = 336 > 44 ORF n = 22 (c) n = 298(d) n = 320 Total n = 205 n = 451 n = 656 Sensitivity = a/(a+c) = 89%; Specificity = d/(b+d) = 66%; Positive Predictive Power = a/(a+b) = 54%; Negative Predictive Power = d/(c+d) = 93%; Overall Correct Classification = (a+d)/n = 73%.

10 2 nd grade Winter ORF as a Predictor of 2 nd Grade Spring State CRTs State Criterion Referenced Test Oral Reading Fluency Nonproficient Proficient Total < 68 ORF n = 185 (a) n = 157 (b) n = 342 > 68 ORF n = 20 (c) n = 294(d) n = 314 Total n = 205 n = 451 n = 656 Sensitivity = a/(a+c) = 90%; Specificity = d/(b+d) = 65%; Positive Predictive Power = a/(a+b) = 54%; Negative Predictive Power = d/(c+d) = 94%; Overall Correct Classification = (a+d)/n = 73%.

11 2 nd grade Spring ORF as a Predictor of 2 nd Grade Spring State CRTs State Criterion Referenced Test Oral Reading Fluency Nonproficient Proficient Total < 90 ORF n = 186 (a) n = 191(b) n = 377 > 90 ORF n = 19 (c) n = 260 (d) n = 279 Total n = 205 n = 451 n = 656 Sensitivity = a/(a+c) = 91%; Specificity = d/(b+d) = 58%; Positive Predictive Power = a/(a+b) = 49%; Negative Predictive Power = d/(c+d) = 93%; Overall Correct Classification = (a+d)/n = 68%.

12 2 nd grade Fall NWF as a Predictor of 3 rd Grade Spring State CRTs State Criterion Referenced Test Nonsense Word Fluency Nonproficient Proficient Total < 50 NWF n = 157 (a) n = 155 (b) n = 312 > 50 NWF n = 68 (c) n = 287 (d) n = 355 Total n = 225 n = 442 n = 667 Sensitivity = a/(a+c) = 70%; Specificity = d/(b+d) = 65%; Positive Predictive Power = a/(a+b) = 50%; Negative Predictive Power = d/(c+d) = 81%; Overall Correct Classification = (a+d)/n = 67%.

13 2 nd grade Fall ORF as a Predictor of 3 rd Grade Spring State CRTs State Criterion Referenced Test Oral Reading Fluency Nonproficient Proficient Total < 44 ORF n = 189 (a) n = 151 (b) n = 340 > 44 ORF n = 36 (c) n = 291 (d) n = 327 Total n = 225 n = 442 n = 667 Sensitivity = a/(a+c) = 84%; Specificity = d/(b+d) = 66%; Positive Predictive Power = a/(a+b) = 56%; Negative Predictive Power = d/(c+d) = 89%; Overall Correct Classification = (a+d)/n = 72%.

14 2 nd grade Winter ORF as a Predictor of 3 rd Grade Spring State CRTs State Criterion Referenced Test Oral Reading Fluency Nonproficient Proficient Total < 68 ORF n = 190 (a) n = 156 (b) n = 346 > 68 ORF n = 35 (c) n = 286 (d) n = 321 Total n = 225 n = 442 n = 667 Sensitivity = a/(a+c) = 84%; Specificity = d/(b+d) = 65%; Positive Predictive Power = a/(a+b) = 55%; Negative Predictive Power = d/(c+d) = 89%; Overall Correct Classification = (a+d)/n = 71%.

15 2 nd grade Spring ORF as a Predictor of 3 rd Grade Spring State CRTs State Criterion Referenced Test Oral Reading Fluency Nonproficient Proficient Total < 90 ORF n = 198 (a) n = 186 (b) n = 384 > 90 ORF n = 27 (c) n = 256 (d) n = 283 Total n = 225 n = 442 n = 667 Sensitivity = a/(a+c) = 88%; Specificity = d/(b+d) = 58%; Positive Predictive Power = a/(a+b) = 52%; Negative Predictive Power = d/(c+d) = 90%; Overall Correct Classification = (a+d)/n = 68%.

16 Conclusion How well do 2 nd grade benchmark scores on DIBELS predict 2 nd and 3 rd grade scores on the State CRTs? DIBELS measures correctly identified 70-91% of students at-risk for not achieving proficiency on the state CRTs.

17 Conclusion General findings: 2nd grade DIBELS predicted performance similarly on 2 nd grade and 3 rd grade CRTs. ORF was a better predictor of CRT performance than NWF. DIBELS was better at predicting those students who were not going to be proficient than those students who were going to be.

18 Conclusion How does our study contribute the literature base? Our study predicted cross-year as well as same year scores on the CRTs. Our study predicted 2 nd and 3 rd grade scores based on 2 nd grade DIBELS. Our study compared the predictability of NWF to the ORF.

Norm Referenced Test (NRT)

22 Norm Referenced Test (NRT) NRT Test Design In 2005, the MSA Mathematics tests included the TerraNova Mathematics Survey (TN) Form C at Grades 3, 4, 5, 7, and 8 and Form D at Grade 6. The MSA Grade 10