The Development of the Grand Rapids Educational Achievement Test

Transcription

1 Western Michigan University ScholarWorks at WMU Master's Theses Graduate College The Development of the Grand Rapids Educational Achievement Test John S. Thatcher Western Michigan University Follow this and additional works at: Part of the Educational Assessment, Evaluation, and Research Commons, and the Special Education and Teaching Commons Recommended Citation Thatcher, John S., "The Development of the Grand Rapids Educational Achievement Test" (1975). Master's Theses This Masters Thesis-Open Access is brought to you for free and open access by the Graduate College at ScholarWorks at WMU. It has been accepted for inclusion in Master's Theses by an authorized administrator of ScholarWorks at WMU. For more information, please contact

2 THE DEVELOPMENT OF THE GRAND RAPIDS EDUCATIONAL ACHIEVEMENT T E S T by John S. T hatcher A P ro je c t Report Subm itted to th e F acu lty o f The G raduate C ollege in p a r tia l fu lfillm e n t o f th e S p e c ia lis t in Education Degree W estern Michigan U n iv ersity Kalamazoo, Michigan December 1975

3 ACKNOWLEDGEMENTS The development of th e Grand Rapids E ducational Achievement T est has b e n e fite d much from th e p a r tic ip a tio n o f P ro fesso rs Galen J. A le ssi, Jack E. A sher, and N eil D. K ent. I must a lso thank Jane W alters who coord in ated th e e f f o r ts o f W estern Michigan U n iv e rs ity 's T e stin g S erv ice and Computer C enter in th e d i f f i c u l t ta s k o f handling 486 v a ria b le s s t a t i s t i c a l l y. My f in a l thanks go to Vern Yetman, who has been a c tiv e ly involved sin c e th e in c ep tio n o f th e GREAT. John S. T hatcher i i

4 INFORMATION TO USERS This material was produced from a microfilm copy of the original document. While the m ost advanced technological means to photograph and reproduce this document have been used, the quality is heavily dependent upon the quality of the original subm itted. The following explanation of techniques is provided to help you understand markings or patterns which may appear on this reproduction. 1.T h e sign or "target" for pages apparently lacking from the document photographed is "Missing Page(s)". If it was possible to obtain the missing page(s) or section, they are spliced into the film along with adjacent pages. This may have necessitated cutting thru an image and duplicating adjacent pages to insure you com plete continuity. 2. When an image on the film is obliterated with a large round black mark, it is an indication that the photographer suspected that the copy may have moved during exposure and thus cause a blurred image. You will find a good image o f the page in the adjacent frame. 3. When a map, drawing or chart, etc., was part of the material being photographed the photographer followed a definite method in "sectioning" the material. It is customary to begin photoing at the upper left hand corner o f a large sheet and to continue photoing from left to right in equal sections with a small overlap. If necessary, sectioning is continued again beginning below the first row and continuing on until com plete. 4. The majority of users indicate that the textual content is of greatest value, however, a som ewhat higher quality reproduction could be made from "photographs" if essential to the understanding of the dissertation. Silver prints o f "photographs" may be ordered at additional charge by writing' the Order Department, giving the catalog number, title, author and specific pages you wish reproduced. 5. PLEASE NOTE: Som e pages may have indistinct print. Filmed as received. Xerox University Microfilms 300 North Zeeb Road Ann Arbor, Michigan 48106

5 MASTERS THESIS M-8013 THATCHER, John Stephen THE DEVELOPMENT OF THE GRAND RAPIDS EDUCATIONAL ACHIEVEMENT TEST. Western Michigan University, Ed.S., 1975 Education, special Xerox University Microfilms, A nn A rbor, M ichigan 48106

6 Table o f Contents C hapter 1 INTRODUCTION What i s th e GREAT?... 1 R atio n ale f o r th e T est and R elated L ite ra tu re Purpose and Use o f th e GREAT C hapter 2 DEVELOPMENT OF THE EXPERIMENTAL FORM C o n s tr u c tio n A dm in istratio n of th e Experim ental GREAT Experim ental P opulation R e l i a b i l i t y V a lid ity C hapter 3 DEVELOPMENT OF THE FINAL FO RM Item Q u ality and C o n s t r u c t i o n A dm in istratio n and S coring of th e F in al Form o f th e GREAT. 28 Performance O b jectiv es GREAT F uture D irectio n s of th e GREAT Appendixes Appendix A Grand Rapids E ducational Achievement T e s t

7 Table o f Contents (continued) Appendix B Experim ental Form, A d m inistration and S c o r i n g Appendix C Grand Rapids E ducational Achievement T est Appendix D F in a l Form, A dm inistration and Scoring R eferences iv

8 In tro d u ctio n What i s th e GREAT? The Grand Rapids E ducational Achievement T e st, in both experimental and f in a l form, has been produced w ith th e in te n t o f providing c lin ic ia n s in th e a re a o f ed u cational measurement w ith an abbreviated read in g and a rith m e tic inventory f o r grades kindergarten through s ix th. I t has a lso been c o n stru cted, but to a le s s e r degree fo r th e s p e c ia l education te acher who views measurement w ithin the classroom as an e s s e n tia l v a ria b le of teach in g and uses th e re s u lts to improve le a rn in g. The b asic in te n t o f th e t e s t is to quickly measure th e areas o f a rith m e tic com putation and word re cognitio n, although o th e r re la te d areas a re review ed. The t e s t has been cons tru c te d in such a manner as to provide inform ation which w ill be h e lp fu l in making d ecisio n s about s p e c ia l education stu d en ts or pros p ectiv e s p e c ia l education stu d en ts and t h e i r placem ent. R atio n ale fo r th e T est and R elated L ite ra tu re The development o f th e Grand Rapids E ducational Achievement T est (h e re a fte r c a lle d th e GREAT) has had much o f i t s beginning ro o ted in th e inadequacies o f th e Wide Range Achievement T est (Ja sta k & J a s ta k, 19&5)* T his t e s t was f i r s t stan d ard ized in 1936 and has subsequently been re v ised in 19^+6 and The authors s ta t e th e i r t e s t has f u l f i l l e d an im portant need based on th e wide p o p u la rity o f th e t e s t. The Wide Range Achievement T est (h e re a fte r c a lle d th e WRAT) in i t s p resen t form measures th e areas of read in g (th e reco g n itio n of 1

9 words and p ro n u n ciatio n ), w ritte n s p e llin g, and a rith m e tic computa tio n. Ja sta k and Ja sta k ( 1965) s t a t e th e ir t e s t has been co n stru cted in such a manner as to accom plish th e fo llo w in g ends: ( l ) to study th e sensory-m otor s k i l l s involved in le a rn in g to read, s p e ll, w rite, and fig u re, (2) to provide sim ple and homogeneous co n ten t, ( 3) to avoid d u p lic a tio n and overlap p in g with t e s t s o f comprehension, judgment, reasoning, and g e n e ra liz a tio n stu d ie d by means o th e r than read in g, s p e llin g, and a rith m e tic, (4) to f r e e d ia g n o stic in feren ces from common confusions due to o p e ra tio n al sem antics, ( 5) to perm it v a lid ity analyses by th e method o f in te rn a l c o n s is t ency. (p. l ) The authors a lso claim th a t t h e ir t e s t c o n stru ctio n lends i t s e l f to fa c to r a n a ly sis in com bination w ith such instrum ents as th e W echsler In te llig e n c e Scale f o r C hildren, which provides fu r th e r v a lid ity. A review o f th e l i t e r a t u r e in term s o f th e claim s made by th e WRAT would suggest th a t th e authors have tr i e d to g et too much from too l i t t l e. In a review o f th e 19^6 e d itio n o f th e t e s t, Courtney ( 19^9) s ta te d th e follow ing: This review er fe e ls th a t th e disadvantages of th e Wide Range Achievement T est and i t s manual overshadow i t s advantages and th a t th e f i r s t approxim ation o f a tra in e d educational c lin ic ia n would f i l l th e need f i l l e d by th is t e s t, i. e., a rough grade placement o f a p a tie n t p rio r to more d e ta ile d d ia g n o stic te s tin g a t s p e c ific grade le v e ls, (p. H6) Courtney c ite d such s p e c ific e rro rs in t e s t c o n stru ctio n as too lim ite d a number of item s per grade, too lim ite d a number o f p u p ils on which th e t e s t was stan d ard ized, and a manual lack in g in s o p h is tic a tio n to th e p o in t th a t m isuse o f th e t e s t might occur q u ite f r e q u en tly. Sims (19^9) in h is review o f th e same e a rly e d itio n o f th e WRAT, suggested th a t th e school p sy ch o lo g ist might tr y th e t e s t but th a t i t was h is convictio n in g en eral th a t th e t e s t had lim ite d u sefu ln ess.

10 He f u r th e r suggested th a t th e r e l i a b i l i t y of th e t e s t had not been adequately e sta b lish e d as w ell as th e method fo r e sta b lis h in g norms, Reviews of l a t e r e d itio n s did not fa re much b e tte r than th e o r ig i n al p u b lic a tio n o f th e WRAT, A review of th e 1965 e d itio n (Merwin, 19?2) suggested th a t th e t e s t was im p ractic al fo r g en eral school and was more s u ita b le f o r a p sy ch o lo g ist working with s p e c ia l c ase s. The au th o r f e l t th e a d m in istra tio n of th e t e s t was much too complex and confusing, th e norms la ck in g in re p re se n ta tiv e sam pling, and claim s made as to r e l i a b i l i t y and v a lid ity lack in g in s t a t i s t i c a l su p p o rt, Thorndike (1972) in h is review o f th e t e s t a lso had some m isgivings about th e domain o f v a lid ity and f e l t th e authors of th e WRAT u t i liz e d r a th e r "ex o tic procedures" in th e ir attem pt to e s ta b lis h val i d i t y. He made th e fo llo w in g statem ent concerning th is a re a of t e s t c o n stru ctio n : They s t a t e f o r example, "V alid ity can be determ ined only by th e comparison o f one t e s t score w ith th o se which measure e n tire ly d if f e r e n t a b i l i t i e s. " Again, " I t i s accep tab le p ra c tic e to use c r i t e r i a o f in te rn a l consisten cy in th e v a lid a tio n o f t e s t s. C r ite r ia of in te r n a l consisten cy, i f properly in te r p r e te d, a re u su ally more m eaningful and more v a lid than a re e x te rn a l c r i t e r i a of com parison." One fin d s i t hard to re c o n c ile th e se statem ents w ith each o th e r o r w ith th e u su al concepts of t e s t v a lid a tio n, (p. 3?)* In s p ite o f claim s l ik e th e se, th e WRAT has enjoyed g re a t popul a r i t y. A survey o f th e frequency of t e s t usage found th e WRAT to be th e second most fre q u e n tly used achievement t e s t in psy ch o lo g ical c lin ic s (Sunberg, 1961). Only th e Stanford Achievement T est s u r passed i t in p o p u la rity. The au thors o f th e WRAT s t a t e th a t th is p o p u la rity is w arranted in view o f th e dem onstrated v a lid ity of th e t e s t. They a re q u ite

11 e x p lic it on t h is m atter and mention sev e ra l ways which have te e n used to dem onstrate th is v a lid ity : (a) th e c o rre la tio n between WRAT sco res and o u tsid e c r i t e r i a such as te a c h e r ra tin g s and chro n o lo g ical age, (b) th e c o rre la tio n between WRAT sco res and s im ila r achievement t e s t s, and (c) th e c o rre la tio n between WRAT scores and mental a b i l i t y o r i n t e l lig e n c e r a tin g s. However, a survey o f th e l i t e r a t u r e concerning th e 19^6 and 1965 e d itio n s o f th e WRAT in d ic a te s th e authors have on occasio n f a lle n sh o rt o f t h e i r g o al. Hopkins and Dobson ( 1962) claim ed c re d it fo r being among th e f i r s t to in v e s tig a te th e v a lid ity o f th e WRAT (19^+6 e d itio n ) beyond th e d a ta provided by th e a u th o rs. T h e ir study w ith elem entary age c h ild re n in re g u la r classroom s y ield ed b o rd e rlin e concurrent v a lid ity f o r grade fiv e and s u b s ta n tia l v a lid ity fo r grade th re e. The t e s t used f o r comparison was th e C a lifo rn ia Achievement T est (CAT) in th e are as of word re c o g n itio n and comprehension. The au thors concluded t h e i r study by s ta tin g th e concurrent v a lid ity o f th e WRAT when compared to th e CAT was not s u p e rio r. They d id fin d th e WRAT to c o rre la te h ig h ly w ith te a c h e r r a tin g s. A nother study (O ldridge, 196^) comparing th e WRAT and th e CAT f o r both read in g and a rith m e tic found a c o rre la tio n (r) of.73 f o r read in g and.80 f o r a rith m e tic. A lthough th e r e s u lts were fav o rable, th e authors mentioned to p o te n tia l u sers of th e t e s t "the tendency o f th e WRAT to sco re low er in grade placem ent th an th e CAT (about 3 months low er in read in g and about 7 months in a rith m e tic ) even though th e d iffe re n c e s were not s ig n ific a n t" (p. 4 l6 ). S tudies comparing th e WRAT w ith in d ices o f in te llig e n c e a re

12 5 c h a ra c te riz e d by R eg er's (1966) fin d in g s of l i t t l e re la tio n s h ip between th e WRAT (19*4-6 e d itio n ) and th e W echsler In te llig e n c e S cale fo r C hildren - F u ll S cale (WISC-FS). C o rre la tio n s (r ' s ) in th is study u tiliz in g educable m entally re ta rd e d youngsters ranged from -.3*4- to.08. Another study (Tobias & G o relick, 1961) using educable m entally re ta rd e d a d u lts f o r su b jects found th e re la tio n s h ip s between the WRAT (S u b test Reading) and th e W echsler A dult In te llig e n c e S cale - F u ll S cale (WAIS-FS) to be minimal ( r =.*t0). I t would seem th a t e v i dence from th e se two stu d ie s would serio u sly jeo p ard ize th e p re d ic tiv e v a lid ity of th e WRAT and n u llif y Jasta k and J a s ta k 's claim th a t stan d ard scores deriv ed from th e WRAT a re comparable to th e IQs of stan d ard in te llig e n c e t e s t s, S tudies comparing th e Peabody P ic tu re Vocabulary T est (PPVT) have fa re d no b e tte r. W olfensberger (1962) found c o rre la tio n s (r ' s ) ranging from.33 to *52 when stu d y in g re ta rd a te s in th e educable range. M ueller (1968) found c o rre la tio n s ran g in g from a low of.02 to a high of.27 when comparing th e same t e s t f o r r e ta rd a te s. F u rth er study by M ueller w ith m entally im paired su b je c ts found th e Stanford B inet, Form L-M (SB L-M) to compare fav o rable w ith only th e WRAT su b je c t A rithm etic. Research d e a lin g w ith th e v a lid ity o f th e 1965 form o f th e WRAT has been much more p r o lif ic and in some cases more encouraging in establis h in g c r e d ib ility. Comparisons between th e WRAT and th e Peabody In d iv id u a l Achievement T est (PIAT) by Soethe (1972) using read in g d isab led and m entally re ta rd e d c h ild re n dem onstrated c o rre la tio n s of.37 to.92 f o r th e d isab le d group and.4 ^ -to.87 f o r th e m entally re ta rd e d group. The concurrent v a lid ity was reasonably high.

13 However, th e study included only a t o t a l o f 40 c h ild re n w ith no s t a t i s t i c a l t e s t s of sig n ific a n c e being re p o rted, A study by Baum (1975) y ielded s im ila r r e s u lts when comparing th e WRAT and th e PIAT employing le a rn in g d isab led ch ild re n between th e ages of 7 to 11, C o rre la tio n s from,49 to,90 were e sta b lish e d. I t is worthy to note th a t only 25 su b je c ts were s e le c te d fo r each o f th e age groups stu d ie d, a seem ingly sm all sample from which to g e n e ra liz e. I t was a lso in te r e s tin g to observe th a t 8% o f th e su b je c ts were male, making g e n e ra liz a tio n s once again d i f f i c u l t, The most d is tu rb in g conc lu sio n from th is study was th e statem ent from th e au th o r th a t th e " u t i l i t y of th e PIAT as a wide-range achievement t e s t is a t le a s t as prom ising as th e WRAT" (p. 492), A study (Cochran & P e d rin i, 19^9) attem p tin g to e s ta b lis h concurrent v a lid ity between th e 19&5 WRAT and th e WAIS-FS re s u lte d in m arginal c o rre la tio n s. The read in g and s p e llin g were not s ig n if ic a n tly re la te d (p <.05) to th e WAIS. C o rre la tio n s ranged around,60, These r e s u lts d i f f e r from th o se o f Ja s ta k and Ja sta k (1965) who found c o rre la tio n s ranging from Such v a r ia b ility in d a ta re p o rtin g could leave th e p ro sp ectiv e c lin ic a l u ser o f th e WRAT in a s ta t e of quandry as to i t s u sefu ln ess as a p re d ic to r o f school achievem ent. I t c e r ta in ly dampens th e statem ent by Jasta k and Ja sta k th a t "th e r e s u lts from th e WRAT t e s t can be d ir e c tly compared w ith th e major in d iv id u a l i n t e l l i gence scale s" (1965, p. 12), This same study by Cochran and P edrin i found c o rre la tio n s between th e SB, L-M and WRAT h ig h er ( ) than th o se o f th e WAIS-FS but th is is not s u rp ris in g in th a t H im elstein (1968) found th e SB, L-M to

14 7 be s ig n if ic a n tly re la te d to sev e ra l te s t s o f academic achievem ent. Cochran and P edrin i a lso e sta b lish e d c o rre la tio n s between th e PPVT and th e WRAT ran g in g from a low of.38 to a high of.47 which a re even b e tte r than th o se re p o rte d e a r lie r in a study (M ueller, 1968) where c o rre la tio n s ranged from.02 to.27. These s t a t i s t i c s suggest th a t th e WRAT can d e riv e l i t t l e proof of v a lid ity from th e PPVT. In co n clu sio n, i t appears th e WRAT has shortcom ings which th e authors o f th e t e s t ig n o re. Why th en, in th e fa c e o f such c o n tra d ic tio n s, has th e WRAT m aintained so much p o p u larity? The reasons seem to l i e in th re e a re a s; (a) th e t e s t 's a p p lic a b ility to a wide span of a b i l i t y (k in d e rg arten to su p e rio r a d u lt), (b) th e speed w ith which i t can be given, and (c) th e r e c a l l form at which does much to e lim in a te chance. The t e s t 's a b i l i t y to q u ickly produce a grade equiv alen t i s most u n fo rtu n ate when th e "score" i s used to pigeonhole a stu d e n t, sometimes to a s p e c ia l education classroom. The WRAT's n e a re st r i v a l, th e PIAT (Dunn & Markwardt, 1970), is s im ila r to th e WRAT w ith th e exception o f a d d itio n a l s u b te s ts. I t has not surpassed th e WRAT in p o p u larity in s p ite of th e f a c t th a t co n siderab le f ie ld te s tin g has gone in to i t over a p erio d o f seven y ears. Lyman (1972) in h is review o f th e PIAT p ra ised th e im pressive amount of work which went in to i t s c o n stru c tio n and f e l t i t only s u ffe r ed when comparing i t to group achievement t e s t s. By comparison, he s ta te d "th e PIAT has le s s dem onstrated v a lid ity, low er t e s t - r e t e s t r e l i a b i l i t y c o e f fic ie n ts a t most le v e ls, sm aller s ta n d a rd iz a tio n sam ples, and fewer s u b te sts" (p. 35)* French (1972) in h is review of th e t e s t mentioned i t s adequacy and was favorab le to th e authors because of th e lim ita tio n s s e t fo r the

15 in te r p r e ta tio n of t e s t s c o re s. He was concerned about th e lack of raw sco re mean d iffe re n c e s between grade le v e ls (in th e s ta n d a rd iz a tio n sample) and th e u n r e lia b ility of th e s u b te s ts when considered in d iv id u a lly. Although th e PIAT enjoyed more p o s itiv e reviews a t th e tim e o f i t s in tro d u c tio n than th e WRAT, th e r e la tiv e ly few stu d ie s now a v a ila b le concerning concurrent v a lid ity have not always been fav o rab le. Comparisons w ith th e WRAT have r e s u lte d in r a th e r low values which might be in th e PIAT's fav o r in view of th e WRAT's q u estio n ab le in te g r ity. I t must be remembered th a t th e WRAT has gained much of i t s so c a lle d " v a lid ity " through h is to r ic a l use being handed down from one genera tio n of c lin ic ia n s to th e next with unquestioning f a i t h. Such a study (th e most fav o rab le) by S itlin g to n (c ite d in Dunn & Markwardt, 1970) found c o rre la tio n s ranging from.58 to,95 when using a group of ^6 educable m entally re ta rd e d a d o le sc e n ts. Comparisons of th e PIAT w ith in te llig e n c e te s t s have re s u lte d in c o rre la tio n s of.57 w ith th e PPVT (Dunn & Markwardt, 1970) using a normal p o p u latio n,.22 w ith th e WISC and m entally re ta rd e d su b je c ts (S oethe, 1972) and.80 u sin g th e WISC and a norm al group (S o eth e, 1972). The most in te n s iv e, re c e n t study o f th e PIAT (Wilson & S pangler, 197*0 using le a rn in g d isab led c h ild re n as su b je c ts found th e t e s t to lend i t s e l f w ell to such an excep tio n al group. The authors c ite d th e speed and ease w ith which th e t e s t could be given, th e lack of required w ritte n r e sponses and th e minimal requirem ent o f lengthy responses as some o f th e advantages of using th e t e s t. However, comparisons w ith th e WISC, SB LM and PPVT d id not go as w ell as th e a d m in istra tio n of th e t e s t to th e le a r n in g d isab le d group. Pearson product-moment c o rre la tio n c o e ffic ie n ts fo r

16 th e WISC, SB L-M and th e PPVT were,31>.54, and.32 re s p e c tiv e ly. P a r tia llin g f o r age r e s u lte d in m oderately in creased re la tio n s h ip s. Although th e au th o rs attem pted to salvage t h e ir d a ta through s t a t i s t i c a l m anipulation, th e PIAT gained l i t t l e c r e d ib ility by comparing i t with popular in te llig e n c e m easures. The authors then go on to warn about th e controversy surrounding th e p ra c tic e o f e s ta b lis h in g achievem ent t e s t v a lid ity by sim ple c o rre la tio n s w ith measures of in te llig e n c e. In summary, i t appears th a t th e PIAT, which was to be a " s o p h is ticated and form idable challen g e to th e WRAT" (P roger, 19?0, p. 467) has not a lte re d th e p a tte rn of in d iv id u a l achievement te s tin g by c lin ic ia n s. The WRAT s t i l l su rp asses i t in use even though th e PIAT appears to have g re a te r fa c e v a lid ity. An a d m in istra tio n tim e o f from 30 to 40 minutes appears to be i t s b ig g e st undoing when co n sidering th e WRAT ta k es only about o n e-h alf th a t tim e. Also th e com plexity (d ire c tio n s ) of th e t e s t has undermined i t s u sefu ln ess as a quick estim ate of achievem ent. I t is out of t h i s background th e id ea fo r a new c h allen g e r to th e world of th e "q u ickies" began to c r y s ta lliz e. There appeared to be a need fo r a t e s t which was more thorough in content than th e WRAT but having a s h o rte r a d m in is tra tio n tim e than th e PIAT (20-30 m inutes). I t was decided th a t a t e s t lim itin g i t s e l f to a rith m e tic and reading but in clu d in g more item s in each of th e areas would be a good p lace to begin, An inform al survey o f c lin ic ia n s revealed th e m ajo rity used th e a rith m e tic and readin g p o rtio n s o f th e WRAT and PIAT w ith g re a te r f r e quency than any o th e r p a rt o f th e t e s t s. In lig h t of t h i s, th e development of th e Grand Rapids E ducational Achievement T est began.

17 10 The Purpose and Use o f th e GREAT The GREAT has "been developed w ith p rim a rily th e d iag n o stic c lin ic ia n in mind. As s ta te d e a r lie r, th e e n tire o rie n ta tio n o f th e GREAT sin ce i t s tim e of conception has been to keep i t s a d m in istra tio n tim e w ithin m inutes w ithout s a c r if ic in g ample sam pling from th e item u n iv erse. I t has been developed w ith th e foreknowledge th a t u sers of th e te s t' w ill be in te re s te d in more than th e academic domain of th e stu d en t and th e re fo re w ill a l l o t only a fra c tio n a l p a rt o f t h e i r tim e fo r academic t e s t in g. To a le s s e r e x te n t, th e GREAT has been developed fo r th e classroom te a c h e r, e sp e c ia lly th e te a c h e r of s p e c ia l education. Such a te a c h e r is more c o n sta n tly bombarded w ith req u ests f o r up to d a te progress re p o rts in th e are as o f a rith m e tic and readin g, and u su a lly because of lack of tim e, cannot become involved in more ex ten siv e achievement te s tin g. E ducational a c c o u n ta b ility c o n sta n tly demands th e use of a sse ssment devices to measure th e progress of s tu d e n ts, classroom s, sch o o ls, and school system s. Although c rite rio n -re f e r e n c e t e s t s a re now th e high fa sh io n, th e re s t i l l remains a need f o r such t e s t s as th e GREAT to f u l f i l l th e follow ing purposes: 1. E stim ate th e stu d en ts a rith m e tic and read in g le v e l to f a c i l i t a t e th e a d a p ta tio n o f m a te ria ls and in s tr u c tio n a l procedures to h is needs and a b i l i t i e s. 2. Diagnose s p e c ific stre n g th s and weaknesses in a stu d en ts a rith m e tic and read in g development. 3. Determ ine th e exten t to which stu d en ts have read in ess a b i l i t i e s and s k i l l s needed to begin a rith m e tic and read in g in s tru c tio n or to proceed to th e nex t s te p in th e in s tru c tio n a l sequence.

18 11 4. Provide inform ation fo r educational planning committees to help in making d e cisio n s concerning s p e c ia l education placements and s e rv ic e s. 5. Provide inform ation u sefu l in making a d m in istra tiv e d ecisio n s in grouping o r programming to b e tte r provide f o r in d iv id u a l d i f f e r ences w ithin re g u la r school programs, 6. M onitor c o n tin u a lly th e progress of in d iv id u a l stu d en ts in r e sp ect to s p e c ific perform ance o b je ctiv es (when a p p lic ab le ) in o rd er to make immediate in s tr u c tio n a l d e c isio n s. In conclusion, th e GREAT's desig n should appeal to th e d ia g n o stic ia n who must be, in a m atter o f moments, an "expert" on th e stu d en ts achievement in th e areas of a rith m e tic and read in g. Although Shoemaker (1975) argues fo r an end to sm all tim e te s tin g e n te rp rise s such as th e GREAT, i t is believed th e purposes o u tlin e d fo r th e GREAT w ill provide a u sefu l serv ic e to consumers of th e t e s t.

19 Development o f th e E xperim ental Form C onstruction The i n i t i a l concern in th e c o n stru ctio n o f th e experim ental form of th e GREAT was i t s relevancy to e x istin g in s tr u c tio n a l programs and i t s v a lid ity from a co n ten t p o in t of view. As Shoemaker (1975) p o in ts out, "An in s tr u c tio n a l program and i t s a sso c iate d item u n iverse a re isomorphic" (p. 128). Keeping th is in mind, a workable item univ erse was sought to supply item s fo r th e GREAT. Such an item domain can ta k e many forms o f c o n stru c tio n but i t was decided th e item -form approach would be used as much as fe a s ib ly p o ssib le. This method reduced th e goals of an in s tr u c tio n a l program to statem ents of h ighly s p e c ific o b je c tiv e s. At th is tim e i t was determ ined th e perform ance o b je ctiv es used by th e Grand Rapids P u b lic Schools would be adopted as a means o f gen eratin g item s fo r th e GREAT. The Reading Performance O b jectives (197*0 were w ritte n by an ex pert team o f elem entary, m iddle school and reading te a c h e rs. These o b je c tiv e s in co rp o rated th e minimal o b je c tiv e s published by th e S ta te o f Michigan. The M athem atical Performance O bjectiv es (1973) were a lso constru cted by an ex p ert team o f lo c a l te a c h e rs. They to o, included th e o b je ctiv es s e t by th e s ta t e. Although Slocum (c ite d in Shoemaker, 1975) i-s c r i t i c a l o f lo c a lly developed o b je c tiv e s and item domains, i t was e sta b lish e d th a t fo r th e purposes of th e GREAT, th e o b jectiv es would be ad equately re p re se n ta tiv e because o f t h e i r san c tio n in g by th e S ta te o f M ichigan. At th is p o in t, item s were generated a t each grade le v e l f o r p lacement on th e GREAT. The q u a n ity o f item s used was determ ined by th e g o al 12

20 13 o f having a t le a s t th re e tim es th e item s appearing on th e experim ental form as would appear on th e f i n a l form. The r a tio n a le f o r th is d e cisio n was to assu re th e e x isten c e o f a ccep tab le item s ( a f t e r item a n a ly s is ) fo r placem ent on th e f in a l form. Although th e e x is tin g o b je c tiv e s u tiliz e d by th e Grand Rapids Schools were im portant in th e productions o f item s, t h e i r m o d ificatio n was sometimes n ecessary so as to b e tte r f i t th e form at of th e GREAT. A lso, i t was im possible to use th e item -form approach w ith much o f th e readin g on th e GREAT because o f th e choice of a word re c o g n itio n form at. In th e absence o f perform ance o b je ctiv es from which to g en erate t e s t ite m s, a word pool was e sta b lis h e d from e x is tin g word re c o g n itio n invent o r i e s. These in v e n to rie s included th e D iagnostic Reading Scales (Spache, 1963)» th e Classroom Reading In v en to ry ( S ilv a r o li, 19&5)» th e D u rre ll A nalysis of Reading D if f ic u lty (D u rre ll, 1955)» and th e Slosson Oral Reading T est (S losson, 1963). The word l i s t s fo r grades one through seven were in co rp o rated f o r use on th e experim ental GREAT by th e random numbers method (Games & K lare, 1967). T his method assigned each word a number which was then used in conju n ctio n w ith a random-number ta b le, th e re fo re a ssu rin g each word an equal o p p o rtu n ity f o r placement on th e t e s t. Fourty words were s e le c te d f o r each grade le v e l w ith th e exception of 80 words used a t th e f i r s t le v e l. Once accep tab le item s had. been e sta b lish e d f o r th e GREAT, th e i r sequencing had to be e sta b lis h e d. For th e a rith m e tic p o rtio n o f th e t e s t, two se c tio n s were used. The f i r s t was composed o f a v erb al s e c tio n covering item s The second was t o t a l l y made up o f com putation problems covering item s Item s w ith in each o f th e se sectio n s were arranged

21 14 according to le v e l of d i f f i c u l t y (easy to d i f f i c u l t ) as d esig n ated by e ith e r th e companion perform ance o b je c tiv e o r th e normal in s tr u c tio n tim e f o r such a problem (a t what le v e l is th a t concept norm ally ta u g h t? ). The read in g component of th e experim ental GREAT was arranged e n tir e ly according to le v e l of d if f i c u l t y as desig n ated by e ith e r th e o b je c tiv e o r th e e sta b lis h e d word l i s t. I t included 354 item s. The form at o f th e t e s t a lso included s tim u li which was not placed on th e f in a l form o f th e GREAT. T his included a s e t o f norm ative ta b le s based on perform ance o b je c tiv e s and e sta b lish e d word l i s t s and a l e t t e r - number assig n ed to each t e s t item fo r computer use in f u r th e r developm ent. The norms were developed so as to give th e experim ental form meaning in term s of a grade equivalency as i t was to be used as p a rt of th e normal t e s t b a tte ry in lie u of such achievem ent t e s t s as th e WRAT. In summary, th e experim ental form of th e GREAT was c re ate d out of th e r a tio n a le th a t a t r i a l t e s t w ith a la rg e number of item s would produce a f i n a l form w ith in creased fa c e and s t a t i s t i c a l v a lid ity. The f i n a l experim ental form was th e end product of two o th e r t r i a l forms (se e Appendix A f o r a com plete p ro to co l of th e experim ental GREAT). A d m in istratio n o f th e Experim ental GREAT At t h is p o in t in th e development of th e GREAT, th e e f f o r ts o f sev e ra l te achers and c lin ic ia n s were s o lic ite d f o r h elp in th e a d m in istra tio n of th e experim ental form. A lthough th e m ajo rity of te s t s were given by one c lin ic ia n, th e le n g th of th e GREAT, e s p e c ia lly a t th e upper elem entary le v e ls, re q u ire d a d d itio n a l h elp in order to m aintain th e e sta b lish e d tim e lin e. As a r e s u lt, a stan d ard ized s e t o f in s tru c tio n s was d eveloped in clu d in g a d m in istra tio n, sco rin g, and tra n sfe re n c e of in fo rm atio n

22 15 to nark sense s h e e ts. These in s tru c tio n s as they appeared in an a tta c h ment to th e experim ental form d ire c te d the examiner to use r a th e r la rg e c e ilin g c r i t e r i a w ith th e o b je c tiv e in mind th a t th e raw sco re obtained would more a c c u ra te ly r e f l e c t th e su b jects a c tu a l r e p e r to ir e. A lso, fa ls e c e ilin g s would be avoided by such a method (see Appendix B f o r a complete s e t o f in s tru c tio n s as used w ith th e experim ental GREAT), Experimental Population The stu d en ts used in o b tain in g d ata on th e experim ental form of th e GREAT re p re se n te d fo u r p u b lic and two p aro ch ial school systems w ith 28 elem entary schools involved. A s t r a t i f i e d r a th e r th an random approach was used because o f th e tim e f a c to r involved in g e ttin g a la r g e r n. The populatio n was f a i r l y eq u ally rep resen ted across socio-econom ic lin e s and n e a rly eq u ally d iv id ed between sexes. Since th e in te n t o f th e GREAT from th e beginning was d ire c te d towards use w ith stu d en ts e ith e r re fe rre d f o r psychological ev alu atio n s o r already in s p e c ia l education classroom s, an attem pt was made to have a t le a s t h a lf th e cases be from th e se so u rces. The rem aining stu d en ts would be from th e re g u la r p o p u latio n. Table 1 provides a grade comparison o f th e number of r e f e r r a l and n o n -re fe rra l stu d en ts in th e experim ental sample. TABLE 1 Grade Comparisons o f th e R e fe rra l and N on-r eferral Groups Grades R e fe rra l (n) N on-r eferral (n) T o tal p re-k, k and and , 5, and T o tals

23 The ra tio n a le fo r such a s t r a t i f i c a t i o n was to provide an other varia b le in term s o f item a n a ly sis so as to more s p e c ific a lly determ ine those item s which were most d iscrim in a tin g between th e r e f e r r a l and n o n -re fe rra l groups (based on item d i f f i c u l t y ). Although th is procedure is probably unique in t e s t c o n stru ctio n, i t was j u s tif ie d on th e basis th a t i t was being applied to an experim ental form. R e lia b ility A high r e l i a b i l i t y c o e f fic ie n t o f.98 was obtained on the a rith m e tic p o rtio n o f th e GREAT (based on t o t a l population) using the Kudar-Richardson Formula 21 method ( KR^1). A c o e f fic ie n t o f.99 was obtained fo r th e read ing using th e same s t a t i s t i c a l procedure. These c o e ffic ie n ts, no doubt, r e f le c te d to some degree th e len g th o f th e experim ental GREAT. A lso, th e se c o e ffic ie n ts were not r e f le c tin g th o se e rro rs normally obtained w ith v a ria tio n s in t e s t a d m in istra tio n o r day-to-day v a ria tio n s in th e in d iv id u a l. C o e ffic ie n ts of r e l i a b i l i t y (SKR^l) obtained on both th e r e f e r r a l and n o n -re fe rra l groups were a lso h ig h, a l l being above. 96. O ther methods of determ ining c o rre la tio n c o e ffic ie n ts were not computed because of th e experim ental form at of th e t e s t. However, i t can be p re d ic ted th a t o th e r techniques would a ls o have re s u lte d in ra th e r high c o e f fic ie n ts because o f th e le n g th o f th e experim ental form. Without a doubt, i t would have been th e o v e rrid in g s t a t i s t i c a l fa c to r. The Standard E rro r of Measurement (SEM)of raw scores fo r th e a r i t h m etic p o rtio n of th e GREAT was J.6. The read in g Standard E rro r of Measurement was 1, 5, The r e l i a b i l i t y c o e f fic ie n t used fo r r^ j in th e equation

24 was Kudar-Richardson Formula 21. Once again th is was re f le c tin g erro rs due only to in te rn a l c o n sid e ra tio n s. In summary, i t appeared th e experim ental GREAT s a tis f ie d s t a t i s t i c a l conditions of r e l i a b i l i t y even though th e main impetus of development was more concerned w ith item a n a ly s is. V alid ity Very l i t t l e d a ta have been e sta b lish e d f o r th e experim ental GREAT in terms of v a lid ity ( c r ite r io n - r e la te d ) due to i t s tr a n s ie n t s ta tu s. However, a comparison of means between r e f e r r a l and n o n -re fe rra l groups equally matched in terms of grade placements in d ic a te d th e experim ental form adequately d iscrim in a ted between th e two groups. For both arith m e tic and read in g, th e mean sco res o f th e n o n -referred group were tw ice as la rg e as those of th e re fe rre d group. Since th e experim ental GREAT was designed as an assessm ent fu n ctio n, i t s a c c e p ta b ility a t th is p o in t in development hinged much on i t s content v a lid ity. When review ing th e method of s e le c tio n of item s fo r th e GREAT, i t appeared th a t a l l th e requirem ents as s e t f o r th in Standards fo r Educational and P sychological T ests (197*0 had been f u l f i l l e d.

25 Development of the Final Form Item Quality and Construction With 481 item s appearing on th e experim ental form of th e GREAT, a d e c isio n had to be made as to what item s should appear on th e f i n a l form. A lso, th e form at and cosm etic appearance of th e t e s t had to be determ ined. The d a ta in terms of item a n a ly sis was stu d ied from th re e p o in ts of view: (a) th e performance of th e e n tir e sample, (b) th e perform ance of th e t o t a l r e f e r r a l groups versus th e n o n -re fe rra l groups, and (c) th e perform ance o f th e r e f e r r a l versus th e n o n -re fe rra l group by th e grade d iv is io n s. The f i r s t d iv is io n was composed o f pre k, k, and 1 s t grade s tu d e n ts, th e second included grades 2 and 3, and th e th ir d grades 4-, 5> and 6. These groupings occurred because of th e overlap of s k i l l s in th e grades being analyzed and a lso to c re a te la r g e r samples (n) f o r d a ta a n a ly s is. The methods u tiliz e d to determ ine item q u a lity were as follow s: (a) th e index o f d if f ic u l ty (percentage c o rre c t) f o r th e r e f e r r a l groups broken in to i t s th re e major d iv is io n s, (b) th e index of d i f f i c u l t y (p e r centage c o rre c t) f o r th e n o n -re fe rra l group broken in to i t s th re e major d iv is io n s, and (c) th e n et D index o f d iscrim in a tio n based on th e t o t a l group. When studying item d if f ic u l ty, each item was assigned to one o f th e th re e m ajor groups f o r a n a ly s is. I t s assignm ent was based on th e grade le v e l as designated by th e companion perform ance o b je c tiv e o r e sta b lish e d word l i s t. A comparison was made fo r each item in terms o f d i f f i c u l t y f o r th e r e f e r r a l and n o n -re fe rra l groups. Those item s w ith th e g re a te s t d is p a r ity were chosen. 18

26 19 Each of th e se item s were then analyzed by th e n et D method o f d is crim in a tio n to determ ine th e degree to which th e item s d iscrim in ated between th e to p 27% o f th e stu d en ts and th e bottom 27%. However, th e f a c t th e sample was h e av ily weighted w ith many young (p re k - l) r e f e r r a l stu d en ts and a lso a la rg e number of n o n -re fe rra l o ld e r (4-6) stu d en ts may have clouded th e n et D values (in a p o s itiv e d ir e c tio n ). N ev ertheless, n et D was s t i l l a u sefu l to o l in determ ining i f th o se stu d en ts who did w ill on th e t o t a l t e s t d id w ell on a p a r tic u la r item and i f th e poorer stu d en ts did poorly on th e same problem. The d a ta f o r determ ining item q u a lity fo r th o se a rith m e tic and read in g item s appearing on th e f in a l form of th e GREAT a re p resented in Table 2. The number assig n ed th e item in th e ta b le corresponds d ir e c tly to th e number of th e item on th e t e s t. TABLE 2 Item Q u ality Data f o r th e GREAT Item Number R e fe rra l (% c o rre c t) N on-r eferral (% c o rre c t) Net D (%) A rithm etic t e s t OO OO

27 20 TABLE 2 (continued) Item Number R e fe rra l (% c o rre c t) Non-RefeiTal {% c o rre c t) Net D {%) A rithm etic t e s t ? no a v a ila b le d a ta

28 21 TABLE 2 (continued) Item Number R e fe rra l {% c o rre c t) N on-r eferral (% c o rre c t) Net D {%) A rithm etic t e s t Reading t e s t

29 22 TABLE 2 ( c o n tin u a l) Item Number Referral {% correct) Non-Referral (% correct) Net D (%) R eading teat

30 23 TABLE 2 (continued) Item Number R e fe rra l (% c o rre c t) N on-r eferral (% c o rre c t) Net D (%) Reading te s t

31 2b TABLE 2 (continued) Item Number R e fe rra l {% c o rre c t) N on-r eferral {% c o rre c t) Net D (%) Reading te s t b b b b b

32 25 TABLE 2 (continued) Item Number Referral {% correct) Non-Referral (% correct) Net D {%) Reading test

33 26 TABLE 2 (continued) Item Number R e fe rra l {% c o rre c t) N on-referral (% c o rre c t) Net D (%) Reading te s t ? ? ? ? ? ? 0 3? ? 21 36

34 27 Although an attmopt was made to choose only th ose Items th a t d i s criminated between the referra l and non-referral group, arithm etic item 29 and reading items 4, 5, 6, 20, and 21 did not f u l f i l l th is requirement. N everth eless, th eso item s were placed on the t e s t because o f th e ir importance as concepts and a lso because the item d if f ic u lt y data was not f e l t to be tru ly rep resen tative. A lso, in looking a t Table 2, i t would appear that many o f the items for the non-referral group were ea sier than what i s usually recommended by such t e s t experts as Nunnally (1972, pp ) or Marshall and Hales (1971, pp ). I t must be remembered, however, th a t th ese percentages are r e fle c tin g the le v e l o f d if fic u lt y as demonstrated by one of the three grade d iv isio n s as opposed to the performance o f th e t o t a l group. The sequencing o f th e t e s t items was based on e ith e r th e le v e l of d if f ic u l ty fo r th e r e f e r r a l and n o n -re fe rra l groups or th e predeterm ined le v e l provided by th e perform ance o b je ctiv e and word l i s t s. Exceptions were made when economy o f space o r t e s t form at would not allow such sequencing. Both th e a rith m e tic and read in g p o rtions of th e GREAT begin w ith excessively easy item s to in su re immediate reinforcem ent to th e more immature stu d en t who is e ith e r academ ically re ta rd e d o r ch ro n o lo g ically young. The a rith m e tic and read in g su b te sts a lso end w ith item s which a re normally found a t th e 7th le v e l so as to help in su re a complete measurement o f th e upper elem entary s tu d e n t's re p e rto ire. The form at o f th e f i n a l form GREAT is very s im ila r to th a t o f th e experim ental form w ith th e number o f t o t a l item s being reduced to 171 from 481. Those item s which u su a lly involve th e younger stu d en t a re p rin te d in

35 prim ary s iz e, a m o d ificatio n which does not appear on th e WRAT. The word re c o g n itio n l i s t contain s a minimum o f 15 words a t each grade with th e exception of 1 s t grade, which contains 30* The WRAT can to a s t no more th an s ix o r seven words f o r many o f i t s grade le v e ls. A lso, many o f th e a rith m e tic item s appearing on th e t e s t a re not found on th e WRAT, g iv in g i t a re fre s h in g uniqueness and a broader d ia g n o stic scope from which to speak from. F in a lly, th e GREAT provides many o f th e d ire c tio n s to th e item s on th e p ro to c o l i t s e l f, an advantage over th e design o f th e WRAT which re q u ire s th e u ser to co n sta n tly r e f e r back to th e stan d ard ized in s tru c tio n s in th e manual (see Appendix C f o r a com plete copy o f th e f in a l form GREAT p ro to c o l). A d m in istratio n and Scoring o f th e F in a l Form o f th e GREAT The in s tru c tio n s f o r use with th e f i n a l form o f th e GREAT were designed w ith e ffic ie n c y and economy of expressio n in mind. The d ire c tio n s were s e r ia liz e d fo r two age le v e ls to d ecrease a d m in istra tio n tim e and a lso in c re a se th e understanding o f s e q u e n tia l step s from a tem poral p o in t o f view. Too o fte n, authors o f manuals w rite t h e i r in s tru c tio n s in th e n a rra tiv e -p a ra g ra p h e d ito r ia l s ty le which can be confusing. The in s tru c tio n s w ill be in corporated in to a manual when norming of th e GREAT i s completed (see Appendix D fo r a d m in istra tio n and sco rin g in s tr u c tio n s ). Performance O b je ctiv es GREAT Since th e w ritin g of b ehavioral o b je c tiv e s is a requirem ent w ith in sp e c ia l education classroom s, i t was decided th a t whenever p o ssib le, an o b je c tiv e should be provided fo r each item on th e GREAT. These o b je ctiv es were d eriv ed from e x is tin g o b jectiv es used a t th e tim e of th e item pool developm ent.

36 29 The number assigned to the following performance objectives (found to the left) corresponds directly to the number of the test item on the GREAT. For arithmetic, the number at the end of the objective is the companion objective number which can be located in the Mathematic Performance Objectives of the Grand Rapids Public Schools. Those objectives ending with the abbreviation n.a, (not available) do not have a companion objective. For reading, those items on the GREAT having corresponding companion objectives will end with the page number where the objective can be located in the Reading Performance Objectives of the Grand Rapids Public Schools. Those objectives ending with the abbreviation n.a. do not have a companion objective. Arithmetic. 1. The learner will place his/her hand over the head as requested, demonstrating an understanding of the concept "over", n.a. 2. The learner will place his/her hand under the table demonstrating an understanding of the concept "under". 3. Given a set of designs sequenced on paper, the learner will point to the one which is first, n.a. Given an assortment of shapes in sequence on paper, the learner will select a given shape as requested. 100, 5. Given an assortment of shapes in sequence on paper, the learner will select a given shape as requested. 100, 6. Given a set of objects on paper with 10 members, the learner will count the members of the set and state the cardinal number of that set. n.a. 7. Given a 2-digit numeral, the learner will identify the number verbally, n.a.

37 8. Given two 1-digit numerals separated by one missing digit, the learner will verbally supply the missing digit, n.a. 9. The learner will verbally supply the correct response when asked, "What coin is worth 10 pennies?", n.a. 10. Given a simple story problem involving money, the learner will be able to solve it (values to 100) Given the directions "Count beginning with 30 to 40" the learner will recite the number names from 30 to 40 in the usual order, n.a. 12. The learner will name orally the days of the week. 93* 13. Given a clock face with the minute hand at 12:00, the learner will state orally the time to the nearest indicated hour. l60, 14. Given a clock face, the learner will tell time orally in half- hour intervals. 3U8, 15. Given a counting sequence of two to four members involving multiples of two, five, and ten, the learner will tell the next number in the sequence. 190o 16. The learner will identify the answer in an addition equation as the sum The learner will state the number of feet in a yard upon request Given a column addition exercise involving two 1-digit addends, the learner will find the sum with or without the aid of counters, n.a. 19. Given a column addition exercise involving two 1-digit addends, the learner will find the sum with or without the aid of counters, n.a. 20. Given a horizontal addition exercise involving two 1-digit addends, the learner will find the sum with or without the aid of counters, n.a.