AVERAGE MARKS SCALING

Transcription

1 TERTIARY INSTITUTIONS SERVICE CENTRE Level 1, 100 Royal Street East Perth, Wester Australia 6004 Telephoe (08) Facsiile (08) Itroductio AVERAGE MARKS SCALING I Wester Australia a wide rage of courses with a exteral exaiatio take at the ed of Year 1 are available for seior secodary studets The exaiatio results are the cobied with school assessets This process ivolves oderatio of school assessets usig the raw exaiatio arks Withi each course, the cobied arks are stadardised 1 Thus, for each studet a coposite uscaled ark is geerated for each course take However, the cohort of studets takig a particular course P ay be acadeically ore able tha the cohort of studets takig course Q This eas that a coposite uscaled ark of, say, 60 i course P represets a higher level of attaiet tha the sae ark i course Q Sice arks i differet courses are added to for a Tertiary Etrace Aggregate (TEA), equity cosideratios require that the arks i courses P ad Q should be scaled This eas that the arks i course P should be scaled up relative to the arks i course Q (or the arks i course Q scaled dow) Of course, the situatio is ore coplex tha just establishig the appropriate relativity of the arks i courses P ad Q I fact there are ay courses that studets ca take ad so it is the relativities of all courses which scalig eeds to address The key cocept is to geerate a easure of acadeic ability, based o studets achieveets, for the cohort of studets takig a particular course The scalig ethod used i Wester Australia 3 is based o the preise that the best easure of a idividual studet s acadeic perforace is that studet s average scaled score across all courses take Suppose that average is deoted by t i That is to say, for a particular studet i defie t i to be the average scaled score obtaied by studet i across all courses take by studet i 1 Prior to 016, the process ivolved oderatig the school assessets agaist stadardised exaiatio arks This is ofte referred to as the fial cobied ark 3 Average Marks Scalig has bee used i New South Wales for ay years The ai architect of that syste was Professor E Seata, Departet of Matheatical Statistics, Uiversity of Sydey The New South Wales syste was adapted for use i Wester Australia by Dr MT Partis, Director, Secodary Educatio Authority, i 1997 ad itroduced i 1998 Prior to that the Australia Scalig Test was used as the achor variable i the scalig process Now cosider the cohort of studets takig a particular course j For each of these studets there will be a average scaled score t i The average of the t i across all the studets takig course j will be deoted by Updated July Curti Uiversity of Techology Edith Cowa Uiversity Murdoch Uiversity The Uiversity of Wester Australia

2 That is to say, for a particular course j defie to be the average of the average scaled scores t i obtaied by all studets takig course j The Average Marks Scalig (AMS) process uses as a proxy easure 4 of the acadeic ability of the cohort of studets takig course j At the ed of the process the value of T P for course P ad the value of T Q for course Q ca be calculated The ea 5 ark for the cohort of studets takig course P is the scaled to T P, whilst the ea ark for the cohort of studets takig course Q is scaled to T Q This gives the appropriate relativity betwee the two courses Adjustets are also ade to the stadard deviatios of the course distributios, but these tur out to be ior copared to the adjustets to the eas It is worth stressig the ai feature of Average Marks Scalig The descriptio above sees to suggest that the scaled scores eed to be kow i order for scalig to be carried out However, at the heart of the process is the equatig of the average scaled score i course j ad the achor variable The atheatics ivolved is set out below, but the key equatio Average scaled score i course j of the cohort of studets takig course j The achor variable for the course j (1) provides the cetral focus of the aalysis To clarify the cocepts ivolved it is worth cosiderig a uerical exaple Suppose that studet i has a scaled score of 63 i course j I what follows this scaled score will be deoted by y That is to say, y 63 Suppose that studet i takes four other courses obtaiig scaled scores i those of 47, 71, 58 ad 66 The the average of this studet s five scaled scores will be 61 That is to say, t i 61 It is iportat to ote that, i this istace, the values of y ad t i are ot the sae This is because they are easurig differet thigs y is easurig the studet s perforace i course j, whilst t i is easurig the studet s perforace across five courses 4 is soeties described as the achor variable for the scalig process 5 Throughout this docuet the ters ea ad average will be regarded as syoyous Now cosider the cohort of all studets takig course j For each of these studets there will be correspodig values of y ad t i Hece, the average values of y ad t i for the whole cohort ca be Updated July 015 Curti Uiversity of Techology Edith Cowa Uiversity Murdoch Uiversity The Uiversity of Wester Australia

3 deteried The scalig process is desiged to ake these averages idetical I siple ters the uscaled arks i course j are oved up or dow to achieve this outcoe The adjustet of the coposite uscaled arks to scaled scores uses a liear coversio, the details of which are explaied below It is iportat to ote the followig poits: the AMS scalig process preserves the rakig of studets ad the shape of the distributio i each course; the ea scaled score across all courses ad all studets (the global ea) is predeteried: set at 60 the stadard deviatio paraeter for the AMS process is predeteried: set at 14 Re-stadardisatio usig z-scores Although exaiatio arks ad school assessets are stadardised at a earlier stage of the process, it is atheatically coveiet to stadardise agai i such a way that the coposite uscaled ark distributio for each course has a ea of 0 ad a stadard deviatio of 1 Let w be the coposite uscaled ark for studet i i course j For each course j let µ j ad σ j be the ea ad stadard deviatio of the w Now put w µ σ j The values geerated i this way are ofte referred to as z -scores By defiitio it follows that the z - scores for each course will have a ea of 0 ad a stadard deviatio of 1 The AMS process uses the values defied above to geerate scaled scores y The coversio to scaled scores ca be regarded as a three-part process First, add 60 to restore the overall ark distributio to the predeteried global ea Secod, add a ter d j for each course j which deteries whether the arks i that particular course are scaled up or dow relative to the global ea This eas that soe d j will be positive ad soe egative Third, add a ter 14 c j, which alters the stadard deviatio for each course fro 1 to 14 c j The value of 14 is predeteried to produce a appropriate global stadard deviatio The c j will be calculated for each course j, but i practice the values tur out to be always close to 1 The cobiatio of the three steps outlied above gives rise to the followig equatio: The scaled score y for studet i i course j is give by y () Updated July Curti Uiversity of Techology Edith Cowa Uiversity Murdoch Uiversity The Uiversity of Wester Australia

4 The paraeters d j ad c j eed to be evaluated for each course j It should be ephasised that whilst equatio () gives a algebraic defiitio of the scaled scores y, the arithetical values of the y ca oly be calculated after the paraeters d j ad c j have bee deteried The way i which the paraeters d j ad c j are evaluated is set out below 3 Calculatig averages I this sectio the techical details of workig out the average of the average scaled scores, that is to say for each course j, are developed For this purpose it is useful to itroduce a fuctio which depeds o whether a particular studet is takig a course or ot Defie 1 if studet i takes course j; 0 if studet i does ot take course j Let be the total uber of studets takig the exaiatios 6 Let be the total uber of courses available i the exaiatios The uber of studets j takig course j is the give by j The uber of courses i take by studet i is give by i j1 For a particular studet i the average scaled score, deoted by t i, over all courses take by that studet is give by t i 1 i α ik y ik 6 I practice a subset of the total uber of studets, kow as the scalig populatio, is used The itetio is to exclude, for exaple, studets takig oly oe course For all the studets takig a particular course j the average of the average scaled scores, deoted by 1 j t i 1 1 α j i α ik y ik Updated July Curti Uiversity of Techology Edith Cowa Uiversity Murdoch Uiversity The Uiversity of Wester Australia

5 1 1 α α j ik ( 60 + d k +14c k z ik ) i The sigificace of t i ad was explaied i 1 is the achor variable for the scalig process 4 Matrix represetatio I 3 the forula for was derived This geerates equatios, correspodig to the values of j ruig fro 1 through to The ext stage i the process is to recast these equatios i a atrix forat Fro the previous sectio it follows that where This leads to the atrix equatio ( 60b jk + d k b jk +14c k a jk ) b jk 1 α ik ad a jk 1 α ik z ik j i j i T ~ 60B 1 ~ + B d ~ +14A c ~ (3) where T ~ is the 1 colu vector [ ]; A ad B are the atrices [ a jk ] ad [ b jk ], respectively; 1 ~ is the 1 colu vector with each etry equal to 1; ad c ~ ad d ~ are the 1 colu vectors [ c j ] ad [ d j ], respectively Aalysis of the b jk which are the eleets of atrix B the gives B1 ~ 1 ~ Hece, equatio (3) siplifies to T ~ 601 ~ + B d ~ +14A c ~ (4) 5 Average scaled score i course j Fro equatio () the scaled score y for studet i i course j is give by Updated July Curti Uiversity of Techology Edith Cowa Uiversity Murdoch Uiversity The Uiversity of Wester Australia

6 y Hece, for a particular course j, the average of the y is give by 1 j y 1 α (60 + d j ) j 1 j sice the z -scores for course j have a ea of 0 Fro equatio (1) this gives Hece, T ~ 60 1 ~ + d ~ (5) Fro equatios (4) ad (5) it follows that 601 ~ + d ~ 601 ~ + B d ~ +14A c ~ Siplifyig this atrix equatio gives (I B) d ~ 14A c ~ (6) where I is the idetity atrix 6 Calculatio of the key paraeters I atrix equatio (6) the oly ukows are the colu vectors c ~ ad d ~ The c j etries which ake up the colu vector c ~ ca be thought of as the stadard deviatios for each course j This ca be evaluated by cosiderig the z -scores for all studets takig course j The stadard deviatio c j for the z -scores i course j is give by a coplicated (but stadard) forula, aely c j 1 α z ik ik α 1 α ik z ik α j α j ik α ik Updated July Curti Uiversity of Techology Edith Cowa Uiversity Murdoch Uiversity The Uiversity of Wester Australia

7 The arithetical values for the c j ca ow be substituted ito equatio (6) Solvig equatio (6) gives the values of d j Equatio () ow allows the scaled scores y to be calculated for all studets i all courses 6 Exaple showig how a studet s scaled score is calculated Cosider a studet whose uscaled cobied ark i course j is 6573 For the cohort of studets takig course j suppose that the ea ad stadard deviatio of the uscaled cobied arks are 5951 ad 10, respectively Now derive the z -score correspodig to the studet s uscaled ark of 6573 This is z w µ j σ j For course j suppose that the scalig paraeters tur out to be c j 10 ad d j 5 The equatio () gives the scaled score y as y (10)(051) 750 Updated July Curti Uiversity of Techology Edith Cowa Uiversity Murdoch Uiversity The Uiversity of Wester Australia