Sesso TA LEARNING ACHIEVEMENT EVALUATION STRATEGY USING FUZZY MEMBERSHI FUNCTION Sughyu Weo ad Jl Km Abstract I ths paper, we suggest a ew learg achevemet evaluato strategy studet's learg procedure. We call ths as fuzzy evaluato. We may assg fuzzy lgual varables to each questo pertag to ts mportace, complexty ad dffculty by usg fuzzy membershp fuctos. The we ca evaluate a score depedg o the membershp degree of ucertaty factors each questo. I addto, we cosder the tme cosumg elemet for solvg a questo. We adapt verse sgmod fucto to cosder tme cosumg elemet, fuzzy cocetrato ad dlato fucto for mportace, a sgmod fucto for complexty, ad fuzzy square method for dffculty. Idex Terms respose accuracy, fuzzy theory, CAI, learg achevemet evaluato, fuzzy membershp grade INTRODUCTION Wth the mared-dow CU prce ad the hghly-developed computer producg techology, the umber of persoal computers(cs) has bee o the costat rse. Ad due to the developmet of multmeda computg techology, the sgle meda -based Computer Asssted Istructo(CAI) system has bee furthered to a mult meda -based CAI system. CAI has bee studed from the early 970s, ad wth the recet assstace of Artfcal Itellgece(AI), the Itellget CAI(ICAI) has become a learers' auxlary study ad[6]. The followg are the four major CAI compoets: doma expertse module, studet model module, tutorg module, ad terface module, all of whch have structured sub-fuctos. Of them the evaluato fucto doma expertse module has a specal mportace, because a proper evaluato has to be made to set a approprate learercetered strategy fosterg the best learg evromet[]. Havg a procedural coveece, the covetoal learg achevemet evaluato method, usg the gradg system (A, B, C, etc), has bee commoly practced spte of ts drawbac the huma teachers dfferet evaluato crtera. However, there has bee a lot of tal about a ew evaluato system at elemetary schools, for t ams at a ehaced evaluato system whch ways of lgustc expressos are prced. Adoptg the Zadeh ad Yager s fuzzy theores[8]-[9], ths study ams to propose a relable fuzzy evaluato method whch ca mae up for the covetoal approaches shortcomgs: these methods barely cosder such vague factors as complexty, mportace, ad dffculty. To ths am, ths study comes up wth the evaluato module, settg a evaluato formula for whch fuzzy membershp fucto s used. Ths module evaluates the learg achevemets automatcally, thus gvg the cocurret evaluato s to studets. INTELLECTUAL EVALUATION Cocepts of Itellectual Evaluato Learg achevemet evaluato dcates how much a learer acheves by a set stadard. Geerally, learg achevemet evaluato ca be defed as a composte ad herarchcal process where each elemet has sub-elemets. I ths study, f E stads for learg achevemet evaluato, the evaluato fucto E of the subevaluato elemets () becomes φ () []. E,,, ) ( φ () Ad evaluato elemets () of E equal ψ ( j ) of the sub-evaluato elemets j (jm). ψ (,,, m ) () Therefore, evaluato seres of E are as follows: E (V ) φ ), φ( ),, φ( ) () ( φ ( ) φ ( ), φ ( ),, φ ( m ) () A teacher frst puts each studet the GOOD or BAD categores, where further there are such sub-evaluato compoets as complexty(comlex ad SIMLE); mportace(imortant ad NOT IMORTANT); ad dffculty(difficult ad EA SY). These sub-compoets (complexty, mportace, ad dffculty) are set as lgustc Sughyu Weo, Catholc Uversty of usa, School of Iformato Egeerg, usa, Korea, 609-77 shwo@cup.ac.r Jl Km, Dogeu Uversty, Departmet of Computer Egeerg, usa, Korea, 6-7 jm@dogeu.ac.r 0-780-6669-7/0/$0.00 00 IEEE October 0 -, 00 Reo, NV st ASEE/IEEE Froters Educato Coferece TA-9
Sesso TA varables, ad at the same tme IMORTANT, MEDIUM, EASY are set to be fuzzy varables. Allottg fuzzy varables to gve lgual varables, ths ew evaluato method maes up for the covetoal formulas ad rules. recedet Studes Focusg o the prevously-metoed compoets: mportace ad complexty, Weo led a studous research o a relable fuzzy evaluato method, whch has a drawbac that caot tegrate each depedet fuzzy evromet[7]. Law also proposed the learg gradg method usg fuzzy umbers, yet ths demads complex mathematcal process, or does t tegrate varous fuzzy evromets, ether[]. Ktaga, Japa, s a propoet of fuzzy evaluato method[]. Nagaoa proposed a fuzzy evaluato method usg a matrx system, whch aalyzes each studet s respose tme[]. Hajme later suggested fuzzy evaluato method that has subjectve evaluato elemets le callgraphy[]. These methods, however, eded up at a expermetal stage or could ot tegrate varous fuzzy evromets. As a cosequece, the developmet of a ew evaluato module that ca reflect the demad of the educatoal feld has bee ecessary. FUZZY EVALUATION METHOD Computato of Respose Accuracy Respose Accuracy wth Lmted Tme oly Assume that a seres of questos, related to a questo doma, are gve le formula (), ad each questo has several sub-questos le formula (6). The the j respose accuracy COR( ) of these s defed by the followg formula (7)., (),, (where, s questo doma, ad, s ay questo ),, m (6) m {, ( µ µ j T )} (7) j j (where, µ s membershp grade that jth sub questo of th problem s j correct or ot (correct :, correct : 0), s algebrac sum, s set, s algebrac product, µ s membershp grade of tme that s eeded solvg T j the problem j.) µ T s also computed by verse sgmod fucto j formula (8). : v α v α ( ) : α < v β (8) γ α µ T j v γ ( ) : β v < γ γ α 0 : v γ (where, v s questo solvg tme for, α s permtted lower lmt j tme ad γ s permtted upper lmt tme for questo solvg, α + γ β.) Respose Accuracy Cosdered Importace Importace s a crtero that dcates how much a studet ca uderstad the questo doma. If a studet wors out a questo that has a relatvely hgh mportace: IMORTANT, the respose accuracy weght factor creases; lewse f a studet solves a relatvely low mportace: NOT IMORTANT, the respose accuracy weght factor decreases; ad last f a studet solves a problem whose mportace s average: MEDIUM, the tal respose accuracy weght factor s mataed. To crease the weght factor, the fuzzy dlato method has bee adopted, ad to decrease t, fuzzy cocetrato method s used. I ths way, assume that the respose accuracy s cosdered, the formula (9) shows that ICOR, a accuracy computato formula, ca be gaed as follows: ICOR( ) (, COR( m {, j ( µ,, (, COR( µ j T )} (9) j (where, s weghed factor, IMORTANT: 0., MEDIUM:, NOT IMORTANT:) Respose Accuracy Cosdered Complexty It s dcated that f a questo grouped uder a COMLEX category s gve, regardless of studets capabltes, ot may of them gve a correct aswer a short tme perod, whle they sort that out whe gve eough tme, though. Because tme permsso s a very mportat elemet ths codto, complexty-related questos are solved wth the tme lmt adjusted properly. Frst, let us brea dow the complexty-related codtos to the followg three categores: COMLEX, MEDIUM, ad SIMLE, the we get a stadard devato σ, the tme dfferece that each studet speds aswerg the gve questos. If a questo s COMLEX, you may crease the tme lmt as much as the stadard devato σ ; lewse f a questo s SIMLE, decrease the tme lmt as much as σ ; ad f MEDIUM, mata the tal lmt tme. I ths way, whe the complexty s cosdered, formula (0) shows that the computato formula respose accuracy CCOR ca be gaed as follows: 0-780-6669-7/0/$0.00 00 IEEE October 0 -, 00 Reo, NV st ASEE/IEEE Froters Educato Coferece TA-0
Sesso TA CCOR( ) (, COR( m {, ( j j T j,, (, COR( µ µ ' )} (0) I ths case, (). µ ca be gaed as the followg formula T ' j : v α v α ( ) : α < v β' γ ' α () µ T' j v γ ' ( ) : β ' v < γ ' γ ' α 0: v γ ' (where, v s questo solvg tme for, α s permtted lower lmtted tme for questo solvg, j γ ' γ + σ, α + γ ' β '.) Respose Accuracy Cosdered Dffculty NORM(DCOR( DCOR( )/m ; (6) NORM(COR( COR( )/(m) (7) NORM(ICOR( ICOR( )/(m) (8) NORM(CCOR( CCOR( )/(m) (9) NORM(DCOR( DCOR( )/(m) (0) (where, s umber of questos, m s umber of sub questos.) Evaluato Fuctos To evaluate the learg achevemet through the respose accuracy ad ormalzed values, the followg fuzzy membershp fucto has to be gaed: 'VERY GOOD' x ; x 'GOOD' x ; 0 < x < 'MEDIUM' x ; 0 < x 0. -x + ; 0. x < 'BAD' - x + ; 0 < x < 'VERY BAD' (- x + ) ; x 0 Tradtoally, to wor out the problem o dffculty, adopted has bee gvg dfferet scores to gve questos. Ths approach, however, has proved to be rrelevat because all the matters cocered are fxed beforehad regardless of studets aswerg propesty, or ca t reflect fuzzy attrbutes exstg the cocept of dffculty. Itroducg dlato operator method fuzzy theores, ths paper proposes a method that gves relatvely dffcult questos to studets who get hgher scores. Formula () s a computato formula to the respose accuracy DCOR whe dffculty s cosdered. DCOR( ) (, COR( m {, ( j j h T j,, (, COR( µ µ )} () (where, h s weghted factor. (Easy :, Medum : 0., Dffcult : 0. Evaluato Fucto ad Fuzzy Evaluato Normalzato The prevously-gaed respose accuracy falls wth the lmt [0, m] because t has m umbers of sub-questos to each questo. Formulas ()(6) are a ormalzato process to the above doma, ad formulas ()(8) are to the whole questo doma. NORM(COR( COR( )/m ; () NORM(ICOR( ICOR( )/m ; () NORM(CCOR( CCOR( )/m ; () Itellectual Evaluato After the respose accuracy s computed by the formulas (7), (9), (0), ad (), each questo s ormalzed by the formulas (7)(0), ad the ormalzed fal respose accuracy from the above s s lgustcally evaluated by oe of the prevously gaed fuzzy membershp fuctos. The ormalzed respose accuracy here ca belog to more tha two fuctos out of the possble fve fuzzy membershp fuctos. I ths case, ths belogs to the fucto that has the maxmum membershp value, ad s evaluated as a sutable fuzzy lgustc varable for that gaed fucto. Formulas ()() show ths process. EVAL( ) {(, NORM ( COR( 0-780-6669-7/0/$0.00 00 IEEE October 0 -, 00 Reo, NV st ASEE/IEEE Froters Educato Coferece TA- { j NORM ( COR( } () (where, FUZSET j(max( ) s fuzzy set that showed maxmum membershp grade whe ormalzed value NORM(COR( of accuracy COR( ) of problem s beloged to several fuzzy sets.) EVAL( ) {(, NORM ( ICOR( { j NORM( ICOR( } EVAL( ) {(, NORM ( CCOR( () ()
Sesso TA { j NORM ( CCOR ( } EVAL( ) {(, NORM ( DCOR( { j NORM ( DCOR( } SIMULATION AND DISCUSSION Smulato Objects () Questo TABLE III TIME ALLOWED Ut : sec, [lower lmt, upper lmt] [,] [,] [,] [,] [,] [,] [,] [,] [,] [,6] [,0] [,0] [,0] [,] [,0] [,0] [,0] [7,] [7,] [0,] [0,0] [0,0] [0,0] [0,0] [0,] Smulato To evaluate a studet's learg achevemet uder the suggested fuzzy evromet as metoed above, ths study prepared the followg evaluato questoare. A elemetary school's fourth grade teacher advsed preparg them, each of whch has complexty, mportace, ad dffculty. Uder the fve questo types, there are fve subquestos; subsequetly, the questoare has questos total. The followg TABLE I s a evaluato questoare preseted to studets. Questo TABLE I EVALUATION QUESTIONS ++7 ++7 ++7 0++7 0+0+7 8-- 8-- 8-- 80-- 8-- 7 7 7 6 60 6 7 8- +0-0 7-6 6 07 ( 6) 7+9 00 7 7 78 Each questo cludes the respectve fuzzy elemets, whch are complexty, mportace, ad dffculty. The followg TABLE II demostrates the s whe the fuzzy attrbutes are gve to each questo, ad the TABLE III shows the s whe the low ad upper tme lmt s gve worg out all the questos. Questo TABLE II FUZZY ATTRIBUTES TO EACH QUESTION X Y Z X Y Z X Y Z X Y Z X Y Z S N E S M E S M E S M E S M E S N E S M E S M E S M E S M E S N M M M M M M M M M M C M D S N M M M M S M M M I M C M D S I D C M D C M D C M D C M D (where, X s complexty, S s SIMLE, M s MEDIUM, C s COMLEX, Y s mportace, N s NOT IMORTANT, I s IMORTANT, Z s dffculty, E s EASY, D s DIFFICULTY.) Results Whe the Tme Allowed s Cosdered I ths codto, the respose accuracy for the s s gaed by the formula (7). Let us loo to the sxth studet's (S6) out of the te. Questo TABLE IV RESONSE ACCURACY(S6) 0.778 0.778 0.778 0.778 0.87 0. 0. 0. 0 0.67 0.0 0. 0.7 0. 0.68 0.969 0.969 0.9 0. 0.08 0.0 0.0 0.08 The followg lst demostrates the respose accuracy of S6 o the questo doma, ad TABLE V shows the s whe hs tellectual evaluato s made o the bass of the gve. COR( (S6 (,.),(,.87),(,0.706),(,.88),(,0.) TABLE V EVALUATION RESULTS WHEN THE TIME CONSIDERED Normal accura Normalze accura zed cy d value cy value. 0.8 GOOD.87 0.7 MEDIUM 0.706 0. BAD.88 0.776 GOOD 0. 0. BAD.0 0. MEDIUM Results Whe Importace s Cosdered Based o the formula (9), whe mportace s cosdered, the respose accuracy s as follows: 0-780-6669-7/0/$0.00 00 IEEE October 0 -, 00 Reo, NV st ASEE/IEEE Froters Educato Coferece TA-
Sesso TA TABLE VI RESONSE ACCURACY OF S6 WHEN IMORTANCE IS CONSIDERED Questo 0.60 0.778 0.778 0.778 0.766 0. 0. 0. 0 0.67 0.0 0. 0.7 0.0 0.68 0.969 0.98 0.9 0.67 0.08 0.0 0.0 0.08 The followg lst s the respose accuracy of S6 o the questo doma. ICOR( (S6(,.99),(,.766),(,0.706),(,.66),(,0.77) TABLE VII shows the s whe hs tellectual evaluato s made o the bass of the gve. TABLE VII EVALUATION OF S6 WHEN IMORTANCE IS CONSIDERED accura Normalz accurac Normalz cy ed value y ed value.99 0.7879 GOOD.766 0. MEDIUM 0.706 0. BAD.66 0.7 GOOD 0.77 0.9 BAD 0.8 0. MEDIUM Results Whe Complexty s Cosdered Usg the formula (0), o the bass of complexty, let us compute the respose accuracy to the gve s. TABLE VIII shows the stadard tme devatos whe the 0 studets are gve the test. Questo TABLE VIII STANDARD TIME DEVIATIONS 0 0. 0. 0.6 0.9 0. 0.8 0.9 0.9.0 0.66.8..6..6.6.6.8.7.8...09.08 Questo TABLE IX ADJUSTED LOWER AND UER TIME LIMIT Ut : sec., [lower lmt, upper lmt] [,] [,.6] [,.6] [,.] [,.] [,.6] [,.7] [,.08] [,.06] [,.96] [,9.] [,0] [,0] [,] [,.] [,8.8] [,0] [7,.7] [7,] [0,7.7] [0,7.8] [0,.] [0,.] [0,.09] [0,9.08] TABLE X RESONSE ACCURACY(S6) Questo 0.778 0.70 0.69 0.68 0.8 0.7 0.6 0 0 0.67 0.0 0. 0. 0.0 0.68 0.9 0.969 0.96 0. 0. 0.6 0.6 0.6 Usg the membershp grade of adjusted tme TABLE IX, TABLE X demostrates the respose accuracy. The followg lst s the respose accuracy of S6 o the questo doma, ad TABLE XI shows the s whe hs tellectual evaluato s made o the bass of the gve. CCOR( (S6 (,.8),(,.9),(,0.867),(,.8),(,.) TABLE XI RESULT S WHEN COMLEXITY IS CONSIDERED accura Normalz accura Normalz cy ed value cy ed value.8 0.77 GOOD.9 0.7 BAD 0.867 0.7 BAD.8 0.77 BAD. 0. BAD Results Whe Dffculty s Cosdered 0.869 0. MEDIUM Based o the formula (), whe dffculty s cosdered, the respose accuracy s as follows: O the bass of TABLE VIII the tme terval lst (TABLE IX) s gve as follows: 0-780-6669-7/0/$0.00 00 IEEE October 0 -, 00 Reo, NV st ASEE/IEEE Froters Educato Coferece TA-
Sesso TA Questo TABLE XII RESONSE ACCURACY(S6) 0.778 0.778 0.778 0.778 0.87 0. 0. 0. 0 0.606 0.0 0. 0.6 0.66 0.8 0.98 0.98 0.99 0.7 0. 0.76 0.76 0. The followg lst s the respose accuracy of S6 o the questo doma, ad TABLE XIII shows the s whe hs tellectual evaluato s made o the bass of the gve. DCOR( (S6 (,.),(,.87),(,.78),(,.8),(,.9) TABLE XIII RESULTS WHEN DIFFICULTY IS CONSIDERED accuracy Normalz accura Normalze ed value cy d value. 0.8 GOOD.87 0.7 MEDIUM.78 0.6 BAD.8 0.86 GOOD.9 0.8 MEDIUM Dscusso.07 0.6 MEDIUM Ths study suggests a ew method that ca get approprate evaluato s from the studets respose. To ths am, ths study uses such fuzzy attrbutes as mportace, complexty, ad dffculty as well as mag the sutable formulas. Ths study has foud the fact that the ew method ths study proposes ca get more flexble evaluato s tha the covetoal methods have doe. TABLE XIV compares a covetoal method wth the proposed method o S6. TABLE XIV COMARISON WITH TRADITIONAL EVALUATION RESULT (S6) Arthmetc score Evaluato wth fuzzy attrbutes Tradtoal method 88 pot B(?) Fuzzy method 0. MEDIUM the proposed method wll be oe of the most approprate evaluato methods accordace wth the recet evaluato propesty. CONCLUSION Ths paper suggests a fuzzy evaluato method sutable for varous fuzzy evromets, leadg a research o the th graders of a elemetary school. Varous fuzzy evromets, as a, are reflected ths expermet to evaluate studets just le a huma teacher does tutvely. The lgual s ths method gves to studets, GOOD or BAD, are useful because t gves more flexble evaluatos to studets. Ths study, accordgly, s of a great educatoal mportace that a ew evaluato stadard s offered to the educatoal feld. Addtoally, ths method has set a steppgstoe leadg to a ehaced evaluato stage. Through the expermet, foud s the fact that tegratg fuzzy attrbutes s easer tha expectedthe attrbutes are tegrated easly whe they go through a smple operato whch tegrates each attrbute. Ths study, however, has a shortcomg that heavly depeds o the studet's respose tme. Afterwards, further studes have to be made to get more relable respose accuracy by aalyzg a studet's respose propesty more. REFERENCES [] Hajme, Y. ad Sect S., "A Educatoal Evaluato System Applg Fuzzy Theory", Tech. Report of JIEICE ET9-(99-0), 99, pp. 9-. [] Kearsley, G., "Artfcal Itellgece ad Istructo : Applcato ad Method", Addso Wesley ublshg Compay, 987. [] Ktaga, I., "Developmet of a Fuzzy Evaluato System", Joural of JIEICE, D-, Vol. J7-D-, No., 99, pp. 0-08. [] Law, C. K., "Usg Fuzzy Numbers Educatoal Gradg System", Fuzzy Sets ad Systems, Vol. 8, No., 996, pp. -. [] Nagaoa, K. ad Wu, Y., "A Aalyss of Learg Respose Tme Matrx Based o Fuzzy Trasformato", Joural of JIEICE, D-, Vol. J7- D-, No., 99, pp. 9-00. [6] Tosho, O., "The Curret Stuatos ad Future Drectos of Itellget CAI Research/Developmet", Joural of JIEICE, Vol. E77-D, No., 99, pp. 9-8. [7] Weo, S. H., S, D. H. ad Chug, H. M., "A Study o Educato Evaluato Method usg Fuzzy Theory", Joural of KFIS, Vol. 6, No., 996, pp. 7-8. [8] Yager, R. R., "A rocedure for Orderg Fuzzy Subsets of the Ut Iterval", Iformato Scece, Vol., 98, pp. -6. [9] Zadeh, L. A., "Fuzzy Sets", Iformato & Cotrol, Vol. 8, 96, pp. 8-. The ultmate objectve of the tradtoal evaluato methods has bee to get exact arthmetc scores as show o TABLE XIV. It oly grades the test s, for stace A or B, by a set stadard; however, the objectve of the proposed method s ot to get exact arthmetc scores. Rather, t ams to come up wth flexble evaluato s, whch are GOOD or BAD. Therefore, ths study has a strog belef that 0-780-6669-7/0/$0.00 00 IEEE October 0 -, 00 Reo, NV st ASEE/IEEE Froters Educato Coferece TA-