Modeling Multiple Subskills by Extending Knowledge Tracing Model Using Logistic Regression

Transcription

1 Modelng Mulple s by Exendng racng Model Usng Logsc Regresson Xuan Zhou Sae Key Lab of Sofware Developen Envronen Deparen of Copuer Scence and Engneerng, Behang Unversy Bejng, Chna zhouxuan@nlsde.buaa.edu.cn Wenjun Wu, Yong Han Sae Key Lab of Sofware Developen Envronen Deparen of Copuer Scence and Engneerng, Behang Unversy Bejng, Chna {wwj, hanyong}@nlsde.buaa.edu.cn Absrac racng (K) s a sandard odel for nferrng suden knowledge asery fro her perforance daa. Generally, here are fve paraeers n K odel we need o esae, hey are ranson paraeers: learnng and forgeng probably, esson paraeers: guessng and slppng probably, and pror probably. K odel s wdely used n sudens learnng oucoe odelng. However, does no suppor ulple subsklls odelng, for he odel works by checkng he hsorcal observaons a a specfc skll. o overcoe hs drawback, hs paper proposes hree odels usng logsc regresson over each sep: KLR-GS, exendng guessng and slppng paraeers; KLR-LFID, nroducng e dffcul no he K odel whle exendng learnng and forgeng paraeers; KLR-FP, exendng boh ranson and esson paraeers. Unlke prevous ehods our odels dscuss he effcency of exendng ranson and esson paraeers whle relaxng he assupons ha subsklls are ndependen. In he end we evaluae how well our odels perfor wh coparson o LR-DBN, K-IDEM and K on wo daases: he open daase ASSISens, and one prvae daase, suden algebra daase colleced hrough our uorng syse. Our odels ouperfor LR-DBN, K-IDEM and K on boh daases. Keywords: knowledge racng; logsc regresson; ulple subsklls. I. INRODUCION Varous knds of e-learnng syses, such as Massvely Open Onlne Courses[] and nellgen uorng syses, are now producng large aouns of feaure-rch daa fro sudens solvng es a dfferen levels of profcency over e. o analyze such educaonal bg daa, researchers ofen use racng Model [2] (K), a 20-year-old ehod ha had becoe he de-faco sandard for nferrng suden knowledge asery hrough her perforance on exercse es. here are fve paraeers n K odel, he ranssson paraeers learnng and forgeng probably, he esson paraeers guessng and slppng probably, and he pror probably ha a suden already know he skll. Unforunaely, he orgnal K odel assues each exercse e only nvolves sngle skll whou consderng laen ulple subsklls n suden hoework and assgnens. Prevous research has explored varous approaches o hs proble. Generally speakng, here are hree groups ncludng conjuncve odels, perforance facor analyss and logsc regresson based odels. Conjuncve odels assue ha a suden us aser all of he subsklls n order o perfor he sep correcly [3][4]. hey calculae he probably of knowng he all by ulplyng he probables of knowng he ndvdual subsklls. Learnng facor analyss odels capure he knowledge coponens of courses as feaures, ake he for of logsc regresson odellng suden perforance [0]. Logsc regresson based odels uses logsc regresson o exend paraeers n K odel separaely [3][4]. However, here are several weaknesses of he odels above. Conjuncve odels and learnng facor analyss odels don consder he e sequence of he quesons answered. Logsc regresson odels haven fully nvesgaed he effec of ranssson and esson paraeers n K odel. hs paper presens hree new ways o exend K odel usng logsc regresson o race ulple subsklls n a Bayesan nework whou assung hey are ndependen. hey are naed KLR-GS, KLR-LFID and KLR-FP. In KLR-GS, we exend he esson paraeers n logsc regresson. In KLR-LFID we exend he ranssson paraeers n logsc regresson, n addon, we ndvdualze he esson paraeer wh e dffculy. KLR-FP cobnes he frs wo odels exendng ranssson and esson paraeers. We have conduced exensve experens o evaluae he perforance of hese odels and copare he wh prevous research odels, especally hose ha also adop logsc regresson based subskll odelng. Experenal analyc resuls confr he perforance proveen of our odels. Furherore, he dealed analyss of he paraeer esaon process based on Expecaon Maxzaon algorh for our odels ndcaes ha he rgh sraegy o choose he odels for dfferen daases. he res of hs paper s organzed as follows: Secon 2 nroduces he K odel, and curren research on ulple subsklls odelng. Secon 3 descrbes our hree odels. Secon 4 nroduces he daases we used n hs paper. Secon 5 repors he experenal resuls of our hree odels [4]; Secon 6 concludes our curren work and dscusses fuure work.

2 II. RELAED WORK A. racng In he Inellgen uorng Syse (IS) [6], here are a varey of ehods o odel sudens learnng oucoes. he racng Model [2] shown n Fgure has been wdely used. he sandard K odel has fve paraeers ha are learned fro daa for each gven skll. hese fve paraeers ndcae he odel s deduced probably wheher he suden has asered he skll gven hs chronologcal sequence of correc and ncorrec responses o a parcular se of quesons. P(G) and P(S) are wo paraeers ha deerne he suden s perforance on a queson based on hs curren asery of hs knowledge. P(G) s he guessng paraeer ha a suden wll answer correcly even f he doesn know hs skll o he queson. P(S) s he slppng paraeer ha a suden wll answer ncorrecly when he already knows hs skll. P() and P(F) paraeers dcae he change beween he saes of he suden knowledge nodes. P() s he learnng paraeer ndcang he probably of ransng fro no aserng o aserng he skll. P(F) s he forgeng rae descrbng he probably of forgeng hs skll ransng fro knowng o no knowng. P(L 0 ) s referred as he pror probably of he nal knowledge node where a suden knows he skll before answerng he frs queson. Snce he K odel has been proposed, has been broadly used o evaluae and predc suden knowledge asery. here are soe oher exensons of he K odel. For exaple, K-IDEM [23] adds he e dffculy whle keepng he odfcaon led o slgh changes o he opology of he K odel. However, when coes o ulple subsklls, K sn able o handle all he sklls sulaneously, because he K odel works by checkng he hsorcal observaons on a specfc skll. Many research effors have been done o address hs proble by ncorporang ulple sklls facors n her odels. hey can be roughly caegorzed no hree ajor groups: conjuncve odels, perforance facor analyss and logsc regresson based odels. Inal PL ( 0) P(G) P(S) K 0 C 0 P() P(F) K C Fgure he Sandard racng Model K n Cn B. Conjuncve Modelng he conjuncve odels, assue sudens us aser all he subsklls o have a grasp of one specfc skll. In [2], Gong proposes wo approaches o hs proble. he frs approach s nspred by he jon probably n Probably heory. I assues ha he laen subsklls for each exercse e are ndependen and descrbes he overall probably of knowng he rgh answer o he e as he ulplcaon of all he probables of every ndvdual subskll. However, such as a probably produc usually underesaes he probably of one suden knowng all he subsklls. he nu of he esaed probably provdes a less pesssc esae based on he assupon ha he lkelhood of a correc answer s donaed by he suden s weakes subskll. he oher approach proposed by Gong uses he nu of he probably of all he subsklls o represen he probably of aserng hs skll. However, when he dfference beween he nu and axu s very large, usng he nu o represen he suden s probably does no see convncng. Inspred by he Ie Response heory (IR) [7], Cen [4] odels he conjuncon as a ulplcaon of subsklls, n whch each subskll has a paraeer represenng s dffculy dscrnaon and oher paraeers represenng profcency of an suden. I s a specal case of Ebreson s ulcoponen laen ra odel [6] and cusozed for he hgh densonal feaure of nellgen uorng syse. However, does no race sklls over a e slce. C. Facor Analyss Facor Analyss (PFA) s an alernave way for suden skll odelng [7]. As a varan of learnng decoposon [8], PFA s bul on reconfgurng Learnng Facor Analyss (LFA) [9]. LFA s a general ehod for cognve odel ha capures he knowledge coponens n a currculu [0]. LFA has hree poran coponens ncludng a sascal odel ha deernes he sklls, he e dffculy ha ay affecs suden perforance, and a cobnaoral search ha perfors odel selecon whn he logsc regresson odel space []. he sascal odel akes he for of logsc regresson wh suden perforance as he dependen varable. Ie dffculy facors can denfy a propery of probles ha causes suden dffcules. Inspred by LFA, [4] proposes wo PFA odels, whch adop an addve odel nsead of conjuncve odel. I assues ha he probably of a suden answerng a queson correcly s proporonal o he aoun of all he requred knowledge he sudens knows plus he easness of he queson and he aoun of learnng ganed for each pracce opporuny. Whle he oher one cobned Ie Response heory and conjuncve odel, supposng he ulple subsklls odelng s a conjuncve odel, whch could perfor by ulplyng all he subskll paraeers. Pavlk [7] reconfgures LFA on odel s ndependen varables by droppng he suden varables and replacng he skll varable wh he quesons deny. I akes he for of sandard logsc regresson wh suden perforance as a dependen varable, and esaes a paraeer for each queson represenng he queson dffculy, and wo paraeers for each skll ndcang he effecs of he pror success and pror falures for ha skll [2]. However, when coes o ulple subsklls odelng, odels we descrbed above don consder he order of he quesons occurred.

3 D. Logsc Regresson based Modelng All he prevous odels gnore he fac ha when a suden answerng quesons, he us work ou n a specfc order. So he eporal learnng procedure s an poran facor for odelng suden perforance. Inspred by he logsc regresson over he learnng and forgeng probables, Gonzalez-Brenes [3] hnks dfferen feaures, such as subsklls, proble s dffculy, and suden ably ec, nfluence paraeers n K odel. Each paraeer could be expressed n logsc regresson wh he feaures we dscussed. Wh dfferen feaures, he K odel could be presened n dfferen ways. So he proposes a general ehod ha allows effcen general feaures no K odel naed Feaure-Aware racng (FAS). Unforunaely, n hs paper, Brenes only focuses on he guessng and slppng paraeers whou descrbng he exenson n he learnng and forgeng paraeers. Yanbo [4] resrucures he K odel usng logsc regresson over each sep s subsklls o odel he learnng and forgeng probables for overall knowledge requred by he sep, he odel s called LR-DBN. In hs paper Yanbo proves he equvalence of hs exenson and odelng he knowledge probables wh logsc regresson. LR-DBN s appled on he Projec LISEN Readng uor [5] durng school year. he resul shows a parcular effcency n he case of asks nvolvng ulple subsklls [6]. However, hs odel only exends he ranson paraeer, lackng he dscusson of esson paraeers. Neher of he above logsc regresson-based on odels have fully nvesgae wheher all he paraeers ncludng learnng rae, forgeng probably, guessng and slppng probably can be negraed n a sngle odel and how o une hese paraeers n dfferen subskll dsrbuons. In he nex secon, we nroduce a group of ul-subskll odels ha negrae all he four paraeers and e dffculy hrough logsc regresson n he K odels. III. EXENDING KNOWLEDGE RACING O MODEL MUIPLE SUBSKILLS WIH LOGISIC REGRESSION In he radonal K Model, we esae he paraeers dependng on he skll ha requres o perfor. We use laen varable o odel he hdden knowledge sae ha changes over seps, and nfer he sae fro sequenal observaons of suden s perforance. If we know or assue whch se of subsklls a sep requres, akes sense o esae overall knowledge and nference of he sep as a funcon of he esaed knowledge of each ndvdual subskll requres. In hs secon we propose hree new odels (ncludng KLR-GS, KLR-LFID and KLR-FP) usng logsc regresson funcons. A. KLR-GS : Exendng Guessng and Slppng Probably In racng Model Usng Logsc Regresson We assue ha he guessng and slppng probably on each sep are based on he suden ably of each subskll. hus he guessng and slppng probably dsnc over each sep. he ore subsklls a sep needs, he less possble suden could guess he rgh answer. he archecure of odel KLR-GS s shown n Fgure 2. he defnon of he varables s lsed as follows: S j (n) : observed varable, he subskll used n sep n; f sep n requres subskll j, 0 oherwse. K n : hdden varable, suden knowledge sae, defne wheher a suden has asered knowledge a hs sep; rue f suden has known, false oherwse. C n : observed varable, suden perforance node, defnes he resul of hs sep; rue f suden answers he queson correcly, false oherwse. P(L 0 ): he pror probably, defnes he probably ha a suden knows hs skll before he sars answerng hs se of queson. P(): he learnng probably; he probably of a suden ransferrng fro no knowng he knowledge n sep n o knowng n sep n +. P(F): he forgeng probably; he probably of a suden ransferrng fro knowng knowledge n sep n o no knowng n sep n +. he radonal K odel assues here are no forgeng, however s possble o keep eory of a specfc skll whou forgeng. So hs odel uses he forgeng probably o descrbe forgeng effec. Inal PL ( 0) K 0 C 0 S0 P() P(F) K C S Fgure 2 Archure of KLR-GS Model he guess probably P(G) n sep n s defned n he for logsc regresson: P( Cn rue Kn false) () exp{ } sn Slarly, he slppng probably P(S) n sep n s defned: () exp{ } 0 s n n n () 0 s n P( C false K rue) exp{ } Here α and δ represen skll j s respecve conrbuons o he guessng and slppng probably, β and γ s he offse of he guess and slp probably. We assue ha α, δ, β and γ s a reflecon of he suden s overall ably n he pracce, so hey do no change over e. K n C n

4 Unlke K odel, changes of guessng and slppng probably on a specfc sep n are no a fxed condonal probably able, hey depend on he dsrbuon of he subsklls and he coeffcen of each subskll. Fgure 3 llusraes he changes of guessng probably on he condon when here are only wo subsklls nvolved on a specfc sep. hs ples ha he group of subsklls has a drec pac on boh guessng and slppng probably: hgher subskll levels resuls n lower guess probably and hgher slp probably. ) Paraeer Esaon: We run Expecaon Maxzaon algorh (EM algorh) o nfer he paraeers, whch s a popular approach o esae he paraeers n he K odel. he E sep uses he curren paraeers ( λ ) as he esaon for he ranson and esson probables o nfer he probably ha he suden has asered he skll a each pracce opporuny. Gven he esaed probables of asery copued n he E sep, he M sep, precopues he paraeer (λ) esaes a) E sep: In hs odel, he suden knowledge nodes are hdden, whle he subsklls node and suden perforance node are observed. he suden knowledge node has wo saes: knowng and no knowng. he suden perforance node has wo saes: answerng correcly and no correcly. he subsklls node also has wo values, 0 eans hs sep doesn conan hs subskll, whle eans hs subskll was nspeced n hs sep. We use o represen he sae of knowledge node n sep, a + represen he probably of ransferrng fro sae o sae +. When = 0 and + =, a + represens he learnng probables, whch eans P(). When = and + = 0, a + represens he forgeng probables, whch eans P(F). π 0 eans he pror probably P(L 0 ). We use b (o ) o represen he probably of when knowledge sae s and he perforance sae s o n he sep. When = 0 and O =, b (o ) eans he guessng probably P(G) n sep n. When = and O = 0, b (o ) eans he slppng probably P(S) n sep n. In hs condon, P(G) could be nferred as Eq, P(S) could be nferred as Eq. he expeced value of he log lkelhood funcon s Q(, )= log P( O, I ) P( O, I ) (3) I O represens he observaon sequence, I represens he hdden saes. λ represens he paraeers descrbed above. λ represens he curren esaed paraeers. P( O, I ) b 0 ( o 0 0) a ( 0b o ) a ( ) a 2 n b n o (4) n n So Eq(3) could be wren as: Q(, )= log P( O, I ) 0 log P( O, I ) N- aa I 0 I N log b ( o ) (, ) P O I I 0 (5) b) M sep: In M sep we need o fnd he paraeer o axze he log lkelhood funcon. So we need o fnd he correc λ o axze he funcon: λ = argax θ Q(λ, λ ). In order o fnd he correc paraeers o axze he log lkelhood funcon, we have o ake he dervave of Eq(5). As we have spl Eq(3) no hree pars, each par has specfc paraeers. So we only need o ake he dervave of he par where he paraeer we wan o esae s n. For exaple, o calculae dervaon of π 0 : n Q ( log (, 0 P O 0 )) (6) d N Under he condon ha = π =, N s he nuber of knowledge node sae, whch eans knowng and no knowng. Inroducng Lagrange ulpler n Eq(6). π can be nferred as: P( O, 0 ) (7) PO ( ) In he sae way, oher paraeers could be nferred as: a j P( O,, j ) 0 P( O, ) P( O, false ) I( o rue) J (, ) 0 * P( O, ) 0 false P( O, false ) I( o false) J (, ) 0 + * P( O, false ) P( O, false ) I( o rue) J (, ) * P( O, false ) 0 0 P( O, false ) I( o false) J (, ) + * P( O, false ) 0 P( O, false ) I( o rue) L(, ) * P( O, rue ) P( O, false ) I( o false) L(, ) + * P( O, rue ) P( O, false ) I( o rue) L(, ) * P( O, rue ) P( O, false ) I( o false) L(, ) + * P( O, rue ) (8) (9) (0) () (2) π 0 = P(O, 0=rue λ ) (3) P(O λ ) Whle J(α, β) and L(δ, γ) are guessng and slppng probably we jus dscussed. J (, )= exp{ () s } 0 () exp{ sn } L(, )= exp{ () 0 sn } n

5 (a). Boh subsklls are nvolved on a sep (b). Only subskll s nvolved (c). Only subskll 2 s nvolved Fgure 3 Guessng probably changes along wh he dsrbuon of wo subsklls, wh β=0.96. X-axs represens he coeffcen of subskll, Y-axs represens he coeffcen of subskll 2, Z-axs represens he value of guessng probably. Algorh : KLR-GS nference algorh If C n s correc P( Cn)* P( Sn ) P( Kn Cn ) P( Cn)* P( Sn)+ P( Cn)*P(G n) Else: P( Cn)* P( Sn ) P( Kn Cn 0) P( Cn)* P( Sn)+ P( Cn)*P( G n) hen we updae suden knowledge node sae n sep n+: P(K n+ ) = P(K n evdence n ) + P( K n evdence n ) P() P(K n evdence n ) P(F) ) Predc preforance he procedure of nferrng he suden knowledge node and perforance n KLR-GS s shown n Algorh. P(G n ) and P( ) are he slppng and guessng probably n sep n, whch could be calculaed as Eq and Eq. he suden perforance sae could also be nferred as: P(C n ) = P(K n ) P( ) + ( P(K n )) P(G n ) B. Exendng Learn and Forge Rae and Ie Dffculy n racng Model(KLR-LFID) KLR-GS only exends guessng and slppng probably whou consderng learnng and forgeng probably. I doesn consder he effec of subsklls on he learnng and forgeng probably. In hs secon, we exend learnng and forgeng probably usng he sae ehod n las secon. In addon, eprcal analyss reveals ha he suden learnng perforance ofen has a srong connecon wh he dffculy of probles. For exaple, would be easer o answer +4 correcly han o answer 5*27-3. Assung ha sudens ay deonsrae a hgher guessng probably and lower slppng probably facng easy probles han facng hard probles, we also nroduce he e dffculy no he K odel. he defnon of odel KLR-LFID s presened n Fgure 4. he defnon of S (n) j, K n, and C n are he sae as KLR-GS. However, P(L 0 ), P(), P(F), P(G) and P(S) are dfferen. he way we calculae P(L 0 ), P(), P(F), P(G) and P(S) s: P(L 0 ) : We exend he pror probably as logc regresson of ulple subsklls requred n hs learnng process. he P(L 0 ) n hs odel can be calculaed as follows: P(L 0 = rue) = 0 + exp{ =0 α s (0) 0 + β 0 } Here α 0 represens skll s conrbuons o he pror probably, β 0 s he offse of he pror probably, s () 0 s an ndcaor of wheher hs subskll s used n sep 0. P(): P() s he ranson probably. Wh dfferen subsklls, has dfferen values n dfferen seps. he learnng probably n sep n can be nferred as: P(K n+ = rue K n = false) = +exp{ () =0 α s n +β} Here α represens skll s conrbuons o he rans probably, β s he offse, s () n ndcaes wheher he skll j s used n sep n. Snce α and β odels he overall knowledge sae of he suden, hey do no change over seps. P(F): P(F) s he forgeng probably. Jus as he ranson probably, changes wh dfferen subsklls n dfferen seps. he forgeng probably n sep n can be nferred as: P(K n+ = false K n = rue) = exp{ =0 δ s n () +γ} +exp{ () =0 δ s n +γ} Here δ represens skll s conrbuons o he forge probably, γ s he offse. P(G) and P(S): P(G) s he guessng probably, P(S) s he slppng probably, hey can be affeced by he dffculy of exercse es. For exaple, f we have 0 probles, dffculy ranges fro o 5, we have 5 guessng probables and 5 slppng probables for sudens o solve each proble. Guessng and slppng probably can be nferred as: P(G) = P(C n = rue K n = false, I n = ), =,2,3,4,5

6 P(S) = P(C n = false K n = rue I n = ), =,2,3,4,5 he process of paraeer esang and perforance predcng of KLR-LFID are slar o he procedure n KLR-GS. he way o odel e dffculy s shown as follows: Inal PL ( 0) K 0 P(G) P(S) C 0 Ie dffculy I 0 S0 K C Ie dffculy I S K n C n Ie dffculy I n C. Exendng Boh Four Paraeers In racng Model(KLR-FP) KLR-GS and KLR-LFID exend he ranson paraeers P(), P(F) and esson paraeers P(G), P(S) separaely. Does ake sense o exend all he four paraeers n he K odel o furher enhance s effecveness of descrbng ulple subsklls n learnng process? Under hs assupon we propose KLR-FP ha exend all he four paraeers n he K odel usng logsc regresson. he defnon of he odel s shown n Fgure 5. he eanngs of S (n) j, K n, and C n are he sae as he oher wo odels. he ranson and esson paraeers are a lle dfferen. P(L 0 ): s defned n he sae way as defned n KLR-LFID. When a suden sar answerng a se of quesons he probably ha he already knows he answer s: P(L 0 = rue) = 0 + exp{ =0 α s (0) 0 + β 0 } Fgure 4 Defnon of KLR-LFID Model ) Ie dffculy Modelng: here are los of ehods o odel e dffcules. In hs paper we use Ie Response heory [7] o odel e dffcules. Ie Response heory s a heory of esng based on he relaonshp beween ndvdual s perforance on a es e and es akers level perforance on an overall easure of ably ha e was desgned o easure. I uses he e response funcon o esae he probably ha a person wh a gven ably level wll answer a queson wh a gven dffculy correcly. Wh dfferen aoun of paraeers n he e response funcon could be caegorzed no hree ypes: PL (one paraeer logsc odel), 2PL and 3PL. In hs paper we chose 2PL odel o esae he e dffculy. he e response funcon for 2PL odel s: P (θ) = + exp{ a (θ b )} he paraeers are nerpreed as follows: a : he dscrnaon of proble. b : he dffculy of proble. θ: he es aker s ably. In order o nroduce e dffculy no our odel, we apply he IR odel on he answerng sequence of all sudens, calculae he proble dffculy, noralze he e dffculy. he P(G) and P(S) could be expressed as: P(G n ) = P(C n = rue K n = false, b n ) P(F n ) = P(C n = false K n = rue, b n ) b n ndcaes he dffculy of e n we odeled n IR odel. P(G n ) eans he guessng paraeer n sep n, P(F n ) eans he forgeng paraeer n sep n. Inal PL ( 0) K 0 C 0 S0 S0 K C S S Fgure 5 Defnon of KLR-FP Model K n C n P(): he ranson probably n sep n can be calculaed as: P(K n+ = false K n = rue) = exp{ =0 δ s () n + γ} + exp{ =0 δ s () n + γ} P(F): he forgeng probably can be nferred as: P(K n+ = false K n = rue) = exp{ =0 ν s () n + ε} + exp{ =0 ν s () n + ε} P(G): Jus as we descrbe n odel KLR-GS, he guessng probably can be nferred as: P(C n = rue K n = false) = + exp{ =0 θ s () n + λ} P(S): he slppng probably can be nferred as:

7 P(C n = false K n = rue) = exp{ =0 μ s () n + ω} + exp{ =0 μ s n () + ω} In hs odel, we use logsc regresson o express he learnng, forgeng, guessng, slppng and pror probably. hs eans, we wll esae ore paraeers han las wo odels. So copared wh oher wo odels, s reasonable o speculae hs odel s ore lkely o be effeced by rando errors happen durng he esaon. IV. DAASES We evaluae KLR-GS, KLR-LFID, and KLR-FP on an open daase ASSISens daase and a prvae daase colleced hrough our own learnng record syse. A. he ASSISens Daase Our frs open daase coes fro ASSISens [8], a web-based ah uorng plafor. I s bes known for s 4 h - 2 h grade ah conen. In he school year soe 600+ sudens use hs plafor for abou wo weeks. If sudens go he e correc hey were allowed o connue wh a new queson. If hey go wrong, hey were provded wh a sall uorng sesson where he proble s broken down no seps for suden o ge correc sep by sep. In our orgnal daases, here are 92 sudens, acons. he longes suden answerng sequence s 768 he shores s. In order o f our odel, we chose sudens who fnshed ore han 20 probles. In he end our acual daase consss 797 sudens, records. records of each suden were logged for 06 sklls (addng-decals, addon, dvson ec.). he overall correc percen s 54.4%. B. s Algebra Daase he prvae daase was colleced fro our learnng record syse sorng 4,037,46 learnng behavor daa fro 40 schools and 9 onlne educaonal copanes. We exraced suden perforance daa of a ah class fro fve junor ddle schools. here were fve chapers n hs class, where eacher assgned weny-fve quesons. s fnshed he hoework hrough a ah uorng syse, afer ha he uorng syse sen he suden perforng daa o our learnng analyss plafor n xapi [9], a brand new specfcaon for learnng echnology o sore and share learnng daa across he plafor. Each suden had hree chances o answer a queson, when a suden aeped a queson for he hrd e, he was allowed o vew he hn. Our syse records he suden s perforance on each aeps, bu we only ark he queson correcly f he suden answers correcly on he frs aep. In he raw daase, here are oally 262,79 acons. Afer flerng he answers afer he frs aep, we obaned 52,500 acons. he overall correc rae of he quesons s 73.27%. he suden nuber n each secon ranges fro 972 o 578, and all of he suden fnshed 25 quesons. Soe of he fnshed he whole fve chapers, whle soe only fnshed par of chapers. V. EXPERIMEN We pleen he odels n he Bayes Ne oolbox for Malab (BN) [20], an open source oolbox for Bayesan nework consrucng. Specfcally, we defne he knowledge node K gven n subsklls S j as a sofax node n he oolbox. o evaluae our odels, we f he separaely for each suden on he frs 80% of all of ha suden s daa, es on he second 20%, and average he es resuls across sudens. We repea he experen for 0 es and average he resul. For coparson, we also apply he LR-DBN, K- IDEM and K odel and esae he suden s perforance on he second 20% daa. he crera we evaluae he perforance of he odels are lsed as follows: MAE: Mean Absolue Error RMSE: Roo Mean Squared Error MSE: Mean Squared Error AUC: Area Under he Curve A. he ASSISens Daase We apply odel KLR-GS, KLR-FP, LR-DBN and K on he ASSISens daase. he KLR-LFID odel canno be appled on hs daase because we noce ha s dffcul o fnd an approprae sequence ha he ajory of sudens have answered o evaluae he e dffculy hrough IR odel. In he sae reason, we ddn apply K- IDEM on hs daase. he resul of he odels s shown n able I. KLR-GS and KLR-FP perfor beer han oher odels. he AUC of KLR-FP s hgher han LR-DBN, whle KLR-GS has 0.35 hgher AUC han LR-DBN. he resuls fro evaluang he odels are srongly n favor of KLR-GS, wh lower MAE, EMSE, MSE and hgher AUC. LR-DBN only consders ranson paraeers, KLR-FP no only exends ranson paraeers bu also exends esson paraeers. Jus as dscussed n Algorh, predcng sae of knowledge and perforance nodes concerns wh guessng and slppng probably. So exendng guessng and slppng probably helps provng he odel. KLR-GS uses logsc regresson o updae guessng and slppng paraeers n he odel. KLR-GS reveals a beer perforance han LR-DBN, so we conclude ha exendng guessng and slppng paraeers proves he odel ore sgnfcanly han exendng learnng and forgeng paraeers. In [3] Brenes exends K odel usng general suden and e feaures on guessng and slppng paraeers, leads o he sae concluson. able I Resuls of odels on he ASSISens plafor daase MAE RMSE MSE AUC KLR-G KLR-FP LR-DBN K B. Algebra Daase he answerng paern n algebra daase s well ordered. So we apply KLR-GS, KLR-LFID, KLR-FP, LR-DBN, K-IDEM and K on hs daase. here are fve chapers n

8 hs daase, we odel each chaper separaely, each chaper we apply he odel for 0 es and average he perforance. Unlke he ASSISens daase, probles have he subsklls labeled for each sep, wha we have abou hs daase s he proble descrpon and answerng oucoes. How o label he subskll for each queson? We nved wo experenced junor school eachers o help us annoang he subsklls. hey suggesed us o separae he nverse funcon no sx subsklls, whch are defnon of nverse funcon, age of nverse funcon, expresson of nverse funcon, propery of nverse funcon, geoerc applcaon of nverse funcon and synhesze of nverse funcon. We use he CFM o esae he effecveness of he dvson of he subskll. ) Why uple subsklls s ore effcen han sngle skll: CFM(conjuncve facor odel) [2] s a popular odel n LFA(Learnng Facor Analyss) [22], whch s a fraework ang o address he proble of dscoverng a beer cognve odel fro suden learnng daa. CFM assues ha when an e requres ulple sklls presen, he e s harder han he es requrng only one of hose sklls. Usng he CFM odel o f he answerng sequence, he cognve odel wh lower errors s he canddaes of beng beer cognve odels. In he radonal K odel, each queson has he sae subskll, whch eans ha here s only one skll. However, n our odel n we assue here are sx subsklls, whch assupon s beer? We use CFM odel o confr hs queson. he overall resuls of sngle skll and sx subsklls are shown n able II. able II Usng CFM o odel sngle skll vs sx subsklls C. Dsscusson In hs secon, we presen he experenal resuls of he odels on each daase. he resuls confr ha all he odels ncludng KLR-FP, KLR-GS and KLR-LFID perfors beer han LR-DBN, K-IDEM and K. KLR- FP underperfors KLR-GS on Asssens daase, bu ouperfors KLR-GS on algebra daase. KLR-LFID ouperfors KLR-FP on algebra daase. I s necessary o nvesgae why KLR-FP underperfors KLR-GS on Asssens daase, bu ouperfors KLR-GS on algebra daase. Fro he begnnng, we exraced he error cases n he predcons ade by boh odels and exaned he sequence lengh of each suden for who our odel fals o predc perforance correcly. he hsogra of sequence lenghs of error predcons can be ploed n Fgure 6.a and Fgure 6.b. he range of error sequence of wo odels s slar, whch eans ha he capacy of wo odels resebles each oher. he dsrbuon of error sequence lengh s anly locaed n he range of 00 o 500, because he shorer sequence can accelerae he convergence of EM-based paraeer fng. Also wh longer sequences, he odels have ore nforaon of suden perforance, hus leadng o a hgher probably o ake correc predcs. We conclude ha wh he sequence range n 50 o 500, srcly speakng 27 o 628, KLR-FP perfors unsably. Sngle Skll Sx s AIC BIC AIC and BIC are easure of he relave qualy of sascal odels for a gven se of daa, he saller he AIC and BIC are, he beer he odel we esae s. I shows ha conjuncve odel wh sx subsklls perfors beer han sngle skll. 2) Resuls: We apply he odels on each secon, hen average he resuls of fve secons, he overall resuls are shown n able III. As shown n able III, boh hree odels perfor beer han LR-DBN, K-IDEM and K. he AUC of KLR-GS s hgher han LR-DBN, he AUC of KLR-LFID s hgher han LR-DBN, and KLR-FP has a hgher AUC han LR-DBN. KLR-FP prefors beer han any oher odels on hs daase. KLR-LFID perfors beer han KLR-GS wh lower MAE, RMSE, MSE and hgher AUC. able III Resuls of Models on he suden algebra daase (a). Lengh of error predcons n KLR-FP MAE RMSE MSE AUC KLR-G KLR-LFID KLR-FP LR-DBN K-IDEM K (b). Lengh of error predcons n KLR-GS Fgure 6 Lengh of error predcons n KLR-FP and KLR-GS. X-axs s he range of sequence lengh separaed no 20 groups, Y-axs s he nuber of sudens whose answerng sequence s n he range.

9 hen we checked he procedure of paraeer esaon and perforance predcon. Afer coparng he ranson and esson paraeers raned n boh correc predcon and error predcon cases, we fnd he ranson and esson paraeers of os error predcons apparenly devae fro noral value ranges, whch are eher oo large or oo sall. Fgure 7 shows such a case where guessng and slppng probably of an error predcaon s exreely saller han hose of a correc predcon along her answerng sequence. (a). Par of he sep subskll arx of he Asssens daase, x- axs represens he subsklls, y-axs represens he seps. (b). Par of he sep subskll arx of he algebra daase, x-axs represens he seps, y-axs represens he subsklls. Fgure 8 Par of he sep subskll arx of he Asssens daase and algebra daase. Blue eans subskll j s no nvolved n sep n, red eans sep n nspecs subskll j. Fgure 7 Guessng and slppng probably of correc and error predcons We esae he coeffcen of ranson and esson paraeers based on Eq(9-2), bu we couldn fnd he correc graden for α j (he coeffcen of subskll j), f s no nvolved n a specfc sep. For S j (n) = 0 n sep n, hen Eq and Eq do no conan coeffcen α j. Under hs condon, we couldn esae α j. So s possble ha we wll sesae he coeffcen of subsklls ha are no used n os seps of he answerng sequence. Wh he os varables beween he odels, KLR-FP wll agnfy he sake durng he paraeer esaon. So we speculae ha KLR-FP wll ake ore sake wh he sparse subskll arx han he dense arx. Fgure 8.a and Fgure 8.b dsplay he subskll dsrbuon of he ASSISens daase and algebra daase respecvely. he ASSISens subskll arx sees o be a sparse arx where here are so any subsklls bu only very few of he are nvolved n a specfc sep. In conras, he subskll arx of algebra daase sees o be uch denser. hus KLR-FP akes ore sakes on he ASSISens daase. In hs way we conclude ha EM process of paraeer esaon s lkely o ake sakes when he subskll arx s a sparse arx. Based on he exanaon we carred ou, we conclude ha KLR-FP underperfors KLR-GS when he answerng sequence s beween 27 and 628, or he subskll arx s a sparse arx. On he condon ha suden answer sequence s under 27 or over 628, or he subskll arx of he probles s dense arx, KLR-FP ouperfors KLR-GS. VI. CONCLUSION In hs paper we nroduce racng Model, and hree groups of exensons n ulple subsklls odelng. hey are conjuncve odels, perforance facor analyss and logsc regresson based odels. However, neher of he odels has fully cobned he ranson and esson paraeers n K odel wh dfferen subsklls. hs paper exends K odel wh logsc regresson whn a dynac Bayesan nework by proposng hree new odels KLR-GS, KLR-LFID and KLR-FP. hey exend dfferen paraeers n K odel, and explore he effec of exenson of dfferen paraeers. he odels were pleened wh a wdely used Bayesan nework oolbox BN. o evaluae our odels, we use wo daases: ASSISens daase, a very faous open daase and suden algebra daase colleced fro our learnng record syse. he odels are copared wh several publshed odels ncludng LR-DBN, K-IDEM and K. Overall, he experenal resuls on boh daases deonsrae ha all he odels proposed n he paper (KLR-GS, KLR-LFID and KLR-FP) perfor beer han LR-DBN, K-IDEM and K. Furherore, we conduc he dealed analyss of he EM based paraeer esaon process for our odels o nvesgae he dfferen perforance for he odel KLR- GS, and KLR-FP under he wo daases. Ineresngly, we observe ha he subskll assocaon arx n he seps of suden proble solvng has neresng pac on he paraeer esaon of hese odels. Prelnary resuls ndcae ha he KLR-GS odel s ore suable n he scenaro of he sparse subskll assocaon arx whereas he KLF-FP sees ore suable n he dense scenaro. ACKNOWLEDGMEN

10 hs work was suppored by gran fro NSFC (Gran No ) and Sae Key Laboraory of Sofware Developen Envronen (Fundng No. SKLSDE-207ZX- 04). REFERENCES [] Wulf J, Bloh I, Leeser J M, e al. Massve Open Onlne Courses [J]. Wrschafsnforak, 204, 6:-4. [2] Corbe A, Anderson J R. racng: Modelng he acquson of procedural knowledge[j]. User Modelng and User- Adaped Ineracon, 994, 4(4): [3] Gong, Yue, J. E. Beck, and N.. Heffernan. "Coparng knowledge racng and perforance facor analyss by usng ulple odel fng procedures." Inernaonal Conference on Inellgen uorng Syses Sprnger-Verlag, 200: [4] Cen, Hao, K. Koednger, and B. Junker. "Coparng wo IR Models for Conjuncve Sklls." Inernaonal Conference on Inellgen uorng Syses Sprnger-Verlag, 2008: [5] Ebreson, Susan E. Mulcoponen Response Models. Handbook of Modern Ie Response heory. Sprnger New York, 997: [6] Goodkovsky V A. Inellgen uorng syse: US, US B2[P] [7] Pavlk Jr P I, Cen H, Koednger K R. Facors Analyss-- A New Alernave o racng[j]. Onlne Subsson, [8] Ghahraan Z, Wolper D M. Modular decoposon n vsuooor learnng[j]. Naure, 997, 386(6623): [9] Cen, Hao, K. Koednger, and B. Junker. Learnng Facors Analyss A General Mehod for Cognve Model Evaluaon and Iproveen. Inellgen uorng Syses. Sprnger Berln Hedelberg, 2006: [0] Cen, Hao, K. Koednger, and B. Junker. Learnng Facors Analyss A General Mehod for Cognve Model Evaluaon and Iproveen. Inellgen uorng Syses. Sprnger Berln Hedelberg, 2006: [] Junker, Bran W., K. R. Koednger, and Maro ron. "Fndng proveens n suden odels for nellgen uorng syses va varable selecon for a lnear logsc es odel." Annual Norh Aercan Meeng of he Psychoerc Socey, Vancouver, BC, Canada. hp://lb. sa. cu. edu/~ bran/bjrs. hl [2] Gong Y, Beck J, Ruz C. Modelng ulple dsrbuons of suden perforances o prove predcve accuracy[j]. User Modelng, Adapaon, and Personalzaon, 202: [3] Gonzalez-Brenes, Jose, Y. Huang, and P. Bruslovsky. "General Feaures n racng o Model Mulple s, eporal Ie Response heory, and Exper." (204). [4] Xu, Yanbo, and J. Mosow. "Usng Logsc Regresson o race Mulple s n a Dynac Bayes Ne." Inernaonal Conference on Educaonal Daa Mnng, Endhoven, he Neherlands, July DBLP, 200: [5] Mosow, Jack, and G. As. Evaluang uors ha lsen: an overvew of projec LISEN. Sar achnes n educaon. MI Press, 200: [6] Desaras M C, Baker R S J D. A revew of recen advances n learner and skll odelng n nellgen learnng envronens[j]. User Modelng and User-Adaped Ineracon, 202, 22(-2):9-38. [7] Ie Response heory[m]. Sprnger Neherlands, 985. [8] Razzaq, Leena, e al. "Blendng Assessen and Insruconal Asssng. " Arfcal Inellgence n Educaon - Supporng Learnng hrough Inellgen and Socally Infored echnology, Proceedngs of he, Inernaonal Conference on Arfcal Inellgence n Educaon, Aed 2005, July 8-22, 2005, Aserda, he Neherlands DBLP, 2007: [9] L, Kn Chew. "Usng xapi and learnng analycs n educaon." elearnng Foru Asa, Shangha, Chna [20] Murphy K P. he Bayes Ne oolbox for Malab[J]. Copung Scence & Sascs, 200, 33:200 [2] Koednger K, Cen H. Generalzed learnng facors analyss:, Iprovng cognve odels wh achne learnng[j]. Dsseraons & heses - Gradworks, 2009, 263: [22] Raffery, Anna N., and M. Yudelson. "Applyng Learnng Facors Analyss o Buld Sereoypc Models." Conference on Arfcal Inellgence n Educaon: Buldng echnology Rch Learnng Conexs ha Work IOS Press, 2007: [23] Pardos Z A, Heffernan N. K-IDEM: Inroducng Ie Dffculy o he racng Model[M]// User Modelng, Adapon and Personalzaon. Sprnger Berln Hedelberg, 20: