KnowledgeZoom for Java: A Concept-Based Exam Study Tool with a Zoomable Open Student Model

Transcription

1 Pete Buslvsky Unvesty f Pttsbugh Pttsbugh, USA peteb@ptt.edu 2013 IEEE 13th Intenatnal Cnfeence n Advanced Leanng Technlges KnwledgeZm f Java: A Cncept-Based Exam Study Tl wth a Zmable Open Student del Dhuba Bashya ON24, Inc. San Fancsc, CA dhuba.bashya@ n24.cm Rya Hssen Unvesty f Pttsbugh Pttsbugh, USA h38@ptt.edu Jul Guea Unvesdad Austal de Chle Valdva, Chle jguea@nf.uach.cl ne Lang Fudan Unvesty Shangha, Chna mchelle_lang@ fudan.edu.cn Abstact-- Ths pape pesents u attempt t develp a pesnalzed exam pepaatn tl f Java/OOP classes based n a fne-ganed cncept mdel f Java knwledge. Ou gal was t exple tw mst ppula student mdel-based appaches: pen student mdelng and pblem sequencng. The esult f u wk s a Java exam pepaatn tl, KnwledgeZm. The tl cmbnes an pen cncept-level student mdel cmpnent, Knwledge Exple and a cnceptbased sequencng cmpnent, Knwledge axmze nt a sngle nteface. Ths pape pesents bth cmpnents f KnwledgeZm, epts esults f ts evaluatn, and dscusses lessns leaned. Keywds-Pblem Sequencng, Open Student delng, Pgessve Zm I. INTRODUCTION Exam pepaatn s a challengng task f cllege students. Wthn a sht ped f tme, typcally a week less, a student needs t evew the cntent that was studed ve the whle semeste, dentfy pssble knwledge gaps and mscnceptns, and fll these gaps. A pesnalzed leanng tl based n a lng-tem student mdel culd be vey helpful n ths pcess. By eflectng students pgess ve the whle semeste, a student mdel can dstngush tpcs that wee leaned and need just a quck efesh f tpcs that wee mssed and may need a thugh evew. Usng ths mdel, a pesnalzed exam pepaatn tl can ndvdually gude each student thugh the study pcess. Supsngly, we wee nt able t fnd any attempt t develp a pesnalzed exam pepaatn tl. Whle a ange f pesnalzed sequencng and adaptve navgatn appaches have been develped (see Sectn II), all appaches knwn t us ae fcused n supptng egula leanng pcess that gudes students thugh the whle pcess f subject leanng statng at the vey begnnng. In u past wk, we expled a numbe f pesnalzed gudance appaches. In patcula, we develped seveal systems t suppt pesnalzed gudance f a cuse n Java and Object Oented Pgammng (OOP) ncludng a tpc-based gudance system JavaGude [8] and a scal gudance system Pgess+ [7]. Whle these tls wee hghly effcent n gudng student pactce ve the duatn f the cuse, we fund that the gudance pvded by ethe f them s nt suffcent f exam pepaatn. Nethe case-ganed tpc-based gudance, n scal gudance was able t ecgnze specfc hles n students knwledge and t ffe the best way t bdge the gap. The expeence wth bth tls caused us t beleve that an exam pepaatn tl eques a fne-ganed cncept-level student mdel and a specfc gap-fcused gudance appach. Ths pape pesents u attempt t develp an exam pepaatn tl f Java/OOP classes based n a fneganed cncept mdel f Java knwledge. Ou gal was t exple tw mst ppula student mdel-based pesnalzed gudance appaches: pen student mdelng and pblem sequencng. The dea f pen student mdelng s t shw the state f a student mdel t the student n de t help he eflect n he knwledge, dentfy gaps, and fcus n fllng these gaps. The dea f adaptve pblem sequencng s t geneate a pesnalzed sequence f pblems that wll help the student t effcently pactce he mssng knwledge. The Java exam pepaatn tl KnwledgeZm (KZ) that we develped cmbnes an pen cncept-level student mdel cmpnent Knwledge Exple (KE) and a cncept-based sequencng cmpnent Knwledge axmze (K) n a sngle nteface. Ths pape pesents bth cmpnents f KZ fcusng n the challenges f cncept-level pen student mdelng and sequencng, epts ts evaluatn, and dscusses lessns leaned. II. RELATED WORK A. Open Student delng Open student mdelng s an mptant eseach dectn n the aea f ntellgent educatnal systems. Unlke the mansteam eseach n ths aea that use a student mdel as a hdden nfmatn suce t adapt the leanng pcess t students needs, pen student mdelng eseaches ague that a student mdel has ts wn pedaggcal value and shuld be vsble and edtable by students. A ange f benefts have been epted n penng the student mdels t the leanes, such as nceasng the leane s awaeness f the develpng knwledge, dffcultes and the leanng pcess, and students engagement, mtvatn, and knwledge eflectn [4; 13; 16]. Vsual pesentatns f the student mdel vay fm dsplayng hgh-level summaes (such as skll metes) [13] t cmplex cncept maps Bayesan Netwks [16]. In patcula, seveal pjects expled Teeaps [14] as a way t pesent heachcal student mdels [2; 5; 10; 11]. Yet, the student mdels expled n eale pjects wee elatvely smple and typcally pesented n ne-sht that elmnated a need t exple the mdel n detal. In cntast, u wk fcuses n easnably cmplex cncept-based use mdels wth hundeds f cncepts and studes a pgessve zm [10] appach t exple these mdels /13 $ IEEE DOI /ICALT

2 B. Adaptve Pblem Sequencng Adaptve pblem sequencng s ne f the ldest technlges n the aea f ntellgent educatnal systems. The gal f ths technlgy s t geneate a pesnalzed sequence f pblems f evey student s that they can acheve the leanng gal n a mst ptmal way. A ange f appaches wee ppsed f adaptve pblem sequencng ncludng appaches based n asscatve mechansms [9], dynamc pblem dffculty [12], and metadata [6]. Cnceptbased pblem sequencng [1] s a subclass f sequencng appaches. It s based n a fne-ganed cncept-level dman mdel that s used t ndex pblems. Althugh all sequencng appaches ty t fnd the mst ptmal pblems f the students, they mght fal when the use mdel s ncect. In such cases the system selectn cannt be eled upn and students shuld be able t select the pblems themselves. In u pevus ntefaces f accessng leanng cntent f Java, we have ted t educe the negatve effects f sequencng es thugh adaptve navgatn suppt technlges that d nt fce the students t wk n a pblem cnsdeed the best by the sequencng mechansm, but pvde anntatn-based navgatn suppt that cmbnes ntellgent gudance wth human decsnmakng [3; 8]. In ths pape, we etun t a me tadtnal pblem sequencng mechansm that we cnsde as a pmsng appach n the exam pepaatn cntext when tme s lmted and an ptmal gudance becmes qute ctcal. III. THE KNOWLEDGEZOO (KZ) STUDY TOOL T nvestgate the value f cncept-based pesnalzatn n the cntext f exam pepaatn, we develped a cnceptbased exam study tl KZ. The gal f KZ s t help the students dentfy the cuse knwledge gaps and pvde tls t bdge these gaps n an effectve way. The fst pat f ths dual gal s suppted by the KE cmpnent, a cncept-based heachcal zmable pen student mdel. The secnd gal s suppted by the K, a cncept-based adaptve pblem sequencng tl. The nteface f KZ (Fg. 1) pvdes dect access t the KE mdel and a buttn t launch the K. Students access the tl thugh a pesnalzed leanng ptal alng wth seveal the study tls such as JavaGude [8] and Pgess+ [7]. A. The Dman del and the Leanng Cntent KZ s based n a cncept-level mdel f knwledge abut Java and OOP. Ths mdel s fmed by a subset f cncepts fm the Java ntlgy bult by the PAWs lab. The Java ntlgy ncludes 344 cncepts ganzed nt an 8-level tee. The leanng cntent n KZ s fmed by 103 paametezed self-assessment questns that wee develped n u team as a pat f an eale pject [8]. Each questn s ndexed wth ntlgy cncepts. The ndexng classfes the peequste cncepts that shuld be knwn befe appachng the questn and the utcme cncepts t be masteed by wkng wth the questn. The numbe f cncepts asscated wth a sngle questn anges fm 5 t 52 (0 t 41 peequstes, 1 t 12 utcmes). These questns cve the 188 mst mptant cncepts f Java whch fm the KZ dman mdel. B. The Knwledge Exple (KE) KE s a mult-level pen student mdel vsualzed wth a zmable Teemap. The nfmatn pesented by KE s an velay mdel f Java Knwledge based n the KZ ntlgcal dman mdel. The velay student mdel n KZ s mantaned by a use mdelng sevce, PERSEUS [15], whch updates the mdel afte evey attempt t answe a questn and changes the knwledge level f cncepts elated t the questn. Fgue 1. The KnwledgeZm nteface shwng the tp level f the Knwledge Exple map and a buttn t launch Knwledge axmze. Fgue 2. Zmng n the nde Expessns (tp left cne as maked n Fg. 1) eveals next level f the cncept heachy. Nw the use can see that the nde LgcExpessn that has ntemedate knwledge as a whle (shwn as yellw) cnssts f seveal well leaned and seveal unknwn cncepts. A zmable Teemap was selected t pesent the student mdel due t ts elatvely lage sze and heachcal natue. The Teemap layut shws nly fu levels f cncept heachy statng fm the cuent tp nde and hdng lwe-level ndes behnd ts ancest nde. The use, hweve, can zm n any nde. Afte zmng n, the nde expands becmng the tp nde and ccupyng the whle vew. Zmng-n mmedately expses pevusly hdden levels f heachy. F example, Fg. 2 shws the esults f zmng nt a secnd level cncept, Expessn shwn n the tp left quadant f Fg. 1. In the Teemap layut, each nde (a cncept n the Java ntlgy) s shwn as a cled ectangle. A leaf cncept f the ntlgy cespnds t a temnal nde f the Teemap. 276

3 The sze f a nde epesents the mptance f a cncept n the cntext f Java language and ts chance t be checked as pat f the exam. We measue t by cuntng hw many questns ae elated t the leaf cncept cespndng t ths leaf nde n the Teemap. Snce the numbe f execses elated t ndes can be qute dffeent, whch leads t a lage dffeence n the nde szes, we use the lg 2 (sze) t mdeate the dffeences. The cl f a nde epesents the level f cncept knwledge demnstated by a student. We use 10 cls fm ed t geen t epesent the pgessn fm weake t stnge knwledge. In a heachcal zmable layut, a leaf nde dectly epesents the mptance and knwledge level f a cncept wth ts sze and cl espectvely, whle each ntemedate nde accumulatvely aggegates mptance and cncept knwledge fm ts chld ndes. As a esult f the aggegatn, the uppe-level vews shw vevews f students state f knwledge n hghe levels (Fg. 1), whle beng able t exple detaled knwledge f evey cncept as zmng nt lwe levels f the ntlgy (Fg. 2). The calculatn f the aggegated sze and cl s mptant t bdge the gaps between lwe and hghe levels f vews. In KE, the sze aggegatn s pvded by Teemap. F the cl aggegatn, the cl f an ntemedate nde s the aveage cl f ts dect chld ndes weghted wth the szes n de t eflect the mptance f the asscated cncepts. C. The Knwledge axmze (K) The gal f the K s t pvde the leane wth a set f questns, whch wll help he acheve he leanng gals by ecmmendng the questns wth the hghest gan. K cnsdes the fllwng facts f selectn f the best actvtes whch ae cnsdeed as questns: Hw much s the student pepaed t d the actvty? The students shuld be pepaed t d the ppsed actvtes. The actvtes f whch the student has lw levels f knwledge f peequste cncepts ae nt gd suggestns. We calculate the leane knwledge f each f the peequste cncepts f an actvty t see hw much the student s pepaed t d t. Equatn (1) shws the fmula: K k w max( k ) w w lg( w ) (1) whee K s the level f the leane s knwledge n the peequstes f the actvty; w s the smthed weght f the actvty-cncept (we d t by pefmng lg functn n the weght); k s the level f the leane s knwledge n the th cncept and s the set f peequste cncepts f the actvty. Hghe knwledge f peequste cncepts f an actvty (lage K) makes t a bette canddate t be selected by the ptmze. What s the mpact f the actvty? The fmula f ths mpact s shwn as (2): I w(1 k ) w whee s the set f cncepts f the utcme f the actvty. Impact I f a cetan actvty shws that when the actvty has hghe mpact and hence t wll be a bette canddate t be selected by the ptmze. Has the use aleady cmpleted the actvty? We use success ate t undestand hw much the leane has leaned fm an actvty. We defne t as (3): (2) s S 1 (3) t 1 whee S s the nvese success ate f the student n the actvty; s s the numbe f the tmes the student has succeeded n the actvty; and t s the ttal numbe f tmes the student has ted the actvty Havng calculated the abve facts, we can smply ank the actvtes usng (4): R K I S (4) whee R s the ank f the actvty and,, ae the weghts assgned t each f the abve mentned facts espectvely. Navgatn Buttn Quz Aea Knwledge Level Fgue 3. The Knwledge axmze nteface Fg. 3 shws the nteface f K. The lst f cncepts cveed by the quz s als shwn n the ght sde f ths panel. The cl next t each cncept epesents the student s cuent knwledge level. IV. THE EVALUATION Quz Cncepts T assess the value f KZ we cnducted a classm study n the cntext f a Java-based undegaduate cuse Intductn t Object Oented Pgammng at the Schl f Infmatn Scences, Unvesty f Pttsbugh. All students enlled n ths cuse wee nvted t use the KZ 277

4 f the fnal exam pepaatn. The study stated n Decembe 4 th 2012 abut a week befe the fnal exam. Nte that the class als used QuzGude and Pgess+ t access Java questns that wee avalable fm the begnnng f the semeste. As a esult, many students leaned a cnsdeable numbe f Java cncepts by the tme they stated wth KZ and wee able t beneft fm the gap fllng natue f the system. Fgue 1 shwed hw knwledge map mght have lked t a typcal student dung the fst sessn f KZ many cncepts wee leaned, yet thee wee stll many ange and ed gaps t fll. A. Lg Analyss We hyptheszed that KZ bdges the exstng gap n the student s knwledge by ecmmendng a set f questns that bng a student t a bette level f knwledge. T examne u hypthess, we cnsdeed the fllwng system usage paametes: Attempts (the ttal numbe f questns attempted) Success Rate (the pecentage f cectly answeed questns) Dstnct Questns (the numbe f dstnct attempted questns) Attempts pe questn (the numbe f attempts f dng a questn) Sessns (the numbe f sessns the students wked wth the systems ) In u analyss we sepaately cunted questn accesses fm KZ and questns accessed fm ethe QuzGude/Pgess+. Attempts made fm KZ wee made by 14 students whle attempts made fm QuzGude/Pgess+ wee made by 17 students. As can be seen n Table I, the ttal numbe f attempts made fm QuzGude/Pgess+ was much lage, whch s natual snce the students wee famla wth QuzGude and Pgess+ fm the begnnng f the class. Yet, t s qute emakable that KZ, whch was ntduced just a week befe the exam, was cnsdeably used. We als bseved that KZ pesented students wth nteestng and challengng questns as shwn by the ncease f attempts pe questn. TABLE I. SYSTE USAGE SUARY Paamete KZ QG,P+ (n=14) (n=17) Attempts Success ate 58% 64% Dstnct questns 119 (27%) 1145 (35%) Attempts pe questns Attempt pe Sessns KZ = KnwledgeZm; QG = QuzGude; P+ = Pgess+. T assess whethe K was successful n maxmzng students steps twads the gals, we guped questns nt thee dffeent cmplexty levels based n the numbe f nvlved cncepts (Easy, deate and Cmplex) [8]. A questn wth 15 fewe cncepts s cnsdeed t be Easy, 16 t 90 as deate, and 90 hghe as Cmplex. Table II lsts the numbe f attempts made t easy, mdeate, and cmplex questns fm KZ and fm QuzGude/Pgess+. The data evealed that althugh n KZ the factn f easy/mdeate questn attempts was smalle than n QuzGude/Pgess+, the numbe f attempts t cmplex questns whch helped students each the gal faste by cveng many cncepts at nce was abut 2.5 tmes geate. Anthe nteestng esult was that despte a emakable ncease n cmplex questns, the success ates acss all systems wee cmpaable. TABLE II. Cmplexty NUBER OF ATTEPTS, SUCCESS RATES BY SYSTE AND COPLEXITY LEVEL Numbe f Attempts KZ (n=14) Success ate Easy 27 (6.2%) 93% deate Cmplex 189 (43.5%) 218 (50.2%) 68% Numbe f Attempts 1123 (34.6%) 1471 (45.3%) QG,P+ (n=17) Success ate 73% 61% 46% 651(20.1%) 55% Ttal % % KZ = KnwledgeZm; QG = QuzGude; P+ = Pgess+. B. Student Feedback Analyss At the end f the evaluatn, students wee asked t pvde feedback abut KZ and the systems used n the cuse. Of 21 students wh etuned the fms, 11 students used KZ, hweve nly 10 f them answeed questns elated t KZ. Snce KZ s the fcus f ths pape, the fllwng analyss s based n the KZ pat f the questnnae and analyzes the answes f these 10 students. The esults ae shwn n Fg 4. Oveall, 80% f the students cnsdeed the KZ system helpful as a whle (A11), whch suggests that t s helpful t cmbne the tw ndvdual cmpnents, KE and K tgethe. F KE, 70% cnsdeed ts nteface helpful t dentfy the knwledge weak pnts (A2), whch pvdes evdence t suppt the man gal f KE. 60% ageed that use f cl f Teemap ndes t shw the cncept knwledge was clea (A5), and 60% ageed that the use f cl aggegatn t shw the hghe-level cncept knwledge was clea (A6); 60% ageed that the use f Teemap nde sze t shw cncept mptance was clea (A7), and 60% ageed that the use f sze aggegatn t shw the mptance f hghe-level cncepts was clea (A8). We need t nvestgate these esults futhe. F K, abut 78% f the students cnsdeed the ablty f K t geneate quzzes that cve many cncepts as helpful (A4) 1, whch pvdes evdence t suppt the man gal f K. Only 30% nted that the quzzes geneated by K wee t smple f them (A9), supptng the lg analyss data that K challenged the students. Hweve, nly 40% cnsdeed that the KZ nteface helped them t access the mst elevant quzzes (A3), and nly 40% cnsdeed that the KZ system acceleated the pepaatn f the fnal exam (A10). 1 Only nne students answeed ths questn. 278

5 The analyss f student feedback ndcated that many students wee fustated that the questns pvded by K cmpnent wee nt affected by the KE zmng actvty. They expected that zmng nt a specfc dffcult cncept shuld allw them t access t questns specfcally elated t that cncept. Pecentage f Students' Answes 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% A1 A2 A3 A4 A5 A6 A7 A8 A9 A10A11 Questns Fgue 4. Subjectve evaluatn: questns and esults V. CONCLUSION AND FUTURE WORK 1: Stngly dsagee 2: Dsagee 3: N pn 4: Agee 5: Stngly agee A1: The KZ nteface helped me t undestand hw the class cntent s ganzed. ( ) A2: The KZ nteface helped me t dentfy my weak pnts. ( ) A3: The KZ nteface helped me t access the mst elevant quzzes. ( ) A4: The ablty f the Knwledge axmze t geneate quzzes that cve many cncepts was helpful. ( ) A5: The use f cl f Teemap ndes t shw my cncept knwledge was clea. ( ) A6: The use f cl aggegatn t shw my hghe-level cncept knwledge s clea. ( ) A7: The use f Teemap nde sze t shw cncept mptance s clea. ( ) A8: The use f sze aggegatn t shw the mptance f hghe-level cncepts s clea. ( ) A9: The quzzes geneated by the Knwledge axmze wee t smple f me. ( ) A10: The KZ system acceleated my pepaatn f the fnal exam. ( ) A11: The KZ system as a whle has been helpful. ( ) In ths pape, we have expled tw cncept-based appaches - an pen zmable student mdel and adaptve pblem sequencng t suppt students t pepae f the fnal exams n a Java pgammng class. The esults f u study shwed that u tl attacted student attentn and was ecgnzed by them as cnsdeably helpful n vsualzng the Java knwledge and n evealng knwledge gaps. KZ was able t geneate challengng questns that shtened the path t students leanng gals. In u futue wk we plan t mpve KZ and mplement bette cnnectns between ts cmpnents by ntegatng cncept zmng and questn access; and t futhe nvestgate hw t epesent me clealy the uses knwledge wth the Teemaps attbutes (cl and sze). ACKNOWLEDGENT Ths eseach was suppted n pat by the Natnal Scence Fundatn unde Gant N Jul Guea s suppted by a Chlean Schlashp (Becas Chle) fm the Natnal Cmmssn f Scence Reseach and Technlgy (CONICYT, Chle) and the Unvesdad Austal de Chle. ne Lang was a Vstng Schla at the Schl f Infmatn Scence, Unvesty f Pttsbugh when she wked n ths pject. REFERENCES [1] P. Buslvsky, A famewk f ntellgent knwledge sequencng and task sequencng. n: Pc. Secnd Intenatnal Cnfeence n Intellgent Tutng Systems, ITS'92 C. Fassn, G. Gauthe and G. ccalla, eds.,spnge-velag, nteal, Canada, 1992, pp [2] P. Buslvsky, I.-H. Hsa and Y. Flajm, Quzap: Open Scal Student delng and Adaptve Navgatn Suppt wth Teeaps. n: Pc. 6th Eupean Cnfeence n Technlgy Enhanced Leanng (ECTEL 2011) Lectue Ntes n Cmpute Scence 6964, Spnge-Velag, 2011, pp [3] P. Buslvsky and S. Ssnvsky, Engagng students t wk wth self-assessment questns: A study f tw appaches. n: Pc. 10th Annual Cnfeence n Innvatn and Technlgy n Cmpute Scence Educatn, ITCSE' , pp [4] S. Bull, Supptng leanng wth pen leane mdels. n: Pc. 4th Hellenc Cnfeence n Infmatn and Cmmuncatn Technlges n Educatn A, Athens, Geece, 2004, pp [5].H. Falakmas, I.-H. Hsa, L. azzla, N. Gant and P. Buslvsky, The Impact f Scal Pefmance Vsualzatn n Students. n: Pc. 12th IEEE Intenatnal Cnfeence n Advanced Leanng TechnlgesIEEE, Rme, Italy, 2012, pp [6] S. Fsche, Cuse and execse sequencng usng metadata n adaptve hypemeda leanng systems. AC Junal n Educatnal Resuces n Cmputng 1(1) (2001). [7] I.-H. Hsa and P. Buslvsky, tvatnal Scal Vsualzatns f Pesnalzed E-Leanng. n: Pc. 7th Eupean Cnfeence n Technlgy Enhanced Leanng (EC-TEL 2012) Lectue Ntes n Cmpute Scence 7563, Saabücken, Gemany, 2012, pp [8] I.-H. Hsa, S. Ssnvsky and P. Buslvsky, Gudng students t the ght questns: adaptve navgatn suppt n an E-Leanng system f Java pgammng. Junal f Cmpute Asssted Leanng, 26(4) (2010) [9] A.N. Kuma, A Scalable Slutn f Adaptve Pblem Sequencng and ts Evaluatn. n: Pc. 4th Intenatnal Cnfeence n Adaptve Hypemeda and Adaptve Web-Based Systems Lectue Ntes n Cmpute Scence 4018, Spnge, 2006, pp [10]. Lang, J. Guea, G.E. aa and P. Buslvsky, Cllabatve E-Leanng thugh Open Scal Student delng and Pgessve Zm Navgatn. n: Pc. COLLABORATECO th IEEE Intenatnal Cnfeence n Cllabatve Cmputng: Netwkng, Applcatns and Wkshang, Pttsbugh, USA, [11] S.N. Lndstaedt, G. Beham, B. Kump and T. Ley, Gettng t Knw Yu Use Unbtusve Use del antenance wthn Wk- Integated Leanng Envnments. n: Pc. 4th Eupean Cnfeence n Technlgy Enhanced Leanng (ECTEL 2009) Spnge-Velag, Nce, Fance, 2009, pp [12] A. tvc and B. atn, Evaluatng adaptve pblem selectn. n: Pc. Thd Intenatnal Cnfeence n Adaptve Hypemeda and Adaptve Web-Based Systems (AH'2004) Lectue Ntes n Cmpute Scence 3137, Spnge-Velag, Beln, 2004, pp [13] A. tvc and B. atn, Evaluatng the Effect f Open Student dels n Self-Assessment. Intenatnal Junal f Atfcal Intellgence n Educatn, 17(2) (2007) [14] B. Shnedeman, Tee vsualzatn wth tee-maps: 2-d space-fllng appach. AC Tansactns n Gaphcs, 11(1) (1992) [15]. Yudelsn, Pvdng sevce-based pesnalzatn n an adaptve hypemeda system. PhD Thess. U. f Pttsbugh, [16] J.-D. Zapata-Rvea and J.E. Gee, Inteactng wth Inspectable Bayesan Student dels. Intenatnal Junal f Atfcal Intellgence n Educatn, 14(1) (2004)