Knowledge Spes nd Lerning Spes 1 Jen-Pul Doignon 2 Jen-Clude Flmgne 3 Astrt How to design utomted proedures whih (i) urtely ssess the knowledge of student, nd (ii) effiiently provide dvies for further study? To produe well-founded nswers, Knowledge Spe Theory relies on omintoril viewpoint on the ssessment of knowledge, nd thus deprts from ommon, numeril evlution. Its ssessment proedures fundmentlly differ from other urrent ones (suh s those of S.A.T. nd A.C.T.). They re dpttive (tking into ount the possile orretness of previous nswers from the student) nd they produe n outome whih is fr more informtive thn rude numeril mrk. This hpter repitultes the min onepts underlying Knowledge Spe Theory nd its speil se, Lerning Spe Theory. We egin y desriing the omintoril ore of the theory, in the form of two si xioms nd the min ensuing results (most of whih we give without proofs). In prtil pplitions, lerning spes re huge omintoril strutures whih my e diffiult to mnge. We outline methods providing effiient nd omprehensive summries of suh lrge strutures. We then desrie the proilisti prt of the theory, espeilly the Mrkovin type proesses whih re instrumentl in unovering the knowledge sttes of individuls. In the guise of the ALEKS system, whih inludes tehing omponent, these methods hve een used y millions of students in shools nd olleges, nd y home shooled students. We summrize some of the results of these pplitions. MSC-lss: 91E45 1 Origin nd Motivtion Knowledge Spe Theory (revited s KST) originted with pper y Doignon nd Flmgne (1985). This work ws motivted y the shortomings of the psyhometri pproh to the ssessment of ompetene. The psyhometri models re sed on the notion tht ompetene n e mesured, whih the two uthors thought ws t lest detle. Moreover, typil pplition of psyhometri model in the form of stndrdized test results in pling n individul in one of few dozen ordered tegories, whih is fr too orse lssifition to e useful. In the se of the S.A.T. 4, for exmple, the result 1 A different version is to pper in The New Hndook of Mthemtil Psyhology, Cmridge University Press. The uthors thnk Lurent Fourny nd Keno Merkx for their reful reding of preliminry drft. 2 Université Lire de Bruxelles, Déprtement de Mthémtique.p. 216, B-1050 Bruxelles, Belgium. doignon@ul..e 3 Deprtment of Cognitive Sienes, University of Cliforni, Irvine, CA 92617, USA, jf@ui.edu 4 A test for ollege dmission nd plement in the U.S. The ronym S.A.T used to men Sholsti Aptitude Test. A few yers go the mening of S.A.T. ws hnged into Sholsti Assessment Test. Tody this ronym stnds lone, without ny ssoited mening. 1
of the test is numer etween 200 nd 800 with only multiples of 10 eing possile sores. In the ited pper, Doignon nd Flmgne proposed fundmentlly different theory. The pper ws followed y mny others, written y them nd other reserhers (see the Biliogrphil Notes in Setion 13). The si ide is tht n ssessment in sholrly sujet should unover the individul s knowledge stte, tht is, the ext set of onepts mstered y the individul. Here, onept mens type of prolem tht the individul hs lerned to mster, suh s, in Beginning Alger: or, in Bsi Chemistry solving qudrti eqution with integer oeffiients; lne hemil eqution using the smllest whole numer stoihiometri oeffiients 5. In KST, prolem type is referred to s n item. Note tht this usge differs to tht in psyhometri, where n item is prtiulr prolem, suh s: Solve the qudrti eqution x 2 x 12 = 0. In our se, the exmples of n item re lled instnes 6. The items or prolem types form possily quite lrge set, whih we ll the domin of the ody of knowledge. A knowledge stte is suset of the domin, ut not ny suset is stte: the knowledge sttes form prtiulr olletion of susets, whih is lled the knowledge struture or more speifilly (when ertin requirements re stisfied) the knowledge spe or the lerning spe. The olletion of sttes ptures the whole struture of the domin. As in Beginning Alger, whih will e our led exmple in this hpter, the domin my ontin s mny s 650 items, nd the lerning spe my ontin mny millions sttes, in shrp ontrst with the few dozen soring tegories of psyhometri test. Despite this lrge numer of possile knowledge sttes, n effiient ssessment is fesile in the ourse of 25 35 questions. In Setions 2 to 8, we review the fundmentl omintoril onepts nd the min xiomtiztions. We introdue the importnt speil se of KST, Lerning Spe Theory (LST). As the olletion of ll the fesile, relisti knowledge sttes my e very lrge, it is essentil to find effiient summries. We desrie two suh summries in Setions 4 nd 7. The theory disussed in this hpter hs een extensively pplied in the shools nd universities. The suess of the pplitions is due in lrge prt to feture of the knowledge stte produed y the ssessment: the stte is preditive of wht student is redy to lern. The reson lies in forml result, the Fringe Theorem (see Setion 6). In some situtions, it is importnt to fous on prt of knowledge struture. We ll the relevnt onept projetion of knowledge struture on suset of the items. This is the sujet of Setion 8. 5 For exmple: Fe 2 O 3 (s) Fe(s)+O 2 (g), whih is not lned. The orret response is: 2Fe 2 O 3 (s) 4Fe(s) + 3O 2 (g). 6 So, the instnes of knowledge spe theory re the items of psyhometris. 2
A next setion is devoted to the desription of the Mrkovin type stohsti ssessment proedure (Setion 10). It relies on the notion of proilisti knowledge struture, introdued in Setion 9. In Setion 11, we give n outline of our methods for uilding the fundmentl struture of sttes for prtiulr sholrly domin, suh s Beginning Alger, Pre-Clulus, or Sttistis. Suh onstrutions re enormously demnding nd time onsuming. They rely not only on dedited mthemtil lgorithms, ut lso on huge ses of ssessment dt. The most extensive pplitions of KST re in the form of the we sed system lled ALEKS 7, whih inludes tehing omponent. Millions of students hve used the system, either t home, or in shools nd universities. Setion 12 reports some results of these pplitions. This hpter summrizes key onepts nd results from two ooks. One is the monogrph of Flmgne nd Doignon (2011) 8. The other ook is the edited volume of Flmgne et l. (2013), whih ontins reent dt on the pplitions of the theory nd lso some new theoretil results. A few dditionl results pper here in Setion 7 nd 11. 2 Knowledge Strutures nd Lerning Spes We formlize the ognitive struture of sholrly sujet s olletion K of susets of si set Q of items. In the se of Beginning Alger, the items forming Q re the types of prolems student must mster to e fully onversnt in the sujet. We suppose tht the olletion K ontins t lest two susets: the empty set, whih is tht of student knowing nothing t ll in the sujet, nd the full set Q of prolems. The next definition st these si notions in set-theoreti terms. We illustrte the definition y few exmples. Definition 1. A knowledge struture is pir (Q, K) onsisting of nonempty set Q nd olletion K of susets of Q; we ssume K nd Q K. The set Q is lled the domin of the knowledge struture (Q, K). The elements of Q re the items, nd the elements of K re the knowledge sttes, or just the sttes. A knowledge struture (Q, K) is finite when its domin Q is finite set. For ny item q in Q, we write K q for {K K q K}, the suolletion of K onsisting of ll the sttes ontining q. A knowledge struture (Q, K) is disrimintive when for ny two items q nd r in the domin, we hve K q = K r only if q = r. We often revite (Q, K) into K (with no loss of informtion euse Q = K). 7 ALEKS is n ronym for Assessment nd LErming in Knowledge Spes. 8 This is muh expnded reedition of Doignon nd Flmgne (1999). 3
K (1) Q Q K (2) {,, } {,, d} {,, d} {,, } {,, d} {, } {, d} {, } {, } {, } {} {d} {} {} {} K (3) Q K (4) Q Q K (5) {,, } {,, d} {,, } {,, d} {,, } {,, d} {, } {, d} {, d} {, } {, d} {} {d} {} {d} {} {} Figure 1: The five exmples of knowledge strutures in Exmple 2. Exmple 2. Here re five exmples of knowledge strutures ll on the sme domin Q = {,,, d}: K (1) = {, {}, {d}, {, }, {, d}, {,, }, {,, d}, Q}, K (2) = {, {}, {}, {}, {, }, {, }, {, }, {,, d}, {,, }, {,, d}, Q}, K (3) = {, {}, {d}, {, }, {, d}, {,, }, {,, d}, Q}, K (4) = {, {}, {d}, {, d}, {,, }, {,, d}, Q}, K (5) = {, {}, {}, {, }, {, d}, {,, }, {,, d}, Q}. The five knowledge strutures re finite, nd ll ut K (4) re disrimintive: we hve K (4) = K (4) = {{,, }, {,, d}, Q} with. (1) In the grphs of these knowledge strutures displyed in Figure 1, the sending lines show the overing reltion of the sttes; tht is, we hve n sending line from the point representing the stte K to the one representing the stte L 4
extly when K is overed y L, tht is when K L nd moreover there is no stte A in K suh tht K A L. We ll representtion of knowledge struture s exemplified in Figure 1 overing digrm of the struture. Note in pssing two extreme ses of knowledge strutures on given domin Q. One is (Q, 2 Q ), where 2 Q denotes the power set of Q, tht is the olletion of ll the susets of Q. The other one is (Q, {, Q}), in whih the knowledge struture ontins only the two required sttes nd Q. These two exmples re trivil nd uninteresting euse they entil omplete lk of orgniztion in the ody of informtion overed y the items in Q. Two requirements on knowledge struture mke good pedgogil sense. One is tht there should e no gps in the orgniztion of the mteril: the student should e le to mster the items one y one. Also, there should e some onsisteny in the items: n dvned student should hve less troule lerning new item thn nother, less ompetent student hs. The two onditions inorported in the next definition formlize the two ides. Definition 3. A lerning spe (Q, K) is knowledge struture whih stisfies the two following onditions: [L1] Lerning smoothness. For ny two sttes K, L with K L, there exists finite hin of sttes K = K 0 K 1 K p = L (2) suh tht K i \ K i 1 = 1 for 1 i p (thus we hve L \ K = p). In words: If the lerner is in some stte K inluded in some stte L, then the lerner n reh stte L y mstering items one y one. [L2] Lerning onsisteny. For ny two sttes K, L with K L, if q is n item suh tht K {q} is stte, then L {q} is lso stte. In words: Knowing more does not prevent lerning something new. Notie tht ny lerning spe is finite. Indeed Condition [L1] pplied to the two sttes nd Q yields finite hin of sttes from to Q. In Exmple 2 (see lso Figure 1), only the two strutures K (1) nd K (2) re lerning spes. The knowledge struture K (3) stisfies Condition [L1] in Definition 3 ut not Condition [L2] (tke K =, L = {} nd q = d). As for K (4), it stisfies [L2] ut not [L1]. Finlly, K (5) does not stisfy either ondition. A simpler wy to hek whether overing digrm s in Figure 1 represents lerning spe is provided in the next setion, just fter Theorem 8. We give one more exmple of lerning spe, whih is relisti in tht its ten items elong to the domin of the very lrge lerning spe of Beginning Alger used in the ALEKS system. A remrkle feture of lerning spe is 5
tht ny suset of the items of lerning spe lso defines lerning spe (see elow Definition 31 nd Theorem 33). Thus we hve lerning spe on these ten items. This exmple will e used repetedly lter on, nd in prtiulr to illustrte the ssessment mehnism, tht is, the questioning lgorithm unovering the knowledge stte of student. Tle 1: The items of the ten-item exmple of Figure 2.. Quotients of expressions involving exponents. Plotting point in the oordinte plne using virtul penil on Crtesin grph e. Solving word prolem using system of liner equtions (dvned prolem) g. Multiplition of deiml y whole numer i. Equivlent frtions: fill the lnk in the eqution =, where, nd re whole numers. Multiplying two inomils d. Writing the eqution of line given the slope nd point on the line f. Grphing line given its eqution h. Integer ddition (introdutory prolem) j. Grphing integer funtions Exmple 4. Ten items in Beginning Alger. Tle 1 lists ten items. Rememer tht items re types of prolems, nd not prtiulr ses (whih re lled instnes). Here is n instne of item d: A line psses through the point (x, y) = ( 3, 2) nd hs slope of 6. Write n eqution for this line. Figure 2 shows 34 knowledge sttes in Q = {,,..., j} whih together form lerning spe. 6
Q {,,, d, f, g, h, i, j} {,, d, f, } {,, d, f, } {,,, d, g, h, i, j g, h, i, j g, h, i, j } {,,, f, g, h, i, j } {,,, f, } {,,, g, } {,, f, g, } {,, f, g, g, h, i h, i, j h, i, j h, i, j } {,,, g,h,i } {,, g, } {,, g, h, i, j h, i, j } {, f, g, h, i, j } {,, g, h, i} {,, g, h, i} {,, g, h, i} {, g, h, i, j} {, g, h, i} {, g, h, i} {, g, h, i} {, h, i} {, g, i} {, g, h} {g, h, i} {, i} {h, i} {, g} {g, i} {g, h} {} {i} {g} Figure 2: The overing digrm of the ten-item lerning spe L of Exmple 4. The mening of the four old lines joining the stte {, g, h, i, j} to four other sttes is explined in Setion 6. 7
3 Knowledge Spes nd Wellgrdedness The nme lerning spe speifilly refers to Conditions [L1] nd [L2] of Definition 3. As mentioned erlier, the two onditions hve n interesting, pedgogil interprettion. Other hrteriztions of the sme omintoril onept fous on some other key onepts, whih we review in the present setion. The symmetri differene etween two sets K nd L is defined y K L = (K \ L) (L \ K). Definition 5. A knowledge spe K is knowledge struture whih is losed under union, or -losed, tht is, C K for ny suolletion C of K. The knowledge struture K is well-grded if for ny two sttes K nd L in K, there exists nturl numer h suh tht K L = h nd finite sequene of sttes K = K 0, K 1,..., K h = L suh tht K i 1 K i = 1 for 1 i h. The knowledge struture K is essile or downgrdle 9 if for ny nonempty stte K in K, there is some item q in K suh tht K \ q K. A downgrdle, finite knowledge spe is lled n ntimtroid 10. The losure under union is ritil property euse it enles (sometime highly effiient) summry of knowledge spe y miniml suolletion of its sttes. The suolletion is lled the se of the knowledge spe nd is one of the topis of our next setion. Note tht well-grded knowledge struture (Q, K) is neessrily essile (given the stte K, tke L = in Definition 5). However, n essile knowledge struture is not neessrily finite nor well-grded. Exmple 6. Tke s items ll the nturl numers, thus the domin is N. As sttes, tke the empty set plus ll the susets of N whose omplement is finite. We denote y G the olletion of sttes: G = { } {K 2 N N \ K < + }. (3) The resulting struture (N, G) is essile. It is infinite, nd not well-grded (onsider for instne the two sttes nd N). Theorem 7. For ny knowledge struture (Q, K), the following three sttements re equivlent. (i) (Q, K) is lerning spe. (ii) (Q, K) is n ntimtroid. (iii) (Q, K) is well-grded knowledge spe. Cosyn nd Uzun (2009) proved the equivlene of Conditions (i) nd (ii) in Theorem 7, while Korte et l. (1991) estlished still nother hrteriztion of ntimtroids (or lerning spes): Theorem 8 elow is Lemm 1.2 of their Chpter 3. We provide omined proof of Theorems 7 nd 8 elow. 9 See Dole et l. (2001) for the ltter term. 10 See Korte et l. (1991) for this use of the word. For other uthors n ntimtroid is losed under intersetion rther thn under union (nd upgrdle rther thn downgrdle), see for exmple Edelmn nd Jmison (1985). 8
Theorem 8. A knowledge struture (Q, K) is lerning spe if nd only if its olletion K of sttes stisfies the following three onditions: () Q is finite; () K is downgrdle, tht is: ny nonempty stte K ontins some item q suh tht K \ {q} K; () for ny stte K nd ny items q, r, if K {q}, K {r} K, then K {q, r} K. Theorem 8 mkes it esy to hek whether (finite) overing digrm suh s tht pitured in Figure 1 represents lerning spe. Assume tht the points representing two sttes re t the sme level (or height) if nd only if they hve the sme numer of items; then it suffies to hek tht: (i) ny sending line onnets points t two suessive levels; (ii) ny point representing nonempty stte is the end of t lest one sending line; (iii) if two sending lines strt from the sme point, then their endpoints re the origins of sending lines hving the sme endpoint. Proofs of Theorems 7 nd 8. For given knowledge struture (Q, K), we show tht (i) (ii) (iii) ( (), () nd () ) (i). First notie tht ny of the three onditions (i), (ii), (iii) entils the finiteness of Q, tht is, Condition (). (i) (ii). Let (Q, K) e lerning spe s in Definition 3. We prove tht (Q, K) is n ntimtroid s in Definition 5, in other words tht (Q, K) is finite knowledge spe whih is moreover downgrdle. Suppose tht K, L re sttes. We first pply Lerning Smoothness to nd L nd derive sequene L 0 =, L 1,..., L l = L of sttes suh tht L i \ L i 1 = 1 for 1 i l. Then pplying Lerning Consisteny to the sttes nd K nd the item forming L 1, we derive K L 1 K. Next, we pply Lerning Consisteny to the sttes L 1 nd K L 1 nd the item forming L 2 \ L 1 to derive K L 2 K. The generl step, for i = 1, 2,..., l, pplies Lerning Consisteny to L i 1 nd K L i 1 nd the item forming L i \ L i 1 to derive K L i K. At the lst step (i = l), we get K L K. On the other hnd, downgrdility of K t the stte K is just prtiulr se of Lerning Smoothness t the sttes nd K. (ii) (iii). If (Q, K) is n ntimtroid, then K is losed under union y definition. To prove the wellgrdedness of K (Definition 5), we tke two sttes K nd L. By ssumption, K L is lso stte, nd moreover y downgrdility there exist sequene of sttes M 0 =, M 1,..., M h = K L with M i \ M i 1 = 1 for 1 i h. Then K M 0, K M 1,..., K M h, fter deletion of repetitions, eomes n inresing sequene K 0, K 1,..., K k from K to K L with inrements onsisting of one item. We derive similr sequene L 0, L 1,..., L l from L to K L. Finlly, K 0, K 1,..., K k = L l, L l 1,..., L 0 is the required sequene from K to L (indeed, k + l = K L ). 9
(iii) ((), () nd ()). Downgrdility () is diret onsequene of wellgrdedness. To prove (), we only need to notie K {q, r} = (K {q}) (K {r}) nd pply the ssumed losure under union. ((), () nd ()) (i). To prove Lerning Smoothness, we onsider two sttes K nd L suh tht K L. By downgrdility, there exist two sequenes K 0 =, K 1,..., K k = K nd L 0 =, L 1,..., L l = L of sttes suh tht K i \ K i 1 = 1 for 1 i k nd L j \ L j 1 = 1 for 1 j l. By repeted pplitions of (), we derive K 1 L 1, K 2 L 1,..., K k L 1 K, next K 1 L 2, K 2 L 2,..., K k L 2 K, et., nd finlly K 1 L l 1, K 2 L l 1,..., K k L l 1 K. Thus K L 0 = K, K L 1,..., K L l 1, L re ll in K (rememer K k = K, L l = L nd K L). After deletion of repetitions, we otin the desired sequene from K to L. To prove Lerning Consisteny, we gin onsider two sttes K nd L with K L together with n item q suh tht K {q} K. In the previous prgrph, we proved the existene of sequene M 0 = K, M 1,..., M h = L of sttes suh tht M i \ M i 1 = 1 for 1 i h. Applying () repetedly, we otin M 1 {q} K, M 2 {q} K,..., M h {q} K, the lst one eing L {q} K s desired. A simple se of lerning spe rises when the olletion of sttes is losed under oth union nd intersetion. Definition 9. A qusi ordinl spe is knowledge spe losed under intersetion. A (prtilly) ordinl spe is qusi ordinl spe whih is disrimintive. In Exmple 2, only the struture K (1) is qusi ordinl spe, nd it is even n ordinl spe. The reson for the terminology in Definition 9 lies in Theorem 10 elow, due to Birkhoff (1937). We rell tht qusi order on Q is reflexive nd trnsitive reltion on Q. A prtil order on Q is qusi order on Q whih is n ntisymmetri reltion (tht is, for ll q nd r in Q, it holds tht qrr nd rrq implies q = r). Theorem 10 (Birkhoff, 1937). There exists one-to-one orrespondene etween the olletion of ll qusi ordinl spes K on set Q nd the olletion of ll qusi orders Q on Q. One suh orrespondene is speified y the two equivlenes 11 for ll q, r in Q : qqr Kq Kr; (4) for ll K Q : K K ( (q, r) Q : r K q K). (5) Its restrition to disrimintive spes links prtilly ordinl spes to prtil orders. Note in pssing tht the losure under intersetion does not mke good pedgogil sense. A vrint of Theorem 10 for knowledge spes ppers elow s Theorem 18; vrint for lerning spes follows from Theorem 19. 11 We rell the formul K q = {K K q K}. 10
4 The Bse nd the Atoms In prtie, lerning spes tend to e very lrge, ounting millions of sttes. For vrious purposes for exmple, to store the struture in omputer s memory suh huge strutures need to e summrized. One suh summry is the se of the struture, whih we define elow. Definition 11. The spn of olletion of sets F is the olletion of sets F ontining extly those sets tht re unions of sets in F. We sy then tht F spns F nd we write S(F) = F. So, S(F) is neessrily -losed. A se of -losed olletion S of sets is miniml suolletion B of S spnning S where miniml refers to inlusion, tht is, if S(H) = S for some H B, then neessrily B H. Note tht y ommon onvention, the empty set is the union of zero set in B. Aordingly, the empty set never elongs to se. It is esily shown tht the se of knowledge spe is unique when it exists. Also, ny finite knowledge spe hs se (see Theorems 3.4.2 nd 3.4.4 in Flmgne nd Doignon, 2011). However, some -losed olletion of sets hve no se; n exmple is the olletion of ll the open susets of the set of rel numers. Any lerning spe hs se euse it is finite nd -losed (f. Theorem 7, (i) (iii)). For exmple, the se of the lerning spe K (2) displyed in Figure 1 is { {}, {}, {}, {,, d}, {,, d} }. (6) The eonomy is not gret in this little exmple ut my eome spetulr in the se of the very lrge strutures enountered in prtie. Another reson for the importne of the se stems from pedgogil onept. The relevnt question is: Given some item q, whih miniml stte, or sttes, must e mstered for q to e mstered?. In more diret words: wht re the miniml sttes ontining given item q?. As one might guess, these miniml sets oinide with the elements of the se. Definition 12. Let K e knowledge spe. For ny item q, n tom t q is miniml stte of K ontining q. A stte K is lled n tom if K is n tom t q for some item q. A knowledge spe is grnulr if for ny item q nd ny stte K ontining q, there is n tom t q whih is inluded in K. Clerly, ny finite knowledge spe is grnulr. On the other hnd, stte K is n tom in knowledge spe K if nd only if ny suolletion of sttes F suh tht K = F ontins K (f. Theorem 3.4.7 in Flmgne nd Doignon, 2011). Note lso tht ny grnulr knowledge spe hs se (f. Proposition 3.6.6 in Flmgne nd Doignon, 2011). Exmple 13. For the ten-item lerning spe pitured in Figure 2 (pge 7), there re two toms t item f, nmely {, f, g, h, i, j}, {,,, f, g, h, i}. (7) 11
Tle 2: The toms t ll the items of the ten-item lerning spe from Exmple 4 (see Exmple 13). Items Atoms {, g, h, i} {, g, h, i} {} d {,, d, f, g, h, i, j}, {,, d, f, g, h, i, j}, {,,, d, g, h, i, j} e Q f {, f, g, h, i, j}, {,,, f, g, h, i} g {g} h {h, i}, {g, h} i {i} j {, g, h, i, j} You n hek from the figure tht these two sets re indeed miniml sttes ontining f nd tht they re the only ones with tht property. Note tht there is just one tom t the item, whih is {, g, h, i}, while there re three toms t d. Tle 2 displys the full informtion on the toms. We onlude this setion with the expeted result ( proof is given in Flmgne nd Doignon, 2011 12 ). Theorem 14. Suppose tht knowledge spe hs se. Then this se is extly the olletion of ll the toms. A simple lgorithm, due to Dowling (1993) nd grounded on the onept of n tom, onstruts the se of finite knowledge spe given the sttes. In the sme pper, she lso desries more elorte lgorithm for effiiently uilding the spn of olletion of susets of finite set. Both lgorithms re skethed in Flmgne nd Doignon (2011, pges 49 50) (nother lgorithm for the seond tsk, in nother ontext, is due to Gnter; see Gnter nd Reuter, 1991). The onept of n tom is losely relted to tht of the surmise system, whih is the topi of our next setion. We omplete the present setion with hrteriztion of lerning spes through their toms (Koppen, 1998). 12 The reder should refer to tht monogrph for most of the proofs omitted in this hpter. 12
Theorem 15. For ny finite knowledge struture (Q, K), the following three sttements re equivlent: (i) (Q, K) is lerning spe; (i) for ny tom A t item q, the set A \ {q} is stte; (i) ny tom is n tom t only one item. 5 Surmise Systems In finite knowledge spe, student msters n item q only when his stte inludes some tom C t q. So, the olletion of ll the toms t the vrious items my provide new wy to speify knowledge spe. We illustrte this ide y the following exmple of knowledge spe. Q {,, d, e} {,,, e} {,,, d} {,, e} {,, d} {,, } {, d} {} Figure 3: The overing digrm of the knowledge spe in Exmple 16. Exmple 16. Consider the knowledge spe H = {, {}, {, d}, {,, }, {,, d}, {,, e}, {,,, d}, {,,, e}, {,, d, e}, {,,, d, e} } (8) on the domin Q = {,,, d, e}. Figure 3 provides its overing digrm, while Tle 3 lists its toms. Tle 3 links eh of items, d nd e to single tom of H, nd items nd to three nd two toms, respetively. So, to mster item, one must first mster either item d, or items nd, or items nd e. Exmple 16 illustrtes the following definition. 13
Tle 3: Items nd their toms in the knowledge spe of Eqution (8) (see lso Figure 3). Items Atoms {} {, d}, {,, }, {,, e} {,, }, {,, e} d {, d} e {,, e} Definition 17. Let Q e nonempty set of items. A funtion σ : Q 2 2Q mpping eh item q in Q to nonempty olletion σ(q) of susets of Q (so, σ(q) ) is lled n ttriution funtion on the set Q. For eh q in Q, ny C in σ(q) is lled luse for q (in σ). A surmise funtion σ on Q is n ttriution funtion on Q whih stisfies the three dditionl onditions, for ll q, q Q, nd C, C Q: (i) if C σ(q), then q C; (ii) if q C σ(q), then C C for some C σ(q ); (iii) if C, C σ(q) nd C C, then C = C. In suh se, the pir (Q, σ) is surmise system. A surmise system (Q, σ) is disrimintive if σ is injetive (tht is: whenever σ(q) = σ(q ) for some q, q Q, then q = q ). Then the surmise funtion σ is lso lled disrimintive. It is esily shown tht ny ttriution funtion σ on set Q defines knowledge spe (Q, K) vi the equivlene K K q K, C σ(q) : C K. (9) In ft, we hve the following extension of Birkhoff s Theorem 10 (it is n extension in the sense tht the one-to-one orrespondene we otin extends the orrespondene in Birkhoff s Theorem). The result is due to Doignon nd Flmgne (1985), who derive it from n pproprite Glois onnetion. Theorem 18. There exists one-to-one orrespondene etween the olletion of ll grnulr knowledge spes K on set Q nd the olletion of ll the surmise funtions σ on Q. One suh orrespondene is speified y the equivlene, for ll q in Q nd A in 2 Q, A is n tom t q in K A σ(q). (10) This one-to-one orrespondene links disrimintive knowledge spes to disrimintive surmise funtions. 14
The orrespondene etween knowledge spes nd surmise funtions is suggestive of prtil method for uilding knowledge spe or even lerning spe, sed on nlyzing lrge sets of lerning dt. We desrie suh method in Setion 11. A hrteriztion of lerning spes through their surmise funtions derives diretly from Theorem 15. Theorem 19. A finite knowledge spe (Q, K) is lerning spe if nd only if the orresponding surmise system (s in Theorem 18) hs the property tht ny luse is luse for only one item. In the se of finite, prtilly ordinl spes, highly effiient summry of the spe tkes the form of the Hsse digrm of the prtil order. Attempts to extend the notion of Hsse digrm from prtilly ordered sets to surmise systems re reported in Doignon nd Flmgne (1999) nd Flmgne nd Doignon (2011). 6 The Fringe Theorem The finl result of stndrdized test is numeril sore 13. In the se of n ssessment in the frmework of lerning spe, the result is knowledge stte whih my ontin hundreds of items. Fortuntely, meningful summry of tht stte n e given in the form of its inner fringe nd outer fringe. In the ten-item Exmple 4, onsider the stte {, g, h, i, j}, whih is printed in old in the overing grph of Figure 2 (pge 7). Figure 4 reprodues the relevnt prt of the grph, in prtiulr ll the djent sttes. From the stte {, g, h, i, j}, only three items re lernle 14, whih re, nd f (we mrk them on their respetive lines). On the other hnd, the only wy to reh stte {, g, h, i, j} is to lern item j from the stte {, g, h, i} (whih is the unique stte giving ess to {, g, h, i, j}). The two sets of items {j} nd {,, f} ompletely speify the stte {, g, h, i, j} mong ll the sttes in the struture 15 ; this is remrkle property of lerning spes whih we now formlize. Definition 20. Let (Q, K) e knowledge struture. stte K in K is the set of items The inner fringe of The outer fringe of stte K is the set of items K I = {q K K \ {q} K}. (11) K O = {q Q \ K K {q} K}. (12) 13 Or ouple of suh sores, in the se of multidimensionl model. 14 We men diretly lernle without requiring the mstery of ny other item outside {, g, h, i, j}. 15 In this prtiulr se, the two-set summry is rely more onise thn the originl stte. However, in relisti lerning spes ontining millions of sttes, the summry my e onsiderly smller thn the stte. 15
{ } {,, g, h, i, j },, g, h, i, j j { }, f, g, h, i, j f {, g, h, i, j} {, g, h, i} Figure 4: Prt of Figure 2 showing the items in the inner fringe nd outer fringe of the stte {, g, h, i, j} (see Definition 20). Note tht the empty stte lwys hs n empty inner fringe, nd tht the whole domin Q hs n empty outer fringe. Theorem 21. In lerning spe, ny stte is speified y the pir formed of its inner fringe nd its outer fringe. Theorem 21 hs the following importnt onsequene. In ny lerning spe, the knowledge stte unovered y n ssessment n e reported s two sets of items: those in its inner fringe nd those in its outer fringe. The outer fringe is espeilly importnt euse, ssuming tht the lerning spe is fithful representtion of the ognitive orgniztion of the mteril, it tells us extly wht the student is redy to lern. We will see in Setion 12 tht this informtion is urte in rel-life: the proility tht student tully sueeds in lerning n item piked in the outer fringe of his or her stte, estimted on the sis of hundreds of thousnd ALEKS ssessments, is out.93 (see pge 45). 7 Lerning Words nd Lerning strings In lerning spe, lerner n reh ny stte y lerning its items one t time ut not in ny order. Let us look t n exmple. Exmple 22. A lerning spe on the domin Q = {,,, d} is desried y its overing digrm in Figure 5. The stte {,, d} n e rehed y mstering the items in three possile suessions, whih we lso ll words (see Definition 23): d, d, d. 16
Q {,, } {,, d} {,, d} {, } {, d} {} {} Figure 5: The overing digrm of the lerning spe in Exmple 22. For the mstery of the whole domin Q, there re 6 strings in ll: d, d, d, d, d, d. All of the words nd strings in Exmple 22 shre self-explntory property: for ny of their prefixes, the items ppering in the prefix form knowledge stte. Let us define the new terminology. Definition 23. Given some finite set Q, word on Q is ny injetive mpping f from {1, 2,..., k} to Q, for some k with 0 k Q ; the se k = 0 produes the empty word. With f(i) = w i for 1 i k, we write the word f s w = w 1 w 2 w k, nd we ll k the length of the word w. A prefix of w is word w 1 w 2 w i, where 0 i k (thus ny word is prefix of itself). If n item q does not pper in w, the ontention of w with q is the word w q = w 1 w 2 w k q. A string on Q is word of length Q. Notie tht our words (nd strings) do not involve repetitions (euse of the required injetivity of f), reson for whih some uthors rther spek of simple words (s for exmple Boyd nd Figle, 1990). Any word w = w 1 w 2 w k determines the set w = {w 1, w 2,..., w k } (if ll words re ounted, k! of them determine the sme set of size k). Let (Q, K) e finite knowledge struture with Q = m. A lerning word (in (Q, K)) is word w on Q suh tht for eh of its prefixes v = w 1 w 2 w i 17
the suset ṽ = {w 1, w 2,..., w i } is stte in K; here 0 i k if k is the length of w. A lerning string is suh lerning word with k = m. Korte et l. (1991) use the expression shelling sequene for our lerning word, nd the expression si word for our lerning string, while Eppstein (2013) uses lerning sequene for lerning string. Generl knowledge spes n e without ny lerning string (for instne, it is the se when K = {, Q} s soon s Q 2). The xioms of lerning spes re onsistent with the existene of (mny) lerning strings nd words. In ft, lerning spes n e reognized from properties of the olletion of their lerning strings (see next theorem) or the olletion of their lerning words (see Theorem 26). Theorem 24. Let Q e finite set, with Q = m. A nonempty olletion S of strings on Q is the olletion of ll lerning strings of some lerning spe on Q if nd only if S stisfies the three onditions elow: (i) ny item ppers in some string of S; (ii) if u nd v re two strings in S suh tht for some k in {1, 2,..., m 1} we hve then { u 1, u 2,..., u k 1 } = { v 1, v 2,..., v k 1 } nd u k v k, (13) is prefix of some string in S; u 1 u 2 u k 1 u k v k (14) (iii) if u nd v re two strings in S suh tht for some k in {0, 1,..., m 1} nd some item q we hve {v 1, v 2,..., v k 1, v k, v k+1 } \ {u 1, u 2,..., u k } = {q}, (15) then u 1 u 2 u k q is prefix of some string in S. Proof. (Neessity.) Assume (Q, L) is lerning spe with Q = m, nd denote y S the olletion of its lerning strings. By Lerning Smoothness S is nonempty nd eh item of Q ppers in some string in S, so Condition (i) is true. If the hypothesis of Condition (ii) holds, then {u 1, u 2,..., u k 1 }, {u 1, u 2,..., u k 1, u k } nd {u 1, u 2,..., u k 1, v k } re ll sttes of L. Hene y Lerning Consisteny {u 1, u 2,..., u k 1, u k, v k } is lso stte, whih we denote y L. On the other hnd, y Lerning Smoothness, there is sequene L k+1, L k+2,..., L m of sttes with L k+1 = L, L m = Q, nd L i \ L i 1 = 1 for i = k + 2, k + 3,..., m. Tking w i s the item in L i \ L i 1, we otin the lerning string u 1 u 2 u k 1 u k v k w k+2 w k+3 w m. Thus Condition (ii) holds. Now suppose the strings u nd v fulfil the ssumption in Condition (iii). Thus {v 1, v 2,..., v k+1 } is stte, tht we ll L. By Lerning Smoothness, there is sequene L k+1, L k+2,..., L m of sttes in L with L k+1 = L, L m = Q 18
nd L i \ L i 1 = 1 for i = k + 2, k + 3,..., m. With {w i } = L i \ L i 1, we otin the lerning string Hene Condition (iii) holds. u 1 u 2 u k 1 u k q w k+2 w k+3 w m. (Suffieny.) Given olletion S of strings on Q stisfying (i) (iii), we ll L the olletion of ll prefixes of strings in S. Then nd Q re in L. Moreover, L lerly stisfies downgrdility, tht is Condition () in Theorem 8. To estlish tht L stisfies Condition (), let K, K {q} nd K {r} e in L. There exist strings u nd v suh tht {u 1, u 2,..., u K } = K nd {v 1, v 2,..., v K +1 } = K {q}. Then y Condition (iii) u 1 u 2 u K q is prefix of some string in S. A similr rgument shows tht u 1 u 2 u K r is string prefix. Then y Condition (ii), u 1 u 2 u K q r is lso prefix of some string. So K {q, r} L. Hene, y Theorem 8, (Q, L) is lerning spe. Moreover, the lerning strings of (Q, L) onstitute extly S (this derives gin from the definition of S together with Condition (iii)). Here is n exmple showing tht Conditions (ii) nd (iii) in Theorem 24 re independent. Exmple 25. On the domin Q = {,,, d}, the two strings d, d form olletion whih stisfies Condition (iii) in Theorem 24 ut not Condition (ii) (tke k = 2, u 1 u 2 = nd v 1 v 2 = ). Conversely, the two strings on the sme domin Q = {,,, d} d, d form olletion whih stisfies Condition (ii) in Theorem 24 ut not Condition (iii) (tke k = 2, u 1 u 2 = nd v 1 v 2 v 3 = ). The next result is Theorem 2.1 in Boyd nd Figle (1990) (ompre with Theorem 1.4 in Korte et l., 1991). Theorem 26. Let Q e finite domin. A olletion W of words on Q is the olletion of ll lerning words of some lerning spe on Q if nd only if W stisfies the following three onditions: (i) ny item from Q ppers in t lest one word of W; (ii) ny prefix of word in W lso elongs to W; (iii) if v nd w re two words of W with ṽ w, then for some item q in ṽ \ w the ontention w q is word gin in W. 19
Proof. (Neessity) Assume (Q, L) is lerning spe, nd denote y W the olletion of ll its lerning words. Then y downgrdility of L nd Q L, the olletion W ontins some string, so Condition (i) is true. By the definition of lerning word w, ny prefix of w is lso lerning word, so Condition (ii) holds. Now tke two words v nd w s in Condition (iii). Then w ṽ L (y Theorem 7, L is -losed). Beuse of Lerning Smoothness, there is sequene L 0 = w, L 1,..., L l = w ṽ with L i \ L i 1 = 1 for i = 1, 2,..., l. Let {q} = L 1 \ L 0. Then q ṽ \ w nd w q W. (Suffiieny) Given olletion W of words s in the sttement, set L = { w w W}. Then L. Repetedly pplying Conditions (i) nd (iii), we infer tht there is string in W, nd so Q L. By (ii), L is downgrdle. To onlude tht (Q, L) is lerning spe it now suffies to prove tht L stisfies Condition (iii) in Theorem 8. Let K, K {q} nd K {r} e sttes in L with q r. There re then some words v nd w in W suh tht ṽ = K {q} nd w = K {r}. Beuse ṽ \ w = {q}, Condition (iii) implies w q W. Now w q = K {q, r}, nd so K {q, r} L. Finlly, it is esily heked tht W onsists of ll lerning words of L. Lerning words nd strings form useful tool for the hndling of lrge lerning spes. For instne, they re impliit in the new representtion in Figure 6 of the lerning spe L from Exmple 22: lerning string onsists of the letters (representing items) on pth from the vertex representing to the vertex representing Q. We ll suh representtion (with letters displyed only to show ddition of single item) lerning digrm. Figure 7 on pge 22 shows similr lerning digrm for our ten-item exmple from Exmple 4. Theorem 24 hrterizes lerning spes through their omplete olletions of lerning strings. In the sme vein s the se whih, ontining only reltively smll numer of sttes of the knowledge struture, gives us ess to the whole olletion, we might wnt to summrize in similr wy the olletion of lerning strings in suolletion. The following definition omes from Eppstein (2013). Definition 27. Let S e olletion of strings on finite domin Q. Form the olletion L S ontining ll possile unions of the sets determined y prefixes of strings in S. Then (Q, L S ) is the lerning spe enoded y S. Tht (Q, L S ) indeed forms lerning spe is esy to verify (for instne, pply Theorem 7(ii)). Now, onversely, given lerning spe (Q, L), we my selet ny nonempty suset S of its olletion of lerning strings; then, in generl, L S L holds, ut there is no reson to hve equlity here (see Theorem 13.5.7 in Eppstein, 2013, for riterion). The se in whih we hve the equlity L S = L is interesting for lgorithmi work on the lerning spe L. Let us denote s S 1, S 2,..., S k the strings forming S. Then ny stte in L is univolly enoded y list of nturl numers n 1, n 2,..., n k : eh of the numers speifies the length of the prefix we need to 20
Q d d d Figure 6: The lerning digrm representing the lerning spe in Exmple 22 (see lso Figure 5). extrt from the orresponding string in S to get the stte t hnd s the union of the prefixes. Eppstein (2013) shows how to exploit the new stte enoding for vrious tsks. The present ontext genertes the following prolem: given lerning spe, how do we ompute the smllest numer of lerning strings needed to enode it? The ensuing invrint ws dued onvex dimension y Edelmn nd Jmison (1985) (see lso Korte et l., 1991). Eppstein (2013) gives n lgorithm to ompute it. Exmple 28. The lerning spe of Exmple 22 (see lso Figure 5) is enoded y the following three of its lerning strings: It is s well enoded y the two strings ut never y just one string. d, d, d. d, d, 21
e Q f d d d f d j f j f f j j f j g h i h g i h g h i i h g g i h i g Figure 7: The lerning digrm of the ten-item lerning spe L of Exmple 4 (see lso Figure 6). 22
8 The Projetion Theorem How lrge is the struture of rel-life lerning spe? For instne, wht is the rtio of the numer of knowledge sttes to the numer of possile susets of the domin? In the ten-item exmple of Tle 1 nd Figure 2 we hve 34 knowledge sttes, whih gives the rtio 34 2 10.03. However, this exmple my e misleding. In rel-life lerning spes, the rtio my eome onsiderly smller s soon s there re ouple of dozen items. As nother illustrtion, we gin tke the 37 items exmple in Beginning Alger. There re 4 615 knowledge sttes in the orresponding (indued) lerning spe. With just 37 items, the rtio of the numer of sttes to the numer of susets is 4 615 2 37.03 10 6. We mention in pssing tht there re 217 different knowledge sttes ontining extly 25 items. Presumly ll these knowledge sttes would e ssigned the sme psyhometri sore in lssil test theory, while KST trets them s different. As mentioned in Setion 1, the full domin of Beginning Alger in the ALEKS system ontins out 650 items. The omplexity of the resulting lerning spe is dunting. It lls for wys of prsing suh huge lerning spes into meningful omponents. One of the gols ould e plement test for whih only prt of the full olletion of items is needed. A more importnt reson rises when we need n ssessment on the full struture. In suh se, the numer of knowledge sttes is so lrge tht strightforwrd pproh eomes infesile. A prtil solution hs een worked out whih onsists in suitly prtitioning the domin nd then rrying on simultneous, prllel, mutully informtive ssessments of the resulting sustrutures, ultimtely followed y the omputtion of the finl stte. We outline this tehnique in Setion 10. Here, we define two useful onepts, tht of projetion nd tht of hildren of lerning spe, given suset of the domin. We show without proof tht suh projetion remins lerning spe, while in generl hildren stisfy only some of the requirements in Definition 3. We egin with n illustrtion sed on smll lerning spe. Exmple 29. Let (Q, K) e the lerning spe on the domin {,,, d} whose overing digrm is provided in Figure 8. Consider the suset Q = {, d} of the domin Q nd form ll the tres K Q, for K ny stte in K. The resulting olletion, the projetion of K on Q, {, {d}, {, d} }, (16) forms gin lerning spe. The generl result ppers in Theorem 33(i) elow. We summrize the onstrution in Figure 9. With the sme spe we now illustrte nother onstrution. This time, we sort out the sttes in K ording to their intersetions with Q = {, d}. The resulting equivlene lsses re displyed row y row in Tle 4. Thus the sttes of K in sme row ll hve the sme tre on Q = {, d} (shown in the seond olumn). It hppens tht they lwys form union-stle, wellgrded knowledge struture (see Theorem 33(ii) elow). However, in view of 23
Q {,, d} {,, d} {, } {, d} {} {} Figure 8: The overing digrm of the lerning spe in Exmple 29. the sene of, the two lst strutures do not onstitute lerning spe. In the third olumn, we show the hildren ; they re otined y sutrting from the sttes (of K shown in tht row) their ommon intersetion. Tle 4: The sttes of the lerning spe (Q, K) in Exmple 29 re sorted ording to their intersetion with {, d}. The third olumn provides the orresponding hildren. Clsses of sttes Intersetions with {, d} Children {{,, d}, Q} {, d} {, {}} {{, d}, {,, d}} {d} {, {}} {, {}, {}, {, }} {, {}, {}, {, }} In the seond row of Tle 4, ll the sttes (of the originl lerning spe) hve in ommon the items d (y onstrution) nd moreover. Removing the two ommon items gives the two sets nd {} whih ltogether form lerning spe on the new domin {}. The lst ssertion is not true in generl. Exmple 30. Let (Q, K) e the lerning spe on the domin {,,, d} whose overing digrm is provided in Figure 10. For Q = {}, one of the two hildren equls {{}, {d}, {, d}}; it does not ontin the empty set. We now define the onept of projetion, nd then tht of hildren. 24
Q {,, d} {,, d} {, d} {, } {, d} {} {} {d} Figure 9: An illustrtion of the two onstrutions in Exmple 29 (with Q = {, d}): the three equivlene lsses re in the rounded, dshed retngles; the tres re displyed on the right. Definition 31. Suppose tht (Q, K) is knowledge struture with Q 2, nd let Q e ny proper nonempty suset of Q. For K in K, the suset K Q of Q is the tre of K on Q. The olletion of ll tres is the projetion of K on Q. K Q = {K Q K K} (17) Figure 11 shows the tre of the ten-item lerning spe on {, d, g, j}. Note tht the sets in K Q my e or not e sttes of K. Definition 32. Suppose gin tht (Q, K) is knowledge struture with Q 2, nd let Q e ny proper nonempty suset of Q. Define the reltion Q on K y K Q L K Q = L Q (18) K L Q \ Q. (19) 25
Q {,, } {,, d} {,, d} {, } {, d} {} Figure 10: The overing digrm of the lerning spe in Exmple 30. (The equivlene etween the right hnd sides of (18) nd (19) is esily verified.) Then Q is n equivlene reltion on K. When the ontext speifies the suset Q, we my simply write for Q. We denote y [K] the equivlene lss of ontining K, nd y K = { [K] K K} the prtition of K indued y (or y Q ). For ny stte K in K, we define the olletion K [K] = { L \ [K] L [K]}. (20) The olletion K [K] is Q -hild of K, or simply hild of K when the set Q is mde ler y the ontext. A hild of K my tke the form of the singleton { }; it is then the trivil hild. We refer to K s the prent struture. Beuse K we hve K [ ] = [ ]. We my hve K [K] = K [L] even when K L. (Exmples re esily uilt: see Tle 4.) Theorem 33. Let (Q, K) e lerning spe, with Q 2. The following two properties hold for ny proper nonempty suset Q of Q. (i) The projetion K Q of K on Q is lerning spe. (ii) The hildren of K re well-grded nd -stle olletions. The ltter mens tht the union of ny nonempty suolletion of the olletion lso elongs to the olletion 16. Exmple 29 shows tht the hildren re not neessrily lerning spes. We n impose some restriting onditions on the set Q tht gurntee ny hild to e lerning spe, provided tht the empty stte is dded to the olletion if neessry (see Flmgne nd Doignon, 2011, Definition 2.4.11 nd Theorem 2.4.12). 16 Notie slight differene etween union-stle nd union-losed ; only the seond ondition entils tht the empty set elongs to the olletion. 26
{, d, g, j} Q {,, d, f g, h, i, j } { {,,, d, f, g, h, i, j},, d, f g, h, i, j } {,,, d g, h, i, j } {,,, f g, h, i, j, g, j } { } {,,, f g, h, i } {,,, g, h, i, j } {,, f, g, h, i, j },, f, g, h, i, j { },,, g,h,i { } {,, g, h, i, j },, g, h, i, j { }, f, g, h, i, j {,, g, h, i} {,, g, h, i} {,, g, h, i} {, g, h, i, j} {, g} {, g, h, i} {, g, h, i} {, g, h, i} {, h, i} {, g, i} {, g, h} {g, h, i} {, i} {h, i} {, g} {g, i} {g, h} {} {i} {g} {} {g} Figure 11: Projetion of the ten-item exmple on {, d, g, j}: the dshed lines delinete the equivlene lsses, the little lk retngles show the tres. 27
The onept of projetion plys n essentil role in designing ssessment lgorithms for relisti lerning spes. In pplitions, the size of the olletion of sttes my e so prohiitively lrge tht the ovious strtegy of grdully nrrowing down, y some method or other, the lss of sttes onsistent with the ssessment results is not prtil. The solution disussed in Susetion 10.1 is to first design suitle prtition of the domin into N mngele lsses. Seond, one uilds the N projetions on these lsses. The ssessment proedure then opertes in prllel on the N projetions. A omintion of the results of the N simultneous ssessments delivers finl knowledge stte. 9 Proilisti Knowledge Strutures The onept of lerning spe is deterministi. As suh, it does not provide relisti preditions of sujets responses to the prolems of test. Proilities must enter in t lest two wys in relisti model. For one, the knowledge sttes will ertinly our with different frequenies in the popultion of referene. So, it mkes sense to postulte the existene of proility distriution on the olletion of sttes. For nother, sujet s knowledge stte does not neessrily speify the oserved responses. A sujet hving mstered n item my e reless in responding, nd mke n error. Also, in some situtions, sujet my e le to guess the orret response to question not yet mstered 17. In generl, it mkes sense to introdue onditionl proilities of responses, given the sttes. Definition 34. A proilisti (knowledge struture) (Q, K, p) onsists of finite knowledge struture (Q, K) with proility distriution p on the olletion K of sttes. Thus p(k) is rel numer with 0 p(k) 1, for ny stte K, nd moreover K K p(k) = 1. A prmetrized (proilisti knowledge struture) (Q, K, p, β, η) is proilisti knowledge struture (Q, K, p) equipped with two funtions β : Q [0, 1] : q β q nd η : Q [0, 1] : q η q. The numer β q represents the reless error proility to item q, nd the numer η q is the luky guess proility for item q. Definition 35. A prmetrized proilisti knowledge struture (Q, K, p, β, η) is stright if β q = η q = 0 for ll items q. We now desrie two types of onstrution of proility distriutions on set of sttes. In the first type, the set of sttes results from projetion. Definition 36. Let (Q, K, p) e proilisti knowledge struture, nd let Q Q. On the projetion K Q, we define the projeted distriution p y setting, for K stte in K Q, p (K ) = {p(k) K K nd K Q = K }. 17 Suh luky guesses hve proility zero or re very unlikely in ssessment systems requiring open responses to the items, insted of multiple-hoie. ALEKS is one of those systems. 28
Then (Q, K Q, p ) is the proilisti projetion on Q of the proilisti knowledge struture (Q, K, p). In the seond se, we strt with proilisti knowledge struture nd extend it on lrger set of sttes. Definition 37. Let (Q, K) e knowledge struture, nd let Q Q. Assume p is proility distriution on the projetion K Q. We define the extended distriution p + to K of p y setting, for K stte in K, p + (K) = p (K Q ) {L K L Q = K Q }. Then (Q, K, p + ) is the uniform extension to (Q, K) of the proilisti knowledge struture (Q, K, p ). 10 The Stohsti Assessment Algorithm The generl ide of the ssessment lgorithm is to grdully updte, fter eh response of the sujet, the distriution of proilities on the olletion of sttes. On eh step of the ssessment, the system selets n item, nd presents rndomly hosen instne of tht item to the student. The student s response is evluted nd lssified s orret or flse. The result serves to updte the proility distriution on the set of sttes. The new distriution is the strting point of the next step. Ultimtely, only one or few sttes will remin with high proility. The system then hooses the finl stte. The ssessment lgorithm we just skethed is pplile in the strightforwrd sitution, tht is, when the lerning spe (or the knowledge struture for tht mtter) is modertely lrge, with domin not exeeding 50 items. Suh lerning spes n serve in the design of some plement tests, for exmple. In Susetion 10.2, we del with the more usul se of domins hving hundreds of items. On suh lrge domins, the pplition of the lgorithm requires onsiderly more sophistition euse the numer of sttes eomes so lrge tht operting on the proility distriution on the set of sttes is unmngele. The solution outlined in Susetion 10.2 is to uild suitle prtition of the domin nd to perform prllel (simultneous, mutully informtive) ssessments on the projetions on ll the sudomins. The finl stte is onstruted y omining the outomes otined in eh of the prllel ssessments. 10.1 Sketh of the lgorithm in the strightforwrd sitution We suppose tht, t the outset of the lgorithm exeution, there exists some proility distriution on the olletion of sttes 18. Suh proility distriution my e inferred from some informtion on the popultion tht the 18 This is one of the resons why the lgorithm nnot e redily pplied in the se of lrge domins: we nnot mnge proility distriution on set ontining illions of sttes. 29