arxiv: v1 [cs.lg] 19 Jan 2018

Transcription

1 Ative Learning of Strit Partial Orers: A Case Stuy on Conept Prerequisite Relations Chen Liang 1 Jianbo Ye 1 Han Zhao 2 Bart Pursel 1 C. Lee Giles 1 1 The Pennsylvania State University 2 Carnegie Mellon University {ul226, jxy198, bkp1, giles}@ist.psu.eu han.zhao@s.mu.eu arxiv: v1 [s.lg] 19 Jan 218 Abstrat Strit partial orer is a mathematial struture ommonly seen in relational ata. One obstale to extrating suh type of relations at sale is the lak of large sale labels for builing effetive ata-riven solutions. We evelop an ative learning framework for mining suh relations subjet to a strit orer. Our approah inorporates relational reasoning not only in fining new unlabele pairs whose labels an be eue from an existing label set, but also in evising new query strategies that onsier the relational struture of labels. Our experiments on onept prerequisite relations show our propose framework an substantially improve the lassifiation performane with the same query buget ompare to other baseline approahes. 1 Introution Pool-base ative learning is a learning framework where the learning algorithm is allowe to aess a set of unlabele examples an ask for the labels of any of these examples (Angluin, 1988; Cohn et al., 1996; Settles, 21). Its goal is to learn a goo lassifier with signifiantly fewer labels by atively ireting the queries to the most valuable examples. In a typial setup of ative learning, the label epeneny among labele or unlabele examples is not onsiere. But ata an knowlege in the real worl are often emboie with prior relational strutures. Taking into onsieration those strutures in builing mahine learning solutions an be neessary an ruial (Getoor an Taskar, 7). The goal of this paper is to investigate the query strategies in ative learning of a strit partial orer, namely, when the groun-truth labels of examples onstitute an irreflexive an transitive relation. In this paper, we evelop effiient an effetive algorithms extening popular query strategies use in ative learning to work with suh relational ata. We stuy the following problem in the ative learning ontext: Problem. Given a finite set V, a strit orer on V is a type of irreflexive an transitive (pairwise) relation. Suh a strit orer is represente by a subset G V V. Given an unknown strit orer G, an orale W that returns W (u, v) = [(u, v) G] { 1, 1}, an a feature extrator F : V V R, fin h : R { 1, 1} from a hypothesis lass H that preits whether or not (u, v) G for eah pair (u, v) V V an u v (using h(f(u, v))) by querying W a finite number of (u, v) pairs from V V. Our main fous is to evelop reasonable query strategies in ative learning of a strit orer exploiting both the knowlege from (non-onsistent) lassifiers traine on a limite number of labele examples an the eutive strutures among pairwise relations. Our work also has a partiular fous on partial orers. If the strit orer is total, a large shool alle learning to rank has stuie this topi (Burges et al., 5; Liu, 9), some of whih are uner the ative learning setting (Donmez an Carbonell, 8; Long et al., 21). Learning to rank relies on binary lassifiers or probabilisti moels that are onsistent with the rule of a total orer. Suh approahes are however limite in a sense to prinipally moeling a partial orer: a lassifier onsistent with a total orer will always have a non-zero lower boun of error rate, if the groun-truth is a partial orer but not a total orer. In our ative learning problem, inorporating the eutive relations of a strit orer in soliiting examples to be labele is non-trivial an important. The hallenges motivating us to pursue

2 this iretion an be explaine in three fols: First, any example whose label an be eterministially reasone from a labele set by using the properties of strit orers oes not nee further manual labeling or statistial preition. Seon, probabilisti inferene of labels base on the inepenene hypothesis, as is one in the onventional lassifier training, is not proper any more beause the eutive relations make the labels of examples epenent on eah other. Thir, in orer to quantify how valuable an example is for querying, one has to ombine unertainty an logi to buil proper representations. Soun an effiient heuristis with empirial suess are to be explore. One relate ative learning work that eals with a similar setting to ours is (Renle an Shmit- Thieme, 8), whereas equivalene relations are onsiere instea. Partiularly, they mae several rue approximations in orer to expeite the expete error alulation to a omputational tratable level. We approah the esign of query strategies from a ifferent perspetive while keeping effiieny as one of our entral onerns. To empirially stuy the propose ative learning algorithm, we apply it to onept prerequisite learning problem (Talukar an Cohen, 212; Liang et al., 215), where the goal is to preit whether a onept A is a prerequisite of a onept B given the pair (A, B). Although there have been some researh efforts towars learning prerequisites (Vuong et al., 211; Talukar an Cohen, 212; Liang et al., 215; Wang et al., 216; Sheines et al., 214; Liu et al., 216; Pan et al., 217), the mathematial nature of the prerequisite relation as strit partial orers has not been investigate. In aition, one obstale for effetive learning-base solutions to this problem is the lak of large sale prerequisite labels. Liang et al. (218) applie stanar ative learning to this problem without utilizing relation properties of prerequisites. Ative learning methos tailore for strit partial orers provie a goo opportunity to takle the urrent hallenges of onept prerequisite learning. Our main ontributions are summarize as follows: First, we propose a new effiient reasoning moule for monotonially alulating the eutive losure uner the assumption of a strit orer. This omputational moule an be useful for general AI solutions that nee fast reasoning in regar to strit orers. Seon, we apply our reasoning moule to exten two popular ative learning approahes to hanle relational ata an empirially ahieve substantial improvements. This is the first attempt to esign ative learning query strategies tailore for strit partial orers. Thir, uner the propose framework, we solve the problem of onept prerequisite learning an our approah appears to be suessful on ata from four euational omains, whereas previous work have not exploite the relational struture of prerequisites as strit partial orers in a priniple way. 2 Reasoning of a Strit Orer 2.1 Preliminary Definition 1 (Strit Orer). Given a finite set V, a subset G of V V is alle a strit orer if an only if it satisfies the two onitions: (i) if (a, b) G an (b, ) G, then (a, ) G; (ii) if (a, b) G, then (b, a) G. Definition 2 (G-Orale). For two subsets G, H V V, a funtion enote as W H (, ) : H { 1, 1} is alle a G-orale on H iff for any (u, v) H, W H (u, v) = [(u, v) G]. The G-orale returns a label enoting whether a pair belongs to G. Definition 3 (Completeness of an Orale). A G- orale of strit orer W H is alle omplete if an only if H satisfies: for any a, b, V, (i) if (a, b) H G, (b, ) H G, then (a, ) H G; (ii) if (a, b) H G, (a, ) H G, then (b, ) H G ; (iii) if (b, ) H G, (a, ) H G, then (a, b) H G ; (iv) if (a, b) H G, then (b, a) H G, where G is the omplement of G. W H is alle omplete if it is onsistent uner transitivity when restrite on pairs from H. Definition 4 (Closure). Given a strit orer G, for any H V V, its losure is efine to be the smallest set H suh that H H an the G-orale W H is omplete. Proposition 1. For any H V V, the losure of H subjet to a strit orer G is unique. 1 Proposition 2. Let G be a strit orer of V. For a omplete G-orale W H, H G is also a strit orer of V. Definition 5 (Desenant an Anestor). Given a strit orer G of V an a V, its anestor subjet 1 Please see the supplemental material for the proofs of propositions an theorems introue hereafter.

3 h g h g h g h g e f e f e f e f a? b a b a b a b (a) (b) Figure 1: Following the notations in Theorem 1: (a) Blak lines are pairs in H, soli lines are pairs in G, an ashe lines are pairs in G. The pair (a, b) in the yan olor is the pair to be labele or eue. (b) If (a, b) G, {(a, b), (e, f), (a, f), (e, b)} N (a,b). () If (a, b) G, {(h, e), (h, a)} T (a,b) an {(b, g), (f, g)} S (a,b). () If (a, b) G, {(a, b), (a, ), (, b), (, )} N (a,b). Likewise, if (x, y) G, s.t.(a, b) S (x,y) T (x,y), {(a, b), (a, ), (, b)} O (x,y). () () to G is A G a := {b (b, a) G} an its esenant is D G a := {b (a, b) G}. 2.2 Reasoning Moule for Closure Calulation With the efinitions in the previous setion, this setion proposes a reasoning moule that is esigne to monotonially alulate the eutive losure for strit orers. Remark that a key ifferene between the traitional transitive losure an our efinition of losure (Definition 3&4) is that the former only fouses on G but the latter requires alulation for both G an G. In the ontext of mahine learning, relations in G an G orrespon to positive examples an negative examples, respetively. Sine both of these examples are ruial for training lassifiers, existing algorithms for alulating transitive losure suh as the Warshall algorithm are not appliable. Thus we propose the following theorem for monotonially omputing the losure. Theorem 1. Let G be a strit orer of V an W H a omplete G-orale on H V V. For any pair (a, b) V V, efine the notation C (a,b) by (i) If (a, b) H, C (a,b) := H. (ii) If (a, b) G H, C (a,b) := H N (a,b) where N (a,b) := {(, ) A G H b an partiularly N (a,b) G. {b}, Da G H {a}}, (iii) If (a, b) G H, C (a,b) := H N (a,b) R (a,b) S (a,b) T (a,b) O (a,b), where N (a,b) := {(, ) A G H a {a}, Db G H R (a,b) := {(, ) (, ) N (a,b) }, S (a,b) T (a,b) := {(, e) A G H a (, e) G H}, {a}, D G H b := {(e, ) A G H a {a}, Db G H (e, ) G H}, O (a,b) := (,) S (a,b) T (a,b) N (,), {b}}, {b}, {b}, N (,) := {(f, e) e A G (H N (a,b) ) {}, f D G (H N (a,b) ) {}}. In partiular, N (a,b) G an R (a,b) S (a,b) T (a,b) O (a,b) G. For any pair (x, y) V V, the losure of H = H {(x, y)} is C (x,y). Figure 1 provies an informal explanation of eah neessary onition (exept for R (a,b) ) mentione in the theorem. If (a, b) is a positive example, i.e. (a, b) G, then (i) N (a,b) is a set of inferre positive examples by transitivity; (ii) R (a,b) is a set of inferre negative examples by irreflexivity; (iii) S (a,b) an T (a,b) are sets of inferre negative examples by transitivity; (iv) O (a,b) is a set of negative examples inferre from S (a,b) an T (a,b). If (a, b) is a negative example, i.e. (a, b) G, then N (a,b) is a set of negative examples inferre by transitivity. 2.3 Computational Effiieny As we will elaborate later, one omputational hurle of our ative learning algorithm is to effiiently alulate the losure set C (x,y) given a omplete G-orale W H. In partiular, among all the formula in Theorem 1, we foun the main bottlenek is to effiiently alulate O (x,y) whose worst time omplexity is O( H 3 ) (followe by Prop. 3), while others an be one in O( H 2 ). Proposition 3. Following the notations in Theorem 1, given S (a,b) T (a,b), the worst time omplexity to alulate O (a,b) is O( H 3 ). Using Prop. 4, one an show that there exists a pruning rule that uts a major proportion of reunant set operations in alulating O (a,b). Proposition 4. Following the notations in Theorem 1, we have N (, ) N (,) if (, ) N (,).

4 We also onut empirial stuies to examine the growth rate of alulating C (x,y). In pratie, we fin the empirial growth rate is loser to linear rate, whih means the worst time omplexity boun presente here is very onservative. 3 Pool-Base Ative Learning The pool-base sampling (Lewis an Gale, 1994) is a typial ative learning senario in whih one maintains a labele set D l an an unlabele set D u. In partiular, we let D u D l = D = {1,..., n} an D u D l =. For i {1,..., n}, we use x i R to enote a feature vetor representing the i-th instane, an y i { 1, +1} to enote its grountruth lass label. At eah roun, one or more instanes are selete from D u whose label(s) are then requeste, an the labele instane(s) are then move to D l. Typially instanes are querie in a prioritize way suh that one an obtain goo lassifiers traine with a substantially smaller set D l. We fous on the poolbase sampling setting where queries are selete in serial, i.e., one at a time. 3.1 Query Strategies The key omponent of ative learning is the esign of an effetive riterion for seleting the most valuable instane to query, whih is often referre to as query strategy. We use s to refer to the selete instane by the strategy. In general, ifferent strategies follow a greey framework: s = argmax min f(s; y, D l), (1) s D u y { 1,1} where f(s; y, D l ) R is a soring funtion to measure the risks of hoosing y as the label for x s D u given an existing labele set D l. We investigate two ommonly use query strategies: unertainty sampling (Lewis an Catlett, 1994) an query-by-ommittee (Seung et al., 1992). We show that uner the binary lassifiation setting, they an all be reformulate as Eq. (1). Unertainty Sampling selets the instane whih it is least ertain how to label. We hoose to stuy one popular unertainty-base sampling variant, the least onfient. Subjet to Eq. (1), the resulting approah is to let f(s; y, D l ) = 1 P (Dl )(y s = y x s ), (2) where P (Dl )(y s = y x s ) is a onitional probability whih is estimate from a probabilisti lassifiation moel traine on {(x i, y i ) i D l }. Query-By-Committee maintains a ommittee of moels traine on labele ata, C(D l ) = {g (1),..., g (C) }. It aims to reue the size of version spae. Speifially, it selets the unlabele instane about whih ommittee members isagree the most base on their preitions. Subjet to Eq. (1), the resulting approah is to let f(s; y, D l ) = C k=1 1[y g(k) (x s )], (3) where g (k) (x s ) { 1, 1} is the preite label of x s using the lassifier g (k). Our paper will start from generalizing Eq. (1) an show that it is possible to exten the two popular query strategies for onsiering relational ata as a strit orer. 4 Ative Learning of a Strit Orer Given G a strit orer of V, onsier a set of ata D V V, where (a, a) D, a V. Similar to the pool-base ative learning, one nees to maintain a labele set D l an an unlabele set D u. We require that D D l D u an D l D u =. Given a feature extrator F : V V R, we an buil a vetor ataset {x (a,b) = F(a, b) R (a, b) D}. Let y (a,b) = [(a, b) G] { 1, 1} be the groun-truth label for eah (a, b) V V. Ative learning aims to query Q a subset from D uner limite buget an onstrut a label set D l from Q, in orer to train a goo lassifier h on D l D suh that it preits aurately whether or not an unlabele pair (a, b) G by h(f(a, b)) { 1, 1}. Ative learning of strit orers iffers from the traitional ative learning in two unique aspets: (i) By querying the label of a single unlabele instane, one may obtain a set of labele examples, with the help of strit orers properties; (ii) The relational information of strit orers oul also be utilize by query strategies. We will present our efforts towars inorporating the above two aspets into ative learning of a strit orer. 4.1 Basi Relational Reasoning in Ative Learning A basi extension from stanar ative learning to one uner the strit orer setting is to apply relational reasoning when both upating D l an preiting labels. Algorithm 1 shows the pseuooe for the pool-base ative learning of a strit orer. When upating D l with a new instane (a, b) D u whose label y (a,b) is aquire

5 Algorithm 1 Pseuooe for pool-base ative learning of a strit orer. Input: D V V % a ata set Initialize: D l {(a s1, b s1 ), (a s2, b s2 ),..., (a sk, b sk )} % initial labele set with k sees D l D l % initial losure D u D\D l % initial unlabele set while D u o Selet (a, b ) from D u % aoring to a query strategy Query the label y (a,b ) for the selete instane (a, b ) D l D l {(a, b )} D u D\D l en while from querying, one first alulates D l, i.e., the losure of D l {(a, b)}, using Theorem 1, an then sets D l := D l an D u := D\D l respetively. Therefore, it is possible to augment the labele set D l with more than one pair at eah stage even though only a single instane is querie. Furthermore, the following orollary shows that given a fixe set of samples to be querie, their querying orer oes not affet the final labele set D l onstrute. Corollary 1.1. Given a list of pairs Q of size m whose elements are from V V, let i 1,..., i m an j 1,..., j m be two ifferent permutations of 1,..., m. Let I = an J =, an I k = I k 1 {q ik }, J k = J k 1 {q jk } for k = 1,..., m, where is efine as the losure set uner G. We have I m = J m, whih is the losure of {q i V V i = 1,..., m}. Corollary 1.1 is a straightforwar result from the uniqueness of losure, whih is also verifie by our experiments. The labele set D l ontains two kins of pairs base on where their labels ome from: The first kin of labels omes iretly from queries, an the seon kin omes from the relational reasoning as explaine by Theorem 1. Suh an approah has a lear avantage over stanar ative learning at the same buget of queries, beause labels of part of the test pairs an be inferre eterministially an as a result there will be more labele ata for supervise training. In our setup of ative learning, we train lassifiers on D D l an use them for preiting the labels of remaining pairs that are not in D l. 4.2 Query Strategies with Relational Reasoning The relational ative learning framework as explaine in the previous setion however oes not onsier inorporating relational reasoning in its query strategy. We further evelop a systemati approah on how to ahieve this. We start from the following formulation: at eah stage, one hooses a pair (a, b ) to query base on (a, b ) = argmax min F (S(y (a,b) = y), D l ), (4) y { 1,1} (a,b) D u S(y (a,b) = y) = (D l {(a, b)}\d l ) D. (5) Again, F is the soring funtion. S(y (a,b) = y) is the set of pairs in D whose labels, originally unknown ( D l ), an now be inferre by assuming y (a,b) = y using Theorem 1. For eah (u, v) S(y (a,b) = y), its inferre label is enote as ŷ (u,v) in the sequel. One an see that this formulation is a generalization of Eq. (1). We now proee to evelop extensions for the two query strategies to moel the epenenies between pairs impose by the rule of a strit orer. Following the same notations as esribe in Setion 3 with the only ifferene that the numbering inex is replae by the pairwise inex, we propose two query strategies tailore to strit orers. Unertainty Sampling with Reasoning. With relational reasoning, one not only an reue the unertainty of the querie pair (a, b) but also may reue that of other pairs eue by assuming y (a,b) =y. The moifie soring funtion reas: F (S(y (a,b) = y), D l ) = 1 P (Dl D)(y (u,v) = ŷ (u,v) x (u,v) ). (6) (u,v) S(y (a,b) =y) Query-by-Committee with Reasoning. Likewise, one also has the extension for QBC, where {g (k) } C k=1 is a ommittee of lassifiers traine on bagging samples of D l D, F (S(y (a,b) = y), D l ) = (u,v) S(y (a,b) =y) C k=1 1(ŷ (u,v) g (k) (x (u,v) )). (7) 4.3 Bouns on the Number of Queries Note that any strit orer G an be esribe as a irete ayli graph (DAG). We show the lower an upper bouns on the number of queries that are neee to learn a onsistent lassifier for G.

6 Domain # Conepts # Pairs # Prerequisites Data Mining Geometry Physis Prealulus Table 1: Dataset statistis. Theorem 2. Given a strit orer G of V, let A be a onsistent learner that makes m queries to the orale before termination, then G m G (8) where G is the transitive reution (Aho et al., 1972) of G an G is the losure (Def. 4) of G. The above lower an upper bouns are tight in the sense that there exist DAGs G 1 an G 2, suh that G 1 = G 1 an G 2 = G 2. With the power of the ative learning paraigm, we want to empirially show that the number of queries neee is muh smaller. 5 Experiments For evaluation, we apply the propose ative learning algorithms to onept prerequisite learning problem (Liang et al., 215; Pan et al., 217). Given a pair of onepts (A, B), we preit whether or not A is a prerequisite of B, whih is a binary lassifiation problem. Here, ases where B is a prerequisite of A an where no prerequisite relation exists are both onsiere negative. 5.1 Dataset We use the Wiki onept map 2 ataset from (Wang et al., 216) whih is ollete from textbooks on ifferent euational omains. For eah omain, the ataset onsists of prerequisite pairs in the onept map. Table 1 summarizes the statistis of the our final proesse ataset. 5.2 Features For eah onept pair (A, B), we alulate two types of features following the popular pratie of information retrieval an natural language proessing: graph-base features an text-base features. Please refer to Table 2 for etaile esription. Note we traine a topi moel (Blei et al., 3) on the Wiki orpus. We also traine a Wor2Ve (Mikolov et al., 213) moel on the same orpus with eah onept treate as an iniviual token. 2 Eah onept orrespons to an English Wiki artile. Feature Desription In/Out Degree The in/out egree of A/B. Common Neighbors # ommon neighbors of A an B. # Links # times A/B links to B/A. Link Proportion The proportion of pages that link to A/B also link to B/A. NGD The Normalize Google Distane between A an B (Witten an Milne, 8). PMI The Pointwise Mutual Information relateness between the inoming links of A an B (Ratinov et al., 211). RefD A metri to measure how ifferently A an B s relate onepts refer to eah other (Liang et al., 215). HITS The ifferene between A an B s hub/authority sores. (Kleinberg, 1999) 1st Sent In Title Title Jaar Length Mention NP Tf-if Sim Wor2Ve Sim LDA Entropy LDA Cross Entropy Whether A/B is in the first sentene of B/A. Whether A appears in B s title. The Jaar similarity between A an B s titles. # wors of A/B s ontent. # times A/B are mentione in the ontent of B/A. # noun phrases in A/B s ontent; # ommon noun phrases. The osine similarity between Tf-if vetors for A an B s first paragraphs. The osine similarity between vetors of A an B traine by Wor2Ve. The Shannon entropy of the LDA vetor of A/B. The ross entropy between the LDA vetor of A/B an B/A (Goron et al., 216). Table 2: Feature esription. Top: graph-base features. Bottom: text-base features. 5.3 Experiment Settings We follow the typial evaluation protool of poolbase ative learning. We first ranomly split a ataset into a training set D an a test set D test with a ratio of 2:1. Then we ranomly selet 2 samples from the training set as the initial query set Q an ompute its losure D l. Meanwhile, we set D u = D\D l. In eah iteration, we pik an unlabele instane from D u to query for its label, upate the label set D l, an re-train a lassifiation moel on the upate D l D. The re-traine lassifiation moel is then evaluate on D test. In all experiments, we use a ranom forests lassifier (Breiman, 1) with trees as the lassifiation moel. We use Area uner the ROC urve (AUC) as the evaluation metri. Taking into aount the effets of ranomness subjet to ifferent initializations, we ontinue the above experimental proess for eah metho repeately with 3 preselete istint ranom sees. Their average sores an onfiene intervals (α =.5) are reporte. We ompare four query strategies: : ranomly selet an instane to query. LC: least onfient sampling, a wiely use unertainty sampling variant. We use logisti regression to estimate posterior probabilities.

7 Metho Use reasoning when upating D l Use reasoning to selet the instane to query Use learning to selet the instane to query LC, QBC,, Table 3: Summary of ompare query strategies. QBC: query-by-ommittee algorithm. We apply query-by-bagging (Mamitsuka, 1998) an use a ommittee of three eision trees. : a simple baseline query strategy esigne to greeily selet an instane whose label an potentially infer the most number of unlabele instanes. Following the previous notations, the soring funtion for is F (S(y (a,b) = y), D l ) = S(y (a,b) = y) whih is solely base on logial reasoning. For experiments, we test eah query strategy uner three settings: (i) Traitional ative learning where no relational information is onsiere. Query strategies uner this setting are enote as, LC, an QBC. (ii) Relational ative learning where relation reasoning is applie to upating D l an preiting labels of D test. Query strategies uner this setting are enote as,, an. (iii) Besies being applie to upating D l, relational reasoning is also inorporate in the query strategies. Query strategies uner this setting are the baseline metho an our propose extensions of LC an QBC for strit partial orers, enote as an, respetively. Table 3 summarizes the query strategies stuie in the experiments. 5.4 Experiment Results Effetiveness Stuy Figure 2 shows the AUC results of ifferent query strategies. For eah ase, we present the average values an 95% C.I. of repeate 3 trials with ifferent train/test splits. In aition, Figure 3 ompares the relations between the number of queries an the number of labele instanes aross ifferent query strategies. Note that in the relational ative learning setting querying a single unlabele instane will result in one or more labele instanes. Aoring to Figure 2 an Figure 3, we have the following observations: First, by omparing query strategies uner the settings (ii) an (iii) with setting (i), we observe that inorporating relational reasoning into ative learning substantially improves the AUC performane of eah query strategy. In aition, we fin the query orer, whih is suppose to be ifferent for eah strategy, oes not affet D l at the en when D D l. Thus, it partly verifies Corollary 1.1. Seon, our propose an QBC- R+ signifiantly outperform other ompare query strategies. Speifially, when omparing them with an, we see that inorporating relational reasoning into ireting the queries helps to train a better lassifier. Figure 3 shows that an lea to more labele instanes when using the same amount of queries than that of an. This partly ontributes to the performane gain. Thir, an are more effetive at both olleting a larger labele set an training better lassifiers than the baseline. In aition, by omparing with,, an, we observe that a larger size of the labele set oes not always lea to a better performane. Suh observations emonstrate the neessity of ombining eterministi relational reasoning an probabilisti mahine learning in esigning query strategies Effiieny Stuy The propose reasoning moule is esigne to be plugge into any algorithm that nees reasoning of strit orers. Thus besies verifying effetiveness, it is also important to investigate its effiieny. We onut empirial stuies on the runtime of the reasoning moule. Figure 4 shows the relation between the average runtime for alulating the new losure using Theorem 1 an the size of the urrent labele losure H. Results for both LBC-R+ an are presente. We an see that as H keeps inreasing uring the pool-base ative learning proess, the average runtime of alulating C (x,y) inreases almost linearly an even ereases a little at the en. Although the worst ase time omplexity for alulating Theorem 1 is O( H 3 ) (for O (a,b) ) an O( H 2 ) (for others), the runtime require is iretly relate to the number of asenants an esenants of elements in V, whih is usually ifferent for the four strit orer atasets use. If the few asenants an esenants effetively ontrol the size of alulations as we have observe, the runtime will be short regarless of a large H.

8 LC (% unlabele ata) LC (% unlabele ata) LC (% unlabele ata) LC (% unlabele ata) (a) Data Mining - LC (b) Geometry - LC () Physis - LC () Prealulus - LC QBC (% unlabele ata) (e) Data Mining - QBC QBC (% unlabele ata) (f) Geometry -QBC QBC (% unlabele ata) (g) Physis - QBC QBC (% unlabele ata) (h) Prealulus - QBC Figure 2: Comparison of ifferent query strategies AUC sores for onept prerequisite learning (a) Data Mining - LC (b) Geometry - LC () Physis - LC () Prealulus - LC (e) Data Mining - QBC (f) Geometry - QBC (g) Physis - QBC (h) Prealulus - QBC Figure 3: Comparison of relations between the number of queries an the number of labele instanes when using ifferent query strategies. Avg. runtime (ms) H Avg. runtime (ms) H Avg. runtime (ms) H Avg. runtime (ms) H (a) Data Mining (b) Geometry () Physis () Prealulus Figure 4: The average runtime for alulating the new losure using Theorem 1 v.s. the size of urrent labele losure H. This might explain why the growth of alulating C (x,y) is near linear. We also empirially evaluate the effets of using Prop. 4 on the effiieny an inlue the results in the supplemental material. 6 Conlusion We propose an ative learning framework tailore to relational ata in the form of strit partial orers. An effiient reasoning moule is propose to exten two ommonly use query strategies unertainty sampling an query by ommittee. Experiments on onept prerequisite learning show that inorporating relational reasoning in both seleting valuable examples to label an expaning the training set signifiantly improves stanar ative learning approahes. Future work oul be to explore the following: (i) apply the reasoning moule to exten other query strategies; (ii) ative learning of strit partial orers from a noisy orale.

9 Referenes Alfre V. Aho, Mihael R Garey, an Jeffrey D. Ullman The transitive reution of a irete graph. SIAM Journal on Computing 1(2): Dana Angluin Queries an onept learning. Mahine learning 2(4): Davi M Blei, Anrew Y Ng, an Mihael I Joran. 3. Latent Dirihlet alloation. Journal of mahine Learning researh 3(Jan): Leo Breiman. 1. forests. Mahine learning 45(1):5 32. Chris Burges, Tal Shake, Erin Renshaw, Ari Lazier, Matt Dees, Niole Hamilton, an Greg Hullener. 5. Learning to rank using graient esent. In Pro. ICML. ACM, pages Davi A Cohn, Zoubin Ghahramani, an Mihael I Joran Ative learning with statistial moels. Journal of Artifiial Intelligene Researh 4(1): Pinar Donmez an Jaime G Carbonell. 8. Optimizing estimate loss reution for ative sampling in rank learning. In Pro. ICML. ACM, pages Lise Getoor an Ben Taskar. 7. Introution to statistial relational learning. Jonathan Goron, Linhong Zhu, Aram Galstyan, Prem Natarajan, an Gully Burns Moeling onept epenenies in a sientifi orpus. In Pro. ACL. Jon M Kleinberg Authoritative soures in a hyperlinke environment. Journal of the ACM 46(5): Davi D Lewis an Jason Catlett Heterogeneous unertainty sampling for supervise learning. In Pro. ICML. pages Davi D Lewis an William A Gale A sequential algorithm for training text lassifiers. In Pro. SIGIR. pages Chen Liang, Zhaohui Wu, Wenyi Huang, an C. Lee Giles Measuring prerequisite relations among onepts. In Pro. EMNLP. pages Chen Liang, Jianbo Ye, Shuting Wang, Bart Pursel, an C Lee Giles Investigating ative learning for onept prerequisite learning. In Pro. EAAI. Hanxiao Liu, Wanli Ma, Yiming Yang, an Jaime Carbonell Learning onept graphs from online euational ata. Journal of Artifiial Intelligene Researh 55: Tie-Yan Liu. 9. Learning to rank for information retrieval. Founations an Trens R in Information Retrieval 3(3): Bo Long, Olivier Chapelle, Ya Zhang, Yi Chang, Zhaohui Zheng, an Belle Tseng. 21. Ative learning for ranking through expete loss optimization. In Pro. SIGIR. ACM, pages Naoki Abe Hiroshi Mamitsuka Query learning strategies using boosting an bagging. In Pro. ICML. Morgan Kaufmann Pub, volume 1. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrao, an Jeff Dean Distribute representations of wors an phrases an their ompositionality. In Pro. NIPS. pages Liangming Pan, Chengjiang Li, Juanzi Li, an Jie Tang Prerequisite relation learning for onepts in moos. In Pro. ACL. ACL, pages Lev Ratinov, Dan Roth, Doug Downey, an Mike Anerson Loal an global algorithms for isambiguation to Wikipeia. In Pro. ACL. ACL, pages Steffen Renle an Lars Shmit-Thieme. 8. Ative learning of equivalene relations by minimizing the expete loss using onstraint inferene. In Pro. ICDM. IEEE, pages Rihar Sheines, Elizabeth Silver, an Ilya Golin Disovering prerequisite relationships among knowlege omponents. In Pro. EDM. pages Burr Settles. 21. Ative learning literature survey. University of Wisonsin, Maison 52(55-66):11. H Sebastian Seung, Manfre Opper, an Haim Sompolinsky Query by ommittee. In Pro. COLT. ACM, pages Partha Pratim Talukar an William W Cohen Crowsoure omprehension: preiting prerequisite struture in Wikipeia. In Proeeings of the Seventh Workshop on Builing Euational Appliations Using NLP. ACL, pages Annalies Vuong, Tristan Nixon, an Brenon Towle A metho for fining prerequisites within a urriulum. In Pro. EDM. pages Shuting Wang, Alexaner Ororbia, Zhaohui Wu, Kyle Williams, Chen Liang, Bart Pursel, an C. Lee Giles Using prerequisites to extrat onept maps from textbooks. In Pro. CIKM. ACM, pages Ian Witten an Davi Milne. 8. An effetive, lowost measure of semanti relateness obtaine from Wikipeia links. In Proeeing of AAAI Workshop on Wikipeia an Artifiial Intelligene: an Evolving Synergy. pages 25 3.

10 A Supplemental Material A.1 Proof of Proposition 1 Proof. For any two supersets H 1 H 2 of H whose orales W H1, W H2 are omplete, W H1 H 2, on a smaller set H 1 H 2, is also omplete (by efinition). A.2 Proof of Proposition 2 Proof. For any (a, b), (b, ) H G, beause W H is omplete, (a, ) H G (by Definition 4 (i)). For any (a, b) H G, (b, ) H G (H G) (by Definition 4 (iv)). Therefore, H G is also a strit orer of V (by Definition 1). A.3 Proof of Theorem 1 A.3.1 Well-efiniteness It is trivial that if W H is omplete, G (H N (x,y) ) is also a strit orer. Therefore, N (,) is well efine, so is O (x,y). A.3.2 Neessity One an easily verify that if (x, y) G H, both N (x,y) H (Definition 3 (i)), R (x,y) H (Definition 3 (iv)), an S (x,y) T (x,y) O (x,y) H (Definition 3 (ii),(iii)) from the efinition of losure, an likewise if (x, y) G, N (x,y) H (Definition 3 (ii), (iii)). In another wor, C (x,y) (H) H. Also, see Fig. 1 for the explanation of eah neessary onition mentione. A.3.3 Suffiieny One an see A G a A G a if a A G a an Da G D G a if a Da G. That is, an anestor of anestor is also an anestor (briefly, AAA), an a esenant of esenant is also a esenant (briefly, DDD). Now we proee to prove C (x,y) is omplete using ontraition whih finalizes the proof of our result C (x,y) = H. If C (x,y) is not omplete, by efinition, one of the four onitions in Definition 3 must fail. If Definition 3 (i) fails, there must exist a, b, suh that (a, b) C (x,y) G, (b, ) C (x,y) G, while (a, ) C (x,y). In this ase, if both (a, b) an (b, ) are from H, beause H is omplete, (a, ) H G ontraits the assumption. Hene at least one of (a, b) an (b, ) is not inlue in H. By the efinition of C (x,y), if one pair belongs to G H, it must ome from N (x,y). Therefore, it implies (x, y) G H. Cases 1: If (a, b) N (x,y) an (b, ) N (x,y), (y, b) G an (b, x) G. That however implies (y, x) G, ontraiting G s efinition as a strit orer (See Definition 1 (ii)). Cases 2: If (a, b) N (x,y) an (b, ) H, a A G H x an Dy G H (by DDD). It implies (a, ) N (x,y) C (x,y). Cases 3: If (a, b) H an (b, ) N (x,y), a A G H x (by AAA) an D G H y. It implies (a, ) N (x,y) C (x,y). In summary, Definition 3 (i) hols for C (x,y). If Definition 3 (ii) fails, there must exist a, b, suh that (a, b) C (x,y) G = G (H N (x,y) ), (a, ) C (x,y) G, while (b, ) C (x,y). In this ase, if both (a, b) an (a, ) are from H, beause H is omplete, (b, ) H G ontraits the assumption. Hene at least one of (a, b) an (a, ) is not inlue in H. We ivie the statement into the following ases to isuss: Cases 1: If (x, y) G, (a, b) N (x,y), an (a, ) H G, (a, x) G H an (y, b) G H. Beause (a, x) H G, an (a, ) H G, (x, ) H G. Beause {(x, y), (y, b)} G (H N (x,y) ), (x, b) G (H N (x,y) ). Therefore, (b, ) N (x,) C (x,y). Cases 2: If (x, y) G an (a, ) R (x,y), (, a) N (x,y), thus (, b) N (x,y). It implies (b, ) R (x,y) C (x,y). Cases 3: If (x, y) G an (a, ) S (x,y), (y, a) G H an (, ) G H suh that (, x) G H. Thus, (x, ) G H (y, ) S (x,y) (b, ) N (y,) C (x,y). Cases 4: If (x, y) G an (a, ) T (x,y), (, x) G H an (a, ) G H suh that (y, ) G H. Thus, (a, y) G H (a, x) T (x,y) (a, b) N (x,y). Beause (a, b) G (H N (x,y) ), one has (a, b) G H (b, x) T (x,y) (b, ) N (b,x) C (x,y). Cases 5: If (x, y) G an (a, ) O (x,y), there exists (, e) S (a,b) T (a,b) suh that (a, ) N (,e). Therefore, {(a, b), (, a), (, e)} G (H N (x,y) ). Hene, (b, ) N (,e) C (x,y). Cases 6: If (x, y) G, we have (a, b) G H an (a, ) N (x,y). Thus (x, b) G H (b, ) N (x,y) C (x,y). In summary, all six ases above ontrait the assumption (b, ) C (x,y). Thus Definition 3 (ii) hols for C (x,y). Given we have verifie the onition of Definition 3 (ii), one an also prove that

11 Definition 3 (iii) hols in a similar way, beause their statements as well as efinitions of S (x,y) an T (x,y) are symmetri. One an easily see that Definition 3 (iv) hols for C (x,y), beause G (H N (x,y) ) is also a strit orer an R (x,y) C (x,y). A.4 Proof of Proposition 3 Proof. If (, ) S (a,b), Db G H {b}. then A G H a {a}, thus D G (H N (a,b)) = D G H (by Definition of N (a,b) ). It hols that {} Db G H {b}. Therefore, one has N (,) N (b,) O (a,b) if (, ) S (a,b). Likewise, N (,) N (,a) O (a,b) if (, ) T (a,b). One has D G (H N (a,b)) O (a,b) = N (b,) (,) S (a,b) whose time omplexity is O( H 3 ). A.5 Proof of Proposition 4 N (,a) (,) T (a,b) Proof. Let (, ) N (,). If, (, ) G (H N (a,b) ). A G (H N (a,b)) A G (H N (a,b)) Likewise, D G (H N (a,b)) Then, A G (H N (a,b)) (AAA). Thus, { } A G (H N (a,b)) { } D G (H N (a,b)) By the efinition of N (, ) N (, ) N (,). A.6 Proof of Theorem 2 {}. {}., an N (,), one has We first introue the notion of transitive reution before we proee: Definition 6 (Transitive Reution (Aho et al., 1972)). Let G be a irete ayli graph. We say G is a transitive reution of G if: (i) There is a irete path from vertex u to vertex v in G iff there is a irete path from u to v in G, an (ii) There is no graph with fewer ars than G satisfying (i). For irete ayli graph G, Aho et al. (1972) have shown that the transitive reution is unique an is a subgraph of G. Let G be a simple irete ayli graph (DAG). In ompliane with Def. 4, we use G to enote the transitive losure of G. Define S(G) as the set of graphs suh that every graph in S(G) has the same transitive losure as G, i.e., S(G) := {G G = G} Aho et al. (1972) have shown that S(G) is lose uner intersetion an union. Further more, for DAG G, the following relationship hols: G = G G G = G G S(G) G S(G) Next, we give the proof of Theorem 2 below. Proof. With the fat that negative labels from the query orale annot help to inue positive labels in the graph, we an boun the number of queries, m, neee to learn a lassifier: m G The proof is by simple ontraition base on the efinition of the transitive reution of G an the fat that A is a onsistent learner. On the other han, there is a learning algorithm A that simply remembers all the queries with positive labels an preit all the other inputs as negative. For this algorithm A, it suffies for A to make G queries. A.7 Experiment Environment All experiments are onute on an Ubuntu 14.4 server with 256GB RAM an 32 Intel Xeon E GHz proessors. Ative learning query strategies are implemente in Python2.7. Coe an ata will be publily available. A.8 Effetiveness of Proposition 4 We also empirially evaluate the effets of using Proposition 4 on the effiieny. Speifially, we measure the total runtime of O (a,b) alulation in Theorem 1 for a full roun of ative learning (until D u = ) with an without applying the pruning rule inue by Proposition 4. The results are shown in Table 4 where the numbers are the average runtime over all 3 ifferent rouns of ative learning for eah ataset. We an see that the pruning an lea to a speeup of 4-55% in the

12 Domain w/o pruning w/ pruning w/o pruning w/ pruning Data mining (-4%) (-53%) Geometry (-54%) (-55%) Physis (-47%) (-49%) Prealulus (-55%) (-55%) Table 4: The effet of Proposition 4 on the total runtime (s) of O (a,b) alulation in Theorem 1 for a full roun of ative learning (until D u = ). experiments an a speeup of 49-55% in the QBC- R+ experiments, whih shows that Proposition 4 is helpful for higher effiieny.