13 Learning with Queries

Transcription

1 13 Lerning with Queries Among the more interesting remining theoreticl questions re: inference in the presence of noise, generl strtegies for interctive presenttion nd the inference of systems with semntics. Jerome Feldmn, in [Fel72] L simplicité n ps esoin d être simple, mis du complexe resserré et synthétisé. Alfred Jrry We descrie lgorithm Lstr introduced y Dn Angluin, nd which hs inspired severl vrints nd dpttions to other clsses of lnguges The Minimlly Adequte Techer A minimlly dequte techer is n Orcle tht cn give nswers to memership queries nd strong equivlence queries. We will nlyse in Section 9.2 the cse where you wnt to lern with less. The min lgorithm tht works in this setting is clled Lstr. The generl ide of Lstr is: - find consistent oservtion tle (representing Df); - sumit it s n equivlence query; - use the counter-exmple to updte the tle; - sumit memership queries to mke the tle closed nd complete; - iterte until the Orcle, upon n equivlence query, tells us tht the correct lnguge hs een reched. 317

2 318 Lerning with Queries The oservtion tle we use is nlogous to tht descried in Section 12.3, so we will use the sme formlism here An oservtion tle An oservtion tle is specific tulr representtion of n utomton. An exmple is given in Tle 13.1(). λ λ () An oservtion tle., q λ q () The corresponding utomton. Fig The oservtion tle nd the corresponding utomton. The mening of the tle cn e mde resonly cler. By conctenting the nme of row r with the nme of column c we get string rc. The string is in the lnguge if the corresponding cell Ot[r][c] contins 1, does not if it is 0. If the tle complies with certin conditions of consistency n utomton cn esily e extrcted from the tle. The utomton corresponding to Tle 13.1() is depicted in Figure 13.1(). We formlly descrie procedure llowing to extrct Df from tle (when possile) in Section An oservtion tle is triple St,Exp,Ot with: - St = Red Blue is set of strings, denoting lels of sttes; - Red Σ is finite set of sttes. - Exp Σ is the experiment set; - Blue = Red Σ \ Red is the set of sttes successors of Red tht re not Red; - Ot : (St) Exp {0,1, } is function such tht: 1 if ue L - Ot[u][e] = 0 if ue L otherwise (not known). Agin, to simplify, Red nd Blue will e sets of strings lso used to lel the sttes. There re numer of key ides one wnts to understnd in order to grsp this lgorithm.

3 13.1 The Minimlly Adequte Techer 319 Definition (Holes) A hole in tle St,Exp,Ot is pir (u,e) such tht Ot[u][e] =. A tle is complete if def it hs no holes. λ λ () An incomplete tle. q q λ q q q q q () The corresponding utomton. Fig The utomton corresponding to n incomplete tle. The prolem with incomplete tles is tht we do not hve ll the informtion needed to extrct Df from tle. In Section 12.3 (pge 286) this ws the key prolem nd no stisfying solution ws given. Consider for instnce Tle 13.2(). There re severl holes tht could e filled in vrious mnners. For exmple, the hole corresponding to Ot[][] indictes tht there is no fixed or known vlue for δ(q,). In order to uild Df from this incomplete tle, we notice tht δ(q,) could e just s well q λ s q. The sitution is represented Figure 13.2() Building from complete nd closed tle Building n utomton from tle St,Exp,Ot cn e done very esily if certin conditions re met: - The set of strings mrking the sttes in St must e prefix-closed; - The set Exp is suffix-closed; - The tle must e complete nd therefore hve no holes; - The tle must e closed. If these conditions hold we cn use Algorithm Lstr-BuildAutomton (13.1, similr to Algorithm 12.5, pge 291). Exmple Consider Tle 13.3(). We cn pply the construction from Algorithm 13.1 nd otin Q = {q λ, q }, F A = {q }, F R = {q λ } nd δ

4 320 Lerning with Queries Algorithm 13.1: Lstr-BuildAutomton. Input: closed nd complete oservtion tle (St,Exp,Ot) Output: Df Σ,Q,q λ, F A, F R,δ Q {q u : u Red v < u Ot[v] Ot[u]}; F A {q u Q : Ot[u][λ] = 1}; F R {q u Q : Ot[u][λ] = 0}; for q u Q do for Σ do δ(q u,) q w Q : Ot[u] = Ot[w] end return Σ,Q,q λ, F A, F R,δ is given y the trnsition Tle 13.3(), then the Automton 13.3(c) cn e uilt: λ λ () The oservtion tle. q λ q λ q q q λ q () The trnsition tle. q λ (c) Automton. q Fig An utomton nd tle Consistency Definition (Consistent tle) Given n utomton A nd n oservtion tle St,Exp,Ot, A is consistent with St,Exp,Ot when the following holds: - Ot[u][e] = 1 = ue L FA (A); - Ot[u][e] = 0 = ue L FR (A). Rememer tht L FA (A) is the lnguge recognised y A y ccepting sttes, wheres L FR (A) is the lnguge recognised y A y rejecting sttes.

5 13.1 The Minimlly Adequte Techer 321 Theorem (Consistency Theorem) Let St, Exp, Ot e n oservtion tle closed nd complete. If St is prefix-closed nd Exp is suffix-closed then Lstr-BuildAutomton( St,Exp,Ot ) is consistent with the dt in St,Exp,Ot. Proof Lstr-BuildAutomton( St, Exp, Ot ) is uilt from the dt from St, Exp, Ot. Completing the tle y sumitting memership queries should llow to solve some of the miguity issues, even if not ll Tles with no holes We consider here the cse where there re no holes in the tle. To rech this sitution, we will hve filled the holes y using memership queries. Definition (Equivlent prefixes nd rows) Two prefixes u nd v re equivlent if def Ot[u] = Ot[v]. We will denote this y u Exp v. The next definition is similr to Definition , pge 290. Definition (Closed tle) A tle St,Exp,Ot ) is closed if def given ny row u of Blue there is some row v in Red such tht u Exp v. Checking if the tle is closed is strightforwrd. But wht cn the lgorithm do once it hs found tht the tle is not closed? Let s e the row (of Blue) tht does not pper in Red, dd s to Red, nd Σ, dd s to Blue. This corresponds to the promotion introduced in Section , pge 285. By repeting this until the tle is closed, we re done. Notice tht the numer of itertions is ounded y the size of the utomton. Exmple In Gold s lgorithm (Section 12.3) ll the Red sttes were oviously different one from the other. Moreover, stte ws moved to the upper prt of the tle only when this condition ws met. Becuse of the lck of control the lerner hs over the Orcle, this is not the cse here. An inconsistent tle is one from which n utomton cnnot e extrcted. This is different from Section 12.3: In this cse it is possile to hve Red prefixes/sttes tht seem equivlent nd need seprting.

6 322 Lerning with Queries λ λ () A tle tht is not closed, ecuse of row. Fig Closing tle. λ λ () Closing the tle. Definition A tle is consistent if def every equivlent pir of rows in Red remins equivlent in St fter ppending ny symol. Ot[s 1 ] = Ot[s 2 ] = Σ, Ot[s 1 ] = Ot[s 2 ]. Wht do we do when we hve n inconsistent tle? If it is inconsistent, then let Σ e the symol for which Ot[s 1 ] = Ot[s 2 ] ut Ot[s 1 ] Ot[s 2 ]. Let e e the experiment for which the inconsistency hs een found (Ot[s 1 ][e] Ot[s 2 ][e]). Then y dding experiment e to the tle, rows Ot[s 1 ] nd Ot[s 2 ] re different. Indeed, Ot[s 1 ][e] Ot[s 2 ][e]. Exmple Tle 13.5() is inconsistent: rows nd look the sme, ut, upon experiment they fil to e equivlent, since rows nd re different. Therefore column (nd experiment) is dded, resulting in Tle 13.5(). On the other hnd, Tle 13.5(c) is consistent, since not only Ot[] = Ot[], ut lso Ot[] = Ot[] nd Ot[] = Ot[]. λ λ () An inconsistent tle (ecuse of nd ). λ λ () The tle hs ecome consistent. Fig Consistency. λ λ (c) A consistent tle.

7 13.2 The lgorithm 323 Once the lerner hs uilt complete, closed nd consistent tle, it cn construct the Df using Algorithm Lstr-BuildAutomton nd mke n equivlence query! Oviously, if the Orcle returns positive nswer to the lgorithm s equivlence query, it cn hlt. If she returns counter-exmple (u), then the lerner should dd s Red sttes ll the prefixes of u, nd complete the Blue section ccordingly (with ll strings p ( Σ) such tht p is prefix of u ut p is not. In this wy t lest one new Red line oviously different from ll the others will hve een dded The lgorithm Algorithm Lstr (13.2) cn now e descried. First the oservtion tle is initilised y Algorithm Lstr-Initilise (13.3). This consists in uilding one Red row (λ) nd s mny Blue rows s there re symols in the lphet. Then the itertive construction egins. When the tle is not closed n extr row is dded (Algorithm Lstr-Close (13.4)), when it is inconsistent n extr column is dded (Algorithm Lstr-Consistent (13.5)). At every moment, memership queries re mde to fill in the holes. When redy nd the tle is closed nd consistent, n equivlence query is mde nd if unsuccessful, new rows re dded (Algorithm Lstr-UseEQ (13.6)). Algorithm 13.2: Lstr Lerning Algorithm. Input: Output: Df A Lstr-Initilise; repet while St,Exp,Ot is not closed or not consistent do if St, Exp, Ot is not closed then St, Exp, Ot Lstr-Close( St, Exp, Ot ); if St, Exp, Ot is not consistent then St, Exp, Ot Lstr-consistent( St, Exp, Ot ) end Answer Eq( St, Exp, Ot ); if Answer Yes then St, Exp, Ot Lstr-UseEQ( St, Exp, Ot, Answer) until Answer= Yes ; return Lstr-BuildAutomton( St, Exp, Ot )

8 324 Lerning with Queries Algorithm 13.3: Lstr-Initilise. Input: Output: tle St, Exp, Ot Red {q λ }; Blue {q : Σ}; Exp {λ}; Ot[λ][λ] Mq(λ); for Σ do Ot[][λ] Mq(); return St, Exp, Ot Algorithm 13.4: Lstr-Close. Input: tle St,Exp,Ot Output: tle St, Exp, Ot updted for s Blue such tht u Red Ot[s] Ot[u] do Red Red {s}; Blue Blue \ {s}; for Σ do Blue Blue {s }; for u,e Σ such tht Ot[u][e] is hole do Ot[u][e] Mq(ue) end return St, Exp, Ot Algorithm 13.5: Lstr-Consistent. Input: tle St, Exp, Ot Output: tle St, Exp, Ot updted find s 1, s 2 Red, Σ nd e Exp such tht Ot[s 1 ] = Ot[s 2 ] nd Ot[s 1 ][e] Ot[s 2 ][e]; Exp Exp { e}; for u,e Σ such tht Ot[u][e] is hole do Ot[u][e] Mq(ue); return St, Exp, Ot

9 13.2 The lgorithm 325 Algorithm 13.6: Lstr-UseEQ. Input: tle St,Exp,Ot, string Answer Output: tle St, Exp, Ot updted for p Pref(Answer) do Red Red {p}; for Σ : p Pref(Answer) do Blue Blue {p} end for u,e Σ such tht Ot[u][e] is hole do Ot[u][e] Mq(ue); return St, Exp, Ot A run of Ä Ø Ö We run Lstr over n exmple. We strt with the empty tle, in which Red = {λ} nd Exp = {λ}. A first memership query is mde with string λ. The nswer is Yes. nd re then dded to Blue nd the memership queries nd re mde. Suppose gin the nswers re Yes. The corresponding Tle 13.6() is closed nd complete, so n equivlence query is mde for the utomton depicted in Figure 13.6(). Let e the negtive counter-exmple returned y the Orcle. The tle is updted to Tle 13.7(). The tle is closed so the holes re filled through memership queries re mde, yielding Tle 13.7(). λ λ () A consistent tle., () The utomton corresponding to the Tle 13.6(). 0 Fig Consistency. But the Tle 13.6() is not closed s rows Ot[] nd Ot[] coincide wheres rows Ot[ ] nd Ot[ ] do not. Experiment is the reson for this so it is dded s n experiment nd the new Tle 13.7(c) hs to e completed. Tle 13.7(d) is therefore otined which is now closed nd complete nd cn e trnsformed into n utomton tht will e proposed s n equivlence query (Figure 13.8). We suppose this time the equivlence query is met with positive nswer so we hlt.

10 326 Lerning with Queries λ λ () Tle fter equivlence query returned (s not in L). λ λ () Memership queries re mde: Tle is not closed. λ λ (c) Adding column to mke the tle closed. λ λ (d) The tle fter filling the holes is closed nd consistent. Fig Running Lstr., q λ q q Fig Automton fter running Lstr Proof of the lgorithm On the first hnd the lgorithm Lstr clerly termintes: Since every regulr lnguge dmits unique miniml Df, let us suppose, without loss of generlity, tht the trget is this minimum Df with n sttes. But since ny Df consistent with tle hs t lest s mny sttes s different rows in Red, nd if tle is closed nd consistent then the construction of consistent Df is unique, therefore the tle cn only grow verticlly until it hs n different rows. Now, ech closure filure dds one different row to Red, ech inconsistency filure dds one experiment, which lso cretes new row in Red. Ech counter-exmple dds lso one different row to Red (notice tht mny rows cn pper ecuse of the prefixes, ut wht mtters is tht t lest one different from the others ppers). If to this you dd tht every time the tle is not consistent or n equivlence query is met y counterexmple, t lest one new row is introduced. Furthermore, the numer of steps etween two of such events is lso finite. So the totl numer of these opertions is ounded. Now, for correction, if the lgorithm hs uilt tle with n oviously

11 13.3 Exercises 327 different rows in Red, nd n is the size of the miniml Df for the trget, then it is the trget. The lgorithm therefore is correct nd termintes. Let us now study the complexity of Lstr: - At most n experiments will e mde (including λ), since n experiment introduces necessrily new different row. So Exp n. - For the sme resons t most n equivlence queries re mde. - The numer of memership queries is ounded y the totl size of the tle which is t most n (the numer of experiments/columns) multiplied y the numer of lines ( nm where m is the length of the longest counterexmple returned y the Orcle). Therefore the totl numer of queries mde is t most n 2 m. A computtion of this numer t ech step of the lgorithm is possile nd gives similr results: The tle only grows with the size of the counter-exmples returned y the Orcle Aout implementtion issues One difficulty with the implementtion of Lstr comes from mintining the redundncy. Actully it is not necessry to implement the ctul tle. A etter ide is to mnge three ssocition tles: A first tle Mq contins the result of the memership queries. It cn e consulted in constnt time to know if prticulr string hs een queried or not nd if it hs if it elongs or not to the lnguge. A second tle Pref contins the different nmes of rows, nd for ech row, the sttus: is it Red or Blue? A third tle just contins the different experiments. The ctul oservtion tle is only simulted y function Ot(u, v) which will return the vlue Mq[uv] Exercises 13.1 Run lgorithm Lstr on the utomton from Figure You will need to simulte the Orcle too! 13.2 Replce the equivlence queries with the use of smpling query Ex() in the ove. Wht vlues of m i should you consider?

12 328 Lerning with Queries 1 0 Fig A trget utomton It ws suggested in Section 9.4 (pge 224) tht equivlence queries cn e replced y smpling. Write the lgorithm llowing to ctully simulte the equivlence query If one chooses to smple insted of mking equivlence query, one prolem is: Wht do we do with ll the exmples tht did fit the hypothesis until counter-exmple ws found? One lterntive is to enter ll this informtion into the oservtion tle. Another is to ignore it. Which is etter? Why? 13.4 Conclusion of the section nd further reding In Chpter 9 we discussed numer of implictions of ctive lerning. In prticulr, negtive results were given Biliogrphicl ckground Algorithm Lstr ws is due to Dn Angluin [Ang90] nd ws lter dpted for rootics scenrio y Ron Rivest nd Roert Schpire [RS93], nd hs led to numer of vrints (oth in description nd in the comintion etween the memership nd the equivlence queries [KV94, BDGW94]). We hve concentrted in this chpter on Df. In the cse of context-free grmmrs the negtive proofs y Dn Angluin nd Michel Khritonov [AK91] with Mts re relted to cryptogrphic ssumptions. On the other hnd, if structurl informtion is ville, Ysuumi Skkir proves the lernility of the clss of context-free grmmrs in this model [Sk90]. Lerning lls of strings from different types of queries hs lso een studied [dlhjt08]. Returning to the Df cse, it should e noticed tht the Orcle hs no reson to return the counter-exmple the lerner relly needs. A more helpful setting ws studied in [BBS00]. Other studies contemplte the fct tht the Orcle is somehow ounded:

13 13.4 Conclusion of the section nd further reding 329 In prctice, it my e difficult to imgine tht the Orcle hs the resources to return n exponentilly long exmple; furthermore, if the Lerner cn find hypothesis correct over the strings of resonle length only, this my e sufficient. These questions re discussed in [Wt94, Cs01] Some lterntive lines of reserch The model hs received considerle ttention nd there re mny ppers on lerning with different sorts of queries. Ysuumi Skkir [Sk87] lerns context-free grmmrs from queries; Tkshi Yokomori [Yok96] lerns 2-tpe utomt from oth queries nd counter-exmples, nd in [Yok94] non-deterministic finite utomt from queries lso in polynomil time, ut depending on the size of the ssocited Df; Jun-Mnuel Vilr extends queries to trnsltion tsks in [Vil96], Oded Mler nd Amir Pnueli [MP91] lern Büchi utomt from queries over infinite strings. A recent ide is tht of comining memership queries nd equivlence queries in some wy. Correction queries [BBBD05, BBDT06] correspond to strings tht the lerner hypotheses s eing in the lnguge. The Orcle then presents some correction of the string if the string does not elong to the lnguge. There re mny wys of defining such corrections, some eing more theoreticl nd other (using the edit distnce) closer to possile pplictions [Tir08, Kin08, BBdlHJT08]. Agin, s in other questions, the prolem of correctly defining topology over the lnguges is here of crucil importnce, s the sort of correction one would expect is one of string close to the queried string. In prctice, getting hold of equivlence queries is considered to e the hrdest of prolems. One wy round this is to smple. An evolutionry lgorithm following this line is proposed in [BL05]. One cn lso consider the cse where the Orcle cn nswer proilistic queries. We visit this question in the corresponding chpters (Chpter 10 for some negtive results, nd Chpter 16 for some positive ones). A typicl ide is to introduce specific smpling queries in order to lern proilistic mchines [dlho04] Open prolems nd possile new lines of reserch We proposed in Section some prolems relting to lerning with queries. More generlly, there re numer of resons for which inventing new query lerning lgorithms re mking the existing ones more efficient (not just in

14 330 Lerning with Queries time, in numer of queries lso) re importnt issues, nd our feeling is tht more reserch in this re should e encourged.