Learning Automata with Hankel Matrices

Transcription

1 Lerning Automt with Hnkel Mtrices Borj Blle [Disclimer: Work done efore joining Amzon]

2 Brief History of Automt Lerning [967] Gold: Regulr lnguges re lernle in the limit [987] Angluin: Regulr lnguges re lernle from queries [993] Pitt & Wrmuth: PAC-lerning DFA is NP-hrd [994] Kerns & Vlint: Cryptogrphic hrdness [9 s, s] Clrk, Denis, de l Higuer, Oncin, others: Comintoril methods meet sttistics nd liner lger [29] Hsu-Kkde-Zhng & Billy-Denis-Rlivol: Spectrl lerning

3 Tlk Outline Exct Lerning Hnkel Trick for Deterministic Automt Angluin s L* Algorithm PAC Lerning Hnkel Trick for Weighted Automt Spectrl Lerning Algorithm Sttisticl Lerning Hnkel Mtrix Completion

4 H P R ˆ p s p s ñ Hpp, sq Hpp, s q f : Ñ R H f pp, sq f pp sq The Hnkel Mtrix s p Hpp, sq.

5 Hnkel Mtrices nd DFA q q q 2 Theorem (Myhill-Nerode 58) The numer of distinct rows of inry Hnkel mtrix H equls the miniml numer of sttes of DFA recognizing the lnguge of H.

6 From Hnkel Mtrices to DFA..

7 Closed nd Consistent Finite Hnkel Mtrices S The DFA synthesis lgorithm requires: Sets of prexes P nd sufxes S Hnkel lock over P = P PΣ nd S Closed: rows(pσ) rows(p) Consistent: row(p) = row(p ) row(p ) = row(p ) P P

8 Lerning from Memership nd Equivlence Queries Setup: Two plyers, Techer nd Lerner Concept clss C of function from X to Y (known to Techer nd Lerner) Protocol: Techer secretly chooses concept c from C Lerner s gol is to discover the secret concept c Techer nswers two types of queries sked y Lerner Memership queries: wht is the vlue of c(x) for some x picked y the Lerner? Equivlence queries: is c equl to hypothesis h from C picked y the Lerner? If not, return counter-exmple x where h(x) nd c(x) differ Angluin, D. (988). Queries nd concept lerning.

9 Angluin's L* Algorithm ) Initilize P = {ε} nd S = {ε} 2) Mintin the Hnkel lock H for P = P PΣ nd S using memership queries 3) Repet: While H is not closed nd consistent: If H is not consistent dd distinguishing sufx to S If H is not closed dd new prex from PΣ to P Construct DFA A from H nd sk n equivlence query If yes, terminte Otherwise, dd ll prexes of counter-exmple x to P Complexity O(n) EQs nd O( Σ n 2 L) MQs Angluin, D. (987). Lerning regulr sets from queries nd counterexmples.

10 Weighted Finite Automt (WFA) Grphicl Representtion Algeric Representtion,.2, 2 q.2, 2,.5,, 2, 3.2, 5 q A x,, ta u P y A A Functionl Representtion Apx x t q J A x A xt

11 Hnkel Mtrices nd WFA Theorem (Fliess 74) The rnk of rel Hnkel mtrix H equls the miniml numer of sttes of WFA recognizing the weighted lnguge of H App p t s s t q J A p A pt A s A st p q s p Appsq

12 From Hnkel Mtrices to WFA H pp, sq Appsq App p t s s p t q q J A p A pt A A s A st p Appsq s H P S H P A S A P` H S `

13 WFA Reconstruction vi Singulr Vlue Decomposition Input: Hnkel H over P = P PΣ nd S, numer of sttes n ) Extrct from H the mtrix H over P nd S 2) Compute the rnk n SVD H = U D V T 3) For ech symol : Extrct from H the mtrix H over P nd S Compute A = D - U T H V Roustness Property }H Ĥ } " ñ }A Â } Op"q Blle, B., Crrers, X., Luque, F. M., & Quttoni, A. (24). Spectrl lerning of weighted utomt.

14 Proly Approximtely Correct (PAC) Lerning Fix clss D of distriutions over X Collect m i.i.d. smples Z = (x,..., x m ) from some unknown distriution d from D An lgorithm tht receives Z nd outputs hypothesis h is PAC-lerner for the clss D if: Whenever m > poly( d, /ε, log /δ), with proility t lest δ the hypothesis stises distnce(d,h) < ε The lgorithm is n efcient PAC-lerner if it runs in poly-time Kerns, M., Mnsour, Y., Ron, D., Ruinfeld, R., Schpire, R. E., & Sellie, L. (994). On the lernility of discrete distriutions. Vlint, L. G. (984). A theory of the lernle.

15 $ & % Estimting Hnkel Mtrices from Smples, /.,,,,,,,, /- Smple,,,,,,, Concentrtion Bound }H Ĥ} O, /. /- ˆ? m. Empiricl Hnkel Mtrix 3 3 Denis, F., Gyels, M., & Hrrd, A. (24, Jnury). Dimension-free concentrtion ounds on hnkel mtrices for spectrl lerning.

16 Spectrl PAC Lerning of Stochstic WFA Algorithm:. Estimte empiricl Hnkel mtrix 2. Use spectrl WFA reconstruction Efcient PAC-lerning: Running time: liner in m, polynomil in n nd size of Hnkel mtrix Accurcy mesure: L distnce on ll strings of length t most L Smple complexity: L 2 Σ n /2 / σ 2 ε 2 Proof: roustness + concentrtion + telescopic L ound Billy, R., Denis, F., & Rlivol, L. (29). Grmmticl inference s principl component nlysis prolem. Hsu, D., Kkde, S. M., & Zhng, T. (29). A spectrl lgorithm for lerning hidden mrkov models.

17 Sttisticl Lerning in the Non-relizle Setting Fix n unknown distriution d over X x Y (inputs, outputs) Collect m i.i.d. smples Z = ((x,y ),...,(x m,y m )) from d Fix hypothesis clss F of functions from X to Y Find hypothesis h from F tht hs good ccurcy on Z Empiricl Risk Minimiztion min hpf m mÿ i `phpx i q, y i q In such wy tht it hs good ccurcy on future (x,y) from d E px,yq d r`phpxq, yqs m ÿ mÿ i `phpx i q, y i q`complexitypz, F q

18 Lerning WFA vi Hnkel Mtrix Completion $ & % (,) (,3) (,) (,2) (,) (,) (,2) (,), /. /-?? 2 3? 2?? 2 3? 2???? Blle, B., & Mohri, M. (22). Spectrl lerning of generl weighted utomt vi constrined mtrix completion.

19 Generliztion Bounds for Lerning WFA The generliztion power of WFA cn e controlled y: Bounding the norm of the weights Bounding the norm of the lnguge (in Bnch spce) ÿ ÿ Bounding the norm of the Hnkel mtrix E px,yq d r`papxq, yqs m mÿ i `papx i q, y i q`õ ˆ}HA } m `? m Blle, B., & Mohri, M. (27). Generliztion Bounds for Lerning Weighted Automt

20 Some Prcticl Applictions L* lgorithm: lern DFA of network protocol implementtions nd compre ginst speciction to nd ugs De Ruiter, J., & Poll, E. (25). Protocol Stte Fuzzing of TLS Implementtions. Spectrl lgorithm: use s initil point of grdient-sed methods, increses speed nd ccurcy Jing, N., Kulesz, A., & Singh, S. P. (2). Improving Predictive Stte Representtions vi Grdient Descent. Hnkel completion: smple-efcient sequence-to-sequence models outperforming CRFs in smll lphets Quttoni, A., Blle, B., Crrers Pérez, X., & Gloerson, A. (24). Spectrl regulriztion for mx-mrgin sequence tgging.

21 Wnt to Lern More? EMNLP 4 tutoril (slides, video, code) Vritions on spectrl lgorithm Extensions to weighted tree utomt Survey ppers B. Blle nd M. Mohri (25). Lerning Weighted Automt M. R. Thon nd H. Jeger (25). Links etween multiplicity utomt, oservle opertor models nd predictive stte representtions F. Vndrger (27). Model Lerning Implementtions: Sp2Lern, LiLern, lilf

22 Thnks! Guillume Russeu Frnco M. Luque Xvier Crrers Mehryr Mohri Prksh Pnngden Pierre-Luc Bcon Pscle Gourdeu Odlric-Amrym Millrd Will Hmilton Lucs Lnger Shy Cohen Amir Gloerson Joelle Pineu Doin Precup Aridn Quttoni

23 Lerning Automt with Hnkel Mtrices Borj Blle [Disclimer: Work done efore joining Amzon]