Speech Recognition Lecture 2: Finite Automt nd Finite-Stte Trnsducers Mehryr Mohri Cournt Institute nd Google Reserch mohri@cims.nyu.com
Preliminries Finite lphet Σ, empty string. Set of ll strings over : Σ Σ (free monoid). Length of string x Σ : x. Mirror imge or reverse of string x = x 1 x n : x R = x n x 1. A lnguge L: suset of Σ. Mehryr Mohri - Speech Recognition pge 2 Cournt Institute, NYU
Rtionl Opertions Rtionl opertions over lnguges: union: lso denoted L 1 + L 2, conctention: closure: L 1 L 2 = {x Σ : x L 1 x L 2 }. L 1 L 2 = {x = uv Σ : u L 1 v L 2 }. L = n=0 L n, where L n = L L. n Mehryr Mohri - Speech Recognition pge 3 Cournt Institute, NYU
Regulr or Rtionl Lnguges Definition: closure under rtionl opertions of Σ. Thus, Rt(Σ ) is the smllest suset L of 2 Σ verifying L ; x Σ, {x} L ; L 1,L 2 L,L 1 L 2 L,L 1 L 2 L,L 1 L. Exmples of regulr lnguges over Σ={,, c} : Σ, ( + ) c, n c, ( +( + c) ) c. Mehryr Mohri - Speech Recognition pge 4 Cournt Institute, NYU
Finite Automt Definition: finite utomton A over the lphet is 4-tuple (Q, I, F, E) where Q is finite set of sttes, I Q set of initil sttes, F Q set of finl sttes, nd E multiset of trnsitions which re elements of Q (Σ {}) Q. pth in n utomton π A =(Q, I, F, E) element of E. pth from stte in I to stte in is n F is clled n ccepting pth. Lnguge L(A) ccepted y A: set of strings leling ccepting pths. Σ Mehryr Mohri - Speech Recognition pge 5 Cournt Institute, NYU
Finite Automt - Exmple 0 1 2 Mehryr Mohri - Speech Recognition pge 6 Cournt Institute, NYU
Finite Automt - Some Properties Trim: ny stte lies on some ccepting pth. Unmiguous: no two ccepting pths hve the sme lel. Deterministic: unique initil stte, two trnsitions leving the sme stte hve different lels. Complete: t lest one outgoing trnsition leled with ny lphet element t ny stte. Acyclic: no pth with cycle. Mehryr Mohri - Speech Recognition pge 7 Cournt Institute, NYU
Normlized Automt Definition: finite utomton is normlized if it hs unique initil stte with no incoming trnsition. it hs unique finl stte with no outgoing trnsition. i A f Mehryr Mohri - Speech Recognition pge 8 Cournt Institute, NYU
Elementry Normlized Automton Definition: normlized utomton ccepting n element Σ {} constructed s follows. 0 1 Mehryr Mohri - Speech Recognition pge 9 Cournt Institute, NYU
Normlized Automt: Union Construction: the union of two normlized utomt is normlized utomton constructed s follows. i 1 A 1 1 f i f i 2 A 2 2 f Mehryr Mohri - Speech Recognition pge 10 Cournt Institute, NYU
Normlized Automt: Conctention Construction: the conctention of two normlized utomt is normlized utomton constructed s follows. i1 A f 1 1 i f 2 A 2 2 Mehryr Mohri - Speech Recognition pge 11 Cournt Institute, NYU
Normlized Automt: Closure Construction: the closure of normlized utomton is normlized utomton constructed s follows. i 0 i A f f 0 Mehryr Mohri - Speech Recognition pge 12 Cournt Institute, NYU
Normlized Automt - Properties Construction properties: ech rtionl opertion require creting t most two sttes. ech stte hs t most two outgoing trnsitions. the complexity of ech opertion is liner. Mehryr Mohri - Speech Recognition pge 13 Cournt Institute, NYU
Thompson s Construction Proposition: let r e regulr expression over the lphet Σ. Then, there exists normlized utomton A with t most 2 r sttes representing r. Proof: (Thompson, 1968) liner-time context-free prser to prse regulr expression. construction of normlized utomton strting from elementry expressions nd following opertions of the tree. Mehryr Mohri - Speech Recognition pge 14 Cournt Institute, NYU
Thompson s Construction - Exmple ε 4 ε 5 ε ε 1 2 ε 3 ε 6 ε 0 ε 7 c 8 ε 9 Normlized utomton for regulr expression + c. Mehryr Mohri - Speech Recognition pge 15 Cournt Institute, NYU
Regulr Lnguges nd Finite Automt Theorem: A lnguge is regulr iff it cn e ccepted y finite utomton. Proof: Let for A =(Q, I, F, E) e finite utomton. (i, j, k) [1, Q ] [1, Q ] [0, Q ] L(A) = is thus regulr. i I,f F X Q if Mehryr Mohri - Speech Recognition pge 16 define Xij 0 is regulr for ll (i, j) since E is finite. y recurrence Xij k for ll (i, j, k) since (Kleene, 1956) X k ij = {i q 1 q 2... q n j : n 0,q i k}. X k+1 ij = X k ij + Xk i,k+1 (Xk k+1,k+1 ) X k k+1,j. Cournt Institute, NYU
Regulr Lnguges nd Finite Automt Proof: the converse holds y Thompson s construction. Notes: more generl theorem (Schützenerger, 1961) holds for weighted utomt. not ll lnguges re regulr, e.g., L = { n n : n N} is not regulr. Let A e n utomton. If L L(A), then for lrge enough n, n n corresponds to pth with cycle: n n = p u q, p u q L(A), which implies L(A) = L. Mehryr Mohri - Speech Recognition pge 17 Cournt Institute, NYU
Left Syntctic Congruence Definition: for ny lnguge L Σ, the left syntctic congruence is the equivlence reltion defined y u L v u 1 L = v 1 L, where for ny u Σ, u 1 L is defined y u 1 L = {w : uw L}. u 1 L of L with respect to u nd denoted L. is sometimes clled the prtil derivtive u Mehryr Mohri - Speech Recognition pge 18 Cournt Institute, NYU
Regulr Lnguges - Chrcteriztion Theorem: lnguge L is regulr iff the set of is finite ( hs finite index). L Proof: let utomton ccepting A =(Q, I, F, E) L e trim deterministic (existence seen lter). let δ the prtil trnsition function. Then, urv δ(i, u) =δ(i, v). u 1 L lso defines n eq. reltion with index Q. since δ(i, u) =δ(i, v) u 1 L = v 1 L, the index of L is t most Q, thus finite. Mehryr Mohri - Speech Recognition pge 19 Cournt Institute, NYU
Regulr Lnguges - Chrcteriztion Proof: conversely, if the set of the utomton Q = {u 1 L: u Σ } ; i = 1 L = L, I = {i} ; F = {u 1 L: u L} ; since ccepts exctly L. u 1 L A =(Q, I, F, E) is finite, then defined y ; is well defined nd E = {(u 1 L,, (u) 1 L): u Σ } u 1 L = v 1 L (u) 1 L =(v) 1 L Mehryr Mohri - Speech Recognition pge 20 Cournt Institute, NYU
Illustrtion Miniml deterministic utomton for ( + ) : L -1 L () -1 L Mehryr Mohri - Speech Recognition pge 21 Cournt Institute, NYU
ε-removl Theorem: ny finite utomton dmits n equivlent utomton with no ε- trnsition. A =(Q, I, F, E) Proof: for ny stte q Q, let [q] denote the set of sttes reched from q y pths leled with. Define A =(Q,I,F,E ) y Q = {[q]: q Q}, I = [q], F = {[q]: [q] F = }. q I E = {([p],,[q]) : (p,,q ) E,p [p],q [q]}. Mehryr Mohri - Speech Recognition pge 22 Cournt Institute, NYU
ε-removl - Illustrtion 0 1 2 3 {0, 1} {0, 2} {0, 1, 3} {0} Mehryr Mohri - Speech Recognition pge 23 Cournt Institute, NYU
Determiniztion Theorem: ny utomton A =(Q, I, F, E) without -trnsitions dmits n equivlent deterministic utomton. Proof: Suset construction: A =(Q,I,F,E ) Q =2 Q. I = {s Q : s I = }. F = {s Q : s F = }. E = {(s,, s ): (q,, q ) E,q s, q s }. with Mehryr Mohri - Speech Recognition pge 24 Cournt Institute, NYU
Determiniztion - Illustrtion 0 1 2 {0} {1} {1, 2} {2} {0, 1} Mehryr Mohri - Speech Recognition pge 25 Cournt Institute, NYU
Completion Theorem: ny deterministic utomton dmits n equivlent complete deterministic utomton. Proof: constructive, see exmple. 0 1 3 0 1 3 2 2 4 Mehryr Mohri - Speech Recognition pge 26 Cournt Institute, NYU
Complementtion Theorem: let A =(Q, I, F, E) e deterministic utomton, then there exists deterministic utomton ccepting L(A). Proof: y previous theorem, we cn ssume A complete. The utomton otined from A y mking non-finl sttes finl nd finl sttes non-finl exctly ccepts L(A). B =(Σ,Q,I,Q F, E) Mehryr Mohri - Speech Recognition pge 27 Cournt Institute, NYU
Complementtion - Ilustrtion 0 1 3 2 4 0 1 3 2 4 Mehryr Mohri - Speech Recognition pge 28 Cournt Institute, NYU
Regulr Lnguges - Properties Theorem: regulr lnguges re closed under rtionl opertions, intersection, complementtion, reversl, morphism, inverse morphism, nd quotient with ny set. Proof: closure under rtionl opertions holds y definition. intersection: use De Morgn s lw. complementtion: use lgorithm. others: lgorithms nd equivlence reltion. Mehryr Mohri - Speech Recognition pge 29 Cournt Institute, NYU
Rtionl Reltions Definition: closure under rtionl opertions of the monoid Σ, where Σ nd re finite lphets, denoted y Rt(Σ ). exmples: (, ), (, ) (, )+(, ). Mehryr Mohri - Speech Recognition pge 30 Cournt Institute, NYU
Rtionl Reltions - Chrcteriztion Theorem: R Rt(Σ ) is rtionl reltion iff there exists regulr lnguge L (Σ ) such tht R = {(π Σ (x),π (x)) : x L} where is the projection of over nd π the projection over. π Σ (Σ ) Σ Proof: use surjective morphism π :(Σ ) (Σ ) x (π Σ (x),π (x)). (Nivt, 1968) Mehryr Mohri - Speech Recognition pge 31 Cournt Institute, NYU
Trnsductions Definition: function trnsduction from Σ to. reltion ssocite to τ : τ :Σ 2 is clled R(τ) ={(x, y) Σ : y τ(x)}. trnsduction ssocited to reltion: x Σ,τ(x) ={y :(x, y) R}. rtionl trnsductions: trnsductions with rtionl reltions. Mehryr Mohri - Speech Recognition pge 32 Cournt Institute, NYU
Finite-Stte Trnsducers Definition: finite-stte trnsducer T over the lphets Σ nd is 4-tuple where Q is finite set of sttes, I Q set of initil sttes, F Q set of finl sttes, nd E multiset of trnsitions which re elements of Q (Σ {}) ( {}) Q. T defines reltion vi the pir of input nd output lels of its ccepting pths, R(T )={(x, y) Σ : I x:y F }. Mehryr Mohri - Speech Recognition pge 33 Cournt Institute, NYU
Rtionl Reltions nd Trnsducers Theorem: trnsduction is rtionl iff it cn e relized y finite-stte trnsducer. Proof: Nivt s theorem comined with Kleene s theorem, nd construction of normlized trnsducer from finite-stte trnsducer. Mehryr Mohri - Speech Recognition pge 34 Cournt Institute, NYU
References Kleene, S. C.1956. Representtion of events in nerve nets nd finite utomt. Automt Studies. Nivt, Murice. 968. Trnsductions des lngges de Chomsky. Annles 18, Institut Fourier. Schützenerger, Mrcel~Pul. 1961. On the definition of fmily of utomt. Informtion nd Control, 4 Thompson, K. 1968. Regulr expression serch lgorithm. Comm. ACM, 11. Mehryr Mohri - Speech Recognition pge 35 Cournt Institute, NYU