General Algorithms for Testing the Ambiguity of Finite Automata and the Double-Tape Ambiguity of Finite-State Transducers

Transcription

1 Interntionl Journl of Foundtions of Computer Science c World Scientific Pulishing Compny Generl Algorithms for Testing the Amiguity of Finite Automt nd the Doule-Tpe Amiguity of Finite-Stte Trnsducers CYRIL ALLAUZEN Google Reserch, 76 Ninth Avenue, New York, NY 10011, US. lluzen@google.com MEHRYAR MOHRI Cournt Institute of Mthemticl Sciences, 251 Mercer Street, New York, NY 10012, US, nd Google Reserch, 76 Ninth Avenue, New York, NY 10011, US. mohri@cs.nyu.edu ASHISH RASTOGI Goldmn, Schs & Co., 200 West Street, New York, NY 10282, US. shish.rstogi@gs.com We present efficient lgorithms for testing the finite, polynomil, nd exponentil miguity of finite utomt with ǫ-trnsitions. We give n lgorithm for testing the exponentil miguity of n utomton A in time O( A 2 E ), nd finite or polynomil miguity in time O( A 3 E ), where A E denotes the numer of trnsitions of A. These complexities significntly improve over the previous est complexities given for the sme prolem. Furthermore, the lgorithms presented re simple nd sed on generl lgorithm for the composition or intersection of utomt. Additionlly, we give n lgorithm to determine in time O( A 3 E ) the degree of polynomil miguity of polynomilly miguous utomton A nd present n ppliction of our lgorithms to n pproximte computtion of the entropy of proilistic utomton. We lso study the doule-tpe miguity of finite-stte trnsducers. We show tht the generl prolem is undecidle nd tht it is NP-hrd for cyclic trnsducers. We present specific nlysis of the doule-tpe miguity of trnsducers with ounded dely. In prticulr, we give chrcteriztion of doule-tpe miguity for synchronized trnsducers with zero dely tht cn e tested in qudrtic time nd give n lgorithm for testing the doule-tpe miguity of trnsducers with ounded dely. Reserch done t the Cournt Institute, prtilly supported y the New York Stte Office of Science Technology nd Acdemic Reserch (NYSTAR). 1

2 2 C. Alluzen, M. Mohri nd A. Rstogi 1. Introduction A finite utomton is miguous if it dmits distinct ccepting pths with the sme lel. The question of the miguity of finite utomt rises in vriety of contexts. In some cses, the ppliction of n lgorithm requires n input utomton to e finitely miguous, in others, the convergence of ound or gurntee relies on finite miguity, or the symptotic growth rte of miguity s function of the string length. Thus, in ll these cses, n lgorithm is needed to test the miguity, either to determine if it is finite, or to estimte its symptotic growth rte. The prolem of testing miguity hs een extensively nlyzed in the pst [10, 8, 17, 3, 7, 19, 16, 18, 20]. The prolem of determining the degree of miguity of n utomton with finite miguity ws shown y Chn nd Irr to e PSPACEcomplete [3]. However, testing finite miguity cn e chieved in polynomil time using chrcteriztion of exponentil nd polynomil miguity given y Irr nd Rvikumr [7] nd Weer nd Seidel [19]. The most efficient lgorithms for testing polynomil nd exponentil miguity, therey testing finite miguity, were given y Weer nd Seidel [18, 20]. The lgorithms they presented in [20] ssume the input utomton to e ǫ-free, ut they re extended y Weer to the cse where the utomton hs ǫ-trnsitions in [18]. In the presence of ǫ-trnsitions, the complexity of the lgorithms given y Weer [18] is O(( A E + A 2 Q )2 ) for testing the exponentil miguity of n utomton A nd O(( A E + A 2 Q )3 ) for testing polynomil miguity, where A E stnds for the numer of trnsitions nd A Q the numer of sttes of A. This pper presents significntly more efficient lgorithms for testing finite, polynomil, nd exponentil miguity for the generl cse of utomt with ǫ- trnsitions. It gives n lgorithm for testing the exponentil miguity of n utomton A in time O( A 2 E ), nd finite or polynomil miguity in time O( A 3 E ). The min ide ehind our lgorithms is to mke use of the composition or intersection of finite utomt with ǫ-trnsitions [14, 13]. The ǫ-filter used in these lgorithms crucilly helps in the nlysis nd test of the miguity. The lgorithms presented in this pper would not e vlid nd would led to incorrect results without the use of the ǫ-filter. We lso give n lgorithm to determine in time O( A 3 E ) the degree of polynomil miguity of polynomilly miguous utomton A nd present n ppliction of our lgorithms to n pproximte computtion of the entropy of proilistic utomton. The notion of miguity is defined in similr wy for finite-stte trnsducers if one is only interested in the miguity with respect to the input lels, or only the output lels, of trnsducer. With tht definition, ll our results for utomt pply directly to the trnsducer cse s well. There is, however, nother notion of interest for trnsducers tht reltes to oth input nd output lels nd tht we refer to s the doule-tpe miguity of trnsducer. A trnsducer is doule-tpe miguous if it dmits two distinct ccepting pths with the sme input lel nd the sme output lel. Doule-tpe miguity cn led to inefficiencies in vriety

3 Testing Automt Amiguity nd Trnsducer Doule-Tpe Amiguity 3 of pplictions where trnsducers re now commonly used, e.g., mchine trnsltion, speech recognition, other lnguge processing res, nd imge processing. This motivtes our study of the doule-tpe miguity of finite-stte trnsducers. We show tht the generl prolem of doule-tpe miguity is undecidle nd tht it is NP-hrd even for cyclic trnsducers. We lso present specific nlysis of the doule-tpe miguity of trnsducers with ounded dely. In prticulr, we give chrcteriztion of doule-tpe miguity for synchronized trnsducers with zero dely tht cn e tested in qudrtic time nd give n lgorithm for testing the doule-tpe miguity of trnsducers with ounded dely. The reminder of the pper is orgnized s follows. Section 2 presents generl utomt nd miguity definitions. In Section 3, we give rief description of existing chrcteriztions for the miguity of utomt nd extend them to the cse of utomt with ǫ-trnsitions. In Section 4, we present our lgorithms for testing finite, polynomil, nd exponentil miguity, nd the proof of their correctness. Section 5 dels with questions relted to the doule-tpe miguity of finite-stte trnsducers. Section 6 shows the relevnce of the computtion of the polynomil miguity to the pproximtion of the entropy of proilistic utomt. 2. Preliminries Definition 1. A finite utomton A is 5-tuple (Σ, Q, E, I, F) where Σ is finite lphet; Q is finite set of sttes; I Q the set of initil sttes; F Q the set of finl sttes; nd E Q (Σ {ǫ}) Q finite set of trnsitions, where ǫ denotes the empty string. We denote y A Q the numer of sttes, y A E the numer of trnsitions, nd y A = A E + A Q the size of n utomton A. Given stte q Q, E[q] denotes the set of trnsitions leving q. For two susets R Q nd R Q, we denote y P(R, x, R ) the set of ll pths from stte q R to stte q R leled with x Σ. We lso denote y p[π] the origin stte, y n[π] the destintion stte, nd y i[π] Σ the lel of pth π. A stte q Q is ccessile if there exists pth from n initil stte to q nd co-ccessile if there exists pth from q to finl stte. A string x Σ is ccepted y A if it lels n ccepting pth, tht is pth from n initil stte to finl stte. A finite utomton A is sid to e trim if ll its sttes lie on some ccepting pth, tht is if every stte is oth ccessile nd co-ccessile. It is sid to e unmiguous if no string x Σ lels two distinct ccepting pths; otherwise, it is sid to e miguous. The degree of miguity of string x in A is denoted y d(a, x) nd defined s the numer of ccepting pths in A leled y x. Note tht if A contins n ǫ-cycle lying long n ccepting pth, there exists x Σ such tht d(a, x) =. Using depth-first serch of A restricted to ǫ-trnsitions, it cn e decided in liner time if A contins such ǫ-cycles. Thus, in the following, we will ssume, without loss of generlity, tht A is ǫ-cycle free.

4 4 C. Alluzen, M. Mohri nd A. Rstogi v v v v v 1 v 1 v 2 v 2 v d v d p p v q p 1 v 1 q 1 u 2 p 2 v 2 q 2 u d p d v d q d () () (c) Fig. 1. Illustrtion of the properties: () (EDA); () (IDA); nd (c) (IDA d ). The degree of miguity of A is defined s d(a) = sup x Σ d(a, x). A is sid to e finitely miguous if d(a) < nd infinitely miguous if d(a) =. It is sid to e polynomilly miguous if there exists polynomil h such tht d(a, x) h( x ) for ll x Σ. The miniml degree of such polynomil is clled the degree of polynomil miguity of A nd is denoted y dp(a). By definition, dp(a) = 0 iff A is finitely miguous. When A is infinitely miguous ut not polynomilly miguous, it is sid to e exponentilly miguous nd dp(a) =. 3. Chrcteriztion of infinite miguity The chrcteriztion nd test of finite, polynomil, nd exponentil miguity of finite utomt without ǫ-trnsitions re sed on the following three fundmentl properties [7, 19, 18, 20]. Definition 2. The properties (EDA), (IDA), nd (IDA d ) for A re defined s follows. () (EDA): there exists stte q with t lest two distinct cycles leled y some v Σ (see Figure 1()) [7]. () (IDA): there exist two distinct sttes p nd q with pths leled with v from p to p, p to q, nd q to q, for some v Σ (see Figure 1()) [19, 18, 20]. (c) (IDA d ): there exist 2d sttes p 1,...,p d, q 1,...,q d in A nd 2d 1 strings v 1,..., v d nd u 2,...,u d in Σ such tht for ll 1 i d, p i q i nd P(p i, v i, p i ), P(p i, v i, q i ), nd P(q i, v i, q i ) re non-empty, nd, for ll 2 i d, P(q i 1, u i, p i ) is non-empty (see Figure 1(c)) [19, 18, 20]. Oserve tht (EDA) implies (IDA) s shown elow. Indeed, ssuming (EDA), let e nd e e the first trnsitions tht differ in the two cycles t stte p, then, since Definition 1 disllows multiple trnsitions etween the sme two sttes with the sme lel, we must hve n[e] n[e ]. Thus, (IDA) holds for the pir (n[e], n[e ]). In the ǫ-free cse, it ws shown tht trim utomton A stisfies (IDA) iff A is infinitely miguous [19, 20], tht A stisfies (EDA) iff A is exponentilly miguous [7], nd tht A stisfies (IDA d ) iff dp(a) d [18, 20]. In the following, we show tht these results cn e extended to the cse of utomt with ǫ-trnsitions. To simplify the proofs, we first consider the cse of multiset utomt. A multiset utomton or m-utomton is 5-tuple (Σ, Q, E, I, F) s defined in Definition 1 except tht E nd F re multisets. We will denote y the union of two

5 Testing Automt Amiguity nd Trnsducer Doule-Tpe Amiguity 5 multisets ({1, 2} {1, 3} = {1, 1, 2, 3}), y the sclr multipliction of multiset y nturl numer (2 {1, 1, 2} = {1, 1, 1, 1, 2, 2}), y X the multiplicity of element in the multiset X ( {1, 1, 2} 1 = 2) nd y X the crdinlity ( {1, 1, 2} = 3) of X. Lemm 3. Let A e trim ǫ-free m-utomton. (i) A is infinitely miguous iff A stisfies (IDA). (ii) A is exponentilly miguous iff A stisfies (EDA). (iii) dp(a) d iff A stisfies (IDA d ). Proof. Given trim m-utomton A = (Σ, Q, E, I, F), we construct finite utomton A = (Σ {#}, Q, E, I, F ) y inserting trnsition leled with # fter ech trnsition nd from ech finl stte s follows: Q = Q Q E Q F with Q E = {q e e E} nd Q F = {q f f F }, E = {(p[e], i[e], q e ), (q e, #, n[e])} {(f, #, q f )}, nd e E F = Q F. Oserve tht the crdinlity of the set Q E (resp. Q F ) is equl to the crdinlity of the multiset E (resp. F). Ech stte q E hs only one incoming nd one outgoing trnsition. The mpping α E : e (p[e], i[e], q e )(q e, #, n[e]) is n injection from E into E 2 nd the mpping α F : f (f, #, q f ) n injection from F into E. Severl key properties follow from the existence of these injections. (1) A is trim since A is trim (follows from the existence of α E nd α F ). (2) There exists n injection β : e 1... e n α E (e 1 )...α E (e n ) from the set of pths in A to the set of pths in A such tht the following conditions re equivlent: () (IDA) (resp. (EDA), (IDA d )) holds for A, () (IDA) (resp. (EDA), (IDA d )) holds for ll pths in the imge of β nd (c) (IDA) (resp. (EDA), (IDA d )) holds for A. (3) The mpping γ : x 1 x 2... x n x 1 #x 2 #... x n ## is ijection from the lnguge ccepted y A to the lnguge ccepted y A nd (4) d(a, x) = d(a, γ(x)) for ll x Σ since the mpping δ : π β(π)α F (n[π]) is ijection etween the sets of ccepting pths of A nd A such tht i[δ(π)] = γ(i[π]). The proposition holds for A since A is stndrd trim utomton s shown in [19, 20] for (i), [7] for (ii) nd [20] for (iii). Hence, it follows from (2) nd (4) tht the proposition lso hold for A. We will now show tht Lemm 3 cn e generlized to the cse of m-utomt with ǫ-trnsitions. f F Lemm 4. Let A e trim ǫ-cycle free m-utomton. (i) A is infinitely miguous iff A stisfies (IDA). (ii) A is exponentilly miguous iff A stisfies (EDA).

6 6 C. Alluzen, M. Mohri nd A. Rstogi 0 1 ε 2 2,1 ε ε ε 1,2 ε 1,1 ε 2,2 0,0 0,0 1,1 ε 2,2 0 1 ε 2 0,1 # 0,2 1,1 () () (c) (d) (e) # # 1,2 2,2 Fig. 2. ǫ-filter nd miguity: () Finite utomton A; () A A without using ǫ-filter, which incorrectly mkes A pper s exponentilly miguous; (c) A A using n ǫ-filter. Weer s processing of ǫ-trnsitions: (d) Finite utomton B; (e) ǫ-free utomton B such tht dp(b) = dp(b ). (iii) dp(a) d iff A stisfies (IDA d ). Proof. The proof is y induction on the numer of ǫ-trnsitions in A. If A does not hve ny ǫ-trnsition, then the proposition holds nd follows from Lemm 3. Assume now tht A hs n+1 ǫ-trnsitions, n 0, nd tht the sttement of the proposition holds for ll m-utomt with n ǫ-trnsitions. Select n ǫ-trnsition e 0 in A such tht there re no outgoing ǫ-trnsitions in n[e 0 ]. Such trnsition must exist since A is ǫ-cycle free. Let A e the m-utomton otined fter ppliction of ǫ-removl to A limited to trnsition e 0. A is otined y deleting e 0 from A nd y dding trnsition (p[e 0 ], l[e], n[e]) for every trnsition e E[n[e 0 ]], i.e. the multiset E of trnsitions of A is defined s: E = (E \ {e 0 }) {(p[e 0 ], l[e], n[e]) e E such tht p[e] = n[e 0 ]}. Finlly, p[e 0 ] is dded to the multiset of finl sttes s mny times s the multiplicity of n[e 0 ] in F, i.e. the multiset F of finl sttes of A is defined s: F = F ( F n[e0] {p[e 0 ]}). It is cler tht A nd A re equivlent nd tht there is lel nd cceptncepreserving ijection etween the pths in A nd A. Thus, () A stisfies (IDA) (resp. (EDA), (IDA d )) iff A stisfies (IDA) (resp. (EDA), (IDA d )) nd () for ll x Σ, d(a, x) = d(a, x). By induction, Lemm 4 holds for A nd thus, it follows from () nd () tht Lemm 4 lso holds for A. The cse of finite utomt with ǫ-trnsitions then follows s corollry of Lemm 4. Proposition 5. Let A e trim ǫ-cycle free finite utomton. (i) A is infinitely miguous iff A stisfies (IDA). (ii) A is exponentilly miguous iff A stisfies (EDA). (iii) dp(a) d iff A stisfies (IDA d ). These chrcteriztions hve een used in [18, 20] to design lgorithms for testing infinite, polynomil, nd exponentil miguity, nd for computing the degree of

7 Testing Automt Amiguity nd Trnsducer Doule-Tpe Amiguity 7 polynomil miguity in the cse of ǫ-free finite utomt. Theorem 6 ([18,20]) Let A e trim ǫ-free finite utomton. (1) It is decidle in time O( A 3 E ) whether A is infinitely miguous. (2) It is decidle in time O( A 2 E ) whether A is exponentilly miguous. (3) The degree of polynomil miguity of A, dp(a), cn e computed in O( A 3 E ). The first result of Theorem 6 hs lso een generlized y [18] to the cse of utomt with ǫ-trnsitions ut with significntly worse complexity. Theorem 7 ([18]) Let A e trim ǫ-cycle free finite utomton. It is decidle in time O(( A E + A 2 Q )3 ) whether A is infinitely miguous. The lgorithms designed for the ǫ-free cse cnnot e redily used for finite utomt with ǫ-trnsitions since they would led to incorrect results (see Figure 2()-(c)). Insted, [18] proposed reduction to the ǫ-free cse. First, [18] gve n lgorithm to test if there exist two sttes p nd q in A with two distinct ǫ-pths from p to q. If tht is the cse, then A is exponentilly miguous (complexity O( A 4 Q + A E)). Otherwise, [18] defined from A n ǫ-free utomton A over the lphet Σ {#} such tht A is infinitely miguous iff A is infinitely miguous, see Figure 2(d)-(e). However, the numer of trnsitions of A is A E + A 2 Q. This explins why the complexity in the ǫ-trnsition cse is significntly worse thn in the ǫ-free cse. The sme pproch cn e used to test the exponentil miguity of A in time O(( A E + A 2 Q )2 ) nd to compute dp(a) when A is polynomilly miguous in O(( A E + A 2 Q )3 ). Note tht we give tighter estimtes of the complexity of the lgorithms of [18, 20] where the uthors gve complexities using the loose inequlity: A E Σ A 2 Q. 4. Algorithms Our lgorithms for testing miguity re sed on generl lgorithm for the composition or intersection of utomt, which we riefly descrie in the following section. Oserve tht A is not the result of pplying the clssicl ǫ-removl lgorithm to A, since ǫ- removl does not preserve infinite miguity nd would led to n even lrger utomton. Insted, [18] used more complex lgorithm where ǫ-trnsitions re replced y regulr trnsitions leled with specil symol while preserving infinite miguity, dp(a) = dp(a ), even though A is not equivlent to A. Sttes in A re pirs (q, i) with q stte in A nd i {1, 2}. There is trnsition from (p, 1) to (q, 2) leled y # if q elongs to the ǫ-closure of p nd from (p, 2) to (q, 1) leled y σ Σ if there ws such trnsition from p to q in A.

8 8 C. Alluzen, M. Mohri nd A. Rstogi , 0 1, 1 0, 1 2, 1 3, 1 3, 2 3, 3 () () (c) Fig. 3. Exmple of finite utomton intersection. () Finite utomt A 1 nd () A 2. (c) Result of the intersection of A 1 nd A Intersection of finite utomt The intersection of finite utomt is specil cse of the generl composition lgorithm for weighted trnsducers [14, 13]. Sttes in the intersection A 1 A 2 of two finite utomt A 1 nd A 2 re identified with pirs of stte of A 1 nd stte of A 2. The following rule specifies how to compute trnsition of A 1 A 2 in the sence of ǫ-trnsition from pproprite trnsitions of A 1 nd A 2 : (q 1,, q 1 ) nd (q 2,, q 2 ) = ((q 1, q 2 ),, (q 1, q 2 )). Figure 3 illustrtes the lgorithm. A stte (q 1, q 2 ) is initil (resp. finl) when q 1 nd q 2 re initil (resp. finl). In the worst cse, ll trnsitions of A 1 leving stte q 1 mtch ll those of A 2 leving stte q 2, thus the spce nd time complexity of composition is qudrtic: O( A 1 A 2 ), or O( A 1 E A 2 E ) when A 1 nd A 2 re trim Epsilon-filtering A strightforwrd generliztion of the ǫ-free cse would generte redundnt ǫ- pths. This is crucil issue in the more generl cse of the intersection of weighted utomt over non-idempotent semiring, since it would led to n incorrect result. The weight of two mtching ǫ-pths of the originl utomt would then e counted s mny times s the numer of redundnt ǫ-pths generted in the result, insted of once. It is lso crucil prolem in the unweighted cse since redundnt ǫ-pths cn ffect the test of infinite miguity, s we shll see in the next section. A criticl component of the composition lgorithm of [14, 13] consists however of precisely coping with this prolem using n epsilon-filtering mechnism. Figure 4(c) illustrtes the prolem just mentioned. To mtch ǫ-pths leving q 1 nd those leving q 2, generliztion of the ǫ-free intersection cn mke the following moves: (1) first move forwrd on n ǫ-trnsition of q 1, or even ǫ-pth, nd remin t the sme stte q 2 in A 2, with the hope of lter finding trnsition whose lel is some lel ǫ mtching trnsition of q 2 with the sme lel; (2) proceed similrly y following n ǫ-trnsition or ǫ-pth leving q 2 while remining t the sme stte q 1 in A 1 ; or, (3) mtch n ǫ-trnsition of q 1 with n ǫ-trnsition of q 2. Let us renme existing ǫ-lels of A 1 s ǫ 2, nd existing ǫ-lels of A 2 s ǫ 1, nd let us ugment A 1 with self-loop leled with ǫ 1 t ll sttes nd similrly, ugment

9 Testing Automt Amiguity nd Trnsducer Doule-Tpe Amiguity 9 ǫ ǫ ǫ 1 ǫ 2 ǫ 1 ǫ 1 ǫ ǫ 2 () ǫ 2 ǫ 2 (0,0) ǫ 2 :ǫ 2 (1,0) ǫ 2 :ǫ 2 ǫ 1 :ǫ 1 ǫ 2 :ǫ 1 ǫ 1 :ǫ 1 ǫ 2 :ǫ 1 (0,1) (1,1) ǫ 1 :ǫ 1 ǫ ǫ 2 :ǫ 2 :ǫ 1 2 ǫ 1 :ǫ 1 (0,2) (1,2) ǫ 2 :ǫ 2 ε2:ε1 x:x ǫ ǫ 2 :ǫ 2 :ǫ 1 2 ǫ 2 :ǫ 0 2 ε1:ε1 x:x ε2:ε2 ε1:ε1 1 ε2:ε2 ǫ 1 ǫ (2,0) ǫ 1 :ǫ 1 (2,1) ǫ 1 :ǫ 1 (2,2) x:x 2 () (c) (d) (e) Fig. 4. Mrking of utomt, redundnt pths nd filter. () Automton A 1 = A 2. () Ã1: self-loop leled with ǫ 1 dded t ll sttes of A 1, regulr ǫs renmed to ǫ 2. (c) Ã2: self-loop leled with ǫ 2 dded t ll sttes of A 2, regulr ǫs renmed to ǫ 1. (d) Redundnt ǫ-pths: strightforwrd generliztion of the ǫ-free cse could generte ll the pths from (0, 0) to (2, 2) for exmple, even when composing just two simple trnsducers (A 1 A 2 ). (e) Filter trnsducer M llowing unique ǫ-pth. Ech trnsition leled x : x represents trnsitions with input nd output x of ll x in Σ. A 2 with self-loop leled with ǫ 2 t ll sttes, s illustrted y Figures 4() nd (). These self-loops correspond to remining t the sme stte in tht mchine while consuming n ǫ-lel of the other trnsition. The three moves just descried now correspond to the mtches (1) (ǫ 2 : ǫ 2 ), (2) (ǫ 1 : ǫ 1 ), nd (3) (ǫ 2 : ǫ 1 ). The grid of Figure 4(c) shows ll the possile ǫ-pths etween intersection sttes. We will denote y Ã1 nd Ã2 the utomt otined fter ppliction of these chnges. For the result of intersection not to e redundnt, etween ny two of these sttes, ll ut one pth must e disllowed. There re mny possile wys of selecting tht pth. One nturl wy is to select the shortest pth with the digonl trnsitions (ǫ-mtching trnsitions) tken first. Figure 4(c) illustrtes in oldfce the pth just descried from stte (0, 0) to stte (2, 1). Remrkly, this filtering mechnism itself cn e encoded s finite-stte trnsducer such s the trnsducer M of Figure 4(d). We denote y (p, q) (r, s) to indicte tht (r, s) cn e reched from (p, q) in the grid. Proposition 8. Let M e the trnsducer of Figure 4(d). M llows unique ǫ-pth etween ny two sttes (p, q) nd (r, s), with (p, q) (r, s). Proof. The proof of this proposition ws previously given in [2]. Let denote (ǫ 1 : ǫ 1 ), denote (ǫ 2 : ǫ 2 ), c denote (ǫ 2 : ǫ 1 ), nd let x stnd for ny (x: x), with x Σ. The following sequences must e disllowed y shortest-pth filter with mtching trnsitions first:,, c, c. This is ecuse, from ny stte, insted of the moves or, the mtching or digonl trnsition c cn e tken. Similrly, insted of c or c, c nd c cn e tken for n erlier mtch. Conversely, it is cler from the grid or n immedite recursion tht filter disllowing these sequences ccepts unique pth etween two connected sttes of the grid. Let L e the set of sequences over σ = {,, c, x} tht contin one of the

10 10 C. Alluzen, M. Mohri nd A. Rstogi x c 0 1 x c c 3 2 c x c {0} {0,1} c x c x {0,2} x c {0,3} x 1 c x 0 x 2 c c x c 3 () () (c) Fig. 5. () Finite utomton A representing the set of disllowed sequences. () Automton B, result of the determiniztion of A. Susets re indicted t ech stte. (c) Automton C otined from B y complementtion, stte 3 is not coccessile. disllowed sequence just mentioned s sustring tht is L = σ (++c+c)σ. Then L represents exctly the set of pths llowed y tht filter nd is thus regulr lnguge. Let A e n utomton representing L (Figure 5()). An utomton representing L cn e constructed from A y determiniztion nd complementtion (Figures 5()-(c)). The resulting utomton C is equivlent to the trnsducer M fter removl of the stte 3, which does not dmit pth to finl stte. Thus, to intersect two finite utomt A 1 nd A 2 with ǫ-trnsitions, it suffices to compute Ã1 M Ã2, using the ǫ-free rules of composition (see section 5 for forml definition of the composition of finite-stte trnsducers). Sttes in the intersection re now identified with triplets mde of stte of A 1, stte of M, nd stte of A 2. A trnsition (q 1, 1, q 1) in Ã1, trnsition (f, 1, 2, f ) in M, nd trnsition (q 2, 2, q 2) in Ã2 re comined to form the following trnsition in the intersection: ((q 1, f, q 2 ),, (q 1, f, q 2)), with = ǫ if { 1, 2 } {ǫ 1, ǫ 2 } nd = 1 = 2 otherwise. In the rest of the pper, we will ssume tht the result of intersection is trimmed fter its computtion, which cn e done in liner time in the size of the result of intersection. Theorem 9. Let A 1 nd A 2 e two finite utomt with ǫ-trnsitions. To ech pir (π 1, π 2 ) of ccepting pths in A 1 nd A 2 shring the sme input lel x Σ corresponds unique ccepting pth π in A 1 A 2 leled with x. Proof. This follows strightforwrdly from Proposition Amiguity Tests We strt with test of the exponentil miguity of A. The key is tht the (EDA) property trnsltes into very simple property for A 2 = A A. A stte in A 2 is triple (p, f, q), denoted y (p, q) f in the following, where p nd q re sttes in A nd f is filter stte.

11 Testing Automt Amiguity nd Trnsducer Doule-Tpe Amiguity 11 Lemm 10. Let A e trim ǫ-cycle free finite utomton. A stisfies (EDA) iff there exists strongly connected component of A 2 = A A tht contins two sttes of the form (p, p) 0 nd (q, q ) f, where p, q nd q re sttes of A with q q. Proof. Assume tht A stisfies (EDA). There exist stte p nd string v such tht there re two distinct cycles c 1 nd c 2 leled y v t p. Let e 1 nd e 2 e the first edges tht differ in c 1 nd c 2. We cn then write c 1 = πe 1 π 1 nd c 2 = πe 2 π 2. If e 1 nd e 2 shre the sme lel, let π 1 = πe 1, π 2 = πe 2, π 1 = π 1 nd π 2 = π 2. If e 1 nd e 2 do not shre the sme lel, exctly one of them must e n ǫ-trnsition. By symmetry, we cn ssume without loss of generlity tht e 1 is the ǫ-trnsition. Let π 1 = πe 1, π 2 = π, π 1 = π 1 nd π 2 = e 2 π 2. In oth cses, let q = n[π 1] = p[π 1] nd q = n[π 2] = p[π 2]. Oserve tht q q. Since i[π 1] = i[π 2], π 1 nd π 2 re mtched y intersection resulting in pth in A 2 from (p, p) 0 to (q, q ) f. Similrly, since i[π 1 ] = i[π 2 ], π 1 nd π 2 re mtched y intersection resulting in pth from (q, q ) f to (p, p) 0. Thus, (p, p) 0 nd (q, q ) f re in the sme strongly connected component of A 2. Conversely, ssume tht there exist sttes p, q nd q in A such tht q q nd tht (p, p) 0 nd (q, q ) f re in the sme strongly connected component of A 2. Let c e cycle in (p, p) 0 going through (q, q ) f, c hs een otined y mtching two cycles c 1 nd c 2. If c 1 were equl to c 2, intersection would mtch these two pths creting pth c long which ll the sttes would e of the form (r, r) 0 mking c distinct from c, nd since A is trim this would contrdict Theorem 9. Thus, c 1 nd c 2 re distinct nd (EDA) holds. Oserve tht the use of the ǫ-filter in composition is crucil for Lemm 10 to hold (see Figure 2). The lemm leds to strightforwrd lgorithm for testing exponentil miguity. Theorem 11. Let A e trim ǫ-cycle free finite utomton. It is decidle in time O( A 2 E ) whether A is exponentilly miguous. Proof. The lgorithm proceeds s follows. We compute A 2 nd, using depth-first serch of A 2, trim it nd compute its strongly connected components. It follows from Lemm 10 tht A is exponentilly miguous iff there is strongly connected component tht contins two sttes of the form (p, p) 0 nd (q, q ) f with q q. Finding such strongly connected component cn e done in time liner in the size of A 2, i.e. in O( A 2 E ) since A nd A2 re trim. Thus, the complexity of the lgorithm is in O( A 2 E ). Testing the (IDA) property requires finding three pths shring the sme lel in A. As shown elow, this cn e done in nturl wy using the utomton A 3 = (A A) A, otined y pplying twice the intersection lgorithm. A stte in A 3 is 5-tuple (p, f, q, g, r), denoted y (p, q, r) f,g in the following, where p, q nd r re sttes in A nd f nd g re filter sttes.

12 12 C. Alluzen, M. Mohri nd A. Rstogi Lemm 12. Let A e trim ǫ-cycle free finite utomton. A stisfies (IDA) iff there exist two distinct sttes p nd q in A with non-ǫ pth in A 3 = A A A from stte (p, p, q) f.f to stte (p, q, q) g,g. Proof. Assume tht A stisfies (IDA). Then, there exists string v Σ with three pths π 1 P(p, v, p), π 2 P(p, v, q) nd π 3 P(q, v, p). Since these three pths shre the sme lel v, they re mtched y intersection resulting in pth π in A 3 leled with v from (p[π 1 ], p[π 2 ], p[π 3 ]) f,f = (p, p, q) f,f to (n[π 1 ], n[π 2 ], n[π 3 ]) g,g = (p, q, q) g,g. Conversely, if there is non-ǫ pth π from (p, p, q) f,f to (p, q, q) g,g in A 3, it hs een otined y mtching three pths π 1, π 2 nd π 3 in A with the sme input v = i[π] ǫ. Thus, (IDA) holds. This lemm ppers lredy s Lemm 5.10 in [9]. Finlly, Theorem 11 nd Lemm 12 cn e comined to yield the following result. Theorem 13. Let A e trim ǫ-cycle free finite utomton. It is decidle in time O( A 3 E ) whether A is finitely, polynomilly, or exponentilly miguous. Proof. First, Theorem 11 cn e used to test whether A is exponentilly miguous y computing A 2. The complexity of this step is O( A 2 E ). If A is not exponentilly miguous, we proceed y computing nd trimming A 3 nd then testing whether A 3 verifies the property descried in Lemm 12. This is done y considering the utomton B on the lphet Σ = Σ {#} otined from A 3 y dding trnsition leled y # from stte (p, q, q) g,g to stte (p, p, q) f,f for every pir (p, q) of sttes in A such tht p q. It follows tht A 3 verifies the condition in Lemm 12 iff there is cycle in B contining oth trnsition leled y # nd trnsition leled y symol in Σ. This property cn e checked strightforwrdly using depth-first serch of B to compute its strongly connected components. If strongly connected component of B is found tht contins oth trnsition leled with # nd trnsition leled y symol in Σ, A verifies (IDA) ut not (EDA) nd thus A is polynomilly miguous. Otherwise, A is finitely miguous. The complexity of this step is liner in the size of B: O( B E ) = O( A 3 E + A 2 Q ) = O( A 3 E ) since A nd B re trim. The totl complexity of the lgorithm is O( A 2 E + A 3 E ) = O( A 3 E ). When A is polynomilly miguous, we cn derive from the lgorithm just descried one tht computes dp(a). Theorem 14. Let A e trim ǫ-cycle free finite utomton. If A is polynomilly miguous, dp(a) cn e computed in time O( A 3 E ). Proof. We first compute A 3 nd use the lgorithm of Theorem 13 to test whether A is polynomilly miguous nd to compute ll the pirs (p, q) tht verify the condition of Lemm 12. This step hs complexity O( A 3 E ).

13 Testing Automt Amiguity nd Trnsducer Doule-Tpe Amiguity 13 We then compute the component grph G of A, nd for ech pir (p, q) found in the previous step, we dd trnsition leled with # from the strongly connected component of p to the one of q. If there is pth in tht grph contining d edges leled y #, then A verifies (IDA d ). Thus, dp(a) is the mximum numer of edges mrked y # tht cn e found long pth in G. Since G is cyclic, this numer cn e computed in liner time in the size of G, i.e. in O( A 2 Q ). Thus, the overll complexity of the lgorithm is O( A 3 E ). Finlly, let us point out tht A 2 cn lso e used to devise simple test for the miguity of A sed on the following oservtion. Lemm 15. Let A e trim ǫ-cycle free finite utomton. A is unmiguous iff every coccessile stte in A 2 = A A is of the form (p, p) 0. Proof. Assume A is unmiguous nd let (p, q) f e coccessile stte in A 2. Since A 2 hs een trimmed, (p, q) f is oth ccessile nd coccessile. Hence, there exist pth π from the initil stte to finl stte of A 2 tht goes through (p, q) f. This pth ws otined y mtching two ccepting pths π 1 nd π 2 with the sme lel with π 1 going through p nd π 2 going through q. If p q or f 0, then π 1 nd π 2 re distinct (y Theorem 9) nd this contrdicts A unmiguous. Hence, p = q nd f = 0. Conversely, let us ssume tht every coccessile stte in A 2 is of the form (p, p) 0. Let us consider two ccepting pths π 1 nd π 2 shring the sme lel. These two pths will e mtched y composition to form n ccepting pth π in A 2. Since there cnnot e multiple trnsitions with the sme lel etween given pir of sttes, the fct tht ll sttes long π re of the form (p, p) 0 implies tht π 1 = π 2. Hence, A is unmiguous. Oserve tht here gin the use of the ǫ-filter in composition is crucil for Lemm 15 to hold (see Figure 2). Theorem 16. Let A e trim ǫ-cycle free finite utomton. It is decidle in time O( A 2 E ) whether A is miguous. Proof. The lgorithms proceeds s follows. We first compute A 2 nd perform depth-first serch to trim it. We cn now check in O( A 2 Q ) time tht ech stte is of the form (p, p) 0. Thus, the complexity of the lgorithm is in O( A 2 E ). 5. Doule-Tpe Amiguity The previous sections presented comprehensive study of the miguity of finite utomt. The notion of miguity is typiclly defined in the sme wy for finitestte trnsducers: trnsducer is sid to e miguous if it dmits two ccepting As mentioned in section 4.2, we lwys trim the result of intersection.

14 14 C. Alluzen, M. Mohri nd A. Rstogi pths with the sme input lel. Thus, the results of the previous sections pply to the trnsducer cse identiclly with tht notion of miguity. There is however nother notion of miguity relted to oth tpes of trnsducer tht is of interest in pplictions, which we refer to s doule-tpe miguity. This section dels with tht notion of doule-tpe miguity. It gives generl decidility nd hrdness results for doule-tpe miguity, nd presents specific nlysis for the cse of trnsducers with ounded dely, including chrcteriztions nd lgorithms for testing the doule-tpe miguity of such trnsducers. We strt with the stndrd definition of finite-stte trnsducer. Definition 17 (Finite-stte trnsducers) A finite-stte trnsducer T is 6- tuple (Σ,, Q, E, I, F) where Σ is finite input lphet of the trnsducer; is finite output lphet; Q is finite set of sttes; I Q the set of initil sttes; F Q the set of finl sttes; nd E Q (Σ {ǫ}) ( {ǫ}) Q finite set of trnsitions. We sy tht the trnsducer T ccepts pir (x, y) Σ if T dmits n ccepting pth with input lel x nd output lel y nd denote this y (x, y) R(T). R(T) is the rtionl reltion defined y T. Given trnsducer T, we define the inverse of T, denoted y T 1, the trnsducer otined y swpping the input nd output lels of ech trnsition in T, thus (x, y) R(T 1 ) iff (y, x) R(T). Let T 1 nd T 2 e two finite-stte trnsducers such tht the input lphet of T 2 coincides with the output lphet of T 1. The result of the composition of T 1 nd T 2 is finite-stte trnsducer denoted y T 1 T 2 nd specified for ll x, y y: (x, y) R(T 1 T 2 ) iff there exists z such tht (x, z) R(T 1 ) nd (z, y) R(T 2 ). The lgorithm to compute the composition of two finite-stte trnsducers is slight modifiction of the intersection lgorithm descried in section 4. The following rule specifies how to compute trnsition of T 1 T 2 from pproprite trnsitions of T 1 nd T 2 in the sence of output-ǫ trnsitions in T 1 nd input-ǫ trnsitions in T 2 : (q 1,,, q 1 ) nd (q 2,, c, q 2 ) = ((q 1, q 2 ),, c, (q 1, q 2 )). The sme epsilon-filtering technique descried in section 4.2 is then used to del with output-ǫ trnsitions in T 1 nd input-ǫ trnsitons in T 2 [14, 13]. The notion of doule-tpe unmiguous trnsducers is defined s follows. Definition 18 (Doule-Tpe Unmiguous Trnsducer) A trnsducer T is sid to e doule-tpe unmiguous if for ll (x, y) Σ, it dmits t most one ccepting pth in T with input lel x nd output lel y. This notion clerly differs from the single-tpe notion discussed in the previous sections for utomt nd often used for trnsducers. A trnsducer dmitting multiple pths with the sme input lel x cn still e doule-tpe unmiguous so long s the output lels of those pths re ll distinct. The generl prolem of determining doule-tpe miguity turns out to e considerly hrder thn tht of determining single-tpe miguity however.

15 Testing Automt Amiguity nd Trnsducer Doule-Tpe Amiguity 15 N:u[N]... 1:u[1] N:v[N]... 1:v[1] 1 2 Fig. 6. The trnsducer constructed corresponding to PCP prolem with lists of strings u i, v i Σ for 1 i N. Both sttes 1 nd 2 re initil nd finl Undecidility Result We show tht the generl prolem of determining if trnsducer T is doule-tpe miguous is undecidle. When we restrict the trnsducer to e cyclic, then the prolem ecomes NP-hrd. Our reduction is from the Post Correspondence Prolem (PCP) [15]. Definition 19 (The Post Correspondence Prolem [15]) Given two list of strings u 1, u 2,...,u N nd v 1, v 2,...,v N, with u i, v i Σ for 1 i N, determine whether there exists sequence of indices (i 1, i 2,..., i K ) with K 1 nd 1 i k N such tht: u i1 u i2... u ik = v i1 v i2...v ik. Theorem 20 ([15, 11]) PCP is undecidle in generl. Furthermore, the prolem remins undecidle even when restricted to fixed numer of strings in (u i ) N, (v i) N, for N 7. Theorem 21. The prolem of determining the doule-tpe miguity of n ritrry finite-stte trnsducer T is undecidle. Proof. Given PCP prolem instnce over the lphet Σ with strings (u i ) N nd (v i ) N, we construct trnsducer T such tht T is doule-tpe miguous if nd only if the PCP prolem hs solution. The trnsducer T is defined s follows (see Figure 6): The set of sttes Q = {1, 2} with I = F = Q. The set of trnsitions E s: E = {(1, i, u i, 1) : 1 i N} {(2, i, v i, 2) : 1 i N}, where (q i,,, q j ) denotes trnsition from stte q i to q j with input lel nd output lel. c If the PCP instnce hs solution (i 1,...,i K ), then T is doule-tpe miguous since the pir i 1...i K : u i1... u ik is ccepted on two pths: one through the trnsitions (1, i k, u ik, 1) for 1 k K, the other through (2, i k, v ik, 2) for 1 k K. c In order to simplify the proof we consider here trnsducer with trnsition outputs in. There strightforwrdly exists n equivlent trnsducer with trnsition outputs in {ǫ}.

16 16 C. Alluzen, M. Mohri nd A. Rstogi Conversely, if T is doule-tpe miguous then there exists two pths π 1 nd π 2 with the sme input nd output lels. A pth in T either remins t stte 1 or t stte 2. It is cler tht if two distinct pths π 1 nd π 2 hve the sme input lels, then they must e t different sttes. Let π 1 e the pth tht remins t stte 1 nd π 2 the pth tht remins t stte 2. Let the input lel on π 1 (nd π 2 ) e the sequence i 1...i K. Since the output lels re the sme on π 1 nd π 2, it follows tht u 1 u 2... u ik = v 1 v 2... v ik. Thus the PCP dmits solution nd the proof is complete. It is nturl to sk how hrd the prolem remins if we restrict our ttention to more specific clsses of trnsducers. We show tht if we restrict ourselves to cyclic trnsducers, the prolem is NP-hrd. Theorem 22. The prolem of determining the doule-tpe miguity of n ritrry cyclic trnsducer T is NP-hrd. Proof. The reduction is from ounded PCP: vrint of PCP in which we seek sequence of indices i 1... i K with K B for some fixed B > 0. The ounded PCP is NP-complete [6]. Insted of hving self-loops t sttes 1 nd 2 in the construction of Theorem 21, we simply unfold the loops B times. This shows tht the prolem for cyclic trnsducers is (t lest) NP-hrd. Note tht this result does not imply tht the prolem is in NP, which in fct, most likely, is not the cse Bounded-dely trnsducers One nturl clss of trnsducers for which more positive results hold is tht of trnsducers with ounded dely. This imposes ound on the mximum difference of length etween the input nd output lel of pth. The following gives forml definition of the notion of dely. Definition 23 (Dely) The dely of pth π is defined s the difference of length etween its input nd output lels: dely(π) = o[π] i[π]. (5) A trim trnsducer T is sid to hve ounded dely if the dely of ll pths of T is ounded. We then denote y dely(t) the mximum dely of ll pths in T. A trnsducer T is synchronized if long ny ccepting pth of T the dely is zero or increses strictly monotoniclly: for ny ccepting pth π = π 1 eπ 2, dely(π 1 ) < dely(π 1 e) or dely(π 1 ) = dely(π 1 e) = 0. A trnsducer with ounded dely is synchronizle, tht is it dmits n equivlent synchronized trnsducer [12]. Given trnsducer T, let T s denote the synchronized trnsducer otined from T using the synchroniztion lgorithm of [12]. The complexity of the synchroniztion

17 Testing Automt Amiguity nd Trnsducer Doule-Tpe Amiguity 17 lgorithm is in O( T s ). However, the size of T s is exponentil in the worst-cse : O( T ( Σ dely(t) + dely(t) )) where Σ is the input lphet of T nd its output lphet. When T is synchronized trnsducer with dely of 0, we cn give chrcteriztion of doule-tpe miguity sed on the form of the identity pths in T 1 T. An identity pth π is n ccepting pth with equl input nd output lels: i[π] = o[π]. Recll tht stte in T T, the composition of two trnsducers T nd T, is of the form (p, q) f, where p is stte of T, q is stte of T, nd f stte of the epsilon-filter. Lemm 24 (Chrcteriztion) Let T e synchronized trnsducer with dely(t) = 0. T is doule-tpe miguous if nd only if there exists successful identity pth in T 1 T going through stte of the form (p, q) f with p q or f 0. Proof. Oserve tht since T is synchronized nd hs dely zero, every trnsition must hve either oth its input nd output lels equl to ǫ, or oth non-ǫ. Assume tht T is doule-tpe miguous. Then, T dmits two ccepting pths π 1 nd π 2 with the sme input nd output lels, sy x nd y respectively. Since these two pths shre the sme input, they re mtched y composition, which results in pth π in T 1 T. Moreover, π is n identity pth since π 1 nd π hve the sme output lel: o[π 1 ] = o[π 2 ]. Let e e the first trnsition long π tht ws otined y mtching two distinct trnsitions e 1 nd e 2 in T. We shll show tht n[e] is stte of the form (p, q) f with p q or f 0. If e 1 is virtul trnsition corresponding to remining t the sme stte without consuming ny symol while e 2 is n ctul ǫ-trnsition in T, then the filter stte of n[e] is not 0, f 0. Assume now tht oth e 1 nd e 2 re ctul trnsitions in T. Since e 1 nd e 2 re distinct nd i[e 1 ] = i[e 2 ], we must hve n[e 1 ] n[e 2 ] or o[e 1 ] o[e 2 ]. Since T hs dely of 0, we must hve o[e 1 ] = o[e 2 ]. Thus n[e 1 ] n[e 2 ] nd n[e] is of the form (p, q) f with p q. Conversely, ssume tht there exists n identity pth π in T 1 T going through stte of the form (p, q) f with f 0 or p q. This pth ws otined y mtching in composition two pths π 1 nd π 2 such tht i[π 1 ] = i[π 2 ] (since they re mtched in composition) nd o[π 1 ] = o[π 2 ] (since π is n identity pth). If π 1 nd π 2 were equl, ll the sttes long π would e of the form (p, p) 0. Thus, π 1 π 2 nd T is doule-tpe miguous. This chrcteriztion directly leds to n lgorithm for testing the doule-tpe miguity of synchronized trnsducers. Theorem 25. The doule-tpe miguity of synchronized trnsducer T cn e decided in time O( T 2 ), where T = Q + E is the totl numer of sttes nd trnsitions of T.

18 18 C. Alluzen, M. Mohri nd A. Rstogi Proof. A key property of synchronized trnsducer T is tht long ny successful pth, trnsition with non-ǫ input nd ǫ output cn only e followed y trnsitions with non-ǫ input nd ǫ output. Similrily, trnsition of with ǫ input nd nonǫ output cn only e followed y trnsitions with ǫ input nd non-ǫ output. By replcing such ǫs with specil symol not lredy in Σ or, sy #, we otin synchronized trnsducer T with dely of 0 such tht T is doule-tpe miguous iff T is doule-tpe miguous. The lgorithm then consists of computing T 1 T, deleting ny trnsitions e such tht i[e] o[e] nd performing depth-first serch to verify tht the sttes tht re oth ccessile nd co-ccessile re ll of the form (p, 0, p). Finlly, we cn use the previous result to devise n effective lgorithm for testing the doule-tpe miguity of ounded-dely trnsducers. Corollry 26. Let T e ounded-dely trnsducer with input lphet Σ nd output lphet. It is decidle in time O( T 2 ( Σ dely(t) + dely(t) ) 2 ) whether T is doule-tpe miguous. Proof. Since T hs ounded dely, we cn use the synchroniztion lgorithm from [12] to compute n equivlent synchronized trnsducer T s. The synchroniztion lgorithms preserves doule-tpe miguity thus T s is doule-tpe miguous iff T is doule-tpe miguous nd y Theorem 25 we cn decide the doule-tpe miguity of T in time O( T s 2 ). 6. Appliction to Entropy Approximtion In this section, we descrie n ppliction in which determining the degree of miguity of proilistic utomton helps estimte the qulity of n pproximtion of its entropy. Weighted utomt re utomt in which ech trnsition crries some weight in ddition to the usul lphet symol. The weights re elements of semiring, tht is ring tht my lck negtion. The following is more forml definition. Definition 27. A weighted utomton A over semiring (K,,, 0, 1) is 7- tuple (Σ, Q, I, F, E, λ, ρ) where Σ is finite lphet, Q finite set of sttes, I Q the set of initil sttes, F Q the set of finl sttes, E Q Σ {ǫ} K Q finite set of trnsitions, λ : I K the initil weight function mpping I to K, nd ρ : F K the finl weight function mpping F to K. Given trnsition e E, we denote y w[e] its weight. We extend the weight function w to pths y defining the weight of pth s the -product of the weights of its constituent trnsitions: w[π] = w[e 1 ] w[e k ]. The weight ssocited y weighted utomton A to n input string x Σ is defined y [A](x) = λ[p[π]] w[π] ρ[n[π]]. (6) π P(I,x,F)

19 Testing Automt Amiguity nd Trnsducer Doule-Tpe Amiguity 19 The entropy H(A) of proilistic utomton A is defined s: H(A) = x Σ [A](x)log([A](x)). (7) The system (K,,, (0, 0), (1, 0)) with K = (R {+, }) (R {+, }) nd nd defined s follows defines commuttive semiring clled the entropy semiring [4]: for ny two pirs (x 1, y 1 ) nd (x 2, y 2 ) in K, (x 1, y 1 ) (x 2, y 2 ) = (x 1 + x 2, y 1 + y 2 ) (8) (x 1, y 1 ) (x 2, y 2 ) = (x 1 x 2, x 1 y 2 + x 2 y 1 ). (9) In [4], the uthors showed tht generlized shortest-distnce lgorithm over this semiring correctly computes the entropy of n unmiguous proilistic utomton A. The lgorithm strts y mpping the weight of ech trnsition to pir where the first element is the proility nd the second the entropy: w[e] (w[e], w[e] log w[e]). The lgorithm then proceeds y computing the generlized shortest-distnce defined over the entropy semiring, which computes the -sum of the weights of ll ccepting pths in A. Here, we show tht the sme shortest-distnce lgorithm yields n pproximtion of the entropy of n miguous proilistic utomton A, where the pproximtion qulity is function of the degree of polynomil miguity, dp(a). Our proofs mke use of the stndrd log-sum inequlity [5], specil cse of Jensen s inequlity, which holds for ny positive rels 1,..., k, nd 1,..., k : ( k k i log i i ) k i log i k. (10) i Lemm 28. Let A e proilistic utomton nd let x Σ + e string ccepted y A on k pths π 1,..., π k. Let w[π i ] e the proility of pth π i. Clerly, [A](x) = k w[π i]. Then, k w[π i ] log w[π i ] [A](x)(log[A](x) log k). (11) Proof. The result follows strightforwrdly from the log-sum inequlity, with i = w[π i ] nd i = 1: ( k k ) k w[π i ] log w[π i ] w[π i ] log w[π i] = [A](x)(log[A](x) log k). (12) k Let S(A) e the quntity computed y the generlized shortest-distnce lgorithm over the entropy semiring or proilistic utomton A. When A is unmiguous, it is shown y [4] tht S(A) = H(A). Theorem 29. Let A e proilistic utomton nd let L denote the expected length of the strings ccepted y A (i.e. L = x Σ x [A](x)). Then,

20 20 C. Alluzen, M. Mohri nd A. Rstogi (1) if A is finitely miguous with d(a) = k for some k N, then H(A) S(A) H(A) + log k; (2) if A is polynomilly miguous with dp(a) = k for some k N, then H(A) S(A) H(A) + k log L. Proof. The lower ound S(A) H(A) follows from the oservtion tht for string x tht is ccepted in A y k pths π 1,..., π k, k ( k ) ( k ) w[π i ] log(w(π i )) w[π i ] log w[π i ]. (13) Since the quntity k w[π i] log(w[π i ]) is string x s contriution to S(A) nd the quntity ( k w[π i])log( k w[π i]) its contriution to H(A), summing over ll ccepted strings x, we otin H(A) S(A). Assume tht A is finitely miguous with degree of miguity k. Let x Σ e string tht is ccepted on l x k pths π 1,...,π lx. By Lemm 28, we hve l x Thus, w[π i ] log w[π i ] [A](x)(log[A](x) log l x ) [A](x)(log[A](x) log k). (14) S(A) = l x x Σ w[π i ] log w[π i ] H(A) + x Σ (log k)[a](x) = H(A) + log k.(15) This proves the first sttement of the theorem. Next, ssume tht A is polynomilly miguous with degree of polynomil miguity k. By Lemm 28, we hve l x w[π i ] log w[π i ] [A](x)(log[A](x) log l x ) [A](x)(log[A](x) log( x k )).(16) Thus, S(A) H(A) + x Σ k[a](x)log x = H(A) + ke A [log x ] (17) H(A) + k log E A [ x ] = H(A) + k log L, which proves the second sttement of the theorem. (y Jensen s inequlity) The theorem shows in prticulr tht the qulity of the pproximtion of the entropy of polynomilly miguous proilistic utomton cn e estimted y computing its degree of polynomil miguity, which cn e chieved efficiently s descried in the previous section. This lso requires the computtion of the expected length L of n ccepted string. L cn e computed efficiently for n ritrry proilistic utomton using the entropy semiring nd the generlized shortest-distnce lgorithms, using techniques similr to those descried in [4]. The only difference is in the initil step, where the weight of ech trnsition in A is mpped to pir of elements y w[e] (w[e], w[e]).

21 Testing Automt Amiguity nd Trnsducer Doule-Tpe Amiguity Conclusion We presented simple nd efficient lgorithms for testing the finite, polynomil, or exponentil miguity of finite utomt with ǫ-trnsitions. We conjecture tht the time complexity of our lgorithms is optiml. These lgorithms hve vriety of pplictions, in prticulr to test pre-condition for the pplicility of other utomt lgorithms. Our ppliction to the pproximtion of the entropy gives nother illustrtion of their usefulness. We lso initited the study of the doule-tpe miguity of finite-stte trnsducers nd gve numer of decidility nd chrcteriztions results s well s lgorithms in the ounded dely cse. These lgorithms cn e of interest in numer of modern pplictions where finite-stte trnsducers re used. Our lgorithms lso demonstrte the prominent role plyed y the intersection of finite utomt or composition of finite-stte trnsducers with ǫ-trnsitions [14, 13] in the design of testing lgorithms. Composition cn e used to devise simple nd efficient testing lgorithms. We hve shown elsewhere how it cn e used to test the functionlity of finite-stte trnsducer, or the twins property for weighted utomt nd trnsducers [1]. References [1] Cyril Alluzen nd Mehryr Mohri. Efficient Algorithms for Testing the Twins Property. Journl of Automt, Lnguges nd Comintorics, 8(2): , [2] Cyril Alluzen nd Mehryr Mohri. 3-wy composition of weighted finite-stte trnsducers. In CIAA 2008, volume 5148 of LNCS, pges Springer, [3] Tt-hung Chn nd Oscr H. Irr. On the finite-vluedness prolem for sequentil mchines. Theoreticl Computer Science, 23:95 101, [4] Corinn Cortes, Mehryr Mohri, Ashish Rstogi, nd Michel Riley. On the computtion of the reltive entropy of proilistic utomt. Interntionl Journl of Foundtions of Computer Science, 19(1): , [5] Thoms M. Cover nd Joy A. Thoms. Elements of Informtion Theory. John Wiley & Sons, Inc., New York, [6] Michel R. Grey nd Dvid S. Johnson. Computers nd Intrctility: A Guide to the Theory of NP-Completeness. W. H. Freemn & Co., New York, NY, USA, [7] Oscr H. Irr nd Bl Rvikumr. On sprseness, miguity nd other decision prolems for cceptors nd trnsducers. In STACS 1986, volume 210 of LNCS, pges Springer, [8] Gérrd Jco. Un lgorithme clculnt le crdinl, fini ou infini, des demi-groupes de mtrices. Theoreticl Computer Science, 5(2): , [9] Werner Kuich. Finite utomt nd miguity. Technicl Report 253, Institute für Informtionsverreitung - Technische Universität Grz und ÖCG, [10] Arnldo Mndel nd Imre Simon. On finite semigroups of mtrices. Theoreticl Computer Science, 5(2): , [11] Yuri Mtiysevich nd Gérud Sénizergues. Decision prolems for semi-thue systems with few rules. In IEEE Symposium on Logic in Computer Science, pges , [12] Mehryr Mohri. Edit-distnce of weighted utomt: Generl definitions nd lgorithms. Interntionl Journl of Foundtions of Computer Science, 14(6): ,