Probabilistic Automata on Finite words: Decidable and Undecidable Problems

Transcription

1 Proilistic Automt on Finite words: Decidle nd Undecidle Prolems Hugo Gimert, Youssouf Oulhdj To cite this version: Hugo Gimert, Youssouf Oulhdj. Proilistic Automt on Finite words: Decidle nd Undecidle Prolems. ICALP 2010, Jul 2010, Bordeux, Frnce. pp , 2010, < / _44>. <hl v3> HAL Id: hl Sumitted on 27 Apr 2010 HAL is multi-disciplinry open ccess rchive for the deposit nd dissemintion of scientific reserch documents, whether they re pulished or not. The documents my come from teching nd reserch institutions in Frnce or rod, or from pulic or privte reserch centers. L rchive ouverte pluridisciplinire HAL, est destinée u dépôt et à l diffusion de documents scientifiques de niveu recherche, puliés ou non, émnnt des étlissements d enseignement et de recherche frnçis ou étrngers, des lortoires pulics ou privés.

2 Proilistic Automt on Finite Words: Decidle nd Undecidle Prolems Hugo Gimert 1 nd Youssouf Oulhdj 2 1 LBRI, CNRS, Frnce hugo.gimert@lri.fr 2 LBRI, Université Bordeux 1, Frnce youssouf.oulhdj@lri.fr Astrct. This pper tckles three lgorithmic prolems for proilistic utomt on finite words: the Emptiness Prolem, the Isoltion Prolem nd the Vlue 1 Prolem. The Emptiness Prolem sks, given some proility 0 λ 1, whether there exists word ccepted with proility greter thn λ, nd the Isoltion Prolem sks whether there exist words whose cceptnce proility is ritrrily close to λ. Both these prolems re known to e undecidle [11, 4, 3]. Aout the Emptiness prolem, we provide new simple undecidility proof nd prove tht it is undecidle for utomt with s few s two proilistic trnsitions. The Vlue 1 Prolem is the specil cse of the Isoltion Prolem when λ = 1 or λ = 0. The decidility of the Vlue 1 Prolem ws n open question. We show tht the Vlue 1 Prolem is undecidle. Moreover, we introduce new clss of proilistic utomt, -cyclic utomt, for which the Vlue 1 Prolem is decidle. Introduction Proilistic utomt on finite words re computtion model introduced y Rin [12]. Like deterministic utomt on finite words, proilistic utomton reds finite words from finite lphet A. Ech time new letter A is red, trnsition from the current stte s Q to new stte t Q occur. In deterministic utomton, t is function of s nd. In proilistic utomton, lottery determines the new stte, ccording to trnsition proilities which depend on the current stte s nd letter. Since the seminl pper of Rin, proilistic utomt on finite words hve een extensively studied, see [6] for survey of 416 ppers nd ooks out proilistic utomt pulished in the 60s nd 70s. Quite surprisingly, reltively few lgorithmic results re known out proilistic utomt on finite words nd lmost ll of them re undecidility results. There re two min lgorithmic prolems for proilistic utomt on finite words: the Emptiness Prolem nd the Isoltion Prolem. The Emptiness Prolem sks, given some proility 0 λ 1, whether there exists word ccepted with proility greter thn λ, while the Isoltion Prolem sks whether there exist words whose cceptnce proility is ritrrily close to

3 λ. Both these prolems were shown undecidle, respectively y Pz [11] nd Bertoni [4, 3]. To our knowledge, most decidility results known for proilistic utomt on finite words re rther strightforwrd: they either pply to one-letter proilistic utomt, in other words Mrkov chins, or to prolems where the proilistic nture of the utomton is not tken into ccount. A notle exception is the decidility of lnguge equlity [13]. In contrst, severl lgorithmic results were proved for proilistic utomt on infinite words. The existence of n infinite word ccepted with proility 1 is decidle [1]. For proilistic Büchi utomt [2], the emptiness prolem my e decidle or not depending on the cceptnce condition. A clss of proilistic Büchi utomt which recognize exctly ω-regulr lnguges ws presented in [7]. In this pper, we only consider utomt on finite words ut severl of our results seem to e extendle to proilistic utomt on infinite words. Our contriutions re the following. First, we provide in Section 2.1 new proof for the undecidility of the Emptiness Prolem. Second, we strengthen the result of Pz: the Emptiness Prolem is undecidle even for utomt with s few s two proilistic trnsitions (Proposition 4). Third, we solve n open prolem: Bertoni s result shows tht for ny fixed cut-point 0 < λ < 1, the Isoltion Prolem is undecidle. However, s stted y Bertoni himself, the proof seems hrdly dptle to the symmetric cses λ = 0 nd λ = 1. We show tht oth these cses re undecidle s well, in other words the Vlue 1 Prolem is undecidle (Theorem 4). Fourth, we introduce new clss of proilistic utomt, -cyclic utomt, for which the Vlue 1 Prolem is decidle (Theorem 5 in Section 4). To our opinion, this is the min contriution of the pper. Moreover, this result is first step towrds the design of clsses of stochstic gmes with prtil oservtion for which the vlue 1 prolem is decidle. These undecidility results show once gin tht proilistic utomt re very different from deterministic nd non-deterministic utomt on finite of infinite words, for which mny lgorithmic prolems re known to e decidle (e.g. emptiness, universlity, equivlence). Surprisingly mye, we remrk tht severl nturl decision prolems out deterministic nd non-deterministic utomt re undecidle s well (Corollries 1 nd 2). Due to spce restrictions most proofs re omitted, they cn e found in [?]. 1 Proilistic Automt AproilitydistriutiononQismppingδ [0,1] Q suchtht s S δ(s) = 1. The set {s Q δ(s) > 0} is clled the support of δ nd denoted Supp(δ). For every non-empty suset S Q, we denote δ S the uniform distriution on S defined y δ(q) = 0 if q S nd δ(q) = 1 S if q S. We denote D(Q) the set of proility distriutions on Q.

4 Formlly, proilistic utomton is tuple A = (Q,A,(M ) A,q 0,F), where Q is finite set of sttes, A is the finite input lphet, (M ) A re the trnsition mtrices, q 0 is the initil stte nd F is the set of ccepting sttes. For ech letter A, M [0,1] Q Q defines trnsition proilities: 0 M (s,t) 1 is the proility to go from stte s to stte t when reding letter. Of course, for every s S nd A, t S M (s,t) = 1, in other word in the mtrix M, the line with index s is proility distriution on Q. Trnsition mtrices define nturl ction of A on D(Q). For every word A nd δ D(S), we denote δ the proility distriution in D(Q) defined y (δ )(t) = s Q δ(s) M (s,t). This ction extends nturlly to words of A : w A, A,δ (w) = (δ w). The computtion of A on n input word w = 0... n A is the sequence (δ 0,δ 1,...,δ n ) D(Q) n+1 of proility distriutions over Q such tht δ 0 = δ {q0} nd for 0 i < n,δ i+1 = δ i i. For every stte q Q nd for every set of sttes R Q, we denote P A (q w R) = r R (δ q w)(r) the proility to rech the set R from stte q when reding the word w. Definition 1 (Vlue nd cceptnce proility). The cceptnce proility of word w A y A is P A (w) = P A (q 0 w F). The vlue of A, denoted vl(a), is the supremum cceptnce proility: vl(a) = sup w A P A (w). 2 The Emptiness Prolem Rin defined the lnguge recognized y proilistic utomton s L A (λ) = {w A P A (w) λ}, where 0 λ 1 is clled the cut-point. Hence, cnonicl decision prolem for proilistic utomt is: Prolem 1 (Emptiness Prolem) Given proilistic utomton A nd 0 λ 1, decide whether there exists word w such tht P A (w) λ. The Strict Emptiness Prolem is defined the sme wy except the lrge inequlity P A (w) λ is replced y strict inequlity P A (w) > λ. The specil cses where λ = 0 nd λ = 1 provide link etween proilistic nd non-deterministic utomt on finite words. First, the Strict Emptiness Prolem for λ = 0 reduces to the emptiness prolem of non-deterministic utomt, which is decidle in non-deterministic logrithmic spce. Second, the Emptiness Prolem for λ = 1 reduces to the universlity prolem for nondeterministic utomt, which is PSPACE-complete [9]. The two other cses re trivil: the nswer to the Emptiness Prolem for λ = 0 is lwys yes nd the nswer to the Strict Emptiness Prolem for λ = 1 is lwys no. In the cse where 0 < λ < 1, oth the Emptiness nd the Strict Emptiness Prolems re undecidle, which ws proved y Pz [11]. The proof of Pz is reduction from n undecidle prolem out free context grmmrs. An lterntive proof ws given y Mdni, Hnks nd Condon [10], sed on reduction from the emptiness prolem for two counter mchines. Since Pz ws focusing

5 on expressiveness spects of proilistic utomt rther thn on lgorithmic questions, his undecidility proof is spred on the whole ook [11], which mkes it rguly hrd to red. The proof of Mdni et l. is esier to red ut quite long nd technicl. In the next section, we present new simple undecidility proof of the Emptiness Prolem. 2.1 New proof of undecidility In this section we show the undecidility of the (Strict) Emptiness Prolem for the cut-point 1 2 nd for restricted clss of proilistic utomt clled simple proilistic utomt: Definition 2 (Simple utomt). A proilistic utomton is clled simple if every trnsition proility is in { 0, 1 2,1}. TheproofissedonresultofBertoni[4]:theundecidilityoftheEqulity Prolem. Prolem 2 (Equlity prolem) Given simple proilistic utomton A, decide whether there exists word w A such tht P A (w) = 1 2. Proposition 1 (Bertoni). The equlity prolem is undecidle. The short nd elegnt proof of Bertoni is reduction of the Post Correspondence Prolem (PCP) to the Equlity Prolem. Prolem 3 (PCP) Let ϕ 1 : A {0,1} nd ϕ 2 : A {0,1} two functions, nturlly extended to A. Is there word w A such tht ϕ 1 (w) = ϕ 2 (w)? Roughly speking, the proof of Proposition 1 consists in encoding the equlity of two words in the decimls of trnsition proilities of well-chosen proilistic utomton. While the reduction of PCP to the Equlity prolem is reltively well-known, it my e less known tht there exists simple reduction of the Equlity prolem to the Emptiness nd Strict Emptiness prolems: Proposition 2. Given simple proilistic utomton A, one cn compute proilistic utomt B nd C whose trnsition proilities re multiple of 1 4 nd such tht: ( w A +,P A (w) = 1 ) 2 ( w A +,P B (w) 1 ) (1) 4 ( w A +,P C (w) > 1 ). (2) 8 Proof. The construction of B such tht (1) holds is sed on very simple fct: rel numer x is equl to 1 2 if nd only if x(1 x) 1 4. Consider the utomton B which is the crtesin product of A with copy of A whose ccepting sttes

6 re the non ccepting sttes of A. Then for every word w A, P A1 (w) = P A (w)(1 P A (w)), thus (1) holds. The construction of C such tht (2) holds is sed on the following ide. Since A is simple, trnsition proilities of B re multiples of 1 4, thus for every 1 word w of length w, P B (w) is multiple of. As consequence, P 4 w B (w) 1 4 if nd only if P B (w) > Adding three sttes to B, one otins esily 4 w proilistic utomton C such tht for every non-empty word w A nd letter A, P C (w) = 1 2 P B(w) , thus (2) holds. To uild C, simply 4 w dd new initil stte tht goes with equl proility 1 2 either to the initil stte of B or to new ccepting stte q f. From q f, whtever letter is red, next stte is q f with proility 1 4 nd with proility 3 4 it is new non-ccepting soring sink stte q. As consequence: Theorem 1 (Pz). The emptiness nd the strict emptiness prolems re undecidle for proilistic utomt. These prolems re undecidle even for simple proilistic utomt nd cut-point λ = 1 2. To conclude this section, we present nother connection etween proilistic nd non-proilistic utomt on finite words. Corollry 1. The following prolem is undecidle. Given non-deterministic utomton on finite words, does there exists word such tht t lest hlf of the computtions on this word re ccepting? We do not know simple undecidility proof for this prolem which does not mke of use of proilistic utomt. 2.2 Proilistic utomt with few proilistic trnsitions Hirvenslo[8] showed tht the emptiness prolem is undecidle for proilistic utomtwhichhvesfews2inputlettersnd25sttes,seelso[5]forsimilr result out the isoltion prolem. On the other hnd, the emptiness prolem is decidle for deterministic utomt. This holds whtever the numer of sttes, s long s there re no proilistic trnsition in the utomton. Formlly, proilistic trnsition is couple (s,) of stte s S nd letter A such tht for t lest one stte t S, 0 < M (s,t) < 1. This motivtes the following question: wht is the miniml numer of proilistic trnsitions for which the emptiness prolem is undecidle? The following undecidility result is rther surprising nswer: Proposition 3. The emptiness prolem is undecidle for proilistic utomt with two proilistic trnsitions. Moreover, slight vrint of the emptiness prolem for proilistic utomt with one proilistic trnsition is undecidle:

7 Proposition 4. The following prolem is undecidle: given simple proilistic utomton over n lphet A with one proilistic trnsition nd given rtionl lnguge of finite words L A, decide whether P A (w) 1 2 for some word w L. For proilistic utomt with unique proilistic trnsition, we do not know whether the emptiness prolem is decidle or not. 3 Undecidility of the Vlue 1 prolem In his seminl pper out proilistic utomt [12], Rin introduced the notion of isolted cut-points. Definition 3. A rel numer 0 λ 1 is n isolted cut-point with respect to proilistic utomton A if: ε > 0, w A, P A (w) λ ε. Rin motivtes the introduction of this notion y the following theorem: Theorem 2 (Rin). Let A proilistic utomton nd 0 λ 1 cutpoint. If λ is isolted then the lnguge L A (λ) = {u A P A (u) λ} is rtionl. This result suggests the following decision prolem. Prolem 4 (Isoltion Prolem) Given proilistic utomton A nd cut-point 0 λ 1, decide whether λ is isolted with respect to A. Bertoni [4] proved tht the Isoltion Prolem is undecidle in generl: Theorem 3 (Bertoni). The Isoltion Prolem is undecidle for proilistic utomt with five sttes. A closer look t the proof of Bertoni shows tht the Isoltion Prolem is undecidle for fixed λ, provided tht 0 < λ < 1. However the sme proof does not seem to e extendle to the cses λ = 0 nd λ = 1. This ws pointed out y Bertoni in the conclusion of [4]: Is the following prolem solvle: δ > 0, x,(p(x) > δ)? For utomt with 1-symol lphet, there is decision lgorithm ound with the concept of trnsient stte [11]. We elieve it might e extended ut hve no proof for it. The open question mentioned y Bertoni is the Isoltion Prolem for λ = 0. The cse λ = 1 is essentilly the sme, since 0 is isolted in n utomton A if nd only if 1 is isolted in the utomton otined from A y turning finl sttes to non-finl sttes nd vice-vers. When λ = 1, the Isoltion Prolem sks whether there exists some word ccepted y the utomton with proility ritrrily

8 close to 1. We use the gme-theoretic terminology nd cll this prolem the Vlue 1 Prolem. The open question of Bertoni cn e rephrsed s the decidility of the following prolem: Prolem 5 (Vlue 1 Prolem) Given proilistic utomton A, decide whether A hs vlue 1. Unfortuntely, Theorem 4. The Vlue 1 Prolem is undecidle. The proofoftheorem4is reduction ofthe Strict Emptiness Prolemto the Vlue 1 Prolem. It is similr to the proof of undecidility of the Emptiness Prolem for proilistic Büchi utomt of Bier et l. [2]. The core of the proof is the following proposition. Proposition 5. Let 0 < x < 1 nd A x e the proilistic utomton depicted on Fig. 1. Then A x hs vlue 1 if nd only if x > 1 2. The proof of Theorem 4 relies on the fct tht there is nturl wy to comine A x with n ritrry utomton B so tht the resulting utomton hs vlue 1 if nd only if some word is ccepted y B with proility strictly greter thn 1 2., 0, 6 4, 1 2, , 1 x, x 5 2, x, 1 x Fig.1. This utomton hs vlue 1 if nd only if x > 1 2. The vlue 1 prolem for simple proilistic utomt cn e strigntforwrdly rephrsed s quntittive decision prolem out non-deterministic utomton on finite words, which shows tht: Corollry 2. This decision prolem is undecidle: given non-deterministic utomton on finite words, does there exists words such tht the proportion of of non-ccepting computtion pthes mong ll computtion pthes is ritrrily smll?

9 4 The clss of -cyclic proilistic utomt In this section, we introduce new clss of proilistic utomt, -cyclic proilistic utomt, for which the vlue 1 prolem is decidle. To get decision lgorithm for the vlue 1 prolem, our strting point is the usul suset construction for non-deterministic utomt, defined y men of the nturl ction of letters on susets of Q. However the quntittive spect of the Vlue 1 Prolem stressed in Corollry 2 suggests tht the suset construction needs to e customized. Precisely, we use not only the usul ction S of letter on suset S Q of sttes ut consider lso nother ction. Roughly speking, deletes sttes tht re trnsient when reding letter forever. Definition 4 (Actions of letters nd -rechility). Let A proilistic utomton with lphet A nd set of sttes Q. Given S Q nd A, we denote: S = {s Q s S,M (s,t) > 0}. A stte t Q is -rechle from s Q if for some n N, P A (s n t) > 0. A stte s Q is -recurrent if it is in ottom strongly connected component of the grph of sttes nd -trnsitions i.e. if for ny stte t Q, (t is -rechle from s) = (s is -rechle from t). A set S Q is -stle if S = S. If S is -stle, we denote: S = {s S s is -recurrent}. The support grph G A of proilistic utomton A with lphet A nd set of sttes Q is the directed grph whose vertices re the non-empty susets of Q nd whose edges re the pirs (S,T) such tht for some letter A, either (S = T) or (S = S nd S = T). Rechility in the support grph of A is clled -rechility in A. The clss of -cyclic proilistic utomt is defined s follows. Definition 5 ( -cyclic proilistic utomt). A proilistic utomton is -cyclic if the only cycles in its support grph re self-loops. Oviously, this cyclicity condition is quite strong. However, it does not forid the existence of cycles in the trnsition tle, see for exmple the utomton depicted on Fig. 2. Note lso tht the clss of -cyclic utomt enjoys good properties: it is closed under crtesin product nd prllel composition. 4.1 The vlue 1 prolem is decidle for -cyclic utomt For -cyclic proilistic utomt, the vlue 1 prolem is decidle:

10 {1, 2} {1} {2}, 1 2 {1, 3} {1, 2, 3, 4} {2, 4} 3 4 {3} {4} {3, 4} Fig.2. A -cyclic utomton (on the left) nd its support grph (on the right). All trnsition proilities re equl to 1 2. Theorem 5. Let A e proilistic utomton with initil stte q 0 nd finl sttes F. Suppose tht A is -cyclic. Then A hs vlue 1 if nd only if F is -rechle from {q 0 } in A. The support grph cn e computed on the fly in polynomil spce thus deciding whether proilistic utomton is -cyclic nd whether n -cyclic utomton hs vlue 1 re PSPACE decision prolems. The rest of this section is dedicted to the proof of Theorem 5. This proof relies on the notion of limit-pths. Definition 6 (Limit pths nd limit-rechility). Let A e proilistic utomton with sttes Q nd lphet A. Given twosusets S,T of Q, we sy tht T is limit-rechle from S in A if there exists sequence w 0,w 1,w 2,... A of finite words such tht for every stte s S: P A (s wn T) 1. n The sequence w 0,w 1,w 2,... is clled limit pth from S to T, nd T is sid to e limit-rechle from S in A. In prticulr, n utomton hs vlue 1 is nd only if F is limit-rechle from {q 0 }. To prove Theorem 5, we show tht in -cyclic utomt, -rechility nd limit-rechility coincide. The following proposition shows tht -rechility lwys imply limit-rechility, my the utomton e -cyclic or not. Proposition 6. Let A e proilistic utomton with sttes Q nd S,T Q. If T is -rechle from S in A then T is limit-rechle from S in A.

11 The converse impliction is not true in generl. For exmple, consider the utomton depicted on Fig. 3. There is only one finl stte, stte 3. The initil stte is not represented, it leds with equl proility to sttes 1, 2 nd 3. The trnsitions from sttes 1, 2 nd 3 re either deterministic or hve proility , #,, # {2} {1, 2, 3}, {1, 2} {1} # {3} {2, 3} # #, # {1, 3}, # Fig.3. This utomton hs vlue 1 nd is not -cyclic. It turns out tht the utomton on Fig. 3 hs vlue 1, ecuse (( n ) n ) n N is limit-pth from {1,2,3} to {3}. However, {3} is not rechle from {1,2,3} in the support grph. Thus, limit-rechility does not imply -rechility in generl. This utomton is not -cyclic, ecuse his support grph contins cycle of length 2 etween {1,2,3} nd {1,3}. It is quite tempting to dd n edge lelled ( ) etween {1,3} nd {3}. Now we prove tht for -cyclic utomt, limit-rechility implies -rechility. Definition 7 (Stility nd -stility). Let A e proilistic utomton with sttes Q. The utomton A is stle if for every letter A, Q is -stle. A stle utomton A is -stle if for every letter A Q = Q. The proof relies on the three following lemmts. Lemm 1 (Blowing lemm). Let A e -cyclic proilistic utomton with sttes Q nd S Q. Suppose tht A is -cyclic nd -stle. If Q is limit-rechle from S in A, then Q is -rechle from S s well. Proof (of the lowing lemm). If S = Q there is nothing to prove. If S Q, we prove tht there exists S 1 Q such tht (i) S 1 is -rechle from S, (ii)

12 S S 1, nd (iii) Q is limit-rechle from S 1. Since S Q nd since there exists limit-pth from S to Q there exists t lest one letter such tht S is not -stle, i.e. S S. Since A is suset-cyclic, there exists n N such tht S n+1 = S n i.e. S n is -stle. We choose S 1 = (S n ) nd prove (i),(ii) nd (iii) First, (i) is ovious. To prove (ii), we prove tht S 1 contins oth S nd S. Let s S. By definition, every stte t of S n is -ccessile from s. Since A is -stle, stte s is -recurrent nd y definition of -recurrence, s is -ccessile from t. Since t S n nd S n is -stle, s S n nd since s is -recurrent s (S n ) = S 1. The proof tht S S 1 is similr. If S 1 = Q the proof is complete, ecuse (i) holds. If S 1 Q, then (iii) holds ecuse S S 1 thus Q is limit-rechle not only from S ut from S 1 s well, using the sme limit-pth. As long s S n Q, we use (iii) to uild inductively n incresing sequence S S 1 S 2... S n = Q such tht for every 1 k < n, S k+1 is -rechle from S k. Since -rechility is trnsitive this completes the proof of the lowing lemm. Next lemm shows tht in -stle nd -cyclic utomt, once computtion hs flooded the whole stte spce, it cnnot shrink ck. Lemm 2 (Flooding lemm). Let A e proilistic utomton with sttes Q. Suppose tht A is -cyclic nd -stle. Then Q is the only set of sttes limit-rechle from Q in A. Now, we turn our ttention to leves of the cyclic support grph. Definition 8. Let A e proilistic utomton with sttes Q. A non-empty suset R Q is clled lef if for every letter A, R = R nd R = R. In stle -cyclic utomton, there is unique lef: Lemm 3 (Lef lemm). Let A e proilistic utomton with sttes Q. Suppose tht A is -cyclic.then there exists unique lef -ccessile from Q. Every set limit-rechle from Q contins this lef. To prove tht limit-rechility implies -rechility, we proceed y induction on the depth in the support grph. The inductive step is: Lemm 4 (Inductive step). Let A e proilistic utomton with sttes Q nd S 0,T Q. Suppose tht A is -cyclic nd T is limit-rechle from S 0. Then either S 0 = T or there exists S 1 S 0 such tht S 1 is -rechle from S 0 in A nd T is limit-rechle from S 1 in A. Repeted use of Lemm 4 gives: Proposition 7. Let A e proilistic utomton with sttes Q nd S 0,T Q. Suppose tht A is -cyclic. If T is limit-rechle from S 0 in A, then T is -rechle from S 0 s well.

13 Thus, limit-rechility nd -rechility coincide in -cyclic utomt nd Theorem 5 holds. Is the mximl distnce etween two -rechle sets in the support grph ounded y polynomil function of A nd Q? The nswer to this question could led to simpler proof nd/or lgorithm. Conclusion Whether the emptiness prolem is decidle for proilistic utomt with unique proilistic trnsition is n open question. The clss of -cyclic utomt cn e proly extended to lrger clss of utomt for which the vlue 1 prolem is still decidle. Acknowledgments We thnk the nonymous referees s well s A. Muscholl nd I. Wlukiewicz for finding flw in the previous version of this mnuscript. We thnk Benedikt Bollig for his vlule comments during the finl preprtion of this mnuscript. References 1. R. Alur, C. Courcouetis, nd M. Ynnkkis. Distinguishing tests for nondeterministic nd proilistic mchines. In STOC, pges ACM, Christel Bier, Nthlie Bertrnd, nd Mrcus Größer. On decision prolems for proilistic üchi utomt. In FoSSCS, pges , A. Bertoni, G. Muri, nd M. Torelli. Some recursive unsolvle prolems relting to isolted cutpoints in proilistic utomt. In Proceedings of ICALP, pges 87 94, London, UK, Springer-Verlg. 4. Alerto Bertoni. The solution of prolems reltive to proilistic utomt in the frme of the forml lnguges theory. In Proc. of the 4th GI Jhrestgung, volume 26 of LNCS, pges Springer, Vincent Blondel nd Vincent Cnterini. Undecidle prolems for proilistic utomt of fixed dimension. Theory of Computing Systems, 36: , R. G. Bukhrev. Proilistic utomt. Journl of Mthemticl Sciences, 13(3): , R. Chdh, A. Prsd Sistl, nd M. Viswnthn. Power of rndomiztion in utomt on infinite strings. In CONCUR, volume 5710 of Lecture Notes in Computer Science, pges Springer, Mik Hirvenslo. Improved undecidility results on the emptiness prolem of proilistic nd quntum cut-point lnguges. In SOFSEM 07, pges , Berlin, Heidelerg, Springer-Verlg. 9. Dexter Kozen. Lower ounds for nturl proofs systems. In Proc. of 18th Symp. Foundtions of Comp Sci., pges , O. Mdni, S. Hnks, nd A. Condon. On the undecidility of proilistic plnning nd relted stochstic optimiztion prolems. Artificil Intelligence, 147:5 34, Azri Pz. Introduction to proilistic utomt (Computer science nd pplied mthemtics). Acdemic Press, Inc., Orlndo, FL, USA, Michel O. Rin. Proilistic utomt. Informtion nd Control, 6(3): , Wen-Guey Tzeng. A polynomil-time lgorithm for the equivlence of proilistic utomt. SIAM J. Comput., 21(2): , 1992.

14 Appendix Proof (of Proposition 1). Given ny instnce ϕ 1,ϕ 2 : A {0,1} of the PCP prolem, we uild n utomton A which ccepts some word with proility 1 2 if nd only if PCP hs solution. Let ψ : {0,1} [0,1] the mpping defined y: ψ( 0... n ) = n n, nd let θ 1 = ψ ϕ 1 nd θ 2 = ψ ϕ 2. Without loss of generlity, y insertion of 1 s in words of the PCP instnce, we cn suppose tht ϕ 1,ϕ 2 : A {10,11}. Tht wy, since Φ is injective on 1{0,1} : w A,(θ 1 (w) = θ 2 (w)) (ϕ 1 (w) = ϕ 2 (w)). (3) Let A 1 = (Q,A,M,q0 1,q1 F ) the proilistic utomton with two sttes Q = {q0 1,q1 F } nd trnsitions: [ ] 1 θ A,M() = 1 () θ 1 () 1 θ 1 () 2 ϕ1() θ()+2 ϕ1(). A simple computtion shows tht: w A, P A1 (w) = θ 1 (w). (4) A very similr construction produces two-sttes utomton A 2 such tht: w A, P A2 (w) = 1 θ 2 (w). (5) Let A e the disjoint union of these two utomt A 1 nd A 2 plus new initil stte tht leds with equl proility 1 2 to one of the initil sttes q1 0 nd q2 0 of A 1 nd A 2. Then for every word w A nd every letter A, ( w A, P A (w) = 1 ) 2 ( w A, 1 2 P A 1 (w)+ 1 2 P A 2 (w) = 1 ) 2 ( w A, θ 1 (w) = θ 2 (w)) ( w A, ϕ 1 (w) = ϕ 2 (w)) PCP hs solution, where the first equivlence is y definition of A, the second is y (4) nd (5), the third holds y (3) nd the fourth is y definition of PCP. This completes the proof of Proposition 1. Proof (of Theorem 1). According to Proposition 1 nd Proposition 2 the emptiness nd the strict emptiness prolems re undecidle for cut-point 1 2 nd utomt whose trnsition proilities re multiples of 1 8. The trnsformtion of such utomt into simple utomt is esy.

15 5 Proof of Proposition 4 Proof. To prove Proposition 4, we provide n lgorithm which tkes s input simple proilistic utomton A on n lphet A (with n ritrry numer of proilistic trnsitions) nd outputs two ojects: simple utomton A over n lphet A with only one proilistic trnsition nd A words {u, A} on the lphet A, ech word u of length 3 Q, such tht: ( w A,P A (w) 1 ) 2 ( w {u, A},P A (w ) 1 ) 2. (6) According to Theorem 1, the Emptiness Prolem for simple proilistic utomt nd cut-point 1 2 is undecidle, hence (6) proves Proposition 4. The lphet A, the utomton A nd the words (u ) A re defined s follows. If A = (Q,A,M,q 0,F) then A = (Q,A,M,q 0,F) is defined y: 1. The lphet A is mde of new letter plus, for ech letter A nd stte s Q, two new letters α(,s) nd β(,s) so tht: A = { } {α(,s),β(,s)}. A,s Q 2. The set of sttes of A is otined from Q y ddition of three new sttes s,s 0,s 1, so tht: Q = Q {s,s 0,s 1 }. 3. The initil stte q 0 nd the set of finl sttes F re left unchnged. 4. The trnsitions of A re s follows. For every letter A nd stte s Q, thenewletterα(,s) hsnoeffectonsttesu s(i.e.m α(,s) (u) = u),while from stte s the trnsition is deterministic to stte s i.e. M α(,s) (s) = s. 5. The new letter hs no effect on sttes u s, while from stte s this is the only proilistic trnsition of A, defined y M (s ) = 1 2 s s For every letter A nd stte s Q, the new letter β(,s) hs no effect on sttes u {s 0,s 1 }. Trnsitions on letter β(,s) from sttes s 0 nd s 1 re deterministic nd depend on M (s). If the trnsition M (s) is deterministic, i.e. ifm (s,r) = 1for somestter then M β(,s) (s 0 ) = r ndm β(,s) (s 1 ) = r. Ifthe trnsitionm (s) is proilistici.e. ifm (s) = 1 2 r+ 1 2 r forsomesttes r,r then M β(,s) (s 0 ) = r nd M β(,s) (s 1 ) = r. Nowwedefinethewordsu, A.Choosesomeenumertion{s 0,s 1,...,s n } = Q of sttes of A nd for ech letter, define the word u = α(,s 0 ) β(,s 0 )α(,s 1 ) β(,s 1 ) α(,s n ) β(,s n ) A. Then (6) holds ecuse for every word A, P A ( ) = P A (u 0 u 1 u 2...). Since utomton A hs only one proilistic trnsition, this completes the proof of Proposition 4.

16 6 Proof of Proposition 3 Proof. The undecidle prolem descried in Proposition 4 reduces to the emptiness prolem for simple proilistic utomt with two proilistic trnsitions: given A nd L, dd new initil stte to A nd from this new initil stte, proceed with proility 1 2 either to the originl initil stte of A to the initil stte of deterministic utomton tht checks whether the input word is in L. This new utomton ccepts word with proility more thn 3 4 if nd only if the originl utomton ccepts word with proility more thn Proof of Proposition 5 Proof (of Proposition 5). We shll prove: ( x > 1 ) ( ε > 0, w A,P Ax (w) 1 ε). (7) 2 In order to prove this equivlence we notice tht: P Ax (1 n 3) = x n nd P Ax (4 n 6) = (1 x) n. Let (n k ) k N n incresing sequence of integers. By reding the word w = n0 n1... ni, we get: ( ) 1 x n k P Ax (1 w 3) = 1 k 0 P Ax (4 w 6) = (1 x) n1 +(1 (1 x) n1 )(1 x) n (8) = 1 (1 (1 x) n k ) k 0 k 0(1 x) n k If x 1 2 then P A x (1 w 3) P Ax (4 w 6) therefore if x 1 2 no word w cn e ccepted with proility greter thn 1 2, in prticulr the utomton does not hve vlue 1. This proves the converse impliction in (7). Assume tht x > 1 2 nd let ε > 0, we exhiit n incresing sequence of integers (n k ) k N such tht we hve: x n k = k 0 (9) (1 x) n k ε k 0 Let C R nd n k = ln x ( 1 k )+C, notice tht k 0 (x)n k = x C. k 0 1 k =. In the other hnd we hve: 1 x = x lnx(1 x) = x ln(1 x) lnx Thereexistsβ > 1suchtht:1 x = x β,hence k 0 (1 x)n k = k 0 xβn k.so: k 0 xβn k = x βc k 0 xβlnx( 1 k ) = x βc k 0 1 k β.sincethisseriesconverges,we stisfy (7) y choosing suitle constnt c.

17 Suppose now tht (7) is stisfied for some sequence n 0,n 1,n 2... Then ccording to (8), for every i N the word n0 n1... ni is ccepted y A x with proility 1 1 (1 x n k ) + 1 (1 (1 x) n k ) k i 0 k i According to, when i goes to, the left opernd converges to 1 2 nd the right operndissymptoticllylrgerthn 1 2 (1 ε).thisprovesthe implictionin (7). 8 Proof of Theorem 4 Proof (of Theorem 4). Given proilistic utomton B with lphet A such tht, B, we comine B nd the utomton A x on Fig.1 to otin n utomton C which hs vlue 1 if nd only if there exists word w such tht P A (w) > 1 2. The input lphet of C is A {} plus new letter. C is computed s follows. First, the trnsitions in A x on letter re deleted. Second, we mke two copies A 4 nd A 1 of the utomton B, such tht the initil stte of A 4 is 4 nd the initil stte of A 1 is 1. From sttes of A 4 nd A 1 other thn the initil sttes, reding letter leds to the sink stte 6. Third, from stte s of A 4 the trnsition on the new letter is deterministic nd leds to 5 if s is finl stte nd to 4 if s is not finl stte. Fourth, from stte s of A 1 the trnsition on the new letter is deterministic nd leds to 1 if s is finl stte nd to 2 if s is not finl stte. Fifth, the finl sttes of C re 5 nd 3. Sixth, sttes 0,3,6,5 nd 2 re soring for letters in A. Then suppose there exists w such tht P A (w) > 1 2 nd let us show tht C hs vlue 1. Let ǫ > 0 nd let u ǫ = i0 i1 i2 i k e word ccepted y B with proility 1 ǫ. Then y construction of C, P C ((w ) i0 (w ) i1 (w ) i2 (w ) i k ) P A (u ǫ ) 1 ǫ, thus C hs vlue 1. Now suppose tht for every w A,P A w 1 2 nd let us show tht C hs not vlue 1. Let w (A {, }). Fctorize w in w = u 0 v 0 u 1 v 1 u k v k such tht u i nd v i A. Then y construction of C nd y hypothesis, P C (w ) P A12 (u 0 u 1 u 2 u k ) vl(a1 ). Thus vl(c) ) nd ccording to 2 vl(a1 2 Proposition 5, vl(c) < 1. 9 Proof of Theorem 5 Proof (of Proposition 6). Proposition 6 is consequence of the two following fcts. First, if there is n edge from S to T in the support grph of A, then T is limit rechle from S: let S,T Q nd A. If S = T, then the sequence

18 constnt equl to is limit pth from S to T. If S = S nd S = T then y definition of S, ( n ) n N is limit pth from S to T. Second, limit-rechility is trnsitive reltion: let S 0,S 1,S 2 Q such tht S 1 is limit-rechle from S 0 nd S 2 is limit-rechle from S 1. Let (u n ) n N limit-pthfroms 0 tos 1 nd(v n ) n N limit-pthfroms 1 tos 2.Then(u n v n ) n N is limit-pth from S 0 to S 2. Proof (of the lef lemm). Since A is -cyclic, there exists t lest one lef S which is -rechle from Q. We prove tht every set limit-rechle from Q contins the lef S. Let R limit-rechle from Q nd (u n ) n N limit-pth from Q to R. Since S is lef then for every A, S is -stle, hence (u n ) n N is fortiori limit-pth from S to R S. Since S is lef, the restriction A[S] of the utomton A to S is -stle. According to the flooding lemm pplied to A[S], S = R S, thus S R. Since -rechility imply limit-rechility, this implies unicity of the lef rechle from Q. Proof (of the Flooding lemm). Let (u n ) n N e limit pth from Q to some set of sttes T Q. We shll prove tht T = Q. Let A T = { A T = T}. First, we provetht for everyletter A T, Q\T is -stle. Otherwisethere would e s Q\T, A T nd t T such tht t is -rechle from s. Since A is -stle, s nd t re oth -recurrent, nd y definition of -recurrence, since t is -rechle from s, s would e -rechle from t s well. But s Q\T nd t T, which contrdicts the -stility of T. Second, we prove tht u n A T for only finitely mny n N. Since for every A T, Q\T is -stle, then during the computtion δ Q = δ 0,δ 1,...,δ un on the word u n, s Q\T δ k(s) is constnt. Thus, for every n N, P A (s un T) = s Q\T (δ Q u n )(s) = s Q\T δ Q (s) = Q T Q > 0. Since (u n ) n N is limit-pth from Q to T, P A (s un T) converges to 0 hence the inequlity cn hold only for finitely mny n N. Now we show tht there exists T 1 Q such tht: (i) T 1 T, (ii) T is -rechle from T 1 in A, (iii) nd T 1 is limit-rechle from Q in A. Since ny infinite susequence of limit-pth is limit-pth, nd since we proved tht u n A T for only finitely mny n N, we cn ssume w.l.o.g. tht for every n N, u n A T. Thus for every n N, there exists v n A, n A\A T nd w n A T such tht u n = v n n w n. W.l.o.g. gin, since A is finite nd D(Q) is compct, we cn ssume tht ( n ) n N is constnt equl to letter A\A T nd tht (δ Q v n ) n N converges to proility distriution δ D(Q). The choice of T 1 such tht (i), (ii) nd (iii) hold depends on Supp(δ).

19 If Supp(δ) = T then we choose T 1 = Supp(δ). Then (i) holds ecuse A T, (ii) holds ecuse T = T 1 nd (iii) holds ecuse (v n ) n N is limit-pth from Q to T 1. If Supp(δ) T then we choose T 1 = Supp(δ). Then (i) clerly holds nd (iii) holds ecuse (v n ) n N is limit pth from Q to T 1 in A. To prove tht (ii) holds, considerthe restrictiona[t,a T ] ofutomtonato sttest nd lphet A T. Then (w n ) n N is limit-pth from T 1 to T in A[T,A T ]. Moreover, since A is -cyclic nd -stle, A[T,A T ] lso is. Thus, we cn pply the lowing lemm to A[T,A T ] nd T 1, which provestht T is -rechle from T 1 in A[T,A T ], thus in A s well. If T 1 = Q, the proof is complete. Otherwise, s long s T n Q, we use condition (iii) to uild inductively sequence T = T 0,T 1,T 2, T n such tht for every 0 k < n, T k T k+1 (condition (i))nd T k is -rechle from T k+1 in A (condition (ii)). Since A is -cyclic, T n = Q fter t most 2 Q inductive steps. Since -rechility is trnsitive, this proves tht T is -rechle from Q. Since A is -stle, the only set -rechle from Q is Q thus T = Q, which completes the proof of the Flooding lemm. Proof (of Lemm 4). Let (u n ) n N e limit-pth from S 0 to T. Let A 0 = { A S 0 = S 0 }. For every n N, let v n e the longuest prefix of u n in A 0. Since every infinite susequence of limit-pth is limit-pth, nd since D(Q) is compct, we cn suppose without loss of generlity tht (δ S0 v n ) n N converges to some distriution δ D(Q). Suppose first tht Supp(δ) = S 0. If u n A 0 for infinitely mny n N then T = S 0. Otherwise, since A is finite we cn suppose w.l.o.g. tht there exists letter A\A 0 such tht for every n N, v n is prefix of u n. Let lso w n such tht u n = v n w n. Let S 1 = S 0. Then S 1 S 0 ecuse A 0 nd S 1 is clerly -rechle from S 0. Moreover (w n ) n N is limit-pth from S 1 to T, this completes the proof. Suppose now tht Supp(δ) S 0. Let A[S 0,A 0 ] the proilistic utomton otined from A y restriction to the lphet A 0 nd to the stte spce S 0. By definition of A 0, A[S 0,A 0 ] is stle nd it is -cyclic ecuse A is. According to the lef lemm, A[S 0,A 0 ] hs unique lef. Let S 1 e this unique lef. Since Supp(δ) is limit-rechle from S 0 in A[S 0,A 0 ], ccording to the lef lemm gin, S 1 Supp(δ) hence S 1 S 0. Moreover, since it is the unique lef, S 1 is -rechle from S 0 in A[S 0,A 0 ] hence in A s well. For every n N, let w n such tht u n = v n w n. Then (w n ) n N is limit-pth from S 1 to T. This completes the proof. Proof (of Proposition 7). Apply gin nd gin Lemm 4 to uild sequence S 0,S 1,S 2,... such tht for every k, S k S k+1, S k+1 is -rechle from S k nd T is limit-rechle from S k+1. As long s S k T, Lemm 4 is used to uild S k+1. Since A is suset-cyclic, the sequence hs length t most 2 Q thus for some k,s k = T. Since -rechility is trnsitive, this proves tht T is - rechle from S 0.