Probabilistic Aspects of Computer Science: Probabilistic Automata

Transcription

1 Probabilistic Aspects of Computer Science: Probabilistic Automata Serge Haddad LSV, ENS Paris-Saclay & CNRS & Inria M Jacques Herbrand Presentation 2 Properties of Stochastic Languages 3 Decidability Results /42

2 Plan Presentation Properties of Stochastic Languages Decidability Results 2/42

3 An introductive example Planning holidays in a foreign country Choosing which plane company to use lowcost or highcost; 2 Renting a room in an hotel by internet or phone; 3 Buying tickets for some exhibitions with agency seeall or dontmiss. Usually these actions must be planned before the holidays. Thus one looks for an a priori optimal policy that maximizes the probability to reach a goal. 3/42

4 Formalisation start 3 4low + high airport 4 low 3 4 internet + 2 phone fail 4 internet + 2 phone 4 seeall + 8 dontmiss hotel 3 4 seeall dontmiss success The probability of success of lowcost internet seeall is /42

5 Probabilistic automata Probabilistic Automata (PA) are a variation of MDP where: The set of possible actions is the same in every state. There are no rewards. There is a subset of final states. More formally, a PA A = (Q, A, {P a } a A, π 0, F ) is defined by: Q, the finite set of states; A, the finite alphabet; For all a A, P a, a probability transition matrix over S; π 0, the initial distribution over states and F Q the final states. 5/42

6 Illustration 2 b a + q 0 q 2 b 2 a + b A = {a, b}; Q = {q 0, q }, F = {q }; π 0 [q 0 ] =. 2 a An edge from a state to another one is labelled by a vector of transition probabilities indexed by A. The vector is denoted by a formal sum. For instance, the transition from q 0 to itself is labelled by a + 0.5b means that: when a is chosen in state q 0, the probability that the next state is q 0, P a [q 0, q 0 ], is equal to. when b is chosen in state q 0, the probability that the next state is q 0, P b [q 0, q 0 ], is equal to /42

7 Policies in PA Words are policies. When some finite word w def = a... a n is selected, we are interested in the probability to be in a final state using w as a policy. Given A a PA and w def = a... a n A a word, the acceptance probability of w by A is defined by: Pr A (w) def = π 0 [q] q Q q F ( n ) P ai [q, q ] i= Notation. Given a word w def = a... a n, the probability matrix P w is defined by P w def = n i= P a i. In particular P ε = Id. With these notations: Pr A (w) = π 0 P w T F where F is the indicator vector of subset F. 7/42

8 Illustration 2 b a + q 0 q 2 b 2 a + b Computation of Pr A (abba) by induction w.r.t. the prefixes. First Pr A (ε) = 0. Pr A (a) = 2 Pr A(ε) = 0 2 a Pr A (ab) = Pr A (a) + 2 ( Pr A(a)) = 2 Pr A (abb) = Pr A (ab) + 2 ( Pr A(ab)) = 3 4 Pr A (abba) = 2 Pr A(abb) = 3 8 More generally, the following recursive equations hold: Pr A (wa) = 2 Pr A(w) and Pr A (wb) = 2 ( + Pr A(w)) from which one can derive an explicit expression of the acceptance probability: n Pr A (a... a n ) = 2 i n ai=b i= Which word maximizes the acceptance probability? 8/42

9 Stochastic languages We are interested in useful policies. This directly leads to the introduction of stochastic languages. Let: A be a probabilistic automaton; θ [0, ] be a threshold also called a cut point; {<,, >,, =, } be a comparison operator. Then L θ (A) is defined by: L θ (A) = {w A Pr A (w) θ} For expressiveness and decidability issues, one also needs the following definitions. A rational PA is a PA with probability distributions over Q Q. A rational stochastic language is a stochastic language specified by a rational PA and a rational threshold. 9/42

10 Counting with PA q 0 a q 2 a 2 a 2 a q 3 q 4 a 2 b b q 2 (a succinct representation with an omitted absorbing rejecting state) Any word z different from a m b n with m > 0, n > 0 cannot be accepted. 2 a Let w def = a m b n with m > 0, n > 0. w can be accepted by: a path q 0, q m, q2 n with probability 2 ; n or by a family of paths q 0, q3, r q4, s q5 n with 0 < r, s and r + s = m with cumulated probability 2 2. m Summing, one obtains: n 2. m Thus: L =0.5 (A) = {a n b n n > 0} b q 5 b 0/42

11 Plan Presentation 2 Properties of Stochastic Languages Decidability Results /42

12 Expressiveness problems Provide a minimal set of comparison operators and thresholds. Position the stochastic languages w.r.t. the Chomsky hierarchy. Study the closure properties of the stochastic languages. 2/42

13 A single threshold is enough A απ 0 [q] α q q 0 A The value α depends on θ 2 in the following way: If θ > 2 then q 0 / F and α def = 2θ so that for all w A, Pr A (w) = 2θ Pr A(w) Thus w L 2 (A ) iff w L θ (A). If θ < 2 then q 0 F and α def = 2( θ) so that for all w A, Pr A (w) = 2θ+Pr A(w) 2( θ) Thus w L (A ) iff w L 2 θ (A). 3/42

14 Getting rid of (dis)equality Given a PA A, we build A as follows. The set of states Q def = Q Q; P a[(q, q 2 ), (q, q 2)] def = P a [q, q ]P a [q 2, q 2]; π 0[q, q 2 ] def = π 0 [q ]π 0 [q 2 ] and F def = F (Q \ F ). Once a word w is selected, the two components of the DES behave independently and so: Pr A (w) = Pr A (w)( Pr A (w)) Consequently Pr A (w) 4 with equality iff Pr A(w) = 2. Thus: L 4 (A ) = L = 2 (A) 4/42

15 Getting rid of lower (or equal) than Given a PA A, we build A by complementing the final states. Then: And so: Pr A (w) = Pr A (w) L θ (A ) = L <θ (A) L >θ (A ) = L θ (A) 5/42

16 The Chomsky hierarchy recursively enumerable context sensitive algebraic regular Class Grammar Device Regular language L ar a ε Finite automaton with L, R, a Σ Algebraic language L R... R n with Stack automaton L and R i Σ Context-sensitive L... L m R... R n Non determ. Turing language m n, (S ε) with L i, R j Σ machine performing in linear space Recursively enumerable language L... L m R... R n avec L i, R j Σ Turing machine 6/42

17 Revisiting the Chomsky hierarchy recursively enumerable stochastic context sensitive algebraic regular rational stochastic 7/42

18 Non recursively enumerable languages 2 b a + q 0 q 2 b 2 a + b Define v a def = 0 and v b def =. The acceptance probability of w... w n is the binary number 0.v wn... v w. So L >θ (A) is the set of representations of numbers (with finite binary development) greater than θ. Thus given 0 θ < θ, 2 a L >θ (A) L >θ (A) So there is an uncountable number of stochastic languages implying that most of them are non recursively enumerable. This result does not hold for rational stochastic languages. 8/42

19 Regular versus stochastic languages A deterministic automaton is a stochastic automaton with probabilities in {0, }. Thus regular languages are stochastic languages. The language {a n b n n > 0} is a rational stochastic non regular language. 9/42

20 Non stochastic context-free languages () L def = {a n ba n2 b... a n k ba i > n i = n } is a non stochastic context-free language. Proof. L is context-free. Use a non deterministic automaton with a counter. With a counter one counts n the number of a s until the first occurrence of b. Then one guesses an occurrence of b and decrements the counter by the occurrences of a until the next occurrence of b. If the counter is zero the word is accepted. Assume that () L = L >θ (A) or (2) L = L θ (A). Let n i=0 c ix i be the minimal polynomial of P a. Since is an eigenvalue of P a, one gets n i=0 c i = 0 and there are positive and negative coefficients. By definition, n i=0 c ip a i = 0 and so for any word w, n i=0 c ip a i w = 0. 20/42

21 Non stochastic context-free languages (2) Proof (continued). Let P os = {i 0 i n c i > 0} and NonP os = {i 0 i n c i 0}. Write P os as {i,..., i k }. Choose w def = ba i b... ba i k b. Case L = L >θ (A). Let 0 i n, by definition of L, π 0 P ai w T F > θ iff i {i,..., i k } So: 0 = n i=0 c iπ 0 P a i w T F = i P os c iπ 0 P a i w T F + i NonP os c iπ 0 P a i w T F > ( i P os c i)θ + ( i NonP os c i)θ = ( n i=0 c i)θ = 0 leading to a contradiction. Case L = L θ (A). Let 0 i n, by definition of L, π 0 P a i w T F θ iff i {i,..., i k } So: 0 = n i=0 c iπ 0 P a i w T F = i P os c iπ 0 P a i w T F + i NonP os c iπ 0 P a i w T F >( i P os c i)θ + ( i NonP os c i)θ = ( n i=0 c i)θ = 0 leading to a contradiction. 2/42

22 Non context-free stochastic languages () L def = {a n b n c n n > 0} is a non context-free rational stochastic language. Proof. Using Ogden s lemma it can be easily proved that L is not context-free. def Observe that L = L L 2 with L = {a n b n c + n > 0} and def L 2 = {a + b n c n n > 0}. So we prove that: for i {, 2}, L i = L = 2 (A i) for some A i the family of languages {L = L = 2 (A)} A is closed under intersection. 22/42

23 Non context-free stochastic languages (2) Proof (continued). L = 2 (A) = {an b n c + n > 0} q 0 2 a 2 a a q 2 a q 3 q 4 a b 2 a b 2 b q 2 c q 6 c c q 7 c q 5 b 23/42

24 Non context-free stochastic languages (3) Proof (ended). Let L = (A ) and L 2 = (A 2) be two arbitrary languages. 2 Using the previous construction, let A and A 2 be automata such that: For any word w, Pr A i (w) 4 ; L = 2 (A i) = L = 4 (A i ). One builds A as follows: The set of states Q def = Q Q 2; P a [(q, q 2 ), (q, q 2)] def = (P ) a [q, q ](P 2) a [q 2, q 2]; π 0[q, q 2 ] def = π,0 [q ]π 2,0 [q 2 ] and F def = F F 2. By construction, Pr A (w) = Pr A (w)pr A 2 (w). So for all word w, Pr A (w) 6 and Pr A(w) = 6 iff Pr A (w) = Pr A 2 (w) = 4. Consequently, L = 6 (A) = L = 2 (A ) L = 2 (A 2) 24/42

25 Inclusion in context-sensitive languages The class of rational stochastic languages is strictly included in the class of context-sensitive languages. Proof. Context-sensitive languages are the languages for which membership checking can be performed by a non deterministic procedure in linear space. A deterministic procedure in linear space (far from being optimal) Pre-computation in constant space. Compute the l.c.m., say b, of denominators of θ, items of matrices {P a } a A and, items of vector π 0. Build the integer matrices P a def = bp a and vector π 0 def = bπ 0. For word w def = a... a n, the problem becomes π 0( n i= P a i ) T F θbn+? Compute θb n+ in space O(n). Compute v def = π 0( n i= P a i ) by initializing v to π 0 and then iteratively multiply it by P a i. The vectors are bounded by b n+. So this is performed in space O(n). The sum and comparison are also done in space O(n). 25/42

26 Operations with regular languages The family of (rational) stochastic languages is closed under intersection and union with regular languages. Proof. Let L θ (A ) be a (rational) stochastic language (with {>, }) and L = (A 2 ) be a regular language. A 2 π 0[q] 2 A A 2 q q 0 L θ 2 (A) = L θ (A ) L = (A 2 ) and L +θ (A) = L θ (A ) L = (A 2 ) 2 26/42

27 A stochastic language a q b 2 b q 2 a 2 a 2 2 a q 3 q 4 a b b a b q 2 2 b 2 b q 5 q 6 b a L = (A) = 2 {am b... ba m k b < k m = m k } ( ( since Pr A (a m b... ba m k b) = ) k+mk ( ) ) k+m /42

28 Concatenation The family of (rational) stochastic languages is not closed under concatenation with a regular language. Proof. Let L def = {a m b... ba m k b < k m = m k } be the previous stochastic language. Then LA = {a m ba m2 b... a m k ba i > m i = m } which is not a stochastic language. 28/42

29 Iteration The family of (rational) stochastic languages is not closed under Kleene star. Proof. Let L def = {a m b... ba m k b < k m = m k } be the previous stochastic language. Assume that L = L θ (A) with {>, }. Let n i=0 c ix i be the minimal polynomial of P a. Since is an eigenvalue of P a, one gets n i=0 c i = 0 and there are positive and negative coefficients. By definition, n i=0 c ip a i = 0 and so for any word w, n i=0 c ip a i w = 0. Let c i,..., c ik be the positive coefficients of this polynomial. Let w def = ba i b(a i2 b) 2... (a i k b) 2. a i w L iff i {i,..., i k }. Case L = L >θ (A). Let 0 i n, π 0 P ai w T F > θ iff i {i,..., i k }. So: 0 = n i=0 c iπ 0 P a i w T F > ( n i=0 c i)θ = 0 leading to a contradiction. Case L = L θ (A). Let 0 i n, π 0 P a i w T F θ iff i {i,..., i k }. So: 0 = n i=0 c iπ 0 P a i w T F > ( n i=0 c i)θ = 0 leading to a contradiction. 29/42

30 A stochastic language a q b 2 a 2 2 a q 3 q 4 a 2 b b a b 2 a q 2 b 2 b q 5 q 6 b b a c q 2 c q 7 A L = 2 (A) = {am b... ba m k bca < k m = m k } 30/42

31 Homomorphism The family of (rational) stochastic languages is not closed under homomorphism. Proof. Let L def = {a m b... ba m k bca < k m = m k } be the previous stochastic language. Define the homomorphism h from A to A def = {a, b} by: h(a) def = a h(b) def = b h(c) def = ε Then h(l) = {a m ba m2 b... a m k ba i > m i = m } which is not a stochastic language. 3/42

32 Plan Presentation Properties of Stochastic Languages 3 Decidability Results 32/42

33 Two decision problems Let A and A be probabilistic automata. First problem Are A and A equivalent? w A Pr A (w) = Pr A (w) Second problem Is L θ (A) equal to L θ (A )? For deterministic automata this is the same problem. It can be solved in polynomial time by a product construction which provides a witness of non equivalence of size less than Q Q. 33/42

34 Linear algebra recalls Let v 0 R n and v,..., v k be linearly independent vectors of R n. How to check whether v 0 is a linear combination of v,..., v k? Solve in O(k 3 + n 2 ) v []... v k [] v [n]... v k [n] x. x k = When v,..., v k are orthogonal def (i.e. for all a b, v a v b = n i= v a[i]v b [i] = 0) Compute in O(kn) the orthogonal projection w 0 = k i= v 0 v i v i v i v i v 0 []. v 0 [n] Check in O(n) whether v 0 = w 0. 34/42

35 Principles of equivalence checking Enumeration of words Looking for a counter-example whose length is increasing starting with word ε. A stack Managing a stack of words w in order to find counter-examples aw for all a A. For efficiency purposes, the stack contains tuples (P w F, P w F, w). An orthogonal family for restricting the enumeration Gen is a set of independent orthogonal vectors of R Q Q. If w is not a counter-example, check if v def = (P w F, P w F ) is generated by Gen. producing v the orthogonal projection of v on subspace spanned by Gen; comparing v to v. If v v then: w is added to the stack; v v is added to Gen. 35/42

36 The algorithm If π 0 F π 0 F then return(false, ε) Gen {( F, F )}; Push(Stack, ( F, F, ε)) Repeat (v, v, w) Pop(Stack) For a A do z P a v; z P av If π 0 z π 0 z then return(false, aw) y 0; y 0 For (x, x ) Gen do (y, y ) (y, y ) + z x+z x x x+x x (x, x ) If (z, z ) (y, y ) then Push(Stack, (z, z, aw)) Gen Gen {(z y, z y )} Until IsEmpty(Stack) return(true) 36/42

37 Complexity Time complexity An item is pushed on the stack iff an item is added to Gen. There can be no more than Q + Q items in Gen. So there are at most Q + Q iterations of the external loop. The index of the first inner loop ranges over A while the index of the most inner loop ranges over Gen. The operations inside the most inner loop are done in O( Q + Q ). This leads to an overall time complexity of O(( Q + Q ) 3 A ). Length of witnesses In addition, the length of the witness is at most Q + Q. (also valid for deterministic automata) 37/42

38 Correctness Assume that the automata are not equivalent and that the algorithm returns true. Let u be a non examined word such that Pr A (u) Pr A (u). Let u def = w w with w( u) the greatest suffix examined by the algorithm. Among such words u, pick one word such that w is minimal. Claim. There exists w that has been inserted in the stack before w such that Pr A (w w ) Pr A (w w ). Let Gen = {w,..., w k } when examining w, there exist λ,..., λ k such that: So: P w F = k i= λ ip wi F and P w F = k i= λ ip w i F Pr A (w w) def = π 0 P w P w F = k i= λ iπ 0 P w P wi F = k i= λ ipr A (w w i ) Similarly: Pr A (w w) = k i= λ ipr A (w w i ) So there exists i, with Pr A (w w i ) Pr A (w w i ). Let w def = w a. aw i is examined by the algorithm. So the word u def = w w i has a decomposition u def = z z where z the greatest suffix examined by the algorithm has for suffix aw i. So z < w : a contradiction. 38/42

39 Undecidability of the equality problem Given A a rational stochastic automaton, the question L = (A) = {ε}? is undecidable. 2 Proof. By reduction of the undecidable Post correspondence problem (PCP): Given an alphabet A and two morphisms ϕ, ϕ 2 from A to {0, } +, does there exist a word w A + such that ϕ (w) = ϕ 2 (w)? Already undecidable for a restriction where the images of letters lie in (0 + ) +. Inserting a before each letter of images reduces the former problem to the latter. A word w def = a... a n (0 + ) + defines a value val(w) [0, ] by: val(w) def = n i= a i 2 n+ i Since every word starts with a, val(w) = val(w ) implies w = w. 39/42

40 Reduction of PCP For w A + and i {, 2}, define val i (w) def = val(ϕ i (w)). a ( val (a))a 2 a val (a)a q 0 q a (val (a) + 2 ϕ(a) )a a ( val 2(a))a a ( val (a) 2 ϕ(a) )a a val 2(a)a 2 q 20 q 2 a (val 2(a) + 2 ϕ2(a) )a a ( val 2(a) 2 ϕ2(a) )a 40/42

41 Illustration of the reduction A a b c ϕ ()0() ()0()0 () ϕ 2 ()0 ()0 ()()() A a b c 3 val 6 val a b c 3 q 0 q 6 a b + 4 c 7 8 a + 2 b + c 8 a + 2 b 4 a + 4 b c 3 q 20 q 4 a b + 64 c 2 2 a + 2 b + c 2 a + 2 b 4/42

42 Correctness of the reduction The recurrence equation: qi0 P wa T q i = qi0 P w T q i (val i (a) + 2 ϕi(a) ) + ( qi0 P w T q i )val i (a) = val i (a) + 2 ϕi(a) qi0 P w T q i By induction we obtain that for all w def = a... a n : qi0 P w T q i = n val i (a j )2 j<k n ϕi(ak) = val i (w) j= So for w A + : Pr A (w) = 2 (val (w) + val 2 (w)). Thus w L = 2 (A) iff val(ϕ (w)) = val(ϕ 2 (w)) implying that ϕ (w) = ϕ 2 (w). 42/42