The complexity of automatic complexity

Transcription

1 The complexity of automatic complexity Bjørn Kjos-Hanssen Department of Mathematics University of Hawai i at Mānoa bjoernkh@hawaii.edu August 0, 205 Abstract We define the automatic complexity A(E) of an equivalence relation E on a finite set S of strings. The minimum complexity is the number of equivalence classes E. We prove that the problem to determine whether A(E) = E is NP-complete, and the problem to determine whether A(E) = E + k for a fixed k is complete for the second level of the Boolean hierarchy for NP, i.e., BH 2 -complete. The definition of A(E) generalizes the notion of automatic complexity of individual strings due to Shallit and Wang. Let L be the language consisting of all strings of maximal nondeterministic automatic complexity. We characterize the complexity of infinite subsets of L by showing that they can be co-context-free but not context-free, i.e., L is CFL-immune, but not cocfl-immune. While we do not know whether L cocfl itself, we show that the following variant L cocfl. Namely, L is the set of all strings of relatively high deterministic automatic complexity, where x has relatively high complexity if A(x) 4 x. Introduction Automatic complexity of strings was introduced by Shallit and Wang [4] as a way to retain some of the power of Kolmogorov complexity while obtaining a computable notion. They raised the question whether automatic complexity is in fact polynomial-time computable. We give partial negative results for that question in two ways. This work was partially supported by a grant from the Simons Foundation (#3588 to Bjørn Kjos-Hanssen).

2 For nondeterministic automatic complexity, introduced by Hyde and Kjos-Hanssen [8,, 2, 3], there is a natural notion of maximally complex strings. We show that the language L consisting of all such strings is not context-free, by virtue of being CFL-immune. This result appears to be at the right level of the complexity hierarchy insofar as we also show that L is not cocfl-immune. While we do not know whether L cocfl, a related language consisting of somewhat complex strings is shown to be non-cocfl. We generalize automatic complexity to a more general notion of automatic complexity of equivalence relations on strings, and show that is not polynomial-time computable. In particular, we show that the set of minimally complex equivalence relations is NP-complete and the set of equivalence relations whose complexity is exactly a constant k above the minimum is BH 2 -complete. In the past, Gold [7] and Angluin [] established NP-completeness for related problems. Heggernes et al. [6] considered parametrized complexity variations such as fixing the number of states at two ( Q = 2) and increasing the alphabet size. As an illustration of the power and computability of automatic complexity, we have created the following web service. To find the complexity of, say, the string 0000, and an illustration of any automaton used in the associated proof, go to Alternatively, play the Complexity Guessing Game at: 2 The set of maximally complex strings is CFL-immune but not cocfl-immune Definition. Let A N denote nondeterministic automatic complexity [9]. For a string x, A N (x) is the minimum number of states of a nondeterministic finite automaton that accepts x, and does not accept any other string of length x, and accepts x via only one computation path. Similarly A denotes deterministic automatic complexity [4]. For a string x, A(x) is the minimum number of states of a deterministic finite automaton that accepts x, and does not accept any other string of length x. 2

3 x m+ q x x 2 x 3 x 4 x m x m start q 2 q 3 q 4... q m q m+ x n x n x n 2 x n 3 x m+3 x m+2 Figure : A nondeterministic finite automaton that only accepts one word x = x x 2 x 3 x 4... x n of length n = 2m +. We do not know whether the restriction of only one computation path in the definition of A N is superfluous. That question was studied in [0] which overlaps with the present paper in also containing Lemma 5. Theorem 2 (Hyde [8, 9]). For a string x of length n, n A N (x) +. 2 An idea of the proof of Theorem 2 is given in Figure. Definition 3. Let b(n) = n 2 + be the canonical upper bound for A N from Theorem 2. Let L k = {x {0,,..., k } : A N (x) = b(n)}. Any x L k is called an maximally complex string. Remark 4. L 3 is known to be infinite (see Theorem 8) but we do not know whether L 2 is infinite. Lemma 5. Suppose x 0, y 0 > 0. The equation ax + py = ax 0 + py 0 () has a unique solution in nonnegative integers (x, y) provided p is prime, p does not divide a, x 0 < p, and y 0 < a. Proof. Reducing mod p we have a(x x 0 ) p 0. Since p does not divide a, a p 0, and so since p is prime, x x 0 p 0. Thus x = x 0 + np for some n Z. If n < 0 then x x 0 p < 0, which contradicts the requirement that x 0. If n > 0 then using y 0, ax + py a(x 0 + p) + p(0) > ax 0 + py 0, contradicting (). Thus n = 0 and the only solution is x = x 0. 3

4 v a x p q u start 0 q v i q w y 2 q 3 q 4 Figure 2: Schematic of the automaton for Case 2 of the proof of Theorem 9. The number of states in the actual automaton is uv a v i wx p y + 2. Definition 6. For any collection of languages MYCLASS, a language L is MYCLASS-immune if it is infinite and contains no infinite subset in MYCLASS. We say that L comyclass if the complement of L belongs to MYCLASS. Let CFL be the class of all context-free languages. Definition 7. Let Σ be a finite alphabet. A function π : Σ Σ is a homomorphism if it respects concatenation: for all x, y, π(xy) = π(x)π(y). Theorem 8. L 3 is not cocfl-immune. Proof. Let t be an infinite square-free word over {0,, 2} generated by, and a fixed point of, a homomorphism. Such a t was constructed by Thue [6]. Let Pref(t) = {x : x is a prefix of t}. By Berstel [3], Pref(t) cocfl. Since every square-free string over {0,, 2} belongs to L 3 (see [9]), Pref(t) L 3. Theorem 9. L 3 is CFL-immune. Proof. Since L 3 is not cocfl-immune (Theorem 8), in particular L 3 is infinite. Suppose L 3 has an infinite subset K CFL. By the Pumping Lemma (Theorem ) there is a pumping length p such that any complex string X K of length at least p can be written as X = uvwxy, where vx and X N := uv N wx N y K L 3 for all N. We denote the length of X N by n N. Since L 3 is infinite, there exists at least one such string X. Case : v x. Then we can use an automaton with a loop at that the longer one z {v, x}. For large enough N, the length of z N will be more than two thirds of the length of X N, which will drive down the complexity of X N to roughly n N /3. 4

5 The remainder of the proof concerns Case 2. Case 2: b := v = x > 0. By Lemma 5, for any i, the equation ar + ps = i(a + p) has only the solution r = s = i provided p is prime, p does not divide a, both p and a are greater than i. In particular, this holds if p = 2k + is prime and with a := 2k > i. Then ar and ps are ai and pi and differ at most by i. We construct an automaton M as follows (Figure 2). We put one loop of length ab and later one of length pb, and add at most ib additional straggling states after the smaller loop of length ab. There are no loops apart from that. Now for the analysis. Each of these loops will be gone through i times. Let U = u + w + y. Let us compare the string length n N = U + 2pbi of the string X N = uv pi wx pi y corresponding to N := pi, to the number of states of M: q = U + pb + ab + ib = U + 2pb + (i )b. (Note that when i =, q = (n N +) 2 as expected as there are 2 repetitions of states.) We would like to have q < n N /2 (so that q < n N /2 + ), i.e., pb + ab + ib + U = (2p + i)b + U < 2 (2pbi + U) = pbi + U 2. Equivalently, i + U 2b b < (i 2)p. To achieve this we can fix i = 3, and use the infinitude of the primes to get p. Thus, M witnesses that X N is not complex, in contradiction to the Pumping Lemma (Theorem ). 3 Somewhat simple strings do not form a CFL Let RE denote the collection of recursively enumerable (or if you prefer, computably enumerable) languages. Recall that L 3 is the set of maximally complex strings for nondeterministic automatic complexity over the alphabet {0,, 2}. Let { R = x : C(x) x } 2 5

6 be the set of random strings for plain Kolmogorov complexity. We have seen (Theorem 8 and Theorem 9) that L 3 is CFL-immune but not cocfl-immune. This is a pleasant analogue of the classical fact that R is RE-immune [5, Section 3.] but not core-immune. Indeed, R core, but we conjecture that the analogous statement L 3 cocfl fails. Conjecture 0. L 3 cocfl. We shall confirm a variant of Conjecture 0 in Theorem 5. First some preliminary results. Theorem (Pumping Lemma for CFL [2]). If a language L is contextfree, then there exists some integer p (a pumping length) such that every string s in L with s p can be written as s = uvwxy where. vwx p, 2. vx, and 3. uv N wx N y is in L for all N 0. Theorem 2. Let A denote deterministic automatic complexity. A(y) A(xyz) for all strings x, y, z. Then Proof. Given an automaton witnessing A(xyz), we merely change the start and end states to obtain an automaton witnessing an upper bound on A(y). Theorem 3. Let A denote deterministic automatic complexity. If π : {0, } {0, } is an injective homomorphism with π(0) = π() then A(x) A(π(x)) for each string x. Proof. Let M be a witnessing automaton for A(π(x)). We can now make an automaton M with the same states as M that uniquely accepts x among strings of length x as follows. Throw out all the edges of M. Put an edge labeled i from q to q 2 in M if δ(q, π(i)) = q 2 in M. Theorem 4 (Shallit and Wang [4, Theorems 0 and 2]). Let A denote deterministic automatic complexity. There is a constant n 0 such that for n n 0, n A(0 n n ) 6 n +. Theorem 5. Let Then S CFL. S = {x {0, } : A(x) < 4 x }. 6

7 Proof. We assume S CFL and derive a contradiction using the Pumping Lemma (Theorem ). Let p be any sufficiently large pumping length. (The meaning of sufficiently large is determined below.) We shall build our unpumpable string as s = r k where r = 0 p p and k is sufficiently large relative to p. Consider any decomposition of s as s = uvwxy where vwx p. The main idea of the proof is that r is the shortest subword R of s such that the simplicity of s comes from repeating R, and the pumpable part vwx of s is shorter than r, hence pumping cannot help but increase the complexity. Thus by choosing k wisely we will have X S and X N := uv N wx N y S for some N, which will be a contradiction. The details are as follows. Case : vx is all 0s. (We omit the case when vx is all s as the proof is identical.) Then so is vwx. Then for N = pk, X N will contain a subword of the form either (0 p 0 p ) k/2 ( p 0 p ) k/2 or (0 p p ) k/2 (0 p 0 p ) k/2. Then using vx p, 3pk = n + pk = n + N n N = uv N wx N y n + Np n + p 2 k = 2pk + p 2 k. By Theorem 2, Theorem 3, and Theorem 4, A(X N ) A(0 k/2 k/2 ) k/2 n N /(4p + 2p 2 ) 4 n N provided i.e., (n N /(4p + 2p 2 )) 2 n N, n N (4p + 2p 2 ) 2, 3pk (4p + 2p 2 ) 2, k (4p + 2p2 ) 2 3p = 4p(2 + p)2. 3 In order to guarantee that A(X ) < 4 n, we require, using Theorem 4: A(X ) A(0 p p ) 6 p + < 4 2pk = 4 n. It suffices to require k 6 4 p 3 and p 2, since then 4 2pk 4 pk 6p > 6 p

8 Case 2: vx contains both 0s and s. Then uv N wx N y will contain a subword of the form 0 N N (using the fact that the blocks 0 p p in s are longer than the pumping length). The analysis is then similar to Case (but without using Theorem 3). Remark 6. There is nothing special about the function n 4 n ; the proof of Theorem 5 can be adapted to other bounds of the form A(x) f( x ), the main requirement being that f(x) = o( x). 4 Automatic complexity of equivalence relations Definition 7. Given a deterministic finite automaton (DFA) M = (Q, Σ, δ, q 0, F ), an equivalence relation D on Q induces an equivalence relation E on a subset S of {0, } if E = {(x, y) S 2 (δ(q 0, x), δ(q 0, y)) D}. A deterministic finite automaton M = (Q, Σ, δ, q 0, F ) coheres with E if there is an equivalence relation D on Q such that D induces E. In words, if D induces E then two strings x, y S are E-equivalent iff M ends in D-equivalent states on input x and on input y. Note that the set of final states F is irrelevant in Definition 7. Definition 8. The automatic complexity A(E) of an equivalence relation E is the least number of states of a DFA that coheres with E. Remark 9. Automatic complexity of a string x (Shallit and Wang [4]) is a special case of automatic complexity of equivalence relations. Namely, the two equivalence classes are {x} and {y : y = x, y x}. 4. Complexity of equivalence relations is BH 2 -complete Definition 20 (inspired by [5, Victor Kuncak s solution to Exercise 7.36]). Given a formula φ with m clauses, we define an equivalence relation E φ. We first define a family of axioms, which are of the form σ q 8

9 where σ is a string and q can be thought of as a state. If σ q and τ q then we let (σ, τ) E φ. ε q 0, q, 2 q 2,..., m q m, m+ q 0, 0 h, 00 v t, 0 v f, 000 l t, 00 l f, 00 l f, 0 l t, 0000 s, 000 r, 000 q 0, 00 r, s, 0000 s, 0000 q 0, 000 q 0, 0 r, 2 0 r,..., m 0 r. For a variable x j, let us define 0 x j = x j and x j = x j (the negation of x j ). For a literal l = b x j we define an encoding string t(l) = j 0b0. For each clause (l l 2 l 3 ) in φ, we add the axiom Finally t(l )t(l 2 )t(l 3 ) s. E φ = {(σ, τ) : σ u, τ u are both axioms, for the same state u}. Thus, we are making all states inequivalent. Theorem 2. {E : A(E) = E } is NP-complete. Proof. We reduce 3-SAT to {E : A(E) = E } using the mapping φ E φ from Definition 20. Consider the partial DFA M in Figure 3, except that the state e and its adjacent edges indicated there are removed. We see that φ is satisfiable iff there is a way to complete M to a DFA realizing E φ without adding any more states (in particular, without adding the state e). The possible ways to complete the DFA are shown in dotted lines. Theorem 22. {E : A(E) = E + } is conp-hard. Proof. We shall reduce 3-SAT to {E : A(E) E + } using the state e in Figure 3. Note that e malevolently maps both 0 and to l t, hence e ensures satisfaction of the axioms j 0 r. If we include e in our DFA then we can add edges to extend the partial DFA to a full DFA, whether or not φ is satisfiable. The state e was not mentioned in the axioms for E φ. Thus φ is satisfiable = A(E φ ) = E φ, φ is unsatisfiable = A(E φ ) = E φ +. 9

10 Definition 23 (Wechsung [8]). The first two levels of the Boolean hierarchy for NP are given by BH = NP, Definition 24. BH 2 = {L \ L 2 : L, L 2 NP}. SAT(2) = {(φ, φ 2 ) : φ is satisfiable and φ 2 is not.}. Theorem 25. SAT(2) is complete for BH 2 with respect to polynomial-time many-one reductions. Theorem 25 can be found in Cai et al. [4, Theorem 5.2]. (In their notation, BH 2 = NP(2).) We will use without proof the extension of Theorem 25 from SAT to 3-SAT. Theorem 26. For each k, {E : A(E) = E + k} is BH 2 -complete. Moreover, {(E, k) : A(E) = E + k} (2) is BH 2 -complete. Proof. For each k, {E : A(E) = E +k} = {E : A(E) E +k}\{e : A(E) E +k } BH 2. Let M and M 2 be automata as indicated in Figure 3 for two 3-SAT instances φ and φ 2, respectively. From M and M 2 we define M in the obvious way so as to achieve L(M) = {0x : x L(M )} {x : x L(M 2 )}. We make corresponding changes in the axioms for E φi (Definition 20). For (2) we include some duplicates of M 2. We define M so that, if say k = 3, L(M) = {0x : x L(M )} {0x : x L(M 2 )} {x : x L(M 2 )}. This way we ensure that A(E) = E + k iff φ 2 is not satisfiable, but φ is. Applying Theorem 25 completes the proof. 0

11 q start 0 q q h 0, 0 v f v t e 0 0 0, l f l t 0 r s Figure 3: Schematic of automata based on Definition 20. The case where the formula φ has m = 2 clauses is shown. The malevolent state e and its adjacent edges are included for the sake of Theorem 22. 0,

12 References [] Dana Angluin. On the complexity of minimum inference of regular sets. Inform. and Control, 39(3): , 978. [2] Y. Bar-Hillel, M. Perles, and E. Shamir. On formal properties of simple phrase structure grammars. Z. Phonetik Sprachwiss. Kommunikat., 4:43 72, 96. [3] Jean Berstel. Every iterated morphism yields a co-cfl. Inform. Process. Lett., 22():7 9, 986. [4] Jin-Yi Cai, Thomas Gundermann, Juris Hartmanis, Lane A. Hemachandra, Vivian Sewelson, Klaus Wagner, and Gerd Wechsung. The Boolean hierarchy. I. Structural properties. SIAM J. Comput., 7(6): , 988. [5] Rodney G. Downey and Denis R. Hirschfeldt. Algorithmic randomness and complexity. Theory and Applications of Computability. Springer, New York, 200. [6] Henning Fernau, Pinar Heggernes, and Yngve Villanger. A multiparameter analysis of hard problems on deterministic finite automata. J. Comput. System Sci., 8(4): , 205. [7] E. Mark Gold. Complexity of automaton identification from given data. Inform. and Control, 37(3): , 978. [8] Kayleigh Hyde. Nondeterministic finite state complexity. Master s thesis, University of Hawaii at Manoa, U.S.A., 203. [9] Kayleigh Hyde and Bjørn Kjos-Hanssen. Nondeterministic automatic complexity of overlap-free and almost square-free words. Electronic Journal of Combinatorics, 205. To appear. [0] Gregory Igusa and Bjørn Kjos-Hanssen. The definition of structure function for nondeterministic automatic complexity. To appear, 206. [] Bjørn Kjos-Hanssen. Kolmogorov structure functions for nondeterministic automatic complexity in computational statistics. In COCOA 204, volume 888 of Lecture Notes in Comput. Sci., pages Springer, Heidelberg,

13 [2] Bjørn Kjos-Hanssen. Kolmogorov structure functions for automatic complexity. Theoret. Comput. Sci., 206. To appear. [3] Bjørn Kjos-Hanssen and Kayleigh Hyde. Nondeterministic automatic complexity of almost square-free and strongly cube-free words. In CO- COON 204, volume 859 of Lecture Notes in Comput. Sci., pages Springer, Heidelberg, 204. [4] Jeffrey Shallit and Ming-Wei Wang. Automatic complexity of strings. J. Autom. Lang. Comb., 6(4): , nd Workshop on Descriptional Complexity of Automata, Grammars and Related Structures (London, ON, 2000). [5] Michael Sipser. Instructor s Solutions Manual: Introduction to the Theory of Computation. Cengage Learning, 3rd edition, 203. [6] A. Thue. Über die gegenseitige Lage gleicher Teile gewisser Zeichenreihen. Norske Vid. Skrifter I Mat.-Nat. Kl., Christiania, : 67, 92. [7] B. A. Trakhtenbrot and Ya. M. Barzdin. Finite automata. North- Holland Publishing Co., Amsterdam-London; American Elsevier Publishing Co., Inc., New York, 973. Behavior and synthesis, Translated from the Russian by D. Louvish, English translation edited by E. Shamir and L. H. Landweber, Fundamental Studies in Computer Science, Vol.. [8] Gerd Wechsung. On the Boolean closure of NP. In Fundamentals of computation theory (Cottbus, 985), volume 99 of Lecture Notes in Comput. Sci., pages Springer, Berlin,

14 q 00 q 0 q 0 q 0 start q ε Figure 4: Illustration of the upper bound A(E) P for Theorem 29. Missing transitions are arbitrary. There is a state q σ for each σ P. Equivalence classes of states are indicated by circle types such as double and dashed. Basic properties of automatic complexity of equivalence relations Remark 27. We use block notation to indicate equivalence relations. For instance, E = [ε, 00][] is the equivalence relation on the set of three strings S = {ε, 00, } according to which the empty string ε and 00 are equivalent to each other, but not to. Definition 28. Given a family of strings S, we define P to be the set of prefixes σ of strings in S such that either σ = ε, or σ (the length σ prefix of σ) has at least two (not necessarily proper) extensions in S. The cardinalities of the sets S, P are denoted S, P. For any equivalence relation E we denote by E its number of equivalence classes. Theorem 29. E A(E) P. Proof. It is immediate that A(E) E. For the other inequality, we were inspired by Trakhtenbrot and Barzdin [7]. Consider for instance E = [ε, 0][00, ][]. Then P = {ε, 0,, 00, } and the automaton witnessing that A(E) P is given in Figure 4. Theorem 30. There are infinitely many equivalence relations E with E = P. 4

15 0, 0, 0, q 0 start q q 2 Figure 5: Proving Theorem 32. Proof. Consider the family S = {0, } n = {x {0, } : x n} and the equivalence relation E on S where all distinct pairs are inequivalent. Then E = 2 n+ = P. Remark 3. In general the following inequalities are best possible: E S P, E A(E) P. That is, in general S and A(E) are incomparable. S > A(E) in the case of an equivalence relation with just one block, whereas A(E) > S in the case E = [00][0000] by similar reasoning to that of Theorem 32 below. Theorem 32. Let k 0. Then A([ε, 0,..., 0 k ][0 k+ ]) = k + 2 = E + k = P. Proof. To see that A([ε, 0][00]) 3 it suffices to study Figure 5, in which the D-equivalence of the states q 0 and q is indicated by double circles. In general, note that each 0-transition from a state corresponding to the big equivalence class B = [ε, 0,..., 0 k ] must go to a fresh state. Indeed, otherwise it will go to a state in B, which leads to all powers of 0 being in B. Notes To facilitate computations, we may use two organizing principles. Proposition 35 and Proposition 36 express the role of conservative extension and refinement, respectively. Definition 33. The equivalence relation E 2 is a conservative extension of the equivalence relation E if E 2 restricted to the strings in E equals E. 5

16 Example 34. [ε, 0][] is a conservative extension of [0][]. Proposition 35. If an equivalence relation E 2 is a conservative extension of an equivalence relation E then A(E ) A(E 2 ). Proposition 36. If E is a finer equivalence relation than E 2 (i.e., E E 2 as sets of pairs from the same underlying set of strings) then A(E 2 ) A(E ). Proof. Consider any automaton M = (Q, Σ, δ, q 0, F ) with A(E ) = Q states and associated equivalence relation D on Q induces E. Let D 2 be a coarsening of D inducing E 2. Then we see that D 2 induces E 2 and hence M also coheres with E 2. So A(E 2 ) Q = A(E ). Proof. Any automaton that coheres with E 2 also coheres with E. Some napkin calculations using Propositions 35 and 36 lead us to a following minimal example of A(E) E, which is obtained by setting k = in Theorem 32. Example 37 (Example for Theorem 9). Imagining a pumping length p = 4, let v = 0 and x = 0, and consider the complex string 00 = vx. Note that U = 0 in this case, and b = v = 2. Let i = 3, p = 5, a = 4, N = pi = 5, and q = pb + ab + ib = = 23, and n N = 2pbi = 60. Thus from Figure 2 we see that (0) pi (0) pi = (0) 5 (0) 5 is not complex. This is not sharp, as numerically we have found that already (0) 2 (0) 2 is not complex. 6