The definition of Kolmogorov structure functions for nondeterministic automatic compleity Gregory Igusa Department of Mathematics University of otre Dame gigusa@nd.edu Bjørn Kjos-Hanssen Department of Mathematics University of Hawai i at Mānoa bjoernkh@hawaii.edu July 8, 2015 Abstract We show that two possible definitions of the Kolmogorov structure function for nondeterministic automatic compleity for a binary word do not coincide. These definitions count either the number of accepting paths, or the number of accepted words. Specifically, we show that the length 7 word 0111000 is a witness of this non-coincidence.there is no shorter eample. In the area of compleity of finite sets of words of the same length, we show that {0110, 1111} has a compleity which depends on whether we are counting accepting paths or accepted words, as well. This work was partially supported by a grant from the Simons Foundation (#315188 to Bjørn Kjos-Hanssen). The authors acknowledge the support of the Institut für Informatik at the University of Heidelberg, Germany during the workshop on Computability and Randomness, June 15 July 9, 2015. 1
m+1 q 1 1 2 3 4 m 1 m start q 2 q 3 q 4... q m q m+1 n n 1 n 2 n 3 m+3 m+2 Figure 1: A nondeterministic finite automaton that only accepts one word = 1 2 3 4... n of length n = 2m + 1. 1 Introduction Shallit and Wang [6] introduced automatic compleity as a computable alternative to Kolmogorov compleity. They considered deterministic automata, whereas Hyde and Kjos-Hanssen [5] studied the nondeterministic case, which in some ways behaves better. Unfortunately, even nondeterministic automatic compleity is somewhat inadequate. The word 00010000 has maimal nondeterministic compleity, even though intuitively it is quite simple. One way to remedy this situation is to consider a structure function analogous to that for Kolmogorov compleity. The latter was introduced by Kolmogorov at a 1973 meeting in Tallinn and studied by Vereshchagin and Vitányi [8] and Staiger [7]. Here we show that some notions in this area, in the non-deterministic setting, depend on whether we are counting accepting words or accepting paths. This is interesting because counting words is most efficient for compression, whereas counting paths is most efficient in terms of computational resources like time. 1.1 Definitions and technical motivation The Kolmogorov compleity of a finite word w is roughly speaking the length of the shortest description w of w in a fied formal language. The description w can be thought of as an optimally compressed version of w. Frustrated by the non-computability of Kolmogorov compleity, Shallit and Wang defined the following finite automaton analogue. In this paper we work only over the binary alphabet {0, 1} and denote by L(M) the language recognized by automaton M. Definition 1 (Shallit and Wang [6]). The automatic compleity of a finite binary word = 1... n is the least number A D () of states of a determin- 2
istic finite automaton M such that L(M) {0, 1} n = {}, that is, is the only word of length n accepted by M. Hyde and Kjos-Hanssen [5] defined a nondeterministic analogue. We assume that our FAs our not generalized, i.e., they have no ɛ-transitions. Definition 2. The nondeterministic automatic compleity A path (w) of a word w is the minimum number of states of an FA M, accepting w such that there is only one accepting path in M of length w. The minimum compleity A path (w) = 1 is only achieved by words of the form a n where a is a single letter. Definition 3. Let n = 2m+1 be a positive odd number, m 0. A finite automaton of the form given in Figure 1 for some choice of symbols 1,..., n and states q 1,..., q m+1 is called a Kayleigh graph. Theorem 4 (Hyde [2]). The nondeterministic automatic compleity A path () of a word of length n satisfies A path () n/2 + 1. Proof. If n is odd, then a Kayleigh graph witnesses this inequality. If n is even, a slight modification suffices [2, 5]. There are two ways to define structure functions of a word, one dual to the other, that are both based on the same set S. The set S still has two versions, as we may count either paths or words. Definition 5. Suppose is a binary word of length n. Let (q, m) S word if there eists a q-state FA M with L(M) and L(M) {0, 1} n 2 m }. We let (q, m) S path if in addition M may be chosen to have at most 2 m accepting paths of length n. We note that S word From S word has the upward closure property q q, m m, (q, m) S word = (q, m ) S word. we can define the structure function h word and the dual struc-. The definition was presented to us by Vereshchagin ture function h word (personal communication, 2014), inspired by [8]. 3
Definition 6. The corresponding notions for S path h path h word (m) = min{k : (k, m) S word }, h word (k) = min{m : (k, m) S word }. we call h path and h path. ote that S path S word and hence h word (k) h path (k) and h word (m) (m) for each k, m, and,. Upper bounds on h word (k), generalizing the k = 0 case covered by Hyde s Theorem 4, were studied in [4]. It may be observed that the proofs given there are based on counting accepting paths and hence apply equally to h path. Coming back to our motivating eample of the word w = 00010000, the automaton in Figure 2 witnesses that the structure function of w is small at some points, despite its compleity attaining the maimum from Theorem 4. 1.2 Applications and further motivation As envisioned by Kolmogorov, structure functions have potential applications in computational statistics. While the problem of determining given a word and a number k whether A () k) is in P and may for all we know be P-complete, one can miniaturize by considering restricted sets of automata, thus obtaining efficiently computable notions of compleity [3]. One demonstration of this idea is the Structure Function Calculator available at the following URL: http://math.hawaii.edu/wordpress/bjoern/structure-function-calculator/ Apart from applications, automatic compleity has intrinsic interest as a toy version of Kolmogorov compleity; one may eplore it by playing the Compleity Guessing Game, at: http://math.hawaii.edu/wordpress/bjoern/software/web/compleity-guessinggame/ A variation on the same idea is the Compleity Option Game, at: http://math.hawaii.edu/wordpress/bjoern/software/web/compleity-optiongame/ 2 Results 2.1 Automatic compleity of doubletons In what follows we consider automata with only a single accept state, although this is not essential. 4
0 1 0 start q 0 q 2 Figure 2: An automaton witnessing a small value h w (2) 3 for the structure function of w = 00010000. There are 8 = 2 3 accepted words of length w. Definition 7. The word-based automatic compleity A word (F) of a finite set F { {0, 1} n to be the minimum number of states of an FA M such that L(M) {0, 1} n = F. The path-based automatic compleity A path (F) is the minimum number of states of an FA M such that in addition M has only F many accepting paths of length n. This generalizes automatic compleity from the case where F is a singleton. Clearly A word A path. We shall see in Theorem 9 that Aword (F) A path (F) in general when F = 2; and we shall conjecture in Conjecture 13 that A word (F) = Apath (F) whenever F = 1. Automatic compleity of doubletons is the case where F is a set with two distinct elements, {, y}, and has been studied by Chambers [1]. It is easy to obtain by calculation that, whether we count accepting paths or accepted words, at length n = 0, there are no doubletons; at length n = 1, the unique doubleton {0, 1} has compleity 1; at length n = 2, all the ( 4 2) = 6 doubletons have compleity 2; at length n = 3 there are ( 2 3 2 ) = 28 doubletons, some of compleity 2 and the rest of compleity 3; and at length n = 4, the realized compleities are 3, 4, and 5. The upper bound of A path (F) 5 for n = 4 also follows from the following theorem, proved by Hyde [2] for f = 1 and by Chambers [1] for f = 2. Theorem 8. The automatic compleity of a set F {0, 1} n (with one accept state allowed) of size f satisfies A path (F) f n/2 + 1. 5
n \ f 1 2 3 4 5 6 7 8 0 1(1) 1 1 1 1 1 1 1 1 1(1) 1 (1) 1 1 1 1 1 1 2 2(2) 3 (2) 4 5(1) 6 7 8 9 3 2(2) 3 (3) 4 5 6 7 8 9(1) 4 3(3) 5 (5) 7 9 11 13 15 17 5 3(3) 5 7 9 11 13 15 17 6 4(4) 7 10 13 16 19 22 25 7 4(4) 7 10 13 16 19 22 25 8 5(5) 9 13 17 21 25 29 33 9 5(5) 9 13 17 21 25 29 33 Figure 3: The Chambers Hyde upper bound on A path (F) from Theorem 8. Word length n and set size f = F. Actual maima in parentheses where known. Combinations not satisfying f 2 n are marked with. 1 1 q 0 start q 1 q 2 1 Figure 4: The optimal automaton C with L(C) {0, 1} 4 = {0110, 1111} by necessity accepts 1111 along two paths, giving Theorem 9. 0 1 Proof sketch. The proof is easy upon considering Figures 1 and 5. The upper bound in Theorem 8, which we can call the Chambers Hyde bound, appears to be sharp for f = 1 and perhaps, in general, for f << 2 n. It is not sharp for (n, f) = (2, 2) as there the maimum is 2 whereas the Chambers Hyde bound is 3. We now show that counting accepting paths is not equivalent to counting accepted words for the automatic compleity of doubletons, i.e., that A path A word. Theorem 9. There is an n and a finite set F {0, 1} n such that A word (F). A path (F) Proof. Let n = 4 and let F = {0110, 1111}. For any FA M such that L(M) {0, 1} 4 = F, M has at least 3 states, and M has at least 3 accepting 6
3 y 3 2 4 y 2 y 4 1 y 1 start Figure 5: An automaton witnessing the Chambers Hyde bound (Theorem 8) for n = 4 and f = 2. paths of length 4; that is, the automaton C in Figure 4 is optimal. This is verified by a computer program available upon request, written in a similar style to that used for Theorem 10. 2.2 Structure functions We now show that the automatic compleity structure function of a word sometimes depends on whether we are counting accepting paths or accepted words. Theorem 10. There is a binary word and a k such that h word (k) < (k). h path Proof. Let = 0111000 and let k = 3. We claim that h word (k) = 2 < 3 = h path (k). Let C 2 be the automaton given by the Kayleigh graph in Figure 6. inspection, L(C 2 ) {0, 1} 7 = {01 3 0 3, 01 5 0, 0 3 1 3 0}. By 7
1 0 1 q 0 start q 1 q 2 0 Figure 6: The automaton C 2 giving us Theorem 10. 1 The automaton C 2 is optimal for the structure function of 0111000 with m = 2. It accepts 3 2 2 words. However, it has 5 > 2 2 accepting paths of length 7. The etra paths are associated with the word 01 5 0. We claim that the number of accepting paths of length 7 is at least 5 in any automaton M with L(C 2 ) {0, 1} 7 = {01 3 0 3, 01 5 0, 0 3 1 3 0} and at most 3 states. This claim is verified by computer using the code in the Appendi (Section 4.1). Remark 11. The length 7 in Theorem 10 is in fact minimal, as can also be verified by computer. 3 Towards resolving the case of compleity of singletons Definition 12. For a word, the word-based nondeterministic automatic compleity of is defined by or equivalently as A word ({}). A word () := h word (0), Conjecture 13. For all and k, A word () = Apath (). Conjecture 13 lies at the crossroads of, and indeed was the inspiration for, our two main results. Theorem 9: A word (F) Apath (F) for the doubleton F = {0110, 1111}. Conjecture 13 states that there is no singleton eample. Theorem 10: h word (k) h path (k) for the word 0111000, at k = 2. Conjecture 13 states that there is no eample with k = 0. Remark 14. We have made a minor computer investigation of Conjecture 13: it holds for binary strings of length at most 8. 8
q b 0 1 0 start q a q c q e 1 0 1 q d Figure 7: An automaton with q = 5 states accepting = (01) 5 and no other word of length n = 10. There are multiple accepting paths, and removing edges cannot remedy that fact. start q a y z q b start q a y q c z q d z q f z Figure 8: Two FAs M 1 (left) and M 2 with accepting a word w = 5 yz 5, where has length 6, y has length 1, and z has length 7. An edge labeled by a word of length greater than 1 represents a concatenated sequence of edges. Since the only solution to the equation 6u+7v+1 = 66 in nonnegative integers is (u, v) = (5, 5), L(M i ) {0, 1} 66 = {w} for both i. In M 1 there are multiple accepting paths, and removing edges cannot remedy that fact or transform M 1 into M 2. The multiple paths diverge, then come together for a while, and then diverge again. 9
We now give some evidence of the difficulty in settling the truth or falsity of Conjecture 13. A naïve first idea may be that we can prove Conjecture 13 simply by removing edges and states from any putative countereample. However, consider the automaton in Figure 7, a Kayleigh graph with all edges labeled 01 accepting only the word = (01) 5 of length 10. While there is an equivalent automaton with fewer edges and only one accepting path, it cannot be obtained from the original automaton in such a simple way. That is, is not accepted on any proper subset of edges and vertices. Using Lemma 15 below we shall be able to produce more interesting eamples intended to cast some doubt upon Conjecture 13. Lemma 15. Suppose 0, y 0 > 0. The equation a + py = a 0 + py 0 (1) has a unique solution in nonnegative integers (, y) provided p is prime, p does not divide a, 0 < p, and y 0 < a. Proof. Reducing mod p we have a( 0 ) p 0 Since p does not divide a, a p 0, and so since p is prime, 0 p 0. Thus = 0 + np for some n Z. If n < 0 then 0 p < 0, which contradicts the requirement that 0. If n > 0 then using y 0, a + py a( 0 + p) + p(0) > a 0 + py 0 contradicting (1). Thus n = 0 and the only solution is = 0. By Lemma 15, 6 + 7y = 65 has the unique solution = y = 5. In fact, a + py = 5(a + p) has a unique solution = y = 5 whenever p > 5 is prime and does not divide a > 5. Figure 8 gives a simple application of this conclusion, but we can go further than that: Theorem 16. For each t there eists an FA M such that 1. M accepts only one word of length n (*), 2. no other automaton obtained from M by deletion of edges and states has the property (*), 3. M accepts along two paths γ 1, γ 2 that diverge from each other, and after coming back together, diverge again, for a total of at least t times of diverging from each other. 10
Proof sketch. We have seen the cases t = 1 in Figure 7 and t = 2 in Figure 8. ow, by substitution we can etend Lemma 15 and the corresponding eample automaton in Figure 8 to more than two variables, and more than two periods of diverging paths, respectively. Starting from the equation 6 + 7y = 65 which by Lemma 15 has the unique solution = y = 5, we find a prime greater than 65, namely 67, and obtain by Lemma 15 that the equation 67 + 6y = 67(5) + 6(65) has the unique solution = 5, y = 65. Putting these together, the equation 67 + 6(6y + 7z) = 67(5) + 6(65) has the unique solution = y = z = 5. So we can create an iterated version of Figure 8 for t = 3 and, by repeating the above argument, for arbitrary t. Theorem 16 still does not really give us reason to doubt Conjecture 13. We finish this article by making a Conjecture 17 epressing the idea that Theorem 16 is, in a sense, optimal. Conjecture 17. Suppose M is an FA which accepts only one word of length n, via several paths. Suppose there is no way to delete edges and paths while maintaining the property of accepting only the one word of length n. Then is of the form = t i=1 u i y k i i v i where denotes concatenation. Moreover, as y i is read by M, y i edges are used each time, i.e., there is no repetition of edges within one of these k i occurrences of y i ; and there is also no repetition of edges during the processing of u i and v i. The edges traversed during the reading of u 1, y k 1 1,..., v t are all distinct; the only repetition occurs during individual subwords y k i i. 11
References [1] Patrick Chambers. Automatic compleity of doubletons. Verbal presentation as a final eam for MATH 657, University of Hawaii at Mānoa, Spring 2014. [2] Kayleigh Hyde. ondeterministic finite state compleity. Master s thesis, University of Hawaii at Manoa, U.S.A., 2013. [3] Bjørn Kjos-Hanssen. Kolmogorov structure functions for nondeterministic automatic compleity in computational statistics. In COCOA 2014, volume 8881 of Lecture otes in Comput. Sci., pages 652 665. Springer, Heidelberg, 2014. [4] Bjørn Kjos-Hanssen. Kolmogorov structure functions for automatic compleity. Theoret. Comput. Sci., 2016. To appear. [5] Bjørn Kjos-Hanssen and Kayleigh Hyde. ondeterministic automatic compleity of almost square-free and strongly cube-free words. In CO- COO 2014, volume 8591 of Lecture otes in Comput. Sci., pages 61 70. Springer, Heidelberg, 2014. [6] Jeffrey Shallit and Ming-Wei Wang. Automatic compleity of strings. J. Autom. Lang. Comb., 6(4):537 554, 2001. 2nd Workshop on Descriptional Compleity of Automata, Grammars and Related Structures (London, O, 2000). [7] Ludwig Staiger. The Kolmogorov compleity of infinite words. Theoret. Comput. Sci., 383(2-3):187 199, 2007. [8] ikolai K. Vereshchagin and Paul M. B. Vitányi. Kolmogorov s structure functions and model selection. IEEE Trans. Inform. Theory, 50(12):3265 3290, 2004. 12
4 Appendi 4.1 Python module automatonreports 1 from automatoncompleitysoda import S i m p l e S t r i n g s 3 d e f c h e c k S i m p l i c i t y ( seq, q, m=0) : 5 n = l e n ( seq ) b = ma( seq ) + 1 7 s = S i m p l e S t r i n g s ( q, n, b, m) s. compleity ( seq ) 9 r e t u r n s. numsimple ( ) 11 d e f getinversestructurefunctionforqequalthree ( myseq ) : 13 p r i n t p r i n t s t r ( myseq ) + = s t r i n g being c o n s i d e r e d 15 myq = 3 i f not (myq i n range ( 1, l e n ( myseq ) /2 + 2) ) : 17 r e t u r n f o r m i n range ( 0, l e n ( myseq ) + 1) : 19 i f c h e c k S i m p l i c i t y ( myseq, myq, m) : i f (myq!= 1) : 21 p r i n t h ˆ( 1) ( + s t r (myq) + )= + s t r (m) break 23 getinversestructurefunctionforqequalthree ( 25 ( 0, 1, 1, 1, 0, 0, 0 ) ) automatonreportssoda.py import numpy 2 import math 4.2 Python module automatoncompleity 4 c l a s s S i m p l e S t r i n g s : 6 d e f i n i t ( s e l f, q, n, b=2, m=0) : s e l f. q = q 8 s e l f. n = n s e l f. b = b 10 s e l f. twotom = 2 m s e l f. emptyttt = f r o z e n s e t ( ) 12 s e l f. matches = s e t ( ) 13
14 d e f compleity ( s e l f, seq ) : t t = numpy. z e r o s ( shape=( s e l f. q, s e l f. q ) ) 16 s e l f. compleitybounds ( ( ), tt, seq ) 18 d e f s l o w i n c r e a s e ( s e l f, seq ) : currma = 1 20 f o r j i n range ( 0, l e n ( seq ) ) : i f seq [ j ] > currma + 1 : 22 r e t u r n F a l s e i f seq [ j ] == currma + 1 : 24 currma += 1 i f currma == s e l f. q 1 : 26 r e t u r n True r e t u r n True 28 d e f checkmatching ( s e l f, seq, seq, t t t ) : 30 newttt = s e t ( t t t ) j = l e n ( seq ) 1 32 i f j >= 0 : p, pp = seq [ j ], seq [ j + 1 ] 34 edgeused = F a l s e f o r bb i n range ( 0, s e l f. b ) : 36 i f (p, pp, bb ) i n t t t : edgeused = True 38 i f bb!= seq [ j ] : r e t u r n 40 break i f not edgeused : 42 newttt. add ( ( p, pp, seq [ j ] ) ) i f l e n ( seq ) < s e l f. n : 44 f o r j i n range ( 0, s e l f. b ) : s e l f. checkmatching ( seq + ( j, ), seq, newttt ) 46 r e t u r n s e l f. matches. add ( seq ) 48 d e f numsimple ( s e l f ) : 50 r e t u r n l e n ( s e l f. matches ) 52 d e f compleitybounds ( s e l f, seq, tt, seq ) : 54 i f not s e l f. s l o w i n c r e a s e ( seq ) : r e t u r n 56 t r a n s i t i o n T a b l e = t t. copy ( ) i f l e n ( seq ) == s e l f. n : 58 newttt = s e t ( s e l f. emptyttt ) f o r nn i n range ( 0, l e n ( seq ) 1) : 60 p, pp = seq [ nn ], seq [ nn + 1 ] newttt. add ( 14
62 (p, pp, seq [ nn ] ) ) 64 f o r bb i n range ( 0, s e l f. b ) : i f (p, pp, bb ) in newttt : 66 i f bb!= seq [ nn ] : t r a n s i t i o n T a b l e [ seq [ nn ], seq [ nn + 1 ] ] = 2 68 j = l e n ( seq ) 1 i f j > 0 : 70 i f ( t r a n s i t i o n T a b l e [ seq [ j 1 ], seq [ j ] ] == 0) : t r a n s i t i o n T a b l e [ seq [ j 1 ], seq [ j ] ] = 1 72 testumber = numpy. l i n a l g. matri power ( t r a n s i t i o n T a b l e, j 74 ) [ 0, seq [ j ] ] i f ( testumber == 0) or ( testumber > s e l f. twotom) : 76 r e t u r n e l s e : 78 i f j == l e n ( seq ) : p r i n t 80 p r i n t s t r ( i n t ( testumber ) ) + = # o f paths matching p r i n t s t r ( seq ) + = s t a t e sequence 82 i f j < s e l f. n : f o r p i n range ( 0, s e l f. q ) : 84 i f p < s e l f. q 1 : s e l f. compleitybounds ( 86 seq + (p, ), t r a n s i t i o n T a b l e, seq ) 88 e l s e : s e l f. compleitybounds ( 90 seq + (p, ), t r a n s i t i o n T a b l e, seq ) 92 r e t u r n i f l e n ( seq ) == 0 : 94 s e l f. checkmatching ( seq, seq, s e l f. emptyttt ) i f l e n ( seq ) == s e l f. n : 96 s e l f. matches. add ( seq ) automatoncompleitysoda.py 15
Contents 1 Introduction 2 1.1 Definitions and technical motivation.............. 2 1.2 Applications and further motivation.............. 4 2 Results 4 2.1 Automatic compleity of doubletons.............. 4 2.2 Structure functions........................ 7 3 Towards resolving the case of compleity of singletons 8 4 Appendi 13 4.1 Python module automatonreports............... 13 4.2 Python module automatoncompleity............. 13 16