On approximate decidability of minimal programs 1

On approximate decidability of minimal programs 1 Jason Teutsch June 12, 2014 1 Joint work with Marius Zimand

How not to estimate complexity What s hard about Kolmogorov complexity? the case of functions tournaments Let C(x) denote Kolmogorov complexity. Definition A binary string x is called random if C(x) x. Sample results 1 No algorithm enumerates more than finitely many random strings. 2 No unbounded, computable function is a lower bound for Kolmgorov complexity (Zvonkin, Levin). 3 Any algorithm mapping a string to a list of values containing its Kolmogorov complexity must, for all but finitely many lengths n, include in the list for some string of length n at least a fixed fraction of the lengths below n + O(1) (Beigel, Buhrman, Fejer, Fortnow, Grabowski, Longpré, Muchnik, Stephan, Torenvliet).

How not to estimate complexity II What s hard about Kolmogorov complexity? the case of functions tournaments Marius and I investigated another form of approximation. Definition A set A is (m, k)-recursive if there exists a computable function f such that for any x 1 <... < x k, the k-bit vector f (x 1,..., x k ) agrees with the characteristic vector A(x 1,... x k ) in at least m places. Theorem (Teutsch, Zimand) The set of Kolmogorov random strings is not (1, k)-recursive for any k.

Short lists for strings What s hard about Kolmogorov complexity? the case of functions tournaments On the other hand, we have the following positive results. Theorem (Bauwens, Mahklin, Vereshchagin, Zimand 2013) There exists an algorithm which maps each binary string x to an O( x 2 )-size list containing a length C(x) + O(1) description for x. Theorem (Teutsch 2013) There exists a polynomial-time algorithm which maps each binary string x to a poly( x )-size list containing a length C(x) + O(1) description for x. Theorem (Bauwens, Zimand 2014) There exists a probabilistic algorithm which, for any positive δ < 1, maps each binary string x to a x -size list which contains a length C(x) + O[log( x /δ)] description for x with probability at least 1 δ. What happens at the semantic level?

Numberings obstacles What s hard about Kolmogorov complexity? the case of functions tournaments A numbering ϕ is a mapping e ϕ e such that the induced mapping e, x ϕ e (x) is partial-computable. Definition Let ϕ be a numbering. ϕ is computably bounded if for any further numbering ψ there exists a computable translation function f such that for any e, ϕ j = ψ e for some j f (e). A computably bounded numbering is said to have the Kolmogorov property if it admits a translation function which is bounded by a linear function. If we were to replace j f (e) with j = f (e) in the definition of computably bounded, we would obtain the usual Gödel numberings from computably theory.

Minimal indices obstacles What s hard about Kolmogorov complexity? the case of functions tournaments Definition For any numbering ϕ, let MIN ϕ = {e : ( j < e) [ϕ j ϕ e ]}, and let min ϕ (e) denote the unique index j MIN ϕ such that ϕ j = ϕ e. Blum (1967) showed that MIN ϕ is not computable for any Gödel numbering ϕ. In fact, Proposition (Teutsch, Zimand) For any computably bounded numbering ϕ, neither MIN ϕ nor min ϕ is computable. Schaefer (1998) asked whether there exists a positive integer k and a Gödel numbering ϕ such that MIN ϕ is (1, k)-recursive.

What s hard about Kolmogorov complexity? the case of functions tournaments Our results using the Kolmogorov property Theorem For any positive integer k and any numbering ϕ with the Kolmogorov property, MIN ϕ is not (1, k)-recursive. Theorem For every numbering ϕ with the Kolmogorov property and any computable function f which maps each index e to a list of indices containing min ϕ (e) we have f (e) = Ω(log 2 e) for infinitely many e. Theorem There exists a numbering ϕ with the Kolmogorov property such that if f is a computable function which maps each index e to a list of indices containing min ϕ (e) then f (e) = Ω(e) for infinitely many e.

Winner-goes-on tournaments What s hard about Kolmogorov complexity? the case of functions tournaments Suppose you are given two strings, one with low complexity, and one with high complexity. Can you tell which is which? Definition A set A is called semirecursive if there exists a computable function g such that if x A or y A, then g(x, y) A {x, y}. Proposition The set of random strings is not semirecursive. Proof. We run a winner-goes-on-tournament on the strings of length n, with the function g above deciding the winner of each match. Eventually we run out of new challengers, and g guarantees that the final winner is random. But we just described the final winner using log n + O(1) bits.

Bootstrapping obstacles What s hard about Kolmogorov complexity? the case of functions tournaments Proposition The set of random strings is not (1, 2)-recursive. Proof. Suppose that some computable function f : N N {L, H} witnesses that the set of random strings is (1, 2)-recursive, where H is the label for random and L is the label for nonrandom. Note that f (x, y) = (H, H) for at most finitely many pairs (x, y) (regardless of whether or not all but finitely many pairs satisfying f (x, y) = (H, H) are both random). Otherwise we could enumerate infinitely many random strings.

Bootstrapping II obstacles What s hard about Kolmogorov complexity? the case of functions tournaments Proof (cont.) Now the set of random strings is semirecursive via the computable function g defined as follows: if f (x, y) = (L, H), then g(x, y) = y, if f (x, y) = (H, L), then g(x, y) = x, if f (x, y) = (L, L), then g waits until one of the strings appears nonrandom and then chooses the other one, and if f (x, y) = (H, H), we hardwire a correct answer for g. Such a function g does not exist. Therefore the set of random strings is not (1, 2)-recursive. In order to go beyond (1, 2)-recursive, another kind of tournament is needed.

The arena method the arena method the champion method Bauwens game What do we do when the odds are not ever in our favor?

The arena method the arena method the champion method Bauwens game What do we do when the odds are not ever in our favor? Answer: We shrink the arena.

Shrinking the arena the arena method the champion method Bauwens game We wish to find simple sets containing high density of minimal indices. We do this as follows. 1 Focus on minimal indices for functions which converge only on input 0. Call this set M. 2 Sufficiently large intervals contain at least a constant fraction of indices from M by the Kolmogorov property. 3 For a sufficiently large interval I, consider the following sets: M I, programs in I that halt only on input 0 but are not minimal, and programs below max I which halt on input 0 and some other input. 4 Use nonuniform advice to guess approximate sizes for these sets, and use these approximations to construct a set that looks very similar to M I.

the arena method the champion method Bauwens game Density boosting with little advice Arena Lemma Let ϕ be a numbering with the Kolmogorov property, and let I n = {2 an + 1,..., 2 a(n+1) }. Then for every ɛ > 0 and n 0, there exists a subset A n I n such that 1 MIN ϕ A n (1 ɛ) A n, 2 C(A n n) O[log(1/ɛ)], and 3 A n = Ω( I n ). With respect to density, minimal indices behave almost like random strings.

The champion method the arena method the champion method Bauwens game A hat makes not a champion

Majority vote tournaments We outline the construction of champions. the arena method the champion method Bauwens game 1 Let f : N {H, L} be a computable function which labels integers as either high or low complexity. 2 Consider f restricted to a finite set A N. The champion is the least integer in A which f labels with H. 3 The complexity of the champion is at most C(A) + O(1), which is low. Therefore f makes a mistake. A more interesting case. When we generalize f to tuples of integers, we select champions by majority vote. Thus we enter the realm of (1, k)-recursive. If there are enough integers with high complexity in A, we can iteratively choose a second place champion, a third place champion, etc. The function f gets an entire tuple wrong.

the arena method the champion method Bauwens game Simple sets with complex strings Champions Lemma Let M be a set of binary strings, let k and m be positive integers, and let f be any unbounded function. Suppose there exist infinitely many finite sets A such that 1 M A ( 1 1 k!(k m+2) ) A, and 2 for all x M A, C(x) C(A) + f [C(A)]. Then M is not (m, k)-recursive. Theorem (reprise) For any positive integer k and any numbering ϕ with the Kolmogorov property, MIN ϕ is not (1, k)-recursive. Proof. Arena Lemma + Champions Lemma. Minimal indices are complex.

The converse fails the arena method the champion method Bauwens game Proposition (Teutsch, Zimand) For every positive integer k, there exists a set of natural numbers M that is not (1, k)-recursive and yet it does not satisfy the conditions of the Champions Lemma. That is, for all finite sets of natural numbers A at least one of the following holds true: 1 M A < ( 1 1 (k+1)! ) A, or 2 there exists x M A such that C(x) < C(A) + log[c(a)].

Bauwens game obstacles the arena method the champion method Bauwens game Theorem (reprise) There exists a numbering ϕ with the Kolmogorov property such that if f is a computable function which maps each index e to a list of indices containing min ϕ (e) then f (e) = Ω(e) for infinitely many e. Proof. Arena Lemma + secondary arena +

Total complexity obstacles the arena method the champion method Bauwens game We outline our approach. Definition (Muchnik) Let U be the universal machine for Kolmogorov complexity. The total complexity of a string y conditioned by x is T (y x) = min{ q : U(q, x) = y and U(q, x) for all x}. If f is computable and min ϕ (e) f (e), then T (min ϕ (e) e) log f (e) + O(1). Thus it suffices to define ϕ so that for infinitely many e, T (min ϕ (e) e) log e O(1).

the arena method the champion method Bauwens game Separating combinatorics from computation Bauwens game isolates the combinatorial aspect of our proof. 1 We build a numbering ϕ on binary indices so that indices with a 1 prefix code for a Kolmogorov numbering, and the indices prefixed with 0 force infinitely many minimal indices to have high total complexity. 2 The action for programs with prefix 0 are determined by a game which uses a constant number of advice bits to determine an arena with a high density of minimal indices and a secondary arena to manage artifact threats which disrupt an otherwise simple arena. The advice is coded into the program index. 3 The game itself keeps track which and how many indices get spoiled against being minimal or having high total complexity. Each time something gets spoiled, we are forced to look for a new witness. A combinatorial argument shows that we don t run out of witnesses.

Computably bounded numberings Numberings without the Kolmogorov property do not have constant size shortlists. Theorem (Teutsch, Zimand) For any computably bounded numbering ϕ and any constant k, there is no computable function f : N N k such that min ϕ (e) f (e) for all e. This theorem is optimal for. Proposition (Teutsch, Zimand) For any computable, unbounded, nondecreasing function g, there exists a Gödel numbering ϕ and a computable function f which maps each index e to a list of size at most g(e) such that min ϕ (e) f (e).

Approximations from above and below Proposition (Teutsch, Zimand) For every computably bounded numbering ϕ, no unbounded, computable function is a lower bound for min ϕ. Let K denote the halting problem. Definition A function f is (K-)approximable from above if there exists a uniform sequence of (K-)computable functions f 0, f 1,... such that for all x, f s (x) f s+1 (x) and lim s f s (x) = f (x). We define (K-)approximable from below similarly, but with the inequalities reversed. Theorem (Teutsch, Zimand) For any numbering ϕ, the function min ϕ is K-approximable from below. If ϕ is computably bounded, then min ϕ is not K-approximable from above.

HELP WANTED obstacles Schaefer s problem short lists Question (Schaefer) Does there exist a Gödel numbering ϕ and a positive integer k such that MIN ϕ is (1, k)-recursive? Lemma (Teutsch, Zimand) For any computably bounded numbering ϕ, the Turing degree of MIN ϕ is. This improve a prior result of Jain, Stephan, and Teutsch. Theorem (Teutsch, Zimand) For any computably bounded numbering ϕ, MIN ϕ is not (1, 2)-recursive. This improves a prior result of Schaefer.

Schaefer s problem short lists The set of shortest descriptions It is sometimes useful to consider a one dimensional analogue of MIN ϕ. Theorem (Stephan, Teutsch) There exists a Gödel numbering ϕ which makes the set to be (1, 2)-recursive. {e : ( j < e) ϕ j (0) ϕ e (0)} One can upgrade the above construction to a polynomially bounded numbering. Theorem (reprise) For every positive integer k and every numbering ϕ with the Kolmogorov property, MIN ϕ is not (1, k)-recursive.

A toy positive result Schaefer s problem short lists Definition For any numbering ϕ, with indices coded as strings, let W ϕ j RW ϕ = {x : ( j < lex x) [max W ϕ j x]}. is the set of values on which ϕ j converges, and < lex is lexicographical order on strings. Let Ω denote 1 minus Chaitin s Omega. Then Ω is approximable from above, Ω s Ω s+1, and every length n prefix of Ω has complexity at least n O(1). We let succ(x) denote the lexicographically least string greater than x. Proposition There exist a Gödel numbering ϕ such that RW ϕ is (1, 2)-recursive.

Proposition There exist a Gödel numbering ϕ such that RW ϕ is (1, 2)-recursive. Proof. Let I n denote the binary strings of length n + 1, and let ψ be a Gödel numbering (on integer indices). We will define ϕ so that ϕ 0 n = ψ n for all n, and for all but finitely many n, RW ψ I n = Ω n. To satisfy the second point, define for all x I n : { Wx ϕ {succ(x), succ[succ(x)]} if succ(x) lex Ω n, = {succ(x)} otherwise. Blue arrows depict indices blocking strings from membership in RW. I n,t : Ω t n

Schaefer s problem short lists Proposition There exist a Gödel numbering ϕ such that RW ϕ is (1, 2)-recursive. Proof (cont.) Indices of the form 0 n coding for the Gödel numbering do not destroy this picture. Indeed if for some j with 0 j < lex Ω n satisfies max W ϕ = Ω n, 0 j then, as the maxes are increasing over time and Ω is decreasing, C( Ω n) 2 log n + 2 log j + O(1) 4 log n + O(1), but this can only happen finitely often, as C( Ω n) n O(1). To see that RW ϕ is (1, 2)-recursive, observe that knowing Ω n gives complete information on membership for RW ϕ at lesser indices. Given two indices, we can label the higher one as being outside RW ϕ, and then, assuming this label is wrong, correctly guess membership on the lesser index.

Schaefer s problem short lists Short lists and the Kolmogorov property Theorem (reprise) For every numbering ϕ with the Kolmogorov property and any computable function f which maps each index e to a list of indices containing min ϕ (e) we have f (e) = Ω(log 2 e) for infinitely many e. Theorem (reprise) There exists a numbering ϕ with the Kolmogorov property such that if f is a computable function which maps each index e to a list of indices containing min ϕ (e) then f (e) = Ω(e) for infinitely many e. These results leave room for improvement. Question Do there exist log 2 -size shortlists for numberings with the Kolmogorov property (as is the case for strings)?

Short lists for strings Schaefer s problem short lists Definition We say that a bipartite graph (L, R, E) admits online matchings up to size s if there exists an algorithm such that for any set of vertices in L of size s, whose vertices are (adversarially) presented to the algorithm one at a time, the algorithm can assign each vertex in order received to one of its neighbors (without knowing what comes next), and the overall assignment after all s elements is a bijection.

Short lists for strings II Schaefer s problem short lists Online Matching Theorem (Bauwens, Mahklin, Musatov, Romashchenko, Shen, Teutsch, Vereshchagin, Zimand) For every k 0, there exists an explicit bipartite graph G = (L, R, E) such that L consist of all binary strings of length at least k, the cardinality of R is less than 2 k+1, the degree of each vertex x L is poly( x ), and G admits (efficient) online matching up to size 2 k. Corollary There exists a computable function which maps each binary string x to an O( x 2 ) size list containing a length C(x) + O(1) description for x.

contains a U-description M, z for x with z C(x) + 1 whenever C(x) x. Corollary There exists a computable function which maps each binary string x to an O( x 2 ) size list containing a length C(x) + O(1) description for x. Proof. We interpret the right hand-side of the graph in the Online Matching Theorem (OMT) to consist of strings of length k + 1. Let U be the universal machine for Kolmogorov complexity, and define a machine M which does the following. 1 Wait for a U-description p to converge (dovetail). 2 Match U(p) L in the p -size OMT graph to some z R, if U(p) is not matched already. At most 2 p strings get matched on this graph. 3 Set M(z) = U(p). Then z = p + 1. Since p can be a U-shortest description for x, the computable set { M, y : y is a neighbor of x in some OMT graph k x + O(1)}

Schaefer s problem short lists We are to admit no more causes of natural things than such as are both true and sufficient to explain their appearances. Therefore, to the same natural effects we must, so far as possible, assign the same causes. Isaac Newton In short, let us not waste time on excessively long computer programs. Thanks-la!