Counting dependent and independent strings

Transcription

1 Counting dependent and independent strings Marius Zimand Towson University MFCS 2010, Brno Marius Zimand (Towson U.) Counting Dependent/Indep. Strings August / 15

2 General principle Background In a large set some relationship emerges among its elements. Marius Zimand (Towson U.) Counting Dependent/Indep. Strings August / 15

3 Background Instances of the general principle n + 1 vectors in an n-dimensional vector space are dependent. Marius Zimand (Towson U.) Counting Dependent/Indep. Strings August / 15

4 Background Instances of the general principle n + 1 vectors in an n-dimensional vector space are dependent. For all k and all n sufficiently large, any subset of [n] of positive density must contain k elements in arithmetic progression. Marius Zimand (Towson U.) Counting Dependent/Indep. Strings August / 15

5 Background Instances of the general principle n + 1 vectors in an n-dimensional vector space are dependent. For all k and all n sufficiently large, any subset of [n] of positive density must contain k elements in arithmetic progression. Any set of 5 points in the plane contains a 4-vertices convex polygon. Marius Zimand (Towson U.) Counting Dependent/Indep. Strings August / 15

6 Background Instances of the general principle n + 1 vectors in an n-dimensional vector space are dependent. For all k and all n sufficiently large, any subset of [n] of positive density must contain k elements in arithmetic progression. Any set of 5 points in the plane contains a 4-vertices convex polygon. Ramsey Theorem,... Marius Zimand (Towson U.) Counting Dependent/Indep. Strings August / 15

7 Main General principle in algorithmical information theory How large should a set of n-bit strings be, to be sure that its elements share some information? Gist of our main results: If A {0, 1} n has size poly(n)2 α, then its elements share α bits of information. Marius Zimand (Towson U.) Counting Dependent/Indep. Strings August / 15

8 Main Kolmogorov complexity Kolmogorov complexity of a string is the length of its shortest description {}}{ has a short description. flipping a coin times: : description bits. Definition: C(x) = min{ p U(p) = x}; C(x y) = min{ p U(p, y) = x}, where U is a fixed universal Turing machine. Marius Zimand (Towson U.) Counting Dependent/Indep. Strings August / 15

9 Main Mutual information, string dependency x, y binary n-bit strings. I (x : y) = quantity of information in x about y. DEFINITION: I (x : y) = C(y) C(y x). Symmetry of Information: I (x : y) = I (y : x) ± O(log n). DEFINITION: y has α-dependency with x if C(y) C(y x) α. Informally: x has α bits of information about y. Marius Zimand (Towson U.) Counting Dependent/Indep. Strings August / 15

10 Main Two notions of independence Pairwise independence The set {x 1,..., x t } of n-bit strings is pairwise independent if C(x i ) C(x i x j ) α, for all i j. Informally: No string has more than α bits of information about any other string. Marius Zimand (Towson U.) Counting Dependent/Indep. Strings August / 15

11 Main Two notions of independence Pairwise independence The set {x 1,..., x t } of n-bit strings is pairwise independent if C(x i ) C(x i x j ) α, for all i j. Informally: No string has more than α bits of information about any other string. Mutual independence The tuple (x 1,..., x t ) of n-bit strings is mutually α-independent if for every permutation π, C(x π(1) x π(2)... x π(t) ) C(x 1 ) + C(x 2 ) C(x t ) α. Informally: x 1,..., x t share jointly at most α bits of information. Marius Zimand (Towson U.) Counting Dependent/Indep. Strings August / 15

12 Main First main result Theorem {x 1, x 2,..., x t } is α-pairwise independent t < poly(n)2 α, (where, poly(n) = 2n 3 ). Marius Zimand (Towson U.) Counting Dependent/Indep. Strings August / 15

13 Main First main result Theorem {x 1, x 2,..., x t } is α-pairwise independent t < poly(n)2 α, (where, poly(n) = 2n 3 ). Theorem The above is tight within a polynomial factor. There exists {x 1, x 2,..., x t } α-pairwise independent with t = 1 poly(n) 2α (provided 5 log n α n). Marius Zimand (Towson U.) Counting Dependent/Indep. Strings August / 15

14 Main Second main result Theorem (x 1,..., x t ) mutually α-independent t < poly(n)2 α, (provided 7 log n + 6 α n). Marius Zimand (Towson U.) Counting Dependent/Indep. Strings August / 15

15 Main Second main result Theorem (x 1,..., x t ) mutually α-independent t < poly(n)2 α, (provided 7 log n + 6 α n). We do not know if the above is tight. We believe it is within a polynomial factor. Marius Zimand (Towson U.) Counting Dependent/Indep. Strings August / 15

16 One proof Some proofs How many strings are dependent with a given string? A x,α : the set of strings about which x {0, 1} n has α bits of information. A x,α = {y {0, 1} n C(y) C(y x) α}. We want to estimate the size of A x,α. Marius Zimand (Towson U.) Counting Dependent/Indep. Strings August / 15

17 Upper bound is easy Some proofs Goal: Upper bound the size of A x,α = {y {0, 1} n C(y) C(y x) α}. If y A x,α, C(y x) C(y) α = n α + O(1). There are at most 2 n α+o(1) such strings y. So, A x,α C 2 n α, for some constant C. Marius Zimand (Towson U.) Counting Dependent/Indep. Strings August / 15

18 Some proofs Lower bound; main technical innovation of the paper Goal: Lower bound the size of A x,α = {y {0, 1} n C(y) C(y x) α}. Normal approach for a lower bound of A x,α : Suppose x is random (the proof can be adapted for non-random strings). x α (the α prefix of x) is also random. Take z of length (n α) and random conditioned by x α. Let y = zx α. There are 2 n α such strings. C(y) C(x α) + C(z x α) O(log n) α + (n α) O(log n) = n O(log n). Also, C(y x) n α + O(log n). So, y A x,α+o(log n). So, A x,α+o(log n) 2 n α. So, A x,α (1/poly(n)) 2 n α. Novel approach: A x,α (1/C) 2 n α poly(n) 2 α. Marius Zimand (Towson U.) Counting Dependent/Indep. Strings August / 15

19 Some proofs Lower bound: new approach Goal: Lower bound the size of A x,α = {y {0, 1} n C(y) C(y x) α}, for x with C(x) α + 8 log n, and α computable from n. Corollary of [Z MFCS 2010] For every constant c, there exists uniformly computable E : {0, 1} n {0, 1} n {0, 1} α+c+o(1) s.t. for all y in {0, 1} n IF C(y) α + 8 log n and C(y n) C(y x) α THEN C(E(x, y) x) c. Let z in Image(f ) with the largest number of preimages in {x} {0, 1} n. z has at least 2 n α c O(1) such preimages. C(z x) < c. So, all the preimages of z fail to satisfy the IF condition. So, {y C(y n) C(y x) α} (1/C) 2 n α poly(n)2 α. Marius Zimand (Towson U.) Counting Dependent/Indep. Strings August / 15

20 Some proofs Another proof What is the size of a min dominating set in the dependency graph? Theorem There exists a set B {0, 1} n of size poly(n) 2 α, such that any string in {0, 1} n is α-dependent with a string in B (i.e., for any string u, there is some string in B that has α bits of information about u). Take x 1,..., x T independently at random in {0, 1} n, for T = poly(n)2 α. Fix u {0, 1} n. At least (1/C)2 n α strings have α bits of information about u. Prob(u is not in any A xi,α) < (1 (1/C)2 α ) T < e n. Prob(some u {0, 1} n is not in any A xi,α) < 2 n e n < 1. So there are some tuplet (x 1,..., x t ) that covers all strings u {0, 1} n. Marius Zimand (Towson U.) Counting Dependent/Indep. Strings August / 15

21 Some proofs Thank you. Marius Zimand (Towson U.) Counting Dependent/Indep. Strings August / 15