Boolean Function Complexity

Transcription

1 Stasys Jukna Boolean Function Complexity Advances and Frontiers August 31, 2011 Springer

2

3 To Daiva and Indrė

4

5 Preface Go to the roots of calculations! Group the operations. Classify them according to their complexities rather than their appearances! This, I believe, is the mission of future mathematicians. Evariste Galois Computational complexity theory is the study of the inherent hardness or easiness of computational tasks. Research in this theory has two main strands. One of these strands structural complexity deals with high-level complexity questions: is space a more powerful resource than time? Does randomness enhance the power of efficient computation? Is it easier to verify a proof than to construct one? So far we do not know the answers to any of these questions; thus most results in structural complexity are conditional results that rely on various unproven assumptions, like P NP. The second strand concrete complexity or circuit complexity deals with establishing lower bounds on the computational complexity of specific problems, like multiplication of matrices or detecting large cliques in graphs. This is essentially a low-level study of computation; it typically centers around particular models of computation such as decision trees, branching programs, boolean formulas, various classes of boolean circuits, communication protocols, proof systems and the like. This line of research aims to establish unconditional lower bounds, which rely on no unproven assumptions. This book is about the life on the second strand circuit complexity with a special focus on lower bounds. It gives self-contained proofs of a wide range of unconditional lower bounds for important models of computation, covering many of the gems of the field that have been discovered over the past several decades, right up to results from the last year or two. More than twenty years have passed since the well-known books on circuit complexity by Savage (1976), Nigmatullin (1983), Wegener (1987) and Dunne (1988) as well as a famous survey paper of Boppana and Sipser (1990) were written. I feel it is time to summarize the new developments in circuit complexity during these two decades. The book is mainly devoted to mathematicians wishing to get an idea on what is actually going on in this one of the hardest, but also mathematically cleanest fields of computer science, to researchers in computer science wishing to refresh their knowledge about the state of art in circuit complexity, as well as to students wishing to try their luck in circuit complexity. vii

6 viii Preface I have highlighted some of the most important proof arguments for circuit lower bounds, without trying to be encyclopedic. To keep the length of the book within reasonable limits, I was forced to focus on classical circuit models results on their randomized or algebraic versions receive less attention here. Also, I often compromise the numerical tightness of results in favor of clarity of argument. My goal is to present the big picture of existing lower bound methods, in the hope that the reader will be motivated to find new ones. More than 40 open problems, marked as Research Problems, are mentioned along the way. Most of them are of a combinatorial or combinatorial-algebraic flavor and can be attacked by students with no knowledge in computational complexity. The book is meant to be approachable for graduate students in mathematics and computer science, and is self-contained. The text assumes a certain mathematical maturity but no special knowledge in the theory of computing. For non-mathematicians, all necessary mathematical background is collected in the appendix of the book. As in combinatorics or in number theory, the models and problems in circuit complexity are usually quite easy to state and explain, even for the layperson. Most often, their solution requires a clever insight, rather than fancy mathematical tools. I am grateful to Miklos Ajtai, Marius Damarackas, Andrew Drucker, Anna Gál, Sergey Gashkov, Dmitry Gavinsky, Jonathan Katz, Michal Koucky, Matthias Krause, Andreas Krebs, Alexander Kulikov, Meena Mahajan, Igor Sergeev, Hans Ulrich Simon, György Turán, and Sundar Vishwanathan for comments and corrections on the draft versions of the book. Sergey Gashkov and Igor Sergeev also informed me about numerous results available only in Russian. I am especially thankful to Andrew Drucker, William Gasarch, Jonathan Katz, Massimo Lauria, Troy Lee, Matthew Smedberg, Ross Snider, Marcos Villagra, and Ryan Williams for proofreading parts of the book and giving very useful suggestions concerning the contents. Their help was crucial when putting the finishing touches to the manuscript. The strong commitment of Andrew Drucker in organizing these final touches and proofreading more than a half of the book by himself cannot be acknowledged well enough. All remaining errors are entirely my fault. My sincere thanks to Georg Schnitger for his support during my stay in Frankfurt. Finally, I would like to acknowledge the German Research Foundation (Deutsche Forschungsgemeinschaft) for giving an opportunity to finish the book while working within the grant SCHN 503/5-1. My deepest thanks to my wife, Daiva, and my daughter, Indrė, for their patience. Frankfurt am Main/Vilnius S. J. August 2011

7 Contents Part I The Basics 1 Our Adversary: The Circuit Boolean functions Circuits Branching programs Almost all functions are complex So where are the complex functions? A 3n lower bound for circuits Graph complexity A constant factor away from P NP? Exercises Analysis of Boolean Functions Boolean functions as polynomials Real degree of boolean functions The Fourier transform Boolean 0/1 versus Fourier ±1 representation Approximating the values 0 and Approximation by low-degree polynomials Sign-approximation Sensitivity and influences Exercises Part II Communication Complexity 3 Games on Relations Communication protocols and rectangles Protocols and tiling Games and circuit depth Exercises ix

8 x Contents 4 Games on 0-1 Matrices Deterministic communication Nondeterministic communication P = NP co-np for fixed-partition games Clique vs. independent set game Communication and rank The log-rank conjecture Communication with restricted advice P NP co-np for best-partition games Randomized communication Lower bound for the disjointness function Unbounded error communication and sign-rank Private vs. public randomness Exercises Multi-Party Games The number-in-hand model The approximate set packing problem Application: streaming algorithms The number-on-forehead model The discrepancy bound Generalized inner product Matrix multiplication Best-partition k-party communication Exercises Part III Circuit Complexity 6 Formulas Size versus depth A quadratic lower bound for universal functions The effect of random restrictions A cubic lower bound Nechiporuk s theorem Lower bounds for symmetric functions Formulas and rectangles Khrapchenko s theorem Complexity is not convex Complexity is not submodular The drag-along principle Bounds based on graph measures Lower bounds via graph entropy Formula size, rank and affine dimension Exercises

9 Contents xi 7 Monotone Formulas The rank argument Lower bounds for quadratic functions A super-polynomial size lower bound A log 2 n depth lower bound for connectivity An n 1/6 depth lower bound for clique function An n 1/2 depth lower bound for matching Exercises Span Programs The model The power of span programs Power of monotone span programs Threshold functions The weakness of monotone span programs Self-avoiding families Characterization of span program size Monotone span programs and secret sharing Exercises Monotone Circuits Large cliques are hard to detect Very large cliques are easy to detect The monotone switching lemma The lower-bounds criterion Explicit lower bounds Circuits with real-valued gates Criterion for graph properties Clique-like problems What about circuits with NOT gates? Razborov s method of approximations A lower bound for perfect matching Exercises The Mystery of Negations When are NOT gates useless? Markov s theorem Formulas require exponentially more NOT gates Fischer s theorem How many negations are enough to prove P NP? Exercises Part IV Bounded Depth Circuits

10 xii Contents 11 Depth-three Circuits Why is depth three interesting? An easy lower bound for Parity The method of finite limits A lower bound for Majority NP co-np for depth-3 circuits Graph theoretic lower bounds Depth-2 circuits and Ramsey graphs Depth-3 circuits and signum rank Depth-3 circuits with parity gates Threshold circuits Exercises Large-Depth Circuits Håstad s switching lemma Razborov s proof of the switching lemma Parity and Majority are not in AC Constant-depth circuits and average sensitivity Circuits with parity gates Circuits with modular gates Circuits with symmetric gates Rigid matrices require large circuits Exercises Circuits with Arbitrary Gates Entropy and the number of wires Entropy and depth-two circuits Matrix product is hard in depth two Larger-depth circuits Linear circuits for linear operators Circuits with OR gates: rectifier networks Non-linear circuits for linear operators Relation to circuits of logarithmic depth Exercises Part V Branching Programs 14 Decision Trees Adversary arguments P = NP co-np for decision tree depth Certificates, sensitivity and block sensitivity Sensitivity and subgraphs of the n-cube Evasive boolean functions Decision trees for search problems Linear decision trees

11 Contents xiii 14.8 Element distinctness and Turán s theorem P NP co-np for decision tree size Exercises General Branching Programs Nechiporuk s lower bounds Branching programs over large domains Counting versus nondeterminism A surprise: Barrington s theorem Oblivious branching programs Exercises Bounded Replication Read-once programs: no replications P NP co-np for read-once programs Branching programs without null-paths Parity branching programs Linear codes require large replication Expanders require almost maximal replication Exercises Bounded Time The rectangle lemma A lower bound for code functions Proof of the rectangle lemma Exercises Part VI Fragments of Proof Complexity 18 Resolution Resolution refutation proofs Resolution and branching programs Lower bounds for tree-like resolution Tree-like versus regular resolution Lower bounds for general resolution Size versus width Tseitin formulas Expanders force large width Matching principles for graphs Exercises Cutting Plane Proofs Cutting planes as proofs Cutting planes and resolution Lower bounds for tree-like CP proofs Lower bounds for general CP proofs

12 xiv Contents 19.5 Chvátal rank Rank versus depth of CP proofs Lower bounds on Chvátal rank General CP proofs cannot be balanced Integrality gaps Exercises Epilogue Pseudo-random generators Natural proofs The fusion method Indirect proofs A Mathematical Background References Index

13 Part I The Basics

14

15 1. Our Adversary: The Circuit Boolean or switching functions map each sequence of bits to a single bit 0 or 1. Bit 0 is usually interpreted as false, and bit 1 as true. Simplest of such functions are the product x y, sum x y mod 2, non-exclusive Or x y, negation x = 1 x. The central problem of boolean function complexity the lower bounds problem is: Given a boolean function, how many of these simplest operations do we need to compute the function on all input vectors? The difficulty in proving that a given boolean function has high complexity lies in the nature of our adversary: the circuit. Small circuits may work in a counterintuitive fashion, using deep, devious, and fiendishly clever ideas. How can one prove that there is no clever way to quickly compute the function? This is the main issue confronting complexity theorists. The problem lies on the border between mathematics and computer science: lower bounds are of great importance for computer science, but their proofs require techniques from combinatorics, algebra, analysis, and other branches of mathematics. 1.1 Boolean functions We first recall some most basic concepts concerning boolean functions. The name boolean function comes from the boolean logic invented by George Boole ( ), an English mathematician and philosopher. As this logic is now the basis of modern digital computers, Boole is regarded in hindsight as a forefather of the field of computer science. Boolean values (or bits) are numbers 0 and 1. A boolean function f(x) = f(x 1,..., x n ) of n variables is a mapping f : {0, 1} n {0, 1}. One says that f accepts a vector a {0, 1} n if f(a) = 1, and rejects it if f(a) = 0. A boolean function f(x 1,..., x n ) need not to depend on all its variables. One says that f depends on its i-th variable x i if there exist constants 3

16 4 1 Our Adversary: The Circuit a 1,..., a i 1, a i+1,..., a n in {0, 1} such that f(a 1,..., a i 1, 0, a i+1,..., a n ) f(a 1,..., a i 1, 1, a i+1,..., a n ). Since we have 2 n vectors in {0, 1} n, the total number of boolean functions f : {0, 1} n {0, 1} is doubly-exponential in n, is 2 2n. A boolean function f is symmetric if it depends only on the number of ones in the input, and not on positions in which these ones actually reside. We thus have only 2 n+1 such functions of n variables. Examples of symmetric boolean functions are: Threshold functions Th n k(x) = 1 iff x x n k. Majority function Maj n (x) = 1 iff x x n n/2. Parity function n (x) = 1 iff x x n 1 mod 2. Modular functions MOD k = 1 iff x x n 0 mod k. Besides these, there are many other interesting boolean functions. Actually, any property (which may or may not hold) can be encoded as a boolean function. For example, the property to be a prime number corresponds to a boolean function PRIME such that PRIME(x) = 1 iff n i=1 x i2 i 1 is a prime number. It was a long-standing problem whether this function can be computed using a polynomial in n number of elementary boolean operations. This problem was finally solved affirmatively by Agrawal, Kayal and Saxena (2004). To encode properties of graphs on the set of vertices [n] = {1,..., n}, we may associate a boolean variable x ij with each potential edge. Then any 0-1 vector x of length ( n 2) gives us a graph Gx, where two vertices i and j are adjacent iff x ij = 1. We can then define f(x) = 1 iff G x has a particular property. A prominent example of a hard-to-compute graph property is the clique function CLIQUE(n, k): it accepts an input vector x iff the graph G x has a k-clique, that is, a complete subgraph on k vertices. The problem of whether this function can also be computed using a polynomial number of operations remains wide open. A negative answer would immediately imply that P NP. Informally, the P vs. NP problem asks whether there exist mathematical theorems whose proofs are much harder to find than verify. Roughly speaking, one of the goals of circuit complexity is, for example, to understand why the first of the following two problems is easy whereas the second is (apparently) very hard to solve: 1. Does a given graph contain at least ( k 2) edges? 2. Does a given graph contain a clique with ( k 2) edges? The first problem is a threshold function, whereas the second is the clique function CLIQUE(n, k). We stress that the goal of circuit complexity is not just to give an evidence (via some indirect argument) that clique is much harder than majority, but to understand why this is so. A boolean matrix or a 0-1 matrix is a matrix whose entries are 0s and 1s. If f(x, y) is a boolean function of 2n variables, then it can be viewed as a

17 1.1 Boolean functions 5 x y x y x y x y x y x x Fig. 1.1 Truth tables of basic boolean operations. boolean 2 n 2 n matrix A whose rows and columns are labeled by vector in {0, 1} n, and A[x, y] = f(x, y). We can obtain new boolean functions (or matrices) by applying boolean operations to the simplest ones. Basic boolean operations are: NOT (negation) x = 1 x; also denoted as x. AND (conjunction) x y = x y. OR (disjunction) x y = 1 (1 x)(1 y). XOR (parity) x y = x(1 y) + y(1 x) = (x + y) mod 2. Implication x y = x y. If these operators are applied to boolean vectors or boolean matrices, then they are usually performed componentwise. Negation acts on ANDs and ORs via DeMorgan rules: (x y) = x y and (x y) = x y. The operations AND and OR themselves enjoy the distributivity rules: x (y z) = (x y) (x z) and x (y z) = (x y) (x z). Binary cube The set {0, 1} n of all boolean (or binary) vectors is usually called the binary n-cube. A subcube of dimension d is a set of the form A = A 1 A 2 A n, where each A i is one of three sets {0}, {1} and {0, 1}, and where A i = {0, 1} for exactly d of the i s. Note that each subcube of dimension d can be uniquely specified by a vector a {0, 1, } n with d stars, by letting to attain any of two values 0 and 1. For example, a subcube A = {0} {0, 1} {1} {0, 1} of the binary 4-cube of dimension d = 2 is specified by a = (0,, 1, ). Usually, the binary n-cube is considered as a graph Q n whose vertices are vectors in {0, 1} n, and two vectors are adjacent iff they differ in exactly one position (see Fig. 1.2). This graph is sometimes called the n-dimensional binary hypercube. This is a regular graph of degree n with 2 n vertices and n2 n 1 edges. Moreover, the graph is bipartite: we can put all vectors with an odd number of ones on one side, and the rest on the other; no edge of Q n can join two vectors on the same side. Every boolean function f : {0, 1} n {0, 1} is just a coloring of vertices of Q n in two colors. The bipartite subgraph G f of Q n, obtained by removing

18 6 1 Our Adversary: The Circuit Fig. 1.2 The 3-cube and its Hasse-type representation (each level contains binary strings with the same number of 1s). There is an edge between two strings if and only if they differ in exactly one position all edges joining the vertices in the same color class, accumulates useful information about the circuit complexity of f. If, for example, d a denotes the average degree in G f of vertices in the color-class f 1 (a), a = 0, 1, then the product d 0 d 1 is a lower bound on the length of any formula expressing f using connectives, and (see Khrapchenko s theorem in Section 6.8). CNFs and DNFs A trivial way to represent a boolean function f(x 1,..., x n ) is to give the entire truth table, that is, to list all 2 n pairs (a, f(a)) for a {0, 1} n. More compact representations are obtained by giving a covering of f 1 (0) or of f 1 (1) by not necessarily disjoint subsets, each of which has some simple structure. This leads to the notions of CNFs and DNFs. A literal is a boolean variable or its negation. For literals the following notation is often used: x 1 i stands for x i, and x 0 i stands for x i = 1 x i. Thus, for every binary string a = (a 1,..., a n ) in {0, 1} n, x 1 i (a) = { 1 if a i = 1 0 if a i = 0 and x 0 i (a) = { 0 if a i = 1 1 if a i = 0. A monomial is an AND of literals, and a clause is an OR of literals. A monomial (or clause) is consistent if it does not contain a contradicting pair of literals x i and x i of the same variable. We will often view monomials and clauses as sets of literals. It is not difficult to see that the set of all vectors accepted by a monomial consisting of k (out of n) literals forms a binary n-cube of dimension n k (so many bits are not specified). For example, a monomial x 1 x 3 defines the cube of dimension n 2 specified by a = (0,, 1,,..., ). Similarly, the set of all vectors rejected by a clause consisting of k (out of n) literals also forms a binary n-cube of dimension n k. For example, a clause x 1 x 3 rejects a vector a iff a 1 = 1 and a 3 = 0. A DNF (disjunctive normal form) is an OR of monomials, and a CNF (conjunctive normal form) is an AND of clauses. Every boolean function f(x) of n variables can be written both as a DNF D(x) and as a CNF C(x):

19 1.1 Boolean functions 7 D(x) = n a:f(a)=1 i=1 x ai i C(x) = n b:f(b)=0 i=1 x 1 bi i. Indeed, D(x) accepts a vector x iff x coincides with at least one vector a accepted by f, and C(x) rejects a vector x iff x coincides with at least one vector b rejected by f. A DNF is a k-dnf if each of its monomials has at most k literals; similarly, a CNF is a k-cnf if each of its clauses has at most k literals. DNFs (and CNFs) are the simplest models for computing boolean functions. The size of a DNF is the total number of monomials in it. It is clear that every boolean function of n variables can be represented by a DNF of size at most f 1 (1) 2 n : just take one monomial for each accepted vector. This can also be seen via the following recurrence: f(x 1,..., x n+1 ) = x n+1 f(x 1,..., x n, 1) x n+1 f(x 1,..., x n, 0). (1.1) It is not difficult to see that some functions require DNFs of exponential size. Take, for example, the parity function f(x 1,..., x n ) = x 1 x 2 x n. This function accepts an input vector iff the number of 1s in it is odd. Every monomial in a DNF for f must contain n literals, for otherwise the DNF would be forced to accept a vector in f 1 (0). Since any such monomial can accept only one vector, f 1 (1) = 2 n 1 monomials are necessary. Thus the lower bounds problem for this model is trivial. Boolean functions as set systems By identifying subsets S of [n] = {1,..., n} with their characteristic 0-1 vectors v S, where v S (i) = 1 iff i S, we can consider boolean functions as set-theoretic predicates f : 2 [n] {0, 1}. We will often go back and forth between these notations. One can identify a boolean function f : 2 [n] {0, 1} with the family F f = {S f(s) = 1} of subsets of [n]. That is, there is a 1-to-1 correspondence between boolean functions and families of subsets of [n]: boolean functions of n variables = families of subsets of {1,..., n}. Minterms and maxterms A 1-term (resp., 0-term) of a boolean function is a smallest subset of its variables such that the function can be made the constant 1 (resp., constant 0) function by fixing these variables to constants 0 and 1 is some way. Thus after the setting, the obtained function does not depend on the remaining variables. Minterms (maxterms) are 1-terms (0-terms) which are minimal under the set-theoretic inclusion. Note that one and the same set of variables may be a 1-term and a 0-term at the same time. If, for example, f(x 1, x 2, x 3 ) = 1 iff x 1 + x 2 + x 3 2, then {x 1, x 2 } is a 1-term of f because f(1, 1, x 3 ) 1, and is a 0-term of f because f(0, 0, x 3 ) 0. If all minterms of a boolean function f have length at most k then f can be written as a k-dnf: just take the OR of all these minterms. But the

20 8 1 Our Adversary: The Circuit converse does not hold! Namely, there are boolean functions f such that f can be written as a k-dnf even though some of its minterms are much longer than k (see Exercise 1.7). Duality The dual of a boolean function f(x 1,..., x n ) is the boolean function f defined by: f (x 1,..., x n ) := f( x 1,..., x n ). For example, if f = x y then f = ( x y) = x y. The dual of every threshold function Th n k (x) is the threshold function Thn n k+1 (x). A function f is self-dual if f (x) = f(x) holds for all x {0, 1} n. For example, the threshold-k function f(x) = Th 2k 1 k (x) of 2k 1 variables is self-dual. Hence, if the number n of variables is odd, then the majority function Maj n is also self-dual. In set-theoretic terms, if S = [n] \ S denotes the complement of S, then the values of the dual of f are obtained by: f (S) = 1 f(s). Thus a boolean function f is self-dual if and only if f(s) + f(s) = 1 for all S [n]. Monotone functions For two vectors x, y {0, 1} n we write x y if x i y i for all positions i. A boolean function f(x) is monotone, if x y implies f(x) f(y). If we view f as a set-theoretic predicate f : 2 [n] {0, 1}, then f is monotone iff f(s) = 1 and S T implies f(t ) = 1. Examples of monotone boolean functions are AND, OR, threshold functions Th n k(x), clique functions CLIQUE(n, k), etc. On the other hand, such functions as the parity function n (x) or counting functions Mod n k(x) are not monotone. Monotone functions have many nice properties not shared by other functions. First of all, their minterms as well as maxterms are just subsets of variables (no negated variable occurs in them). In set-theoretic terms, a subset S [n] is a minterm of a monotone function f if and is a maxterm of f if f(s) = 1 but f(s \ {i}) = 0 for all i S, f(s) = 0 but f(s \ {i}) = 1 for all i S. Let Min(f) and Max(f) denote the set of all minterms and the set of all maxterms of f. Then we have the following cross-intersection property: S T for all S Min(f) and all T Max(f). Indeed, if S and T were disjoint, then for the vectors x with x i = 1 for all i S, and x i = 0 for all i S, we would have f(x) = 1 (because S is a minterm) and at the same time f(x) = 0 (because T S is a maxterm of f). The next important property of monotone boolean functions is that every such function f has a unique representation as a DNF as well as a CNF: