Polynomial threshold functions and Boolean threshold circuits

Transcription

1 Electronic Colloquium on Computational Complexity, Report No. 21 (2013) Polynomial threshold functions and Boolean threshold circuits Kristoffer Arnsfelt Hansen 1 and Vladimir V. Podolskii 2 1 Aarhus University, arnsfelt@cs.au.dk 2 Steklov Mathematical Institute, podolskii@mi.ras.ru February 5, 2013 Abstract We study the complexity of computing Boolean functions on general Boolean domains by polynomial threshold functions (PTFs). A typical example of a general Boolean domain is {1, 2} n. We are mainly interested in the length (the number of monomials) of PTFs, with their degree and weight being of secondary interest. We show that PTFs on general Boolean domains are tightly connected to depth two threshold circuits. Our main results in regard to this connection are: PTFs of polynomial length and polynomial degree compute exactly the functions computed by THR MAJ circuits. An exponential length lower bound for PTFs that holds regardless of degree, thereby extending known lower bounds for THR MAJ circuits. We generalize two-party unbounded error communication complexity to the multi-party number-on-the-forehead setting, and show that communication lower bounds for 3-player protocols would yield size lower bounds for THR THR circuits. We obtain several other results about PTFs. These include relationships between weight and degree of PTFs, and a degree lower bound for PTFs of constant length. We also consider a variant of PTFs over the max-plus algebra. We show that they are connected to PTFs over general domains and to AC 0 THR circuits. ISSN

2 1 Introduction Let f : X n { 1, 1} be a Boolean function on a domain X R n. We say that a real n-variate polynomial P is a polynomial threshold function (PTF) computing f if for all x X it holds that f(x) = sgn(p (x)). Other terminology has been used in the literature for the same notion. We may alternatively say that P is a sign representation of f, or that P is a voting polynomial computing f. Polynomial threshold function have been studied intensively for decades. Much of this work was motivated by questions in computer science [27], and PTFs are now an important object of study in areas such as Boolean circuit complexity [4, 11, 2, 19], learning theory [16, 17], and communication complexity [28]. The main motivation of this paper is Boolean circuit complexity. A major and long-standing open problem is to obtain an explicit super-polynomial lower bound for depth two threshold circuits. A long line of research have established lower bounds for several subclasses of depth two threshold circuits. The largest subclass for which super-polynomial lower bounds are known is the class THR MAJ of depth two threshold circuits, where all gates except the output gate is required to compute threshold functions with polynomially bounded weights [9]. We shall see that PTFs on general Boolean domains are tightly connected to both these classes of circuits. For a PTF P we will be interested in the several measures of complexity. The length of P, denoted by len(p ), is the number of monomials of P. The degree of P, denoted by deg(p ), is the usual total degree of P. Finally, note that in the case that X is a finite domain, without loss of generality one may assume that the coefficients of P are integers, and can thus speak of the weight of P, meaning the largest magnitude of a coefficient of P. We restrict our focus to the case of computing Boolean functions with Boolean inputs. More precisely we only consider the case when the domain X is a Boolean n-cube, X = {a, b} n, for distinct a, b R. Such sign representations of Boolean functions have been studied intensively due to their fundamental nature and vast number of applications. This research has almost exclusively focused on the two Boolean n-cubes, {0, 1} n and { 1, 1} n, sometimes denoted as the standard basis and the Fourier basis, respectively. Indeed, most often the notion of PTFs is defined specifically for the case of the domain { 1, 1} n. This choice is, however, of little consequence when one disregards the length as a parameter and focuses on the degree, as is the case in many applications of PTFs. Note also that for these two domains any PTF can without loss of generality be assumed to be multilinear, meaning that all variables have individual degree at most 1. Focusing on the length of a PTF rather than the degree, the choice of domain becomes crucial already for the case of the two domains {0, 1} n and { 1, 1} n. This was studied in depth by Krause and Pudlák [19]. Minksy and Papert [20] has shown that the parity function requires exponential length over the domain {0, 1} n (cf. [10, 1]), whereas it can be computed by a PTF of length 1 over the domain { 1, 1} n. Conversely, Krause and Pudlák construct a PTF on domain {0, 1} n of length n that require length 2 nω(1) on domain { 1, 1} n. For this construction, large weight is crucial. Indeed, Krause and Pudlák also show that any function computed by a polynomial length and polynomial weight PTF on the domain {0, 1} n can also be computed by a polynomial length and polynomial weight PTF on the domain { 1, 1} n. A notable exception to the focus on the domains {0, 1} n and { 1, 1} n is the work of Basu et al. [1] that consider representing the parity function (or rather, a natural generalization of the parity function) on domains of the form X = A n, for a set A Z. They especially focus on the cases 1

3 A = {0, 1,..., m} and A = {1, 2,..., m}, where m 2. It is important to note that on most Boolean domains {a, b} n, it is not without loss of generality to assume that polynomials are multilinear. One may easily convert a given PTF into a multilinear PTF computing the same function, but such a conversion may change both the length as well as the weight significantly. Indeed, Basu et al. show that the parity function provides such an example. Namely they show that on the domain {1, 2} n there is a PTF of length n+1 and degree n 2 computing the parity function, whereas any multilinear PTF computing the same function must have length 2 n. Thus evaluating PTFs on general Boolean domains has the effect that allowing high degree (meaning polynomial, exponential, or perhaps even higher), may help to greatly reduce the length needed to compute a given Boolean function. In this paper our aim is to investigate in detail the computational power of PTFs of polynomial length over a general Boolean domain of the form {a, b} n, a b. Some of these domains essentially corresponds to the two usual domains {0, 1} n and { 1, 1} n, namely those that are simple scalings {0, a} n and {a, a} n, and we shall hence not consider these further. In particular we shall by general Boolean domains, refer to any other Boolean domain {a, b}. For most of our results it turns out the precise choice of general domain does not matter (in fact all our results hold when sgn(a) = sgn(b)), and we shall henceforth develop our results in terms of the domain {1, 2} n. 1.1 Our results Over the usual Boolean domains {0, 1} n and { 1, 1} n PTFs are basic extensions of linear threshold functions that are still very limited in expressive power. Indeed, in these cases PTFs require exponential length to compute simple functions such as symmetric Boolean functions [4, 18]. Over a general Boolean domain the situation changes drastically. We show that in this case PTFs of just constant length can actually compute interesting classes of functions (see Proposition 1 and Proposition 4). More importantly, when moving to polynomial length PTFs obtain computational power right at the frontier of known circuit lower bounds for threshold circuits. Namely we show in Theorem 5 that PTFs of polynomial length and polynomial degree compute exactly the functions computed by polynomial size THR MAJ circuits. This circuit class is the largest depth two threshold circuit class for which superpolynomial lower bounds are known. These lower bounds were obtained by sign rank lower bounds of matrices, or equivalently lower bounds for unbounded error communication complexity [9], and this is still the only lower bound method known for this class of circuits. In Section 3.2 we show that this lower bound method applies to PTFs, even with no degree restriction. We tend to believe that allowing exponential or perhaps even larger degree allows for more Boolean functions to be computed by PTFs, and we relate this in Proposition 10 to a question about simulating large weights by small weights in threshold circuits in a very strong way. This in turn also gives an indication that the power of the sign rank lower bound method extends beyond THR MAJ circuits. Let us also note in the passing that polynomial length PTFs over domains {0, 1} n and { 1, 1} n also correspond to circuit classes, namely the subclass of THR AND circuits with no negations of inputs in the case of domain {0, 1} n and THR XOR circuits in the case of domain { 1, 1} n. These are thus strictly less powerful than PTFs of polynomial length and polynomial degree over a general Boolean domain. Our study of PTFs on general Boolean domains leads also to a possible way to approach the major open problem of proving lower bounds for THR THR circuits. Just as is the case of THR MAJ circuits, most lower bounds for classes of threshold circuits have been obtained using various models of communication complexity [12, 15, 11, 23, 9, 29, 26]. In Section 3.3 and Section 6 we generalize the notion of sign complexity to higher order tensors and unbounded error communication complexity 2

4 to the multiparty number-on-the-forehead setting, and we show that sufficiently good lower bounds for these (for order 3 tensors, or equivalently, 3-party communication protocols) would yield circuit lower bounds for THR THR circuits. An important technical ingredient in this connection is a previous result showing that the threshold gates at the second level can be exchanged with exact threshold gates [14]. While we currently know no lower bounds for this communication model 1, we feel this relation is significant, given the previous successes of communication complexity for lower bounds for threshold circuit classes, and deserves further study. Multi-party communication complexity have been used earlier for threshold circuit classes, but in the bounded error setting. In particular, lower bounds have been obtained for depth 3 unweighted threshold circuits with small bottom fanin [15]. In the unbounded error setting we can additionally address depth 3 weighted threshold circuits with small bottom fanin. The above relations between PTFs on general Boolean domains and threshold circuits further motivate an in-depth study of PTFs, besides them being a fundamental way to represent Boolean functions. For instance, it is tempting to conjecture that PTFs can only compute functions computable by constant depth threshold circuits. It seems that before such questions can be addressed, one needs more insight into PTFs. We currently don t know how large PTF degree can be useful for computation. In Section 4 we show that the minimal degree of a PTF within a given length bound can be bounded in terms of its integer weights, and conversely the integer weights can be bounded in terms of its degree. These bounds are obtained by setting up suitable linear programs and integer linear programs, where the variables are exponents or weights respectively, and then using known bounds on feasible basic solutions and small integer feasible solutions. In addition to PTFs on general domains we also consider a max-plus version of PTFs. The maxplus algebra works over the max-plus semiring, which is the set of integers with the max operation playing the role of addition and the usual addition playing the role of multiplication [30, 6]. This setting arises as a limit case in several areas of mathematics and turns out to be helpful. In our case it turns out that max-plus PTFs are connected with PTFs over the the general domains and are moreover connected to the hierarchy of AC 0 THR circuits. Finally we study the relations between PTFs over different general domains. Though we are unable to completely resolve the questions arising here, we still can prove some nontrivial relations. For example, we show that PTFs over domains {1, 2} n and PTFs over {1, 2} n are essentially equivalent. 2 Preliminaries 2.1 Polynomial threshold functions For given length bound l(n) and degree bound d(n), we let PTF a,b (l(n), d(n)) denote the class of Boolean functions on domain {a, b} n computed by polynomial threshold functions of length l(n) and degree d(n). That is f PTF a,b (l(n), d(n)) if and only if there is a polynomial p(x) Z[x] with l(n) monomials of degree at most d(n) and such that for all x {a, b} n we have f(x) = 1 if and only if p(x) 0. Of particular interest is the case when l(n) is a polynomial in n. For this reason we will abbreviate PTF a,b (poly(n), d(n)) by PTF a,b (d(n)). If we do not wish to impose a degree bound we write this as PTF a,b (l(n), ) and PTF a,b ( ), respectively. As mentioned in the 1 Unlike the case of bounded error communication complexity, even obtaining non-explicit lower bounds pose a challenge, since counting arguments fail [24]. 3

5 introduction we state our results in terms of the specific domain {1, 2} n. We remark that in most of our results one may replace {1, 2} be any other domain {a, b}, where a b, and a, b 0. The exceptions to this are our results about PTFs of constant length 2, namely Propositions 1, 4, and 16 as well as Theorem 17. These results hold instead assuming sgn(a) = sgn(b). See Section 7 for further discussion of differences and equivalences between domains. 2.2 Exponential form of PTFs We shall find it convenient to switch back to the standard domain {0, 1} n even when considering PTFs over the domain {1, 2} n. Given variables y 1,..., y n {1, 2}, we define corresponding variables x 1,..., x n {0, 1} by x i = log 2 (y i ). Correspondingly we have y i = 2 x i. Under this change of variables monomials turn into exponential functions, y a yan n = 2 a 1x a nx n and more generally a polynomial P (y) = l j=1 c j, turns into a weighted sum of exponential functions: n i=1 ya ij i P (y) = l j=1 c j 2 n i=1 a ijx i, where a ij 0 are the non-negative integer exponents of the polynomial. Rewriting a PTF in this way, we shall say it is in exponential form. We shall in general allow also for negative integer coefficients a ij in the exponents. By simply multiplying the entire expression with the term 2 n i=1 b ix i, where b i = max(0, max j a ij ) we can make all the coefficients to be positive. This in turn requires us to redefine the degree of the polynomial to be max j ( n i=1 max(0, a(j) i ))+max j ( n i=1 max(0, a(j) i )). At times it may also be convenient to move the absolute value of the coefficients c j to the exponents as an additive term log 2 ( c j ) in order to have all coefficients of the exponential form be of magnitude Boolean functions and Circuit classes We give here briefly for the most part standard definitions of Boolean functions and circuit classes. As is usual, when considering a Boolean function f : {0, 1} n {0, 1}, unless otherwise specified we always have a family of such functions in mind, one for each input length. The functions of most importance for this work are linear threshold functions defined by linear inequalities and their relatives defined by linear equations [14]. Let x 1,..., x n {0, 1} be Boolean variables. For w = (w 1,..., w n ) R n and t R, we define the function THR w,t by THR w,t (x) = 1 if and only if n i=1 w ix i t. Similarly we define the function ETHR w,t by ETHR w,t (x) = 1 if and only if n i=1 w ix i = t. We call w the weights and t the threshold. The case when all weights are 1 and the threshold is n/2 is of special interest. Define the function MAJ by MAJ(x) = 1 if and only if n i=1 x i n/2 and the function EMAJ by EMAJ(x) = 1 if and only if n i=1 x i = n/2. Besides these functions we consider the usual AND, OR, and XOR functions of n Boolean variables. Let THR and ETHR denote the class of THR w,t and ETHR w,t functions for all w and t. Let MAJ, EMAJ, AND, OR, XOR denote the class of all MAJ, EMAJ, AND, OR and XOR functions. We may by adding a subscript to these functions and classes denote the number of inputs to them, e.g. by AND k we denote the Boolean AND function of k Boolean variables. By ANY we denote the class 2 Note that the XOR function can be computed by a length 1 PTF over the domain {1, 2} but not over the domain {1, 2}. 4

6 of all Boolean functions. This latter class will only be used with a subscript specifying a constant number of inputs, and is hence in this case equivalent to constant sized Boolean combinations. From these Boolean functions we define unbounded fan-in Boolean circuits. Inputs are allowed to be Boolean variables or their negation, as well as the Boolean constants 0 and 1. As with Boolean functions we always have a family of Boolean circuits in mind, one for each input length n. The size of a Boolean circuit is the number of wires. Unless otherwise specified, we consider circuit families of polynomial size. A class of Boolean functions immediately define a class of Boolean circuits as families of single gate circuits. Given two classes of circuits C 1 and C 2 we denote by C 1 C 2 the class of circuits consisting of circuits from C 1 that is fed as inputs the output of circuits from C 2. 3 PTFs and threshold circuits 3.1 Circuit characterizations We first note that PTFs already of length 2 on domain {1, 2} n can compute the class of linear threshold functions, and this is in fact an exact characterization. Furthermore, the case of polynomial degree corresponds to the case of polynomial weights. Proposition 1. PTF 1,2 (2, ) = THR and PTF 1,2 (2, poly(n)) = MAJ. Proof. Consider first the linear threshold function THR w,t given by the linear inequality n i=1 w ix i t 0. By raising each side of the inequality to the power of 2 immediately results in the exponential form of the desired length 2 PTF. Conversely, consider a length 2 PTF defined by the exponential form inequality c 1 2 n i=1 a ix i + c 2 2 n i=1 b ix i 0. If sgn(c 1 ) = sgn(c 2 ) the Boolean function is constant, and hence is trivially a linear threshold function. So assume sgn(c 1 ) sgn(c 2 ). Without loss of generality let c 1 > 0 and c 2 < 0. The inequality is then equivalent to the linear inequality n i=1 (a i b i )x i + log 2 c 1 log 2 ( c 2 ) 0, defining a linear threshold function. The specialization to polynomial degree of the PTF and polynomial weights of the linear threshold function follows immediately. We can generalize one direction of Proposition 1 as follows. Lemma 2. Any function computed by an AND k THR circuit or an OR k THR circuit can be computed by a PTF on domain {1, 2} n of length k + 1. Similarly, any function computed by an AND k MAJ circuit or an OR k MAJ circuit can be computed by a PTF on domain {1, 2} n of length k + 1 and degree poly(n). Proof. We prove the case of OR k THR circuits. The case of AND k THR circuits then follow by using De Morgan s rules. Let the k threshold functions be given by the sign of the linear expressions l 1 (x),..., l k (x), all with integer coefficients. Let c = log 2 k + 2, and define new linear expressions l1 (x),..., l k (x) by l i (x) = c(l i (x) + 1). We can then define the PTF by the exponential form E(x) = 2 l 1 (x) l k (x) 2k. In case there exist i such that l i (x) 0, then l i (x) c, and hence E(x) 2 c 2k > 0. In case for all i we have l i (x) < 0, then we also have l i (x) 0 for all i, and hence E(x) k 2k < 0. Again the specialization to polynomial degree of the PTF and polynomial weights of the linear threshold functions follows immediately. 5

7 A basic closure property of PTFs, being sign representations, is that the parity of PTFs is also a PTF. Lemma 3. If the Boolean functions f 1,..., f k are computed by PTFs on domain {1, 2} n of length s, then XOR(f 1,..., f k ) is computed by a PTF on domain {1, 2} n of length s k. Furthermore, if each of the k given PTFs are of degree at most d, the resulting PTF is of degree at most kd. Proof. If E 1 (x),..., E k (x) are the exponential forms of the PTFs computing the Boolean functions f 1,..., f k then k j=1 ( E j(x)) is an exponential form of a PTF computing the function XOR(f 1 (x),..., f k (x)). Remark. We note that classes THR THR, MAJ MAJ and so on are also closed under the parity function of constant fan-in. To see this, one can switch to ETHR and EMAJ gates on the bottom level and use the fact that they are closed under AND k using results of [14]. We can now characterize the Boolean functions computed by constant length PTFs on domain {1, 2} n as the class of constant size Boolean combinations of linear threshold functions. This class of functions was considered earlier in the setting of learning in [16]. Proposition 4. PTF 1,2 (O(1), ) = ANY O(1) THR, and PTF 1,2 (O(1), poly(n)) = ANY O(1) MAJ Proof. Consider a function g(f 1 (x),..., f k (x)), where each f i is a linear threshold function. First express g(y) as a k-variate polynomial p(y) over Z 2. By Lemma 2, the function of x defined by each monomial of p can be computed by a PTF of length k + 1. Using Lemma 3 we may now express the sum over Z 2 of these at most 2 k functions by a PTF of length (k + 1) 2k. Conversely, we will show how to evaluate the sign of a PTF of length k by a constant size Boolean combination of linear threshold functions. We will describe this evaluation by a decision tree, where each decision node is determined by a linear threshold function. Consider the PTF in its exponential form s 1 2 l 1(x) + + s k 2 l k(x) where s i { 1, 1}, and the coefficients of every l i, except possibly for the constant term, are integer. Define c = log 2 k, and consider an input x. First we determine using a decision tree of depth O(k), indices i 1 and i 2 such that l i1 (x) l i2 (x) l j (x) for all j / {i 1, i 2 }. In case l i1 (x) c + l i2 (x) the sign of E(x) is precisely s i1. Otherwise we have that l i1 (x) = d + l i2 for some number 0 d < c. Note that there are at most c such possible d, since the coefficients of l i1 (x) and l i2 (x) are, except possibly for the constant term, integer. We may consider each of these at most c cases separately after using a depth log 2 c decision tree. For such a fixed d we may now rewrite the exponential form as (s i1 2 d + s i2 )2 l i (x) 2 + s j 2 lj(x) = s2 l(x) + s j 2 l j(x) j / {i 1,i 2 } j / {i 1,i 2 } for appropriate s and l, depending on s i1, s i2, and l i2, but otherwise independent of the specific x. Since this is an expression of the same type, but of length k 1 we may continue as above. Doing this we obtain a decision tree of depth O(k 2 ) that determines the sign of the expression. This gives in turn a Boolean combination of 2 O(k2) linear threshold functions. Again, the specialization to polynomial degree of the PTF and polynomial weights of the linear threshold functions follows immediately. 6

8 In the case of polynomial degree we can characterize the Boolean functions computed by polynomial length PTFs on domain {1, 2} n as a class of depth 2 threshold functions with polynomially bounded weights on the bottom level. Theorem 5. PTF 1,2 (poly(n)) = THR MAJ Proof. We first construct a PTF given a THR MAJ circuit. Suppose that the output threshold gate is given by the inequality ( s k=1 w ky k ) t 0. Let l 1 (x),..., l s (x) be the linear expressions defining the s majority gates with integer coefficients. Let p(n) be a polynomial such that l k (x) p(n) for all x {0, 1} n and all k. Let m = 2p(n) + 1 and define the m m matrix A = (a ij ) by a ij = 2 (i p(n) 1)(j 1) for i, j = 1,..., m. Note that A is a Vandermonde matrix with distinct rows, and hence A is invertible. Define u = (0,..., 0, 1,..., 1) }{{}}{{} T, and let v = A 1 u. We now define PTFs by the exponential forms p(n) p(n)+1 E 1 (x),..., E s (x) given by m E k (x) = v j 2 (j 1)lk(x). (1) j=1 By construction we have E k (x) = u lk (x)+p(n)+1. In other words, whenever l k (x) < 0 we have E k (x) = 0 and whenever l k (x) 0 we have E k (x) = 1. We then obtain a PTF for the entire circuit by the exponential form E(x) = ( s k=1 w ke k (x)) t. Conversely consider a PTF given by its exponential form E(x) = s k=1 c k2 lk(x), where the coefficients of l k (x) are positive integers of polynomial magnitude. Thus there is a polynomial p(n) such that 0 l k (x) p(n) for all x {0, 1} n and all k. We now construct a THR EMAJ circuit is follows. For every k {1,..., s} and for every j {0,..., p(n)}, we take an EMAJ gate deciding whether l(x) = j, and then feed the output of this gate into the output THR gate with weight c k 2 j. This THR EMAJ circuit is then easily converted into a THR MAJ circuit [14]. As a byproduct of this result we get the following interesting structural result about depth 2 threshold circuits. Lemma 6. Any polynomial size circuit in THR MAJ is equivalent to a polynomial size circuit of the same form such that all majority gates on the bottom level are monotone. Proof. Starting with THR MAJ circuit we first switch to a PTF in exponential form, then by multiplying the whole expression by a large enough term 2 c i x i we make all coefficients in the exponents positive and then we switch back to THR MAJ circuit. Note that the proof of Theorem 5 gives the circuit with monotone bottom level. The previous lemma can be actually proved directly by the same techniques as in the proof of Theorem 5. This direct proof actually gives also the analogous result for MAJ MAJ circuits. We present this in Appendix A. 3.2 Lower bounds for PTFs The sign rank of a real matrix A = (a ij ) with nonzero entries is the minimum possible rank of a real matrix B = (b ij ) of same dimensions as A satisfying sgn(a ij ) = sgn(b ij ) for all i, j. We are interested 7

9 in the sign rank of matrices defined from Boolean functions. Let f : {0, 1} n {0, 1} n { 1, 1} be a Boolean function of 2n bits partitioned in two blocks each of n bits. We associate with f a 2 n 2 n matrix M f, the communication matrix, indexed by x, y {0, 1} n and defined by (M f ) x,y = f(x, y). Lemma 7. Assume f : {0, 1} n {0, 1} n { 1, 1} is computed by a PTF on domain {1, 2} n {1, 2} n of length s. Then the matrix M f has sign rank at most s. Proof. Consider the PTF computing f in exponential form E(x, y) = s j=1 c j2 lj(x,y). For each j, define the 2 n 2 n matrix B j, indexed by x, y {0, 1} n, defined by (B j ) x,y = 2 lj(x,y). From this definition we immediately have that the matrix B = s j=1 c jb j is a sign representation of M f. Now note that we can write l j (x, y) = l (1) j (x) + l (2) j (y). Hence B j is an outer product, B j = b (1) j b (2) j where b (1) j, b (2) j {0, 1} n are defined by (b (1) j ) x = 2 l(1) j (x) and (b (2) j ) y = 2 l(2) j (y). It follows that B j is of rank at most 1, and hence rank(b) s j=1 rank(b j) s. Thus lower bounds on the sign rank of communication matrices of Boolean functions directly implies length lower bounds for PTFs on domain {1, 2} n not depending on the degree and weights. Strong lower bounds are now known for several of Boolean functions. We mention two of particular interest. Forster [8] proved that the sign rank of the 2 n 2 n matrix corresponding to the inner product mod 2 function, IP 2 (x, y), has sign rank 2 n 2. Razborov and Sherstov [26] proved that the sign rank of the 2 m3 2 m3 matrix corresponding to the Boolean function f m (x, y) = m i=1 y ij ) is 2 Ω(m). Combining these results with Lemma 7 we have the following. T, m 2 j=1 (x ij Corollary 8. Any PTF on domain {1, 2} n {1, 2} n computing IP 2 requires length 2 n 2. Any PTF on domain {1, 2} m3 {1, 2} m3 computing f m requires length 2 Ω(m). Sign rank was previously used to give the first lower bounds for THR MAJ circuits and sign rank remains the only known method for obtaining such lower bounds. Since PTFs can compute all functions computed by THR MAJ circuits already with polynomial degree by Theorem 5, Corollary 8 indicates that the lower bound technique of sign rank is applicable to more general models of computation. Showing that these models are indeed stronger would require a different lower bound method for THR MAJ circuits. Instead we will relate the question whether PTFs with no degree restrictions are more expressive than PTFs of polynomial degree to a question about threshold circuits. For this we will need the following lemma. Lemma 9. For any s we have THR s ETHR PTF 1,2 (s + 1, ) AND 2. In particular, THR THR PTF 1,2 ( ) AND 2. Proof. From [14] we have THR THR = THR ETHR, so the second statement follows from the first. Consider a THR s ETHR circuit C with ETHR gates g 1,..., g s defined by integer linear expressions l 1 (x),..., l s (x), and suppose the output THR gate is given by sgn( s j=1 w jy j t), where w j 0 for all j. Let 0 < m min y {0,1} s s j=1 w jy j t be such that m/(2s w j ) < 1 for all j, and let also m j = min {x {0,1} n l j (x) 0} l j (x) 2. Define polynomials p 1,..., p s by p j (x) = log 2 (m/(2s w j ))/m j l j (x) 2. Then when l j (x) = 0 we have 2 pj(x) = 1 and when l j (x) 0 we have 8

10 2 pj(x) m/(2s w j ). Let x {0, 1} n and define y {0, 1} s be y j = 1 if and only if l j (x) = 0. Then we have y j 2 pj(x) m/2s w j and further s s w j y j w j 2 p j(x) m 2. j=1 j=1 from which it follows that the sign of the expression s j=1 w j2 p j(x) t corresponds to the output of the circuit. On the other hand, it is easy to see that after opening the brackets in the exponents this expression corresponds to a PTF 1,2 (s + 1, ) AND 2 circuit. Proposition 10. PTF 1,2 (poly(n)) PTF 1,2 ( ) unless THR THR THR MAJ AND 2. Proof. Assume PTF 1,2 (poly(n)) = PTF 1,2 ( ). Then THR THR PTF 1,2 ( ) AND 2 = PTF 1,2 (poly(n)) AND 2 = THR MAJ AND 2, where the first inclusion follows from Lemma 9 and the last equality follows from Theorem 5. We tend to consider the inclusion THR THR THR MAJ AND 2 as being unlikely to hold. Note that this would also mean THR THR AND = THR MAJ AND. 3.3 Sign complexity of tensors and depth 2 threshold circuits In this section we define the notion of sign complexity of an arbitrary order tensor, generalizing sign rank of matrices. The definition is made with the aim of capturing the notion of k-party unbounded error communication complexity given in Section 6. For simplicity we give the definition for the special case of order 3 tensors. The extension to tensors of any order k is direct. Let A = (a ijk ) be an order 3 tensor. We say that A is a cylinder tensor if there is an order 2 tensor A = (a ij ) such either a ijk = a jk, for all i, j, k, a ijk = a ik for all i, j, k, or a ijk = a ij, for all i, j, k. In other words an order 3 tensor is a cylinder tensor if there are two indices such that every entry depends only on the value of these two indices. An order 3 tensor A is a cylinder product if it can be written as a Hadamard product A 1 A 2 A 3 where A 1,A 2, and A 3 are cylinder tensors. That is, a ijk = a (1) jk a(2) ik a(3) ij, for all i, j, k, where A 1 = (a (1) jk ),A 2 = (a (2) ik ), A 3 = (a (3) ij ). The sign complexity of an order 3 tensor A = (a ijk ) is the minimum r such ( that there exist ) cylinder product tensors B 1,..., B r, with B l = (b (l) ijk ), such that sgn(a ijk) = sgn b (1) ijk + + b(r) ijk, for all i, j, k. The uniform sign complexity is defined in the same way, except we require that each cylinder product tensor B 1,..., B r is a Hadamard product of cylinder tensors in which all entries have the same sign. Remark. Recalling that the rank of a matrix A is also the minimum number r such that A can be written as a sum of r matrices of rank 1, we see that the sign complexity of a tensor specializes to sign rank for matrices. In Section 6 we generalize unbounded error communication complexity to the multi-party number-on-the-forehead setting, and show that uniform sign complexity of tensors precisely captures communication complexity in this setting, in the same way that sign rank captures the communication complexity in the two-party setting. This also means that the sign complexity essentially captures communication complexity in this setting. 9

11 Lemma 11. The uniform sign complexity an order 3 tensor is at most 8 times its sign complexity (for a tensor of general order k the factor is 2 k ). Proof. Simply rewrite each Hadamard product of cylinder tensors A 1 A 2 A 3 into a sum of 8 such products by expanding (A + 1 A 1 ) (A+ 2 A 2 ) (A+ 3 A 3 ), where A+ l, A l for l = 1, 2, 3. 0 with A l = A + l A l We are interested in the sign complexity of tensors defined from Boolean functions. Namely, for a Boolean function f : {0, 1} n {0, 1} n {0, 1} n { 1, 1} we associate with f a 2 n 2 n 2 n tensor T f, the communication tensor, indexed by x, y, z {0, 1} n and defined by (T f ) xyz = f(x, y, z). Proposition 12. Assume that f : {0, 1} n {0, 1} n {0, 1} n { 1, 1} is computed by a THR s ETHR circuit. Then the sign complexity of T f is at most s + 1. Proof. From Lemma 9 we have that s+1 sgn w j 2 p j(x,y,z) = f(x, y, z), j=1 for all x, y, z {0, 1} n, where p j (x, y, z) are degree 2 polynomials. Now notice that we can rewrite each p j as p j (x, y, z) = p (1) j (y, z) + p (2) j (x, z) + p (3) j (x, y) and we can rewrite the above as s+1 sgn w j 2 p(1) j (y,z) 2 p(2) j (x,z) 2 p(3) j (x,y) = f(x, y, z). j=1 Since each of the exponential expressions w j 2 p(1) j (y,z), 2 p(2) j (x,z), and 2 p(3) j (x,y) define 2 n 2 n 2 n cylinder tensors, this shows that the sign complexity of T f is at most s + 1. Using the result of [14] that THR THR = THR ETHR this translates to a statement about THR THR circuits. Inspection of the proof of [14, Theorem 7] shows that a THR THR circuit where the output gate has fan-in s may be converted to an equivalent THR ETHR circuit where the output gate has fan-in O(sn 3 log n). From this, Proposition 12 above, and the results of Section 6 we obtain the following. Corollary 13. Assume that f : {0, 1} n {0, 1} n {0, 1} n { 1, 1} has unbounded error 3-player NOF communication complexity c. Then every THR ETHR computing f must contain 2 c /poly(n) THR gates. Remark. The above result can be generalized to THR THR AND k circuits by considering communication protocols with 2k + 1 parties. If lower bounds for such circuits could be obtained for increasing k, they could using the switching lemma be generalized to THR THR AND circuits, or even THR THR AC 0 circuits (cf. [25, 13]). 10

12 4 Weights and Degree In this section we among other things address the question of the minimal degree of a PTF computing a given Boolean function. Currently we are unable to give an upper bound on the degree required to compute a Boolean function given a bound on the length. That is, we don t know if PTF 1,2 (poly(n), d(n)) = PTF 1,2 ( ) for any function d(n), and in particular we don t know if PTF 1,2 (poly(n)) = PTF 1,2 ( ), though as we have indicated we believe the latter to be false. We first show that the degree can be bounded in terms of the weight, and conversely the weight can be bounded in terms of the degree. We currently know of no method to bound the degree and weight simultaneously in terms of the length. Note that the proof of Proposition 4 together with the upper bound n O(n) on the weight of linear threshold functions [21] imply that for any PTF P of constant length there is another PTF P of constant length and degree n O(n) computing the same function. However the precise length of P is exponential in the length of P. We are able to avoid the exponential increase only for length 3, and we feel the proof of this gives some indications about the difficulty of the general problem. The final result of this section is an exponential degree lower bound for PTFs of constant length. 4.1 Relationship between minimal weight and minimal degree The proof by Muroga, Toda and Takasu [22] (cf. [21]), showing that linear threshold functions needs integer weights of magnitude no more than (n + 1) (n+1)/2 /2 n can readily be adapted to PTFs on domain {1, 2} n to give a bound on weight in terms of degree. Proposition 14. Suppose P is a PTF of degree d and length s. Then there is another PTF P of degree d and length s having weight at most s s/2 2 ds such that sgn(p (x)) = sgn(p (x)) for all x {1, 2} n. Furthermore the set of monomials of P is the same of P. n i=1 xa ij i Proof. Suppose P (x) = s j=1 w j w 1,..., w s, with an inequality for each x {1, 2} n.. We construct system of linear inequalities in variables Let P (x) = s n j=1 w j i=1 xa ij i. For each x {1, 2} n, if P (x) > 0 we take the inequality P (x) 1, and if P (x) < 0 we take the inequality P (x) 1. This set of inequalities has a solution (namely any sufficiently large multiple of w), and any solution of the inequalities gives a set of weights defining an equivalent PTF P. Let w be a solution that maximizes the number of inequalities satisfied with equality. Then it is easy to see that w is uniquely determined by these equalities. In other words, w is a solution to a linear system Aw = b, where A is a s s matrix with entries from {0, 1,..., 2 d } and b { 1, 1} s. By Cramer s rule, w j = det(a j)/ det(a), where A j is obtained from A by replacing column j by b. By Hadamard s inequality det(a j ) s s/2 2 ds, and we can now just clear the common denominator obtaining the claimed integer weights. A more complicated proof can give us a bound in another direction. Proposition 15. Suppose P is a PTF having integer coefficients, weight W and length s with n variables. Then there is another PTF P of weight W and length s having degree at most (sn + 1)(sn) sn/2 log 2 s + log 2 W sn such that sgn(p (x)) = sgn(p (x)) for all x {1, 2} n. Furthermore the (multi)set of weights of P is the same of P. 11

13 Proof. Consider P in its exponential form s j=1 w j2 lj(x), where l j (x) = a 1j x a nj x n, with a ij 0, and w j Z with w j W. We construct a system of linear inequalities and equations in variables a ij, with a number of inequalities and equations produced for each x {0, 1}n. Let l j (x) = a 1j x 1 + +a nj x n, and P be the PTF given by the exponential form s j=1 w j2 l j (x). Define c = log 2 s + log 2 W. Consider a fixed x {0, 1} n. We will consider the values l 1 (x), l 2 (x),..., l s (x) in sorted order. For simplicity of notation, assume that in fact we have l 1 (x) l 2 (x)... l s (x). We partition this ordered sequence into blocks of consecutive values. Let 1 = i 1 i 2... i m i m+1 = s + 1 be the endpoint of m such blocks, where block k consists of the indices i k j < i k+1. We choose the partition in such a way that for i k j < i k+1 1 with 1 k m, and l j (x) < l j+1 (x) + c, l ik 1(x) l ik (x) + c, for 1 < k m. In other words, in each block the difference between consecutive values is less than c, whereas the difference between the endpoints values of different blocks is at least c. We form an equality for consecutive values from the same block and an inequality otherwise. For i k j < i k+1 1 we take the equality and for 1 < k m we take the inequality l j(x) l j+1(x) = l j (x) l j+1 (x), l i k 1(x) l i k (x) c. In total this gives a system of 2 n (s 1) equations and inequalities in sn variables. We add to this positivity constraints, a ij 0. All coefficients are either 1 or 1, and the right-hand sides are all integers in the range {0, 1,..., c}. Let A be the (2 n (s 1) + sn) sn matrix of coefficients and let b Z 2n (s 1)+sn be the right-hand sides. We know already that the system of equations and inequalities has an integer solution, namely that given by P. Furthermore, any integer solution gives a set of coefficients to the linear forms defining an equivalent PTF. To see this, note the inequalities guarantee that for any block, when its contribution is nonzero it is guaranteed to dominate the contribution of all following blocks. We can now use a bound of von zur Gathen and Sieveking [31] to estimate an integer solution with small entries. For this let M be the maximum magnitude of a m m minor of the matrix [ A b ], where m sn. Then the system has an integer solution a with a ij (sn + 1)M. By Hadamard s inequality, M (sn)sn/2 c sn, and hence the entries of a are bounded by (sn + 1)(sn) sn/2 c sn. From Propositions 14 and 15 we for example have that for a PTF of length poly(n), if the degree is at most 2 poly(n) then the weight can be assumed to be 2 2poly(n), and vise versa. 12

14 4.2 Exponential upper bound for the degree of size 3 threshold gates. Proposition 16. Any PTF of length 3 over the domain {1, 2} n is equivalent to a length 3 PTF with degree n O(n). Proof. Consider a PTF of length 3 in its exponential form E(x) = w 1 2 l 1(x) + w 2 2 l 2(x) + w 3 2 l 3(x) When all w i s have the same sign, the Boolean function computed is constant, so assume this is not the case. Thus either 1 or 2 of the w i s are positive. We consider the case when 2 are positive. The other case is analogous. We can thus rewrite E(x) in the form E(x) = 2 A(x)+α + 2 B(x)+β 2 C(x)+γ where the A(x),B(x), and C(x) are affine linear forms with positive integer coefficients, and 0 α, β, γ < 1. In fact, by dividing by 2 γ, we may assume γ = 0. For each x {0, 1} n we produce a number of linear equations and inequalities. We have 3 cases. In case A(x) C(x) or B(x) C(x) we take that inequality. Note that A(x) C(x) or B(x) C(x) alone implies that E(x) > 0. In case both A(x) C(x) 2 and B(x) C(x) 2 we take both these inequalities. Note that A(x) C(x) 2 and B(x) C(x) 2 together implies that E(x) < 0. The last case is that either A(x) = C(x) 1 or B(x) = C(x) 1. Consider first the subcase of A(x) = C(x) 1. We may then rewrite Thus E(x) 0 if and only if which holds if and only if E(x) = 2 B(x)+β 2 C(x) (1 2 α 1 ) 2 B(x) 2 C(x) (1 2 α 1 )2 β B(x) C(x) + log 2 (1 2 α 1 ) β. Since the coefficients of B(x) and C(x) are integer, this holds if and only if B(x) C(x) + log 2 (1 2 α 1 ) β. Note that log 2 (1 2 α 1 ) β might be of large magnitude, and thus we do not want to have such a number in our system. Instead we introduce a new integer variable k A and add to our system the equality A(x) = C(x) 1 and the inequalities B(x) C(x) k A and k A 1. The sub-case of B(x) = C(x) 1 is similar, and E(x) 0 if and only if A(x) C(x) + log 2 (1 2 β 1 ) α. We add here in a similar way the equality B(x) = C(x) 1 and the inequalities A(x) C(x) k B and k B 1. Finally we add positivity constraints to all coefficients of variables in exponents. In this way we end up with a system of at most 2 2 n + 3n + 2 equations and inequalities in 3(n + 1) + 2 = 3n + 5 variables. Similarly as in the proof of Proposition 15 we may find an integer solution with entries 13

15 of bounded magnitude. The bound we can achieve here by the result of von zur Gathen and Sieveking [31] is (3n + 6)(3n + 5) (3n+5)/2 2 3n+5 = n O(n). Given such an integer solution A (x), B (x), C (x), k A, k B we now wish to find α and β solving the equations k A = log 2 (1 2 α 1 ) β (2) k B = log 2 (1 2 β 1 ) α or equivalently 2 k A 1 < (1 2 α 1 )2 β 2 k A 2 k B 1 < (1 2 β 1 )2 α 2 k B This can be done as follows. First, let log 2 (3) 1 α 0, β 0 < 1 be given by α 0 = log 2 (1 2 k A 1 )+1 and β 0 = log 2 (1 2 k B 1 ) + 1. Then we have 1 2 α 0 1 = 2 k A 1 and 1 2 β 0 1 = 2 k B 1. Choose 0 < ɛ < min(1 α 0, 1 β 0 ), and let α = log 2 (1 2 k A ɛ ) + 1 and β = log 2 (1 2 k B ɛ ) + 1. Then we have 1 2 α 1 = 2 k A ɛ and 1 2 β 1 = 2 k B ɛ. Note also that α < α 0 and β < β 0 which implies 0 < α, β < 1 ɛ. Combining this we see that α and β satisfies the equations (3). Remark. One may in fact solve the equations (2) without the ceiling operation except when exactly one of k A and k B is 1. In case k A = k B = 1 we have the solution α = β = 0. In case k A, k B > 1 we have the solution ( 2 2 α 2 k A ) = log k A k B ( 2 2 β 2 k B ) = log k. A k B 4.3 Exponential lower bound for the degree of constant-size threshold gates The ODD-MAX-BIT function (abbreviated here by OMB) was defined by Beigel [3] as OMB(x 1,..., x n ) = 1 if and only if (max{i x i = 1} mod 2) = 1. In this section we prove a degree lower bound of the form 2 Ω(nɛ) for any PTF of constant length s computing OMB, where ɛ depends on s. By Proposition 4 we have PTF 1,2 (O(1), ) = ANY O(1) THR. Inspecting the proof we can observe that a PTF of individual degree at most d and length k is in fact turned into a Boolean combination of 2 O(k2) threshold functions, where these threshold functions have integer weights of magnitude at most d. We can rewrite this Boolean combination of linear threshold functions by a constant size DNF, i.e. we obtain an OR 2 2 O(k2 ) AND 2 O(k 2 ) THR circuit where all threshold gates are computed with integer weights of magnitude at most d. We can thus just give weight lower bounds for OR O(1) AND O(1) THR circuits. Theorem 17. Any circuit in the class OR k AND l THR computing OMB function on n variables require weights of size 2 Ω(n1/kl). Proof. In this proof we will identify inputs x {0, 1} n to the function OMB with the subsets X [n] given by X = {i x i = 1}, and vice versa. Define m = n 1/kl. In the course of the proof we will construct a sequence of disjoint subsets A 0, A 1,..., A m [n] such that a 0 <... < a m whenever a 0 A 0,..., a m A m. In addition to these we will identify a particular THR gate in the circuit such that on inputs of the form (3) 14

16 X i = A 0 A i A i 1 A i 3 A i 5..., the THR-gate is 1 for odd i and is 0 for even i. With this sequence in hand we will be able to prove the weight lower bound in a standard way. Break the set [n] into m equal-sized blocks B 1,..., B m of consecutive elements. Let A 0 =. Consider the first AND gate in the circuit and the first of its THR gates. Let us denote it by t 1. We shall attempt to find the sequence of sets A 1,..., A m for t 1. We shall choose these one by one, i = 1,..., m, until we either succeed or fail for some i. When i is odd we wish to find A i B i such that OMB(X i ) = 1 and t 1 (X i ) = 1. When i is even we wish to find A i B i such that OMB(X i ) = 0 and t 1 (X i ) = 0. If on each step we can find such A i we are done. Suppose that on some step i we fail to find A i. We shall then add to A 0 the sets A i 1, A i 3, A i 5,... and repeat the whole argument restricted to block B i, which we divide into m new blocks. The point is that before doing this we may simplify the circuit. Namely, when i is odd, we have for all A i B i for which OMB(X i ) = 1 that t 1 (X i ) = 0. Thus we may eliminate t 1 together with its parent AND gate from the circuit when restricting input to B i and the bits in the set A 0. Since the circuit actually computes the OMB function there will still be other AND THR sub-circuits remaining. Similarly, when i is even, we have for all A i B i for which OMB(X i ) = 0 that t 1 (X i ) = 1. Thus we may eliminate t 1 from the circuit when restricting input to B i and the bits in the set A 0. Since the circuit actually computes the OMB function there will still be another THR gate in the circuit that is a sibling of t 1. We need to proceed to the sub-block at most kl 1 times since each time we reduce the number of threshold gates by at least 1. It is not hard to see that by the choice of parameters we finally get the sequence of sets as desired. Let us now consider the threshold gate corresponding to this sequence. Suppose it is given by the inequality j w jx j > t, where the variables range only over the proper block, and all variables outside are fixed. Define W Ai = j A i w j, where i = 1,..., m. Then from the inputs X 1,..., X m we get the sequence of inequalities W A1 > t, W A2 + W A1 < t, W A3 + W A2 > t, W A4 + W A3 + W A1 < t,..., and from these inequalities the desired lower bound 2 Ω(n1/kl) follows easily. 5 Max-plus PTFs In this section we introduce one more complexity class which is a limit class of our PTF classes in the flavor of max-plus algebra. It turns out this class is related to the Boolean circuits of the form AC 0 THR. Consider some integer polynomial p(x) Z[x] and consider some domain {1, b}. This polynomial computes some function in PTF 1,b (l, d), where l is the length of p and d is the degree of p. We start with some preliminary work to make p be in general position. First we can assume that p(x) 0 for all x {1, b} n. For this just consider 2p(x) + 1 instead of p(x). Next by varying the coefficients of p by small rational numbers and multiplying the whole polynomial by large enough integer to make the coefficients integer again we can assume that for each x {1, b} n none of the two monomials are equal. Now consider the exponential form of p P (x) = l 1 b L i(x) l 2 b M j(x), L i (x) = n n v ki x k + v 0i, M j (x) = u kj x k + u 0j. i=1 j=1 k=1 k=1 15

17 Note that here we put the coefficients of the polynomial to the exponents. Suppose now that we start to increase b (for convenience assume that b is positive). Then the function computed by p might start changing. Since all linear forms are different for all inputs x {0, 1} n it is easy to see that for large enough b and for any x the sign of P (x) is equal to the sign of the largest monomial. That is, P (x) 0 if and only if max (L i (x)) max (M j (x)) (4) i=1,...,l 1 j=1,...,l 2 Definition 1. By max-plus PTFs we denote expressions of the form (4). The length of the PTF (4) is l 1 + l 2, the degree is the maximal sum of absolute values of all coefficients of L 1,..., L l1 and M 1,..., M l2 except the constant term. Note that max-plus PTFs are essentially just polynomial inequalities in the max-plus algebra. We let mpptf(l(n), d(n)) to be the set of Boolean functions computable by max-plus PTFs of length l(n) and degree d(n). As before we denote by mpptf(d(n)) the set of Boolean functions computable by max-plus PTFs of length poly(n) and degree d(n). Here d(n) might be. It turns out that the class mpptf( ) is related to the circuits of the form AC 0 THR. More specifically, we can consider the hierarchy of polynomial size circuits of the form Σ d THR and Π d THR, where Σ d and Π d are the classes of depth d circuits of interchanging layers of AND and OR gates of unbounded fan-in, Σ d having OR gate at the top and Π d having AND gate at the top. In this hierarchy lower bounds are known only for the case d = 1, that is for the circuits of the form AND THR and OR THR (the standard proof of the lower bound on the size of DNF for the parity function works). For the circuits of the form AND OR THR and OR AND THR no superpolynomial lower bounds for explicit functions are known. It turns out that the class mpptf( ) lies between these two levels of the hierarchy. First we note some trivial property of max-plus PTF classes. Proposition 18. mpptf( ) and mpptf(poly(n)) are closed under negation. Proof. This follows directly from the definition of max-plus PTFs and the fact that any max-plus PTF can be easily reconstructed in such a way that for all x {0, 1} n the equality does not hold in (4). Lemma 19. AND THR, OR THR mpptf( ), and mpptf( ) AND OR THR, OR AND THR Proof. Assume that we have a circuit of the form OR THR and l 1 0,..., l k 0 are the threshold gates on the bottom level. Then it is clear that the max-plus PTF max(l 1,..., l k ) 0 computes the same function. For the class AND THR an analogous proof works. Alternatively, the inclusion follows from Proposition 18. Now consider the max-plus PTF (4). It is true if and only if there is i [l 1 ] such that for all j [l 2 ] it is true that L i (x) M j (x). That is, this PTF is equivalent to the formula l 1 l 2 i=1 j=1 (L i (x) M j (x)), 16