Energy-Efficient Threshold Circuits Coputing Mod Functions Akira Suzuki Kei Uchizawa Xiao Zhou Graduate School of Inforation Sciences, Tohoku University Araaki-aza Aoba 6-6-05, Aoba-ku, Sendai, 980-8579, Japan. Eail: {a.suzuki, uchizawa, zhou}@ecei.tohoku.ac.jp Abstract We prove that the odulus function MOD of n variables can be coputed by a threshold circuit C of energy e size s = O(e(n/ 1/(e 1 for any integer e 2, where the energy e is defined to be the axiu nuber of gates outputting 1 over all inputs to C, the size s to be the nuber of gates in C. Our upper bound on the size s alost atches the known lower bound s = Ω(e(n/ 1/e. 1 Introduction Neuronal signals play fundaental role in inforation processing of the brain. A neuron eitting a signal is said to be firing. Recent biological studies report the following fact about the energy consuption of the neuronal firing: the energy cost of a neuronal firing is high while energy supplied to the brain is liited, hence neural networks ust have low firing activity (Attwell & Laughlin, 2001; Lennie, 2003. Consequently, any neuroscientists consider that the etabolic liit ust influence the way in which inforation is processed, the brain has countered this etabolic constraint by adopting energy-efficient circuit designs (Földiak, 2003; Laughlin & Sejnowski, 2003; Olshausen & Field, 2004; Vinje & Gallant, 2000. Uchizawa, Douglas Maass consider the proble posed above fro the view point of theoretical coputer science, introduce a new coplexity easure called the energy coplexity of threshold circuits (Uchizawa et al., 2006, where a threshold circuit, is a cobinatorial circuit consisting of threshold gates, is a theoretical odel of neural circuit (Minsky & Papert, 1988; Parberry, 1994; Sia & Orponen, 2003; Siu et al., 1995. Based on the biological fact above, the energy e of a threshold circuit C is defined as the axiu nuber of threshold gates outputting 1 over all inputs to C. In previous research, several facts are known on the coputational power of threshold circuits with sall energy (Uchizawa et al., 2006, 2009a; Uchizawa & Takioto, 2008; Uchizawa et al., 2009b. Particularly, Uchizawa et al. (2006 find a non-trivial circuit structure that benefits energy-efficiency, provide threshold circuits of polynoial size energy O(log n for a fairly large class of Boolean functions Supported by MEXT Grant-in-Aid for Young Scientists (B No.21700003 Copyright c 2011, Australian Coputer Society, Inc. This paper appeared at the 17th Coputing: The Australasian Theory Syposiu (CATS 2011, Perth, Australia, January 2011. Conferences in Research Practice in Inforation Technology (CR- PIT, Vol. 119. Alex Potanin Taso Viglas, Ed. Reproduction for acadeic, not-for-profit purposes peritted provided this text is included. of n variables. However, their construction is not specialized for a particular task, hence it soeties gives a redundant circuit. In this paper, we consider one of the fundaental well-studied Boolean functions in the theory of circuit coplexity, the odulus function, as a particular task. The odulus function MOD of n variables for two positive integers n is defined as follows: MOD (x = 0 if the nuber of 1 s in an input x {0, 1} n is a ultiple of, otherwise, MOD (x = 1. Although the odulus function ay be far fro real tasks that neural networks in the brain perfor, we believe that considering such a siple fundaental task akes an iportant step for understing what circuit structure benefits the energy-efficiency of threshold circuits. Uchizawa et al. (2009b proved that size energy of a threshold circuit coputing the odulus function cannot be siultaneously sall: Any threshold circuit C of energy e coputing MOD of n variables has size ( ( n 1/e s = Ω e. (1 We prove in this paper that MOD of n variables can be coputed by a threshold circuit of energy e size ( ( n 1/(e 1 s = O e (2 for every integer e 2. Coparing the right-h side of Eq. (1 with one of Eq. (2, we can find the difference between the ters only in the exponent of n/. our upper bound alost atches the lower bound, iplies that there exists a tight tradeoff between size energy of threshold circuits coputing odulus function. We obtain the result by construction of the desired threshold circuits, hence it exhibit a circuit design of energy-efficient threshold circuits. In addition, we consider an extree case where threshold circuits have energy e = 1. In this case, we prove that any threshold circuit C coputing the PARITY of n variables ust have an exponential nuber of gates in n. On the other h, Eq. (2 iplies that PARITY (i.e., MOD 2 can be coputed by a threshold circuit of size s = O(n energy e = 2. we know fro these facts that there exists a significant gap of coputational power between threshold circuits of e = 1 ones of e = 2. The rest of the paper is organized as follows. In Section 2, we define soe ters on threshold circuits the odulus function. In Section 3, we first provide our ain theore. We then give a technical lea, prove the theore using the lea. In
Section 4, we prove the technical lea given in Section 3. In Section 5, we give the lower bound for threshold circuits of energy one. In Section 6, we conclude with soe rearks. 2 Preliinaries A threshold circuit C is a cobinatorial circuit of threshold gates, is expressed by a directed acyclic graph. Let n be the nuber of input variables to C. Then each node of in-degree 0 in C corresponds to one of the n input variables x 1, x 2,..., x n, the other nodes correspond to threshold gates. We define the size s(c of a threshold circuit C as the nuber of threshold gates in C. Let g1 C, g2 C,..., gs(c C be the gates in a threshold circuit C. For each gate gi C, 1 i s(c, let z(x = (z 1 (x, z 2 (x,..., z ki (x {0, 1} ki be the k i inputs of gi C with weights w 1, w 2,..., w ki a threshold t i for x {0, 1} n. Then the output gi C (z(x of the gate gi C is defined as follows: k i gi C (z(x = sign w j z j (x t i, where sign(z = 1 if z 0 sign(z = 0 if z < 0. Siply, gi C(z(x is denoted by gc i [x]. Let f : {0, 1} n {0, 1} be a Boolean function of n variables. Let gs C be a gate of out-degree 0 in C, let the output gs C [x] of g s be the output C(x of C. C(x = g s [x] for every input x {0, 1} n. The gate gs C is called the top gate of C. A threshold circuit C coputes a Boolean function f : {0, 1} n {0, 1} if C(x = f(x for every input x {0, 1} n. We define the energy e(c of a threshold circuit C as e(c = gi C [x]. s(c ax x {0,1} n the energy e(c is the axiu nuber of gates outputting 1 over all inputs x {0, 1} n to C. Trivially, we have 0 e(c s(c. For an input variable x = (x 1, x 2,..., x n {0, 1} n, we define x as the haing weight of the inputs x, that is, x = x i. Then, for an integer 2, the odulus function MOD of n variables is defined as follows: For every input variable x = (x 1, x 2,..., x n {0, 1} n, { 1 if x is not a ultiple of ; MOD (x = 0 otherwise. If n 1, then MOD (x = 1 for all inputs x except x = (0, 0,..., 0. a circuit consisting of a single threshold gate with weight ones for all the inputs threshold one coputes MOD. One ay thus assue that n in the reainder of the paper. 3 Energy-Efficient Circuits Our ain result is the following theore that yields an energy-efficient threshold circuit coputing the odulus function. Theore 1. Let, n be any two positive integers, let e 2 be any integer. Then there is a threshold circuit coputing MOD of n variables such that its energy is at ost e its size is at ost (n 1/(e 1 + 1 (e 1 ( ( n 1/(e 1 = O e. (3 Uchizawa et al. (2009b prove that the size s energy e of a threshold circuit C coputing MOD of n variables cannot be siultaneously sall, as described in the following theore. Theore 2 (Uchizawa et al. 2009b. Let C be a threshold circuit coputing the function MOD of n variables. Then the size s energy e of C satisfy n 1 + 1 1 2πe where c = 2.718 is a constant. ( e 2c s (4 By a siple odification of Eq. (4, we can obtain fro Theore 2 the following lower bound on the size of threshold circuits coputing MOD of n variables. Corollary 1. Let C be any threshold circuit of energy e coputing MOD of n variables. Then ( ( n 1/e s(c = Ω e. (5 Observe the asyptotic ters in the right-h side of Eqs. (3 (5. We can find the difference between the ters only in the exponent of n/: the ter in Eq. (3 has 1/(e 1, while the ter in Eq. (5 has 1/e. Hence, the upper bound in Theore 1 alost atches the lower bound in Corollary 1. In the rest of the section, we prove Theore 1. We say that a threshold circuit C is regular if the inputs of every gate in C includes all the inputs x 1, x 2,..., x n with weight ones. In other words, every gate in C receives all the unweighted inputs x 1, x 2,..., x n. The following technical lea plays key role in our proof. Lea 1. Let, n, n be positive integers such that n n + 1. Let C be a regular threshold circuit coputing MOD of n variables. Then, there is a regular threshold circuit C coputing MOD of n variables such that e(c e(c + 1 s(c s(c + e n + 1 1. n +1 We will prove the lea in the next section. Using the lea, we prove Theore 1 below. Proof of Theore 1. Let e be an arbitrary integer at least 2. We prove the theore by constructing a regular threshold circuit C coputing MOD of n variables such that e(c e (n 1/(e 1 + 1 s(c (e 1. (6
We provide our construction by induction on e 2. That is, we construct a threshold circuit of energy e + 1 fro a threshold circuit of energy e. We start fro the case of e = 2 as the basis. Basis: e = 2. Consider a regular threshold circuit C consisting of a single threshold gate g with threshold t = 1 1 input variables. Clearly, e(c = 1, s(c = 1, C coputes MOD of n = 1 variables. Therefore, Lea 1 iplies that there is a regular threshold circuit C coputing MOD of n variables such that e(c e(c + 1 ( n + 1 s(c s(c + 1. Since e(c = 1, we have e(c e(c + 1 = 2 = e. Since s(c = 1, we have ( n + 1 s(c s(c + 1 = (n 1/(e 1 + 1 = (e 1. Inductive Step: e 3. By the induction hypothesis, there is a regular threshold circuit C coputing MOD of n = γ e 1 1 variables for each positive integer γ where e(c e (n s(c 1/(e 1 + 1 (e 1 ((γ e 1 1/(e 1 1 + 1 (e 1 (e 1γ. (7 We will construct a regular threshold circuit C coputing MOD of n variables, show that C has the energy e(c e + 1 (8 the size (n 1/e + 1 s(c e. (9 Since C coputes MOD of n = γ e 1 1 variables, by Lea 1 there is a regular threshold circuit C coputing MOD of n variables such that We choose e(c e(c + 1 = e + 1 s(c s(c + γ e 1 1. (10 (n 1/e + 1 γ =, then γ e (n + 1/, hence γ e 1 1 γ. (11 Therefore, by Eqs. (7, (10 (11, we have s(c s(c + γ e 1 1 4 Proof of Lea 1 (e 1γ + γ eγ (n 1/e + 1 = e. In the section, we prove Lea 1. Let, n, n be positive integers such that n n +1. Let C be a regular threshold circuit coputing MOD of n variables, s = s(c. We denote by g 1, g 2,..., g s the threshold gates in C. One ay assue without loss of generality that g 1, g 2,..., g s are topologically ordered with respect to the underlying directed acyclic graph of C, that each gate g i, 1 i s, receives exactly (i 1+n inputs fro the outputs of the gates g 1, g 2,..., g i 1 the n inputs x 1, x 2,..., x n. If there is soe gate g i, 1 j i 1, such that g i has no input fro the output of g j, then one connects input of g i with weight 0 for the output of g j. Therefore, for each index i, 1 i s, let w i,1, w i,2,..., w i,i 1 be the weights of the gate g i for the outputs of the gates g 1, g 2,..., g i 1, respectively, denote by t i the threshold of g i. Since C is regular, each of the gates g 1, g 2,..., g s has weight ones for the n input variables. the output of the gate g i for each input x {0, 1} n can be recursively coputed by the following threshold function: sign ( x t 1 if i = 1; ( g i[x ] = sign x + i 1 w i,j g j [x ] t (12 i otherwise. We show that, for any positive integer n n + 1, MOD of n variables can be coputed by a regular threshold circuit C of energy e(c e(c + 1 size s(c s(c + β, where β = n + 1 α 1 n + 1 α =. We construct the desired threshold circuit C fro C, as described below. To obtain C, we add new input variables x n +1, x n +2,..., x n to C, connect each of the new input variables to each of the gates g 1, g 2,..., g s with weight one. Besides, for each index i, 1 i β, we add a new threshold gate ĝ i with weight ones
for the inputs x 1, x 2,..., x n a threshold αi to C, connect the output of the gate ĝ i to the gates g 1, g 2,..., g s with weight αi. For each index i, 1 i s, we denote by g i the gate in C that corresponds to the gate g i in C, denote by x = (x 1, x 2,..., x n {0, 1} n an input to C. Then, the output of the gate g i for an input x {0, 1} n is now represented as ( sign x β αj ĝ j [x] t 1 if i = 1; ( g i [x] = sign x + i 1 w i,j g j[x] β αj ĝ j [x] t i otherwise. (13 Moreover, for each index i, 2 i β, we connect the output of the gate ĝ i to the gates ĝ 1, ĝ 2,..., ĝ i 1 with weight αi. we have ( sign x αj ĝ j [x] αi ĝ i [x] = if 1 i β 1; sign ( x αβ if i = β. β for x {0, 1} n. Clearly, C is a regular circuit, s(c s(c + β = s(c n + 1 + 1. n +1 (14 Below we prove that C coputes MOD of n variables, e(c e(c + 1. Let x {0, 1} n be an arbitrary input to C. Note that 0 x x n α 1 α α(β + 1 1 for any input x {0, 1} n. Let x i =, α then trivially αi x α(i + 1 1. (15 We prove the following clai. Clai 1. The following (i, (ii (iii hold. (i ĝ i [x] = 0 for each i, i + 1 i β; (ii ĝ i [x] = 1 if i = i ; (iii ĝ i [x] = 0 for each i, 1 i i 1. In other words, if 0 x α 1, none of ĝ 1, ĝ 2,..., ĝ β outputs one; otherwise, only the gate ĝ i outputs one. Proof of Clai. For each index i, 1 i β, let β x αj ĝ j [x] αi p i (x = if 1 i β 1, x αβ if i = β. (16 Clearly, p i [x] is the value in the sign function of the right h side of Eq. (14 for x {0, 1} n, that is, ĝ i [x] = sign(p i (x. We evaluate p i (x, prove (i, (ii (iii. (i ĝ i [x] = 0 for each i, i + 1 i β. If i β 1, then by Eqs. (15 (16 p i (x α(i + 1 1 αj ĝ j [x] αi α(i + 1 i 1 1. (17 If i = β, then by Eqs. (15 (16 we siilarly have p i (x = x αβ α(i + 1 1 αβ 1. (18 Since i + 1 β, Eqs. (14, (17 (18 iply that ĝ i [x] = sign(p i (x = 0. (ii ĝ i [x] = 1 if i = i. In this case, we have 1 i β. By (i above, if i β 1, j=i +1 αj ĝ j [x] = 0, hence we have by Eqs. (15 (16 p i (x = x αi αi αi = 0. Thus Eq. (14 iplies that ĝ i [x] = sign(p i (x = 1. (iii ĝ i [x] = 0 for each i, 1 i i 1. In this case, we have 2 i+1 i β, hence ĝ j [x] ĝ i [x]. By (ii above, we have g i [x] = 1, hence αj ĝ j [x] αi ĝ i [x] = αi. (19 Since i + 1 β, we have i β 1. Therefore, by Eqs. (16 (19 p i (x x αi αi. (20
By Eqs. (15 (20, we have p i (x α(i + 1 1 αi αi α 1 αi 1 hence Eq. (14 iplies that ĝ i [x] = sign(p i (x = 0. We are now ready to prove the lea by the clai. There are the following two cases to consider. Case 1: 0 x α 1. In this case, the clai iplies that none of ĝ 1, ĝ 2,..., ĝ β output one. Besides, we have n + 1 α 1 = 1 n. Therefore, Eqs. (12 (13 iply that, for every index i, 1 i s, the output of g i for x {0, 1} n equals to the output of g i for an input x {0, 1} n such that x = x. Thus the nuber of gates outputting one is at ost e. Since C coputes MOD, C(x equals to MOD (x. Case 2: α x α(β + 1 1. In this case, the clai iplies that only the gate ĝ i of ĝ 1, ĝ 2,..., ĝ β outputs one, hence Eq. (13 iplies that, for every index i, 1 i s, the output of g i can be represented as i 1 g i [x] = sign x + w i,jg j [x] αi t i. (21 Eq. (15 iplies that 0 x αi α 1, hence, for every index i, 1 i s, we have that g i [x] for x {0, 1} n equals to the output of g i for an input x {0, 1} n such that x αi = x. at ost e gates of the gates g 1, g 2,..., g s output one, consequently the nuber of gates outputting one in C is at ost e+1. The circuit C coputes MOD of n variables, x αi is a ultiple of if only if x is a ultiple of. Hence, C(x equals to MOD (x. 5 Circuits of Energy One In this section, we consider an extree case where threshold circuits have energy e = 1. While we know fro Theore 1 that PARITY (i.e., MOD 2 of n variables can be coputed by a threshold circuit of size s = O(n energy e = 2, we can prove that any threshold circuit of energy e = 1 coputing PARITY of n variables ust have an exponential nuber of gates in n, as follows. Theore 3. If a threshold circuit C of energy one coputes PARITY of n variables, then the size s of C is at least 2 n 1. Proof. Let C be a threshold circuit of size s energy e = 1 that coputes PARITY of n variables. We denote by g 1, g 2,..., g s the threshold gates in C, let g s be the top gate of C. Let X 0 = {z {0, 1} n z is even}, n 0 be the cardinality of X 0. Clearly, n 0 = 2 n 1. We prove that s n 0 = 2 n 1 as follows. For the sake of contradiction, assue that s n 0 1. Since the top gate g s outputs zero for any input z X 0, we have, for each input z X 0, either exactly one of g 1, g 2,..., g s 1 outputs one or none of the gates outputs one. Since s n 0 1, the pigeon hole principle iplies that there exists a pair of inputs x = (x 1, x 2,..., x n, y = (y 1, y 2,..., y n X 0 that satisfies one of the following conditions: (i there exists only an index k, 1 k s 1, such that the gate g k outputs one for each of the inputs x y; (ii none of the gates g 1, g 2,..., g s outputs one for each of the inputs x y. For each of (i (ii, we derive a contradiction as follows. We first consider (i. Let the gate g k have weights w 1, w 2,..., w n for the n input variables a threshold t. Since only the gate g k outputs one for each of x y, we clearly have w i x i t 0 w i y i t 0. w i y i 2t 0. (22 Since x y, there exists an index j such that x j y j. Consider a pair of inputs x y obtained fro x y by exchanging the jth coponents of x for that of y: x = (x 1, x 2,..., x j 1, y j, x j+1,..., x n (23 y = (y 1, y 2,..., y j 1, x j, y j+1,..., y n (24 Both x y are even we have either 0 = x j y j = 1 or 1 = x j y j = 0, hence x y are odd. only the top gate g s outputs one for each of x y, consequently g k outputs zero for each of x y, which iplies that w i x i w j x j + w j y j t < 0 w i y i w j y j + w j x j t < 0. w i y i 2t < 0. (25 We obtain a contradiction fro Eqs. (22 (25 We next consider (ii, derive a contradiction in a siilar way to (i. Let the top gate g s have weights w 1, w 2,..., w n for the n input variables
a threshold t. Since none of the gates outputs one for x y, we clearly have w i x i t < 0 w i y i t < 0. w i y i 2t < 0. (26 Let j be an index such that x j y j, then consider a pair of inputs x y obtained fro x y by switching the jth coponents of the inputs as in Eqs (23 (24. Clearly, x y are both odd. the top gate g s outputs one for each of x y, which iplies that w i x i w j x j + w j y j t 0 w i y i w j y j + w j x j t 0. w i y i 2t 0. (27 We obtain a contradiction fro Eqs. (26 (27 Theores 1 3 iply that there exists a significant gap of coputational power between threshold circuits of e = 1 ones of e = 2. 6 Conclusions In the paper, we design energy-efficient threshold circuits coputing the odulus function MOD, show that MOD of n variables can be coputed by a threshold circuit of size s = O(e(n/ 1/(e 1 energy e for any integer e 2. The upper bound on the size s = O(e(n/ 1/(e 1 alost atches the known lower bound Ω(e(n/ 1/e. We also show that any threshold circuit of energy e = 1 needs at least 2 n 1 threshold gates to copute PARITY of n variables. A Boolean function f : {0, 1} n {0, 1} is called syetric if f(x depends only on the nuber of 1s in x for every input x {0, 1} n. the odulus function is syetric. A generalization of the result to syetric functions reains open. Földiak, P. (2003, Sparse coding in the priate cortex, The Hbook of Brain Theory Neural Networks 1, 1064 1068. Laughlin, S. B. & Sejnowski, T. J. (2003, Counication in neuronal networks, Science 301(5641, 1870 1874. Lennie, P. (2003, The cost of cortical coputation, Current Biology 13, 493 497. Minsky, M. & Papert, S. (1988, Perceptrons: An Introduction to Coputational Geoetry, MIT Press, Cabridge, MA. Olshausen, B. A. & Field, D. J. (2004, Sparse coding of sensory inputs, Current Opinion in Neuro Biology 14, 481 487. Parberry, I. (1994, Circuit Coplexity Neural Networks, MIT Press, Cabridge, MA. Sia, J. & Orponen, P. (2003, General-purpose coputation with neural networks: A survey of coplexity theoretic results, Neural Coputation 15, 2727 2778. Siu, K. Y., Roychowdhury, V. & Kailath, T. (1995, Discrete Neural Coputation; A Theoretical Foundation, Prentice-Hall, Inc., Upper Saddle River, NJ. Uchizawa, K. & Takioto, E. (2008, Exponential lower bounds on the size of constant-depth threshold circuits with sall energy coplexity, Theoretical Coputer Science 407(1-3, 474 487. Uchizawa, K., Douglas, R. & Maass, W. (2006, On the coputational power of threshold circuits with sparse activity, Neural Coputation 18(12, 2994 3008. Uchizawa, K., Nishizeki, T. & Takioto, E. (2009a, Energy coplexity depth of threshold circuits, in Proceedings of the 17th International Syposiu on Fundaentals of Coputation Theory, Springer Lect. Notes in Coputer Science 5699, pp. 335 345. Uchizawa, K., Takioto, E. & Nishizeki, T. (2009b, Size energy of threshold circuits coputing od functions, in Proceedings of the 34th Int. Syp. on Matheatical Foundations of Coputer Science, Aug. 24-28, 2009, High Tatras, Slovakia, Springer Lect. Notes in Coputer Science 5734, pp. 724 735. Vinje, W. E. & Gallant, J. L. (2000, Sparse coding decorrelation in priary visual cortex during natural vision, Science 287(5456, 1273 1276. Acknowledgents We would like to thank Takao Nishizeki for his fruitful discussion the referees for their helpful coents. References Attwell, D. & Laughlin, S. B. (2001, An energy budget for signaling in the gray atter of the brain, Journal of Cerebral Blood Flow Metabolis 21, 1133 1145.