Lecture 11: Polynomial Approximations

Transcription

1 CS 710: Complexity Theory 10/13/011 Lecture 11: Polyomial Approximatios Istructor: Dieter va Melkebeek Scribe: Cheta Rao I the previous lecture, we discussed Boolea circuits ad their lower bouds. I particular, we discussed costat-depth circuits with ubouded fa-i (i-degree) that ca compute certai operatios such as biary additio i polyomial-size. We also highlighted that biary multiplicatio ad parity ( - defied i last lecture) caot be decided i this framework. I today s lecture, we will focus o the two differet approaches to show that parity, ad i tur multiplicatio, caot be decided by sub-expoetial-sized costat-depth circuits. The followig sectio (Sectio 1) provides isight o two methods - polyomial approximatios ad radom restrictios. Sectios ad 3 coclude with a lower bouds for parity. 1 Lower Bouds for o costat-depth circuits The literature has cocetrated o two methods to establish lower bouds for parity o costatdepth circuits - Radom Restrictios method - This approach uses a p-radom restrictio (defied i previous lecture) to prove a tight lower boud for circuits - C d ( ) Ω( d 1 1 ). This method also provides similar results for MAJORITY ad Modular gates (MOD k ). Defiitio 1 (Modular gate). A MOD k gate is a gate that outputs 0 if the sum of its iputs (mod k) is 0, ad 1 otherwise. The precise defiitio is as follows - { 0 if MOD k (x) = i x i 0 (mod k) 1 otherwise where x is the gate iput. Polyomial Approximatios method - This method makes use of low-degree polyomial approximatios (over fiite fields) to show that C d ( ) Ω( d 1 ) (d is circuit depth). Although this is a weaker boud i compariso with the radom restrictios method, the proof techique is very iterestig ad it allows the use of MOD 3 gates. Polyomial Approximatios method Theorem 1. Give ay circuit C d of depth d that decides, its size is at least Ω( 1 d ). This result holds eve if we allow additioal MOD 3 gates. Proof. We prove the theorem i two steps. I the first step, we approximate the output of costatdepth circuits usig a low degree polyomial over the field Z 3 with a small error. The choice of Z 3 aids i expressig MOD 3 gates as low degree polyomials. I geeral, for ay prime p, MOD p 1

2 gates have low degree polyomials i the field Z p. I the subsequet step, we show that for the parity fuctio such a low-degree polyomial does ot exist. Step 1: Cosider a circuit C made of AND, OR, NOT ad MOD 3 gates. C ca be represeted as a multivariate polyomial of degree where is the size of the iput usig a polyomial coefficiet for each iput. However, our aim is to express C as a lower degree polyomial, allowig reasoably small errors. To achieve this objective, we must represet every literal ad gate of the circuit i terms of a approximate low-degree polyomial over Z 3 i.e. fid polyomial P Z 3 s.t. o may iputs X {0,1}, P(X) is equal to the value of the gate. We costruct the polyomial iductively from the bottom to top. The base case would be a simple literal: LITERAL (X): I this case, the polyomial is trivially idempotet - P i (X) = X i. The degree of the polyomial is 1 ad there are o errors iduced by this substitutio. I the iductive step, we cosider each of the possible gates. We cosider the followig sequece of gates: NOT ( ): If the iput of a NOT gate is polyomial P(X), the P (X) = 1 P(X) represets the output. This does ot icrease the degree of the polyomial or itroduce ay additioal error. MOD 3 : The output P (X) of the gate is 0 whe m i=1 P i(x) 0 (mod 3). The output is 1 if the resultat sum is either 1 or 1(). Upo squarig the sum of iput polyomials, the polyomial exactly behaves like the MOD 3 gate. Hece, the output polyomial is P (X) = ( m i=1 P i(x)). However, the degree of the polyomial doubles with this gate ad there are o additioal errors. OR (): The output P (X) of the OR gate is 0 iff ( i) P i (X) = 0, or equivaletly, ( i)(1 P i (X) = 1). This ca be represeted as follows: α : P (X) = 1 m (1 P i (X)) (1) The output polyomial is precise but its degree is depedet o the fa-i m ad could be ubouded. I the presece of multiple hierarchies of OR gates the polyomial will have a comparable degree to that of the trivial boud. To obviate this issue, we pick a radomized liear combiatio of iputs. This process is repeated t times for a better approximatio of the OR gate. Let r i Z 3 be the coefficiet associated with P i (X) ad let the output fuctio be defied as: i=1 m β : P (X) = ( r i P i (X)) () i=1 Note that we square the liear combiatio to keep the output P (X) as Boolea. If the actual output of the OR gate is 0, the approximatio (P (X)) is always 0. O the other had, if the output of OR gate is 1, the approximatio errs (outputs 0 whe some P j (X) = 1) with a probability 1/3: Pr(P (X) = 0 m i=1 P i(x) = 1) = 1 3 To verify that this holds, assume that we pick r j at the ed. Sice P j (X) = 1, there is exactly oe value of r j (amog {0,1,} = Z 3 ) that makes the polyomial to output 0.

3 A few key observatios of this approximatio is that we have costructed a polyomial whose degree doubles istead of fa-i (m) of the OR gate. The error itroduced by the approximatio is atmost 1/3. As with ay radomized algorithm, we ca repeat the above procedure for t idepedet trials ad see if the output of at least oe of the them is oe. Combiig the formulatios 1 ad, the fial approximate polyomial is: P (X) = P α(p β 1 (X),...,P β t (X)), (3) where P α ad P β i are the applicatios of α-formulatio ad β-formulatio (for ith trial) respectively. This resultat polyomial has a degree of t times the iput as each applicatio of equatio β- formulatio doubles the degree; ad has a error of atmost µ = 1 3 t as each of the idividual t trials must result i a error. AND (): We ca simulate a AND gate usig the NOT ad OR gates. The resultig approximatio P (X) has characteristics of the OR gate approximatio (t degree icremet ad a error of µ). P P P P MOD 3 m m m P P 1... P m P 1... P m P 1... P m (a) NOT gate (b) MOD 3 gate (c) OR gate (d) AND gate Figure 1: Approximatio of Boolea gates If the depth of the circuit is d, the degree of the polyomial P(X) represetig the etire circuit will be at most (t) d. P(X) gives the wrog value oly if the output of at least oe of the gates i C is wrog. This happes with probability at most µ C (tighter bouds would deped o the umber of AND/OR gates i C). If we cosider all possible iputs (iput size ), the expected umber of iputs for which P(X) will output the wrog value is atmost (µ C ). This shows the existece of radom Z 3 coefficiets for which P(X) is wrog i o more tha the expected umber. More formally, Lemma 1. There exists a choice of r i s such that the umber of iput possibilities for which the approximate polyomial P(X) is ot equivalet to the behavior of the circuit is atmost C 3 t. Proof. For every fixed iput X ad circuit C, the error probability at ay OR/AND gate would be µ = 1 3 t. Hece the upper boud of the error i the whole circuit is µ C - Pr(P(X) C(X)) µ C = C 3 t 3

4 The expected umber of errors give all iput possibilities is - E[#X s.t. (P(X) C(X))] C 3 t Thus, for some choice of r i s we have that - #X s.t. (P(X) C(X)) C 3 t This costructio could be geeralized i ay field Z p, for prime p, usig the MOD p gates. From Fermat s Little Theorem, we ca use the fact that a p 1 1 (mod p) a Z p \{0} to esure Boolea values for AND/OR gate approximatios. This would result i a approximate polyomial of degree of atmost ((p 1) t) d ad maximum error 1 p t. Step : I this step, give a polyomial P(X) of some degree that approximates o a subset G of iputs, we establish a upper boud for the degree of the approximate polyomial P(X) below which every fuctio of iputs has a correspodig polyomial approximatig it over G. By equatig the umber of such fuctios to the umber of polyomials with degrees ot greater tha the established upper boud, we derive the lower boud o the depth of circuit C. Iitially, we trasform the Boolea iputs to a coveiet domai. Usig a simple liear trasformatio from Boolea values {0,1} {1, 1}, we reduce to a simple product of literals. We assume that this polyomial P(X) has degree atmost ad computes correct values o more tha (1 µ) fractio of iputs. Let us represet this set of good iputs as G { 1,1} ad hece G 1 µ. Lemma. Let P be a polyomial of degree at most that represets i=1 x i i a set G { 1,1}. The each fuctio f : G Z 3 has a multivariate polyomial Q over Z 3 of degree at most + such that it represets f, i.e. ( x G)f(x) = Q(x). Proof. Every fuctio f o iputs has a multivariate polyomial of degree at most. This is easy to see as we ca represet every possible iput usig moomials of degree. Let us start from oe such polyomial Q (such that f = Q o G). Cosider a moomial i Q of the form i I x i where I is a subset of the iput bits {1,,...,}. Sice we represet multiplicatio with ±1 iputs, we ca rewrite the moomial as: x i = i I = = x i = i I ( i I x i )( ) x i i I ( )( ) x i x i i I i=1 (4) ( x i )P(X) (5) i I 4

5 Equatio 4 holds for ay iput X of bits as multiplicatio with squares of each ±1 iput does ot chage the value of the expressio. Equatio 5 is a approximatio of multiplicatio ad hece, holds oly for the set of good iputs (G). The LHS of (5) has degree I ad the RHS has a degree at most + Ī = + I. Choosig the miimum degree expressio amog them helps us express ay moomial i Q with a maximum degree of + (average of the degrees). Further, we combie the result of Lemmas 1 ad to prove the theorem. Suppose there exists a circuit C of depth d computig. From Lemma 1, there exists a polyomial P of degree at most = (t) d that computes parity o a set G of relative size at least 1 C 3 t. Cosequetly, from Lemma, all fuctios f : G Z 3 for some good iput set (G) ca be represeted usig a multivariate polyomial of degree at most +. The total umber of such polyomials must be at least the umber of fuctios f from G to Z 3. The umber of multivariate polyomials with degree at most + is exactly 3 M where M is the umber of moomials of degree at most +. There are ( i) moomials of degree i, ad hece M = + i=0 ( ) i ( The umber of moomials of degree will be 1 (half of the possible moomials as ( i) = ) i ). Each of the remaiig = Θ( ) terms i the summatio will be lower tha ( ) ( ( ) i max = ( ). Usig Stirlig s approximatio, we ca show that: ) ( ) ( ) = Θ Thus, ( ) M = 1 + Θ ( ( )) 1 = + Θ The umber of fuctios of the form G Z 3 is 3 G as each elemet of G is assiged to oe of 3 possible values of Z 3. As the umber of fuctios of this form must be at most the umber of polyomials of degree at most (+ )/, 3 G 3 M or, i other words, G M. This gives us the followig boud o the size of G. 1 µ C = G M 1 ( ) + Θ From Lemma 1, µ 1 3 t whe = (t) d. Thus, 1 C 3 t 1 µ C 1 ( ) (t) d + Θ [ ( )] 1 (t) = C 3 t d Θ (6) (7) 5

6 For equatio (6) to be meaigful, the RHS should be less tha 1. Thus, settig (t) d = O( ) gives a appropriate value for the RHS i the equatio (7). Thus, t = Θ( d) 1 ad this gives C Ω( d 1 ). The oly part of the above aalysis that chages whe workig over Z p rather tha Z 3 is that = ((p 1) t) d rather tha (t) d. Thus, the result holds with the same lower boud o C for Boolea circuits with MOD p gates for ay prime p. I fact, the argumet i the above proof ca be geeralized to give a lower boud for circuits with MOD p gates to compute MOD q for distict primes p ad q (recall that parity is the special case of q = ). This is achieved by viewig Step as harmoic aalysis over Z ad the geeralizig that to harmoic aalysis over Z q. As this geeralizatio takes a bit of work to prove, we leave it at that. We further use the proof above to give a lower boud o circuits that eve approximate parity. Corollary 1. A depth d ubouded fa-i circuit that agrees with parity ( ) o a fractio at least of {0,1} must have size Ω(ǫ/d). (1 ǫ)/ Proof. Let C be a circuit that is correct o at least ( 1 + ρ) of the iputs for ρ > 0. Similar to Theorem 1, we ca prove that there exists a polyomial of degree = (t) d that is correct o a set G that is at least 1 + ρ C 3 of {0,1}. From Step of the proof above, t 1 + ρ C 3 t 1 ( (t) d ) + Θ = ρ C ( ) 3 t Θ (8) ( )] (t) = C 3 [ρ t d Θ (9) Note that the (t) d / term is Ω(1/ ), so ρ must also be Ω(1/ ) to esure the lower boud we get is eve positive. If we let ρ = 1/ (1 ǫ)/, we set (t) d = Θ( ǫ/ ) to optimize the RHS of equatio (9). Hece, t = Θ( ǫ/(d) ), ad we get that C Ω(ǫ/(d)). The above corollary proves the iapproximability of the parity fuctio usig costat-depth circuits. There is a stroger result that ca be proved usig radom restrictios. 3 Radom Restrictios method I this sectio, we provide a alterate proof to Theorem 1. The boud for the circuit size is tighter tha what was achieved by the polyomial approximatios method i Sectio. However, the proof does ot allow the circuit to cotai additioal MOD 3 gates. A restrictio fixes some iputs of a circuit to costat values, ad leaves other iputs free. More formally, we defie radom restictios as the followig: Defiitio (Radom Restrictios). A p-radom restrictio o variables is a radom fuctio ρ : {x 1,...,x } {,0,1}, such that for each i, idepedetly, Pr[ρ(x i ) = ] = p ad Pr[ρ(x i ) = 0] = Pr[ρ(x i ) = 1] = 1 p. If ρ(x i) =, the we leave x i as a variable. Otherwise we set it to the result of ρ(x i ). Note that if we apply a radom restrictio to a parity fuctio, we get a parity fuctio or its complemet depedig o the iputs correspodig to. 6

7 Theorem. Give ay circuit C d of depth d that computes, its size is at least Ω( 1 d 1 ) Proof. To prove this theorem, let us assume a eat structure for the circuit C d that we cosider - NOT ( ) gates do t cout towards the size or depth of the circuit. The circuit gates are orgaized i alteratig layers of AND ad OR gates. We further show that these assumptios do t affect the bouds. Lemma 3. If C d is a circuit of size C d ad depth d, the there is a circuit C d of size at most C d ad depth d that computes the same fuctio ad such that all the NOT gates are applied at the iput level. Proof. We prove the stroger statemet that, for every gate g of C d, there is a gate g i C d whose output is the complemet of g. The we just let the output of C d be the complemet of the output gate of C d. Let us order the gates of C d as g 1,...,g Cd i such a way that if the gate g i uses the output of gate g j as a iput the j < i. We describe a iductive costructio. I the base case, if g 1 is a AND gate (OR gate), the g 1 is a OR gate (AND gate), whose iputs are the complemets of the iputs of g 1. It follows from De Morga s law that the output of g 1 is the complemet of the output of g 1. I the iductive step, if we have costructed gates g 1,...,g i whose outputs are the complemet of g 1,...,g i, the g i+1 has similar complemeted iputs as g i+1 ad from De Morga s law, it follows that the output of g i+1 is the complemet of the output of g i+1. Lemma 4. If C d is a circuit of size C d ad depth d, the there is a circuit C d of size at most d C d ad depth d that computes the same fuctio ad such that the gates are arraged i d sequetial layers which have alteratig AND ad OR gates. Proof. The associativity of AND ad OR esures that every iput-output path i the circuit has a alteratio betwee AND gates ad OR gates. If there is a AND gate (OR gate) g i the circuit oe of whose iputs is comig from aother AND gate (OR gate) g : the we ca coect the iputs of g directly to g. This does ot alter the size or depth of the circuit. Repeatedly mergig the similar gates, we evetually get a alteratig circuit C d which has the same size ad depth of C d. Fially, we arrage the gates i d layers so that each layer is etirely AND or OR gates, ad wires go oly from lower-umbered layers to higher-umbered layers. Fially, we replace all wires that jump may layers by a path of alteratig fa-i 1 AND gates ad fa-i 1 OR gates. This step icreases the size of the circuit by atmost a factor of d. Further, we use the followig lemma (Switchig Lemma) to establish a cotradictio for a polyomial-sized circuit C d that decides PARITY. Lemma 5 (Switchig Lemma). Let ϕ be a k-cnf formula. We the apply a ρ-radom restrictio that leaves a fractio p of variables uassiged. For the remaiig fractio (1 p) of the variables, idepedetly hardwire 0 or 1 based o the radom restrictio. The, for every k, Pr[ϕ ρ caot be expressed as a k -DNF ] (5pk) k 7

8 Note that the statemet is trivial if the restrictio ca set all variables (p = 1). For practical purposes, the values of p, k ad k should cocur with the iequality - C (5pk) k < 1. Proof Idea. Cosider a AND of ORs, where each OR has size at most k. Notice that a radom restrictio is ot very likely to set a OR to 0 sice all literals ivolved eed to be set to 0 for that to happe. But if k is small, there is a otrivial probability that this happes. There are two cases: 1. There are a large umber of pairwise disjoit ORs. I that case, there are may idepedet evets that ca set the AND gate to 0, amely each of those pairwise disjoit ORs beig set to 0. Sice each of those evets happes with a otrivial probability, the odds are that the radom restrictio will set the AND gate to 0, i which case it ca trivially be writte as a DNF will small bottom fa-i.. There is ot a large umber of pairwise disjoit ORs. Let V be a miimal set of variables such that each OR queries at least oe variable from V. Sice there is a lot of overlap amog the ORs, V is small. If we query all the variables i V, the we have essetially reduced our problem to a simpler oe of the same type, amely the trasformatio of a CNF with bottom fa-i at most k 1. This is because each of the ORs cotais at least oe literal i V. We the repeat the case distictio to that simpler problem, depedig o the settig of the variables i V. Alog every brach of this process, we will evetually ed up i case 1. Sice there are at most k steps ad each step ivolves queryig a small umber of variables, we ed up with a decisio tree of small depth that represets the give CNF uder a radom restrictio with high probability. A decisio tree of small depth ca be tured ito a DNF with small bottom fa-i by writig dow a OR over all paths i the decisio tree that lead to acceptace of the AND of all the coditios that defie the path..... ρ.... k k Figure : k CNF to k DNF 8

9 Give a circuit C d that decides PARITY, we ca rewrite it i terms of alterate levels of AND or ORs without much chage i size (Lemmas 3 ad 4). Note that we ca also switch from a DNF to a CNF by cosiderig the egatio of the circuit. Depth d.... ρ.... Depth 3 Depth MERGE Depth 1 k k Figure 3: Decreasig the depth by mergig adjacet levels while maitaiig a small bottom fa-i Next, we use repeated ivocatios of Switchig Lemma to reduce the circuit depth by 1 i each iteratio (as show i Figure 3) util we are left with a circuit of depth. Suppose that the bottom gates are ANDs (ORs). To apply the switchig lemma, we eed to esure the gates at the bottom of the circuit have small fa-i. To esure this, we isert dummy OR (AND) gates below the AND (OR) gates. Namely, for each iput x to the AND (OR) gate, we replace that with x OR x. Now, we apply the switchig lemma to the AND of ORs (OR of ANDs) we have created. With high probability, each applicatio is successful i creatig a OR of ANDs (AND of ORs) with small bottom fa-i ad without settig too may variables. Now the secod bottom-most ad third bottom-most levels are both ORs (ANDs) ad ca be merged. This reduces the depth of the circuit by 1 (back dow to d sice we added a level iitially) without addig extra gates. Now the circuit still has small bottom fa-i, so we ca apply the switchig lemma agai. We repeat this process util we get a circuit C of depth. At this poit, if C computed, the C computes m o some m-subset of the variables (those that were uset by the radom restrictios; E(m) = p d 1 ). 9

10 This umber of uset variables m is larger tha the small bottom fa-i of the resultig k CNF (or k DNF) circuit C ad hece, PARITY caot be decided by C. Therefore, we arrive at a cotradictio. Corollary. The size of circuit C d of costat depth d required to compute correctly o atleast ( /d ) fractio of the iputs, is atleast Ω( 1 d ). C d ( ) Ω( 1 d ) Proof. The proof follows similar ideas as that of Corollary 1, but is more ivolved. 4 Next Lecture I the ext lecture, we will focus o parallelism. We will use circuits to model parallel computatios. 5 Refereces M. Furst, J.B. Saxe, M. Sipser. Parity, circuits, ad the polyomial-time hierarchy. Mathematical Systems Theory, 17(1), p.13-7, J. Hastad. Computatioal limitatios for small depth circuits. MIT Press, Ph.D. thesis. N. Liial, Y. Masour, N. Nisa. Costat depth circuits, Fourier trasform, ad learability. I Proceedigs of the 30th Aual Symposium o Foudatios of Computer Sciece (FOCS 89). p A.A. Razborov. Lower bouds o the size of bouded depth etworks over a complete basis with logical additio. Matematicheskie Zametki, 41, p , Roma Smolesky. Algebraic methods i the theory of lower bouds for boolea circuit complexity. I Proceedigs of the 19th ACM Symposium o Theory of Computig, pages 77-8, Ackowledgemets I writig the otes for this lecture, I perused the otes by Seeu William Umboh ad Piramaayagam Arumuga Naiar for Lecture 08 ad Lecture 09 from the Sprig 007 offerig of CS 810, ad the otes by Chris Hopma ad Phil Rydzewski for Lecture 08 ad Lecture 09 respectively from the Sprig 010 offerig of CS